Compiler automatic tuning method based on discrete particle swarm algorithm

Abstract

The application relates to a compiler automatic tuning method based on a discrete particle swarm algorithm, which improves the particle swarm algorithm to adapt to a discrete solution space of a compiler optimization problem, and proposes a speed updating strategy combining a crowded distance and fitness, balances the moving speed of particles by evaluating the crowded distance and fitness of solution samples in a region where the current particles are located, selects the best optimization sequence, and thus improves the performance during compiler automatic tuning; solves the premature convergence problem existing in many current discrete particle swarm algorithms and the problem of insufficient global search capability when the problem dimension is high, and significantly improves the tuning effect of compiler automatic tuning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer program source program compilation optimization, in particular to a compiler automatic tuning method based on a discrete particle swarm algorithm. BACKGROUND

[0002] Compilers such as GCC and LLVM convert source programs written in programming languages such as C and C++ into executable programs, and support optimization through compilation optimization options to improve the runtime performance of programs. However, the pre-defined optimization levels (such as -O1, -O2, -O3) usually cannot achieve the best performance for each specific program, so it is necessary to carefully tune the optimization flags and order for each program, especially for runtime performance sensitive applications.

[0003] But it is obviously very time-consuming and unrealistic to manually analyze and tune each new program, because it is a combinatorial optimization problem, and the ordering combination of hundreds of optimization options constitutes an exponential level of search space. Therefore, the research on compiler automatic tuning becomes very important, which has important significance for improving the intelligence of the compiler and improving the efficiency of the program. In particular, in the field of high-performance computing (HPC), performance optimization is particularly important, because these systems usually run compute-intensive tasks such as climate simulation, genome sequencing, and high-energy physics computing. These tasks often need to run for a long time and occupy a large amount of computing resources, so even a small performance improvement can bring significant cost savings and computing efficiency improvements. In the context of HPC, compiler tuning can fully utilize hardware features and architectural advantages, reduce computing time and energy consumption, and improve the efficiency of scientific research and engineering computing. Therefore, in the context of HPC, compiler automatic tuning is not only the key to improving the performance of individual applications, but also the necessary means to improve the overall system performance and resource utilization.

[0004] In the prior art, many methods have been proposed to automatically adjust compiler optimization flags using various heuristic algorithms to achieve the required runtime performance of a given program. Typically, these methods use specific search strategies to iteratively test different combinations of optimization flags, and eventually output the sequence with the best performance when the termination condition is reached. Although these methods have proven to be effective to some extent, they still have some limitations. More specifically, since heuristic algorithms usually set optimization flags to generate sequences to evaluate their fitness, these methods tend to ignore the characteristics of the program's intermediate representation, and are prone to premature convergence and insufficient global search capability when the dimension is high. SUMMARY

[0005] To overcome at least one of the deficiencies in the prior art, the present application provides a compiler automatic tuning method based on a discrete particle swarm algorithm.

[0006] In a first aspect, a compiler automatic tuning method based on a discrete particle swarm algorithm is provided, comprising:

[0007] Step S1, generating an initial search space; the samples in the initial search space are feature vector sets corresponding to optimization flag sequences;

[0008] Step S2, randomly selecting K samples from the initial search space and determining the execution speedup ratio corresponding to each sample; constructing a training set based on the K samples and the execution speedup ratio, and training a multi-layer perceptron based on the training set;

[0009] Step S3, in the current iteration, for each particle, selecting a sample closest to the particle in the initial search space as a to-be-evaluated sample; inputting the to-be-evaluated sample into the multi-layer perceptron to obtain the predicted speedup ratio of the to-be-evaluated sample;

[0010] Step S4, sorting all predicted speedup ratios from large to small, and selecting the to-be-evaluated samples corresponding to the top three predicted speedup ratios in the sorting result as samples with better prediction effect;

[0011] Step S5, selecting the sample with the maximum execution speedup ratio from the samples with better prediction effect as the best sample of the current iteration; determining whether the current iteration reaches the maximum number of iterations, if yes, outputting the best sample, if no, executing step S6;

[0012] Step S6, adding the best sample and its corresponding execution speedup ratio to the training set; determining an optimization flag sub-sequence that has an important influence on optimization effect according to the samples with better prediction effect, and adding the optimization flag sub-sequence to the initial search space;

[0013] Step S7, calculating the crowding distance of each particle, and updating the position and moving speed of each particle based on the crowding distance of each particle and the fitness of each particle;

[0014] Step S8, determining whether the current iteration is a round of search space update, if yes, returning to step S2, if no, returning to step S3.

[0015] In one embodiment, in step S1, the initial search space is generated, comprising:

[0016] Randomly generating a plurality of optimization flag sequences according to the optimization flag set of the compiler;

[0017] Each optimization flag sequence compiles the source program into a set of intermediate representations;

[0018] Extract a feature vector from the intermediate representation set to generate a feature vector set;

[0019] The feature vector set corresponding to all the optimization flag sequences constitutes an initial search space.

[0020] In one embodiment, in step S2, determining the execution speedup ratio corresponding to each sample comprises:

[0021] Compiling the source program into a binary executable file based on the optimization flag sequence corresponding to each sample; executing the executable file to obtain an execution time; and determining the execution speedup ratio based on the execution time and the execution time after optimization at the highest optimization level provided by the compiler.

[0022] In one embodiment, in step S6, determining the optimization flag subsequence that has a significant impact on the optimization effect according to the sample with a better prediction effect comprises:

[0023] Constructing a directed weighted graph according to the optimization flag sequence corresponding to the sample with a better prediction effect; the nodes of the directed weighted graph are optimization flags, the directed edges are the order relationships of the optimization flags in the sequence, and the weights of the directed edges are the number of occurrences of the order relationships;

[0024] Removing the directed edges with weights not meeting a set value multiple times in the directed weighted graph to obtain multiple subgraphs;

[0025] The optimization flag sequence corresponding to each subgraph is the optimization flag subsequence that has a significant impact on the optimization effect.

[0026] In one embodiment, in step S7, calculating the crowding distance of each particle comprises:

[0027] Using the DBSCAN clustering algorithm to cluster the samples in the initial search space and dividing all the samples into multiple categories;

[0028] For each particle, calculating the distance between the particle and each category and selecting the category corresponding to the minimum distance as the neighbor set of the particle;

[0029] Calculating the distance between the particle and each sample in the neighbor set, sorting the distances from small to large, and calculating the average of the first k distances in the sorting result as the crowding distance of the particle.

[0030] In one embodiment, in step S7, updating the position and moving speed of each particle based on the crowding distance of each particle and the fitness of each particle comprises:

[0031] Determining the moving direction of the particle:

[0032] vec(d) = v(d-1) + r1-(pbest(d-1)-x(d-1)) + r2-(gbest(d-1)-x(d-1))

[0033] Wherein, vec(d) is the moving direction of the particle in the current iteration d, v(d-1) is the moving speed of the particle in the previous iteration d-1, pbest(d-1) is the individual optimal position in the previous iteration d-1, x(d-1) is the position of the particle in the previous iteration d-1, gbest(d-1) is the global optimal position in the previous iteration d-1; r1 and r2 are the individual optimal coefficient and the global optimal coefficient respectively;

[0034] Determine the moving speed of the particle:

[0035]

[0036] Wherein, v(d) is the moving speed of the particle in the current iteration d, cd(d) is the crowding distance of the particle in the current iteration d, cur_score is the fitness of the particle in the previous iteration d-1.

[0037] Update the position of the particle:

[0038] x(d) = x(d-1) + v(d)

[0039] Wherein, x(d) is the position of the particle in the current iteration d, x(d-1) is the position of the particle in the previous iteration d-1.

[0040] In the second aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the automatic tuning method of the compiler based on the discrete particle swarm algorithm is realized.

[0041] In the third aspect, a computer program product is provided, and the computer program product includes computer program / instructions. When the computer program / instructions are executed by a processor, the automatic tuning method of the compiler based on the discrete particle swarm algorithm is realized.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] 1. The application improves the particle swarm algorithm to adapt to the discrete solution space of the compiler optimization problem, and proposes a speed update strategy combining congestion distance and fitness. The speed of the particle is dynamically balanced by evaluating the congestion distance and fitness of the solution sample in the area where the particle is located, so as to select the best optimization sequence and improve the performance of the compiler automatic tuning. The premature convergence problem of many discrete particle swarm algorithms and the problem of insufficient global search ability when the problem dimension is high are solved, and the tuning effect of the compiler automatic tuning is significantly improved.

[0044] 2. The application has good scalability and can find an optimization strategy suitable for most programs in a complex and variable optimization environment. The running performance of the source program optimized by the application is improved more than that of the source program optimized by the traditional method, and the application has a broad application prospect and market potential. BRIEF DESCRIPTION OF DRAWINGS

[0045] The application can be better understood by referring to the following description in conjunction with the accompanying drawings, which form a part of this specification and are included to further illustrate the application. In the drawings:

[0046] Figure 1 A flow chart of a compiler automatic tuning method based on a discrete particle swarm algorithm according to an embodiment of the application is shown;

[0047] Figure 2 An acceleration result graph of the automatic tuning result of the method of the application compared with O3 is shown;

[0048] Figure 3 A program acceleration ratio result graph with the increase of the number of iterations is shown. DETAILED DESCRIPTION

[0049] In the following, exemplary embodiments of the application will be described with reference to the accompanying drawings. In the specification, all features of the actual embodiments are not described for the sake of clarity and brevity. However, it should be understood that many embodiment-specific decisions can be made in the process of developing any such actual embodiment to achieve the specific goals of the developer, and these decisions can vary from embodiment to embodiment.

[0050] It should also be noted here that, in order to avoid obscuring the application due to unnecessary details, only the device structure closely related to the scheme according to the application is shown in the drawings, and other details not closely related to the application are omitted.

[0051] It should be understood that the present application is not limited to the described embodiments as a result of the following description with reference to the drawings. In this context, the embodiments can be combined with one another, features can be replaced or borrowed between different embodiments, or one or more features can be omitted in one embodiment, if possible.

[0052] The embodiment of the present application provides a compiler automatic tuning method based on a discrete particle swarm algorithm, Figure 1 A flow chart of the compiler automatic tuning method based on the discrete particle swarm algorithm is shown according to the embodiment of the present application, referring to FIG., the method comprises the following steps:

[0053] In step S1, an initial search space is generated; samples in the initial search space are feature vector sets corresponding to optimization flag sequences.

[0054] The optimization flag set O of the compiler is represented as:

[0055] O={O1,O2,…O m}

[0056] Each element in the optimization flag set O represents an optional optimization flag.

[0057] A plurality of optimization flag sequences are randomly generated according to the optimization flag set of the compiler; each optimization flag sequence S can be represented as S={S1,S2,…S i …S n},wherein S i ∈O is the i th optimization flag in the optimization flag sequence S.

[0058] Each optimization flag sequence compiles a source program into a set of intermediate representations; each optimization flag sequence corresponds to an intermediate representation, so that the conversion from the optimization sequence to the intermediate representation is completed, and the set of intermediate representations IRSet is represented as:

[0059] IRSet={I1,I2,…,I n}

[0060] Feature vectors are extracted from the set of intermediate representations to generate a feature vector set Vec={V1,V2,…,V n};

[0061] The feature vector set corresponding to all optimization flag sequences constitutes the initial search space. In this way, the mapping from the optimization flag sequence to the feature vector is completed, and the problem of finding an optimal optimization flag sequence is converted into the problem of finding an optimal feature vector of the intermediate representation.

[0062] Step S2, randomly picking K samples from the initial search space, and determining an execution speedup ratio corresponding to each sample; constructing a training set based on the K samples and the execution speedup ratios, and training a multi-layer perception (MLP), i.e., a cost model, based on the training set.

[0063] Specifically, the execution speedup ratio corresponding to each sample is determined, including:

[0064] Compiling the source program into a binary executable file based on the optimization flag sequence corresponding to each sample; executing the executable file to obtain an execution time; determining the execution speedup ratio based on the execution time and the execution time after O3 optimization of the compiler.

[0065] Step S3, in the current iteration, for each particle, selecting a sample closest to the particle in the initial search space as an evaluation sample; inputting the evaluation sample into the multi-layer perception to obtain a predicted speedup ratio of the evaluation sample.

[0066] Here, if the current iteration is the first iteration, the number of particles needs to be set first, and the position and moving speed of each particle are initialized. By calculating the distance between the position of each sample in the initial search space and the position of the particle, a sample closest to the particle is selected as the evaluation sample.

[0067] Step S4, sorting all predicted speedup ratios from large to small, and selecting the evaluation samples corresponding to the top three predicted speedup ratios in the sorting result as samples with better prediction effect.

[0068] Step S5, selecting a sample with the largest execution speedup ratio from the samples with better prediction effect as the best sample of the current iteration; determining whether the current iteration reaches the maximum number of iterations, if yes, outputting the best sample, if not, executing step S6.

[0069] Here, the optimization flag sequence corresponding to the output best sample is the best optimization flag sequence, and the execution performance of the executable file generated by compiling the source program using the best optimization flag sequence is the best.

[0070] Step S6, adding the best sample and the execution speedup ratio corresponding thereto to the training set; determining an optimization flag sub-sequence having an important influence on the optimization effect according to the samples with better prediction effect, and adding the optimization flag sub-sequence to the initial search space.

[0071] Here, the training set is updated so that the prediction effect of the trained multilayer perceptron is more accurate. The optimization flag subsequence is added to the initial search space to provide more reliable options for generating new samples subsequently.

[0072] In step S7, the crowding distance of each particle is calculated, and the position and moving speed of each particle are updated based on the crowding distance of each particle and the fitness of each particle.

[0073] In step S8, it is determined whether the current iteration is a round of search space update. If yes, the process returns to step S2. If no, the process returns to step S3. Here, the round of search space update can be, for example, a multiple of 10, that is, the process returns to step S2 after every 10 iterations so as to reselect samples from the updated initial search space.

[0074] The embodiment improves the particle swarm algorithm to adapt to the discrete solution space of the compiler optimization problem, and proposes a speed updating strategy combining the crowding distance and the fitness. The moving speed of the particle is dynamically balanced by evaluating the crowding distance and the fitness of the solution samples in the region where the current particle is located, so as to select the best optimization sequence and improve the performance of the compiler automatic tuning. The early convergence problem existing in many discrete particle swarm algorithms and the problem of insufficient global search capability when the problem dimension is high are solved, and the tuning effect of the compiler automatic tuning is significantly improved.

[0075] In one embodiment, in step S6, the optimization flag subsequence that has an important influence on the optimization effect is determined according to the sample with a better prediction effect, including:

[0076] A directed weighted graph is constructed according to the optimization flag sequence corresponding to the sample with a better prediction effect. The nodes of the directed weighted graph are optimization flags, the directed edges are the order relations of the optimization flags in the sequence, and the weights of the directed edges are the number of times of the occurrence of the order relations.

[0077] In the directed weighted graph, the directed edges with weights not meeting a set value are removed multiple times to obtain multiple subgraphs. Here, the directed edges with lower weights can be removed, for example, the directed edges with weights of 1 are removed to obtain a subgraph, and the directed edges with weights of 1 and 2 are removed to obtain a second subgraph. The set value can be set according to actual conditions and is not specifically limited.

[0078] The optimization flag sequence corresponding to each subgraph is the optimization flag subsequence that has an important influence on the optimization effect.

[0079] In one embodiment, in step S7, the crowding distance of each particle is calculated, including:

[0080] The DBSCAN clustering algorithm is used to cluster the samples in the initial search space, and all the samples are divided into multiple categories.

[0081] For each particle, the distance between the particle and each category is calculated, and the category corresponding to the minimum distance is selected as the neighbor set of the particle;

[0082] The distance between the particle and each sample in the neighbor set is calculated, and the distances are sorted from small to large. The average of the top k distances in the sorting result is calculated as the crowded distance of the particle. The following formula is used to represent it:

[0083]

[0084] Wherein, cd(i) is the crowded distance of particle i, and j is the sample label in the neighbor set of particle i.

[0085] In one embodiment, in step S7, the position and moving speed of each particle are updated based on the crowded distance of each particle and the fitness of each particle, including:

[0086] Determine the moving direction of the particle:

[0087] vec(d) = v(d-1) + r1 · (pbest(d-1) - x(d-1)) + r2 · (gbest(d-1) - x(d-1))

[0088] Wherein, vec(d) is the moving direction of the particle in the current iteration d, v(d-1) is the moving speed of the particle in the previous iteration d-1, pbest(d-1) is the individual optimal position in the previous iteration d-1, x(d-1) is the position of the particle in the previous iteration d-1, gbest(d-1) is the global optimal position in the previous iteration d-1; r1 and r2 are the individual optimal coefficient and the global optimal coefficient respectively; here, gbest(d-1) refers to the position of the best sample output in the previous iteration d-1.

[0089] Determine the moving speed of the particle:

[0090]

[0091] Wherein, v(d) is the moving speed of the particle updated in the current iteration d, cd(d) is the crowded distance of the particle in the current iteration d, and cur_score is the fitness of the particle in the previous iteration d-1.

[0092] Update the position of the particle:

[0093] x(d) = x(d-1) + v(d)

[0094] Wherein, x(d) is the position of the particle updated in the current iteration d, and x(d-1) is the position of the particle in the previous iteration d-1.

[0095] To further verify the effectiveness of the method of the present application, the following experimental analysis is performed.

[0096] The method of the present application and the highest optimization option O3 level of the LLVM compiler are compared, and the optimal optimization option sequence found by the method of the present application and the O3 option are used to compile the source program, Table 1 shows the list of source programs, and then the running time of the executable files generated is compared. It can be understood from the -mllvm-debug-pass=Arguments instruction that the optimization flag sequence of O3 uses a total of 256 optimization options (the options have repeated occurrences), so the length of the generated sequence is adjusted to be the same as the O3 sequence. And each time the running time performance is evaluated, each executable file is executed three times to take the average value to eliminate the influence of accidental events.

[0097] Figure 2 The acceleration result graph of the automatic tuning result of the method of the present application compared with O3 is shown, in which the horizontal axis is different source programs, and the vertical axis is the running time when executing how many times the acceleration is generated compared with O3. It can be seen from the graph that the tuning method of the present application can find a very good optimization sequence for most programs, in which the highest acceleration reaches 1.5 times, but the minimum is less than 1.1. It can be seen that although the tuning method of the present application can significantly improve the performance in some cases.

[0098] Table 1

[0099]

[0100] Figure 3 The program acceleration ratio result graph with the increase of the number of iterations is shown. With the increase of the number of iterations, the acceleration ratio of most programs is improved. For example, the acceleration ratio of program P1 is 1.08 at 30 iterations, and is improved to 1.22 at 50 iterations. Similarly, the acceleration ratio of program P6 is improved from 1.11 to 1.38.

[0101] The embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned compiler automatic tuning method based on the discrete particle swarm algorithm.

[0102] The embodiment of the present application provides a computer program product, including computer programs / instructions, which are executed by a processor to implement the above-mentioned compiler automatic tuning method based on the discrete particle swarm algorithm.

[0103] The above merely provides the various embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A compiler auto-tuning method based on a discrete particle swarm algorithm, characterized in that, The method comprises the following steps: Step S1, generating an initial search space; The samples in the initial search space are feature vector sets corresponding to optimization flag sequences; Step S2, randomly selecting K samples from the initial search space, and determining the execution speedup ratio corresponding to each sample; constructing a training set based on the K samples and the execution speedup ratio, and training a multilayer perceptron based on the training set; Step S3, in the current iteration, for each particle, select a sample closest to the particle in the initial search space as a to-be-evaluated sample; input the to-be-evaluated sample into the multilayer perceptron to obtain the predicted speedup ratio of the to-be-evaluated sample; Step S4, sort all the predicted speedup ratios from large to small, and select the to-be-evaluated samples corresponding to the top three predicted speedup ratios as the samples with better prediction effect; Step S5, select the sample with the maximum execution speedup ratio from the samples with better prediction effect as the best sample of the current iteration; determine whether the current iteration reaches the maximum iteration number, if yes, output the best sample, if not, execute step S6; Step S6, add the best sample and the execution speedup ratio corresponding to the best sample to the training set; determine the optimization flag subsequence that has an important influence on the optimization effect according to the samples with better prediction effect, and add the optimization flag subsequence to the initial search space; Step S7, calculate the crowding distance of each particle, and update the position and moving speed of each particle based on the crowding distance of each particle and the fitness of each particle; Step S8, determine whether the current iteration is the round of search space update, if yes, return to step S2, if not, return to step S3; In step S1, the initial search space is generated, comprising: Randomly generating a plurality of optimization flag sequences according to the optimization flag set of the compiler; Each optimization flag sequence compiles the source program into a set of intermediate representations; Extracting feature vectors from the set of intermediate representations to generate a set of feature vectors; The set of feature vectors corresponding to all optimization flag sequences constitutes the initial search space; In step S2, the execution speedup ratio corresponding to each sample is determined, comprising: Compiling the source program into a binary executable file based on the optimization flag sequence corresponding to each sample; executing the executable file to obtain the execution time; determining the execution speedup ratio based on the execution time and the execution time after optimization of the highest optimization level provided by the compiler.

2. The method of claim 1, wherein, In step S6, the optimization flag subsequence that has an important influence on the optimization effect is determined according to the samples with better prediction effect, comprising: Constructing a directed weighted graph according to the optimization flag sequence corresponding to the samples with better prediction effect; the nodes of the directed weighted graph are optimization flags, the directed edges are the order relationship of optimization flags in the sequence, and the weight of the directed edge is the number of times of the order relationship appearing; Removing the directed edges with weights not meeting the set value in the directed weighted graph multiple times to obtain a plurality of subgraphs; The optimization flag sequence corresponding to each subgraph is the optimization flag subsequence that has an important influence on the optimization effect. In step S6, the optimization flag subsequence that has an important influence on the optimization effect is determined according to the samples with better prediction effect, comprising:

3. The method of claim 1, wherein, Constructing a directed weighted graph according to the optimization flag sequence corresponding to the samples with better prediction effect; the nodes of the directed weighted graph are optimization flags, the directed edges are the order relationship of optimization flags in the sequence, and the weight of the directed edge is the number of times of the order relationship appearing; Removing the directed edges with weights not meeting the set value in the directed weighted graph multiple times to obtain a plurality of subgraphs; The optimization flag sequence corresponding to each subgraph is the optimization flag subsequence that has an important influence on the optimization effect. In step S7, the crowded distance of each particle is calculated, including: Using the DBSCAN clustering algorithm, the samples in the initial search space are clustered, and all samples are divided into multiple categories; For each particle, the distance between the particle and each category is calculated, and the category corresponding to the minimum distance is selected as the neighbor set of the particle; The distance between the particle and each sample in the neighbor set is calculated, and the distances are sorted from small to large. The average of the first k distances in the sorting result is calculated as the crowded distance of the particle.

4. The method of claim 1, wherein, Wherein, In step S7, the position and moving speed of each particle are updated based on the crowded distance of each particle and the fitness of each particle, including: Determining the moving direction of the particle: wherein, is the current iteration is the moving direction of the particle, is the previous iteration is the moving speed of the particle, is the previous iteration is the individual optimal position of the previous iteration, is the previous iteration is the position of the particle, is the previous iteration is the global optimal position of the previous iteration; and are the individual optimal coefficient and the global optimal coefficient, respectively. Determining the moving speed of the particle: wherein, is the current iteration updated velocity of the particle, is the current iteration crowding distance of the particle, is the previous iteration fitness of the particle; Updating the position of the particle: wherein, is the current iteration updated position of the particle, previous iteration position of the particle.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the compiler automatic tuning method based on the discrete particle swarm algorithm in any one of claims 1-4.

6. A computer program product, characterised in that, Including computer programs / instructions, which are executed by the processor to implement the compiler automatic tuning method based on the discrete particle swarm algorithm in any one of claims 1-4.

Images