A parallel compilation method based on intermediate representation
By sorting and grouping the executable units of the target intermediate representation by weight, the problem of excessive compilation time in the prior art is solved, and the load balancing and efficiency improvement of the compilation task are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING NORI INTEGRATED CIRCUIT DESIGN CO LTD
- Filing Date
- 2026-04-16
- Publication Date
- 2026-07-10
AI Technical Summary
In existing parallel compilation methods, the uneven distribution of workload among executable units leads to excessively long compilation times and low compilation efficiency.
By obtaining the weights of the executable units in the intermediate representation of the target, sorting them from largest to smallest weight, and assigning them to n groups according to their weights, ensuring that the average weight of each group is close, and then assigning them to n threads for parallel execution.
Load balancing of compilation tasks was achieved, reducing overall compilation time and improving compilation efficiency.
Smart Images
Figure CN122363705A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of compiler technology, and in particular to a parallel compilation method based on intermediate representation. Background Technology
[0002] In existing technologies, parallel compilation typically involves splitting the work into multiple threads based on the number of executable units. This approach suffers from the problem that the workload allocated to each independent thread is not evenly distributed. Larger executable units require more compilation time and can easily become bottlenecks. For example, if the compilation of a large executable unit (e.g., executable unit A) and a small executable unit (e.g., executable unit B) are assigned to the same thread, and that thread compiles executable unit A first, the increased compilation time for unit A might mean that other threads have already finished compiling while the compilation of executable unit B has yet to begin, resulting in a longer overall compilation time and lower compilation efficiency. Summary of the Invention
[0003] The purpose of this invention is to provide a parallel compilation method based on intermediate representation, so as to reduce compilation time and improve compilation efficiency.
[0004] According to the present invention, a parallel compilation method based on intermediate representation is provided, the method comprising the following steps: S100: Obtain the total number of executable units included in the target intermediate representation. If the total number is greater than the preset number of groups n, proceed to S200; otherwise, group each executable unit included in the intermediate representation separately and determine that the grouping is complete.
[0005] S200, the executable units included in the intermediate representation are sorted according to the weights corresponding to the executable units included in the target intermediate representation; the weight corresponding to any executable unit is positively correlated with the time required for the compiler to compile the executable unit.
[0006] S300, according to the sorting result, the executable units included in the intermediate representation are allocated to n groups; wherein, the allocation process includes: when the current group is not the last group, if the maximum weight of the executable units in the current executable unit list is greater than or equal to the average weight avg of the current group, then the executable unit func1 with the largest weight in the current executable unit list is set as the current group alone; if it is less, then func1 is added to the current group, and the executable units in the executable unit list except func1 are traversed in descending order of weight, and the executable units with weights not exceeding the current remaining target difference are selected in turn and added to the current group, and the current remaining target difference is updated until the current remaining target difference is not greater than 0 or the traversal ends, and the executable units added to the current group are removed from the executable unit list; the initial value of the current remaining target difference is equal to avg, and is updated to the current remaining target difference minus the weight of the added executable unit after each executable unit is added to the current group; avg is the ratio of the total weight of all executable units in the target intermediate representation to n, or the ratio of the total weight of the remaining ungrouped executable units to the number of remaining groups.
[0007] S400 distributes the compilation tasks of the executable units in n groups to n threads for parallel execution.
[0008] Compared with the prior art, the present invention has at least the following beneficial effects: This invention sorts the executable units included in the intermediate representation according to their weights, and then allocates them to n groups based on the sorting results. The compilation tasks of the executable units in the n groups are distributed to n threads for parallel execution. The allocation process uses the weight of the executable units as the quantification of the compilation workload, and the average weight of the current group as the benchmark for the total weight of the current group. Specifically, during grouping, the executable unit with the largest weight is selected first. If the weight of this executable unit is greater than or equal to the target average weight, it is divided into a separate group to avoid load imbalance caused by excessively large weights. If the weight of this executable unit is less than the target average weight, suitable executable units are selected in descending order of weight and added to the current group. The remaining target difference is dynamically updated to strictly control the total weight of the current group to approach the average weight of the current group, until the current group is constructed. Therefore, by using this dynamic balancing allocation strategy, the executable units included in the target intermediate representation are roughly divided into several equal parts, so that the weight of each part is approximately equal. This results in load balancing of the threads compiling each executable unit, reducing the overall compilation time of the executable units included in the target intermediate representation and improving compilation efficiency. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 A flowchart of a parallel compilation method based on intermediate representation provided in an embodiment of the present invention. Detailed Implementation
[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0012] According to this embodiment, a parallel compilation method based on intermediate representation is provided, the method including the following steps, such as... Figure 1 As shown: S100: Obtain the total number of executable units included in the target intermediate representation. If the total number is greater than the preset number of groups n, proceed to S200; otherwise, group each executable unit included in the intermediate representation separately and determine that the grouping is complete.
[0013] In this embodiment, the target intermediate representation is the intermediate representation of the source code that needs to be compiled. Those skilled in the art will understand that an intermediate representation (IR) is an internal representation generated by the compiler after scanning and analyzing the source code. It represents the semantic and syntactic structure of the source program, and each stage of the compiler performs analysis or optimization transformations on the IR. The source code is a hardware description language, such as SystemVerilog or Verilog, and after compilation, it becomes a binary file. The IR is the intermediate representation between the source code and the binary file. This intermediate representation is a data structure built in memory, rather than a persistently stored intermediate file.
[0014] In this embodiment, an executable unit refers to an execution entity that is triggered by an event, call, or continuous mechanism during the simulation process and participates in signal calculation or state update, such as an always procedure block, a continuous assignment statement, a function, a task, etc. Those skilled in the art know that the process of identifying and dividing executable units from IR is prior art and will not be described in detail here.
[0015] In this embodiment, the preset number of groups n is also the number of threads executing the compilation task, and n is a known value.
[0016] S200, the executable units included in the intermediate representation are sorted according to the weights corresponding to the executable units included in the target intermediate representation; the weight corresponding to any executable unit is positively correlated with the time required for the compiler to compile the executable unit.
[0017] In one specific implementation, the sorting is a sorting of weights from largest to smallest.
[0018] In this embodiment, the weight of any executable unit is positively correlated with the compilation time required by the compiler to compile that executable unit. That is, the more and more complex the expression nodes included in an executable unit, the longer the compilation time required for that executable unit, and the greater the weight of that executable unit. The weight of any executable unit is the cumulative sum of the weights of all expression nodes included in that executable unit, and the weight of any executable unit is greater than or equal to 0.
[0019] In this embodiment, the weight corresponding to any executable unit needs to be obtained by traversing each expression node of that executable unit, and by traversing its child expression nodes in different ways according to different types of expression nodes, until the leaf expression node is recursively reached and the weight is accumulated. As a specific implementation, the process of obtaining the weights corresponding to the executable units included in the target intermediate representation includes: S210, construct an access context, which is used to accumulate weights during traversal and includes a weight attribute for recording the current weight value and an operation interface for updating the weight attribute.
[0020] S220, For different types of expression nodes in the intermediate presentation layer, configure corresponding access processing functions. The access processing functions are used to perform weight calculation and child node traversal operations on the expression nodes during the traversal process.
[0021] In this embodiment, to achieve traversal, a visit function is added to the class corresponding to each type in the IR layer, as shown below: void visit(VisitContext&cnt); The VisitContext contains weighted attributes that need to be accumulated, for example: class VisitContext { public: void addWeight() { _weight++;} void setWeight(unsigned long weight) { _weight = weight;} unsigned long getWeight() { return _weight;} ... private: unsigned long _weight; ... } S230, for any executable unit in the target intermediate representation, recursively traverse the expression nodes in the intermediate representation corresponding to the executable unit, call the access processing function corresponding to each expression node, and perform weight accumulation operation according to the preset differentiated weight rules during the traversal process; after the traversal is completed, the accumulated weight value in the access context is determined as the weight of the executable unit; wherein, the weight of the executable unit is related to its internal structural complexity, and the higher the number of expression nodes and the nesting level, the greater its corresponding weight value.
[0022] In this embodiment, the differentiation rules can be set independently based on the compilation time required by the compiler to compile different types of expression nodes. As a specific implementation, the differentiation rules include: for flow control type expression nodes, only their child expression nodes are traversed and their corresponding access processing functions are called, without performing weight accumulation operations; for computation type expression nodes or call type expression nodes, the weight operation interface in the access context is called to perform weight accumulation operations, and if the expression node contains child expression nodes, its child expression nodes are further traversed and their corresponding access processing functions are called; for expression nodes that do not contain child expression nodes, including constant expression nodes, only weight accumulation operations are performed through their access functions.
[0023] In this embodiment, for non-leaf expression nodes of the process class, the child elements are traversed in their visit function, and the visit function of each child element is called. For example: void Always::visit(VisitContext&cnt) { getEventControl()->visit(cnt); getBody()->visit(cnt); } void EventControlExpr::visit(VisitContext&cnt) { for (unsigned i = 0; i <getExprList().size(); i++) { getExprList().at(i)->visit(cnt); } } The above example uses the access function of the Always expression node. The first line of code, `getEventControl()->visit(cnt)`, retrieves the event control child element of the current expression node and calls its access function to complete the recursive access. The second line of code, `getBody()->visit(cnt)`, retrieves the function body child element of the current expression node and calls its access function to complete the recursive access. Taking the access function of the EventControlExpr expression node as an example, the for loop iterates through the list of all child expression nodes of the current expression node. The code `getExprList().at(i)->visit(cnt)` within the loop retrieves each child expression node sequentially and calls its corresponding access function to complete the recursive traversal.
[0024] In this embodiment, for non-leaf expression nodes of calculation and calling types, since their execution requires resources, their weights are increased before traversing their child elements and calling the child element's visit function. For example: void BinaryExpr::visit(VisitContext&cnt) { cnt.addWeight(); getExpr1()->visit(cnt); getExpr2()->visit(cnt); } void SysFunctionCallExpr::visit(VisitContext&cnt) { cnt.addWeight(); for(unsigned i = 0; i <getExprList().size(); i++) { getExprList().at(i)->visit(cnt); } } The above example uses the access function of the binary expression node `BinaryExpr`. The first line of code, `cnt.addWeight()`, calls the weight accumulation interface of the access context to accumulate and calculate the weight of the current expression node. The second line of code, `getExpr1()->visit(cnt)`, retrieves the first sub-expression node and calls its access function. The third line of code, `getExpr2()->visit(cnt)`, retrieves the second sub-expression node and calls its access function. Taking the access function of the system function call expression node `SysFunctionCallExpr` as an example, the first line of code, `cnt.addWeight()`, completes the weight accumulation of the current expression node. The `for` loop iterates through the list of parameter expression nodes in the function call, and the loop body calls the access function of each parameter sub-expression node in turn to complete the recursive traversal.
[0025] In this embodiment, only weights need to be added for constant expression nodes and leaf expression nodes. For example: void StringExpr::visit(VisitContext&cnt) { cnt.addWeight(); } void ContinueStmt:: visit(VisitContext&cnt) { cnt.addWeight(); } The code above uses the access function of the string expression node StringExpr as an example. The code cnt.addWeight() only calls the weight accumulation interface of the access context to complete the weight calculation of the current expression node, without any subsequent child element traversal operations. Taking the access function of the continuation statement ContinueStmt as an example, the code cnt.addWeight() also only performs the weight accumulation operation. Since this expression node has no subordinate child elements, there is no need to execute child element traversal and access logic.
[0026] In this embodiment, the specific value of the weight accumulation in S230 is a preset fixed value or a configurable dynamic value, and the accumulation value corresponding to different types of expression nodes can be set separately.
[0027] Based on S210-S230, this embodiment can obtain the weight corresponding to each executable unit included in the target intermediate representation, and the larger the weight of an executable unit, the longer it takes to compile the executable unit.
[0028] S300, according to the sorting result, the executable units included in the intermediate representation are allocated to n groups; wherein, the allocation process includes: when the current group is not the last group, if the maximum weight of the executable units in the current executable unit list is greater than or equal to the average weight avg of the current group, then the executable unit func1 with the largest weight in the current executable unit list is set as the current group alone; if it is less, then func1 is added to the current group, and the executable units in the executable unit list except func1 are traversed in descending order of weight, and the executable units with weights not exceeding the current remaining target difference are selected in turn and added to the current group, and the current remaining target difference is updated until the current remaining target difference is not greater than 0 or the traversal ends, and the executable units added to the current group are removed from the executable unit list; the initial value of the current remaining target difference is equal to avg, and is updated to the current remaining target difference minus the weight of the added executable unit after each executable unit is added to the current group; avg is the ratio of the total weight of all executable units in the target intermediate representation to n, or the ratio of the total weight of the remaining ungrouped executable units to the number of remaining groups.
[0029] In this embodiment, the allocation process further includes: in the construction of the last group, all executable units in the executable unit list are directly used as the group.
[0030] In one specific implementation, the allocation process also includes: executing n loops, building one group in each loop, until all n groups are built. It should be understood that the first group is built first, then the second group, and so on, until all n groups are built.
[0031] In a preferred embodiment, the allocation process further includes: before traversing the executable units in the executable unit list excluding func1 in descending order of weight, determining whether the minimum weight of each executable unit in the executable unit list excluding func1 is greater than the current remaining target difference. If it is greater, then func1 is grouped separately as the current group; otherwise, traversing the executable units in the executable unit list excluding func1 in descending order of weight is performed. Based on this preferred embodiment, when the minimum weight of each executable unit in the executable unit list excluding func1 is greater than the current remaining target difference, it is unnecessary to traverse the executable units in the executable unit list excluding func1 in descending order of weight, thus improving the efficiency of the allocation process.
[0032] As an optional implementation, if the current group is the first group, then avg is the ratio of the total weight of all executable units in the target intermediate representation to n; if the current group is not the first group, then avg is the ratio of the total weight of the remaining ungrouped executable units to the number of remaining groups.
[0033] S400 distributes the compilation tasks of the executable units in n groups to n threads for parallel execution.
[0034] This embodiment sorts the executable units included in the intermediate representation according to their weights (positively correlated with the compilation time), and allocates them to n groups based on the sorting results. The compilation tasks of the executable units in the n groups are then distributed to n threads for parallel execution. The allocation process uses the weight of the executable unit as the quantification of the compilation workload, and the average weight of the current group as the benchmark for the total weight of the current group. Specifically, during grouping, the executable unit with the largest weight is selected first. If the weight of this executable unit is greater than or equal to the target average weight, it is grouped separately to avoid load imbalance caused by excessively large weights. If the weight of this executable unit is less than the target average weight, suitable executable units are selected in descending order of weight and added to the current group. The remaining target difference is dynamically updated to strictly control the total weight of the current group to approach the average weight of the current group, until the current group is constructed. Therefore, by using this dynamic balancing allocation strategy, the executable units included in the target intermediate representation are roughly divided into several equal parts, so that the weight of each part is approximately equal. This results in load balancing of the threads compiling each executable unit, reducing the overall compilation time of the executable units included in the target intermediate representation and improving compilation efficiency.
[0035] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims
1. A parallel compilation method based on intermediate representation, characterized in that, The method includes the following steps: S100: Obtain the total number of executable units included in the target intermediate representation. If the total number is greater than the preset number of groups n, proceed to S200; otherwise, group each executable unit included in the intermediate representation separately and determine that the grouping is complete. S200, the executable units included in the intermediate representation are sorted according to the weights corresponding to the executable units included in the target intermediate representation; the weight corresponding to any executable unit is positively correlated with the time required for the compiler to compile the executable unit; S300, according to the sorting result, the executable units included in the intermediate representation are allocated to n groups; wherein, the allocation process includes: when the current group is not the last group, if the maximum weight of the executable units in the current executable unit list is greater than or equal to the average weight avg of the current group, then the executable unit func1 with the largest weight in the current executable unit list is set as the current group alone; if it is less, then func1 is added to the current group, and the executable units in the executable unit list except func1 are traversed in descending order of weight, and the executable units with weights not exceeding the current remaining target difference are selected in turn and added to the current group, and the current remaining target difference is updated until the current remaining target difference is not greater than 0 or the traversal ends, and the executable units added to the current group are removed from the executable unit list; the initial value of the current remaining target difference is equal to avg, and is updated to the current remaining target difference minus the weight of the added executable unit after each executable unit is added to the current group; avg is the ratio of the total weight of all executable units in the target intermediate representation to n, or the ratio of the total weight of the remaining ungrouped executable units to the number of remaining groups; S400 distributes the compilation tasks of the executable units in n groups to n threads for parallel execution.
2. The parallel compilation method based on intermediate representation according to claim 1, characterized in that, The allocation process also includes: in the construction of the last group, all executable units in the executable unit list are directly used as that group.
3. The parallel compilation method based on intermediate representation according to claim 1, characterized in that, The process of obtaining the weights corresponding to each executable unit included in the target intermediate representation includes: S210, construct an access context, which is used to accumulate weights during traversal and includes a weight attribute for recording the current weight value and an operation interface for updating the weight attribute. S220, Configure corresponding access processing functions for different types of expression nodes in the intermediate presentation layer. The access processing functions are used to perform weight calculation and child node traversal operations on the expression nodes during the traversal process. S230, for any executable unit in the target intermediate representation, recursively traverse the expression nodes in the intermediate representation corresponding to the executable unit, call the access processing function corresponding to each expression node, and perform weight accumulation operation according to the preset differentiated weight rules during the traversal process; after the traversal is completed, the accumulated weight value in the access context is determined as the weight of the executable unit; wherein, the weight of the executable unit is related to its internal structural complexity, and the higher the number of expression nodes and the nesting level, the greater its corresponding weight value.
4. The parallel compilation method based on intermediate representation according to claim 3, characterized in that, The differentiation rules include: for flow control type expression nodes, only their child expression nodes are traversed and the corresponding access processing functions are called, without performing weight accumulation operations; for calculation type expression nodes or call type expression nodes, the weight operation interface in the access context is called to perform weight accumulation operations, and if the expression node contains child expression nodes, its child expression nodes are further traversed and the corresponding access processing functions are called; for expression nodes that do not contain child expression nodes, including constant expression nodes, only weight accumulation operations are performed through their access functions.
5. The parallel compilation method based on intermediate representation according to claim 3, characterized in that, The specific value of weight accumulation in S230 is a preset fixed value or a configurable dynamic value. The accumulation value corresponding to different types of expression nodes can be set separately.
6. The parallel compilation method based on intermediate representation according to claim 1, characterized in that, The allocation process further includes: before traversing the executable units in the executable unit list other than func1 in descending order of weight, first determining whether the minimum weight of the executable units in the executable unit list other than func1 is greater than the current remaining target difference. If it is greater, then func1 is treated as the current group; otherwise, the executable units in the executable unit list other than func1 are traversed in descending order of weight.
7. The parallel compilation method based on intermediate representation according to claim 1, characterized in that, If the current group is the first group, then avg is the ratio of the total weight of all executable units in the target intermediate representation to n; if the current group is not the first group, then avg is the ratio of the sum of the weights of the executable units in the executable unit list after the previous group was grouped to the number of remaining groups.
8. The parallel compilation method based on intermediate representation according to claim 1, characterized in that, The allocation process also includes: executing n loops, building one group in each loop, until all n groups are built.
9. The parallel compilation method based on intermediate representation according to claim 1, characterized in that, The sorting is based on weights from largest to smallest.