An incremental compilation method based on intermediate representation
By using an incremental compilation method based on intermediate representation, and utilizing checksum comparison and preset threshold judgment, only executable units that fail to match are compiled, thus solving the problem of long waiting times in existing technologies and improving compilation speed and user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING NORI INTEGRATED CIRCUIT DESIGN CO LTD
- Filing Date
- 2026-04-16
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies, users need to recompile the entire design every time they make a design modification or a minor change, resulting in excessively long waiting times, especially for large designs that can take several hours to compile. How can we reduce user waiting time and improve user experience?
An incremental compilation method based on intermediate representation is adopted. An intermediate representation is generated by scanning and analyzing the target source code, calculating the checksum of each executable unit, comparing it with the checksum in the index file, and determining whether to recompile based on the cache matching degree and a preset threshold. Only executable units that fail to match are compiled, and the functional entities of the cached binary file are reused.
It improves compilation speed, reduces user waiting time, and enhances user experience. By using incremental compilation technology to compile only the necessary parts instead of compiling the entire dataset, it saves compilation time.
Smart Images
Figure CN122064324B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of compiler technology, and in particular to an incremental compilation method based on intermediate representation. Background Technology
[0002] In current technologies, compilation often starts from scratch. Even if the design remains unchanged or only a small part is modified, users have to recompile the entire design. While recompiling every time ensures data consistency, it often leads to excessively long waiting times, especially for large designs, where a single compilation can take several hours or even tens of hours. How to reduce user waiting time and improve user experience is an urgent problem to be solved. Summary of the Invention
[0003] The purpose of this invention is to provide an incremental compilation method based on intermediate representation to improve compilation speed, reduce user waiting time, and enhance user experience.
[0004] According to the present invention, an incremental compilation method based on intermediate representation is provided, the method comprising the following steps: S100 uses a compiler to scan and analyze the target source code to obtain an intermediate representation of the target source code.
[0005] S200, obtain the checksum of each executable unit in the intermediate representation of the target source code.
[0006] S300, compare the checksum of each executable unit in the intermediate representation of the target source code with the checksum in the index file to obtain the cache matching degree; the checksum in the index file includes the checksum of each executable unit in the intermediate representation of the previous source code of the same design.
[0007] S400, compare the cache matching degree with a preset threshold to determine whether to recompile all executable units in the intermediate representation of the target source code; if not, perform incremental compilation processing, including: performing matching judgment based on the executable units in the intermediate representation of the target source code, dividing the executable units in the intermediate representation of the target source code into successfully matched executable units and unmatched executable units; reusing the corresponding functional entities in the cache binary file for the successfully matched executable units; performing compilation for the unmatched executable units, appending the generated functional entities to the cache binary file, and appending the corresponding checksum to the index file.
[0008] Compared with the prior art, the present invention has at least the following beneficial effects: This invention first obtains an intermediate representation of the target source code, then acquires the checksum of each executable unit within it, and compares it with the checksum of each executable unit in the index file to obtain the cache matching degree. It then determines whether to recompile all executable units in the intermediate representation of the target source code based on a preset threshold and the relative magnitude of the cache matching degree. If not recompiling, only the executable units that did not match the index file are compiled, and the corresponding functional entities in the cached binary file corresponding to the index file are reused. Thus, this invention achieves incremental compilation by calculating a checksum for each executable unit at the intermediate representation layer, caching the previously compiled cached binary file of the same design, and reusing the corresponding functional entities in these cached binary files during the current compilation when the cache matching degree is satisfied, thereby improving compilation speed. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 A flowchart of an incremental compilation method based on intermediate representation provided in an embodiment of the present invention. Detailed Implementation
[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0012] According to this embodiment, an incremental compilation method based on intermediate representation is provided. The method includes the following steps, such as... Figure 1 As shown: S100 uses a compiler to scan and analyze the target source code to obtain an intermediate representation of the target source code.
[0013] Those skilled in the art will understand that the Intermediate Representation (IR) is an internal representation generated by the compiler after scanning and analyzing the source code, representing the semantics and syntactic structure of the source program. Each stage of the compiler performs analysis or optimization transformations on the IR. The advantages of this embodiment, which implements incremental compilation based on IR rather than on the source file, include: 1. It can filter out content that does not affect the compilation result, such as spaces, newlines, and comments. 2. It can reuse the results of the parser's checks on the syntax and semantics of the source file. 3. It can reuse the results of various optimizations based on the IR layer. For example, if an executable unit is not called during runtime, the IR object represented by that executable unit will be deleted during invalid code optimization at the IR layer, thus eliminating the need to calculate checksums and generate cached binary files for that executable unit.
[0014] In this embodiment, the source code is a hardware description language, such as SystemVerilog or Verilog, and the binary file is a binary object file compiled based on the hardware description language. The object file can participate in subsequent linking processing as an independent compilation unit to generate an executable file. IR is an intermediate representation between the source code and the binary file. The intermediate representation is a data structure built in memory, rather than an intermediate file that is persistently stored.
[0015] Those skilled in the art will recognize that the process of using a compiler to scan and analyze the target source code (i.e., the hardware description language source code corresponding to the current design) to obtain the corresponding intermediate representation is prior art and will not be elaborated here.
[0016] S200, obtain the checksum of each executable unit in the intermediate representation of the target source code.
[0017] In this embodiment, an executable unit refers to an execution entity that is triggered by an event, call, or continuous mechanism during the simulation process and participates in signal calculation or state update, such as an always procedure block, a continuous assignment statement, a function, a task, etc. Those skilled in the art know that the process of identifying and dividing executable units from IR is prior art and will not be described in detail here.
[0018] In one specific implementation, S200 includes: for any executable unit, traversing the internal structure of the executable unit according to its intermediate representation, extracting elements belonging to a preset semantic type, performing deterministic transformation processing on the attribute information of the elements to construct a checksum calculation context, and processing the context using a preset checksum generation algorithm to obtain the checksum of the executable unit. Optionally, the checksum generation algorithm is CRC checksum, MD5 algorithm, etc. Optionally, the preset semantic type includes control flow structure, calculation expression, symbol and data dependency, data type and attribute, and timing and triggering conditions, etc. The deterministic transformation processing includes: uniformly encoding attribute values, sorting attributes according to preset rules, and filtering non-semantic related information, etc.
[0019] S300, compare the checksum of each executable unit in the intermediate representation of the target source code with the checksum in the index file to obtain the cache matching degree; the checksum in the index file includes the checksum of each executable unit in the intermediate representation of the previous source code of the same design.
[0020] In this embodiment, the previous source code of the same design refers to the most recent historical version of the source code belonging to the same design as the current design. It should be understood that the same design refers to a hardware or software design project with the same design goals, the same functional specifications, and the same top-level module structure, whose source code forms multiple historical versions during the development iteration process, and each version belongs to the same design.
[0021] In this embodiment, the index file includes the compiler version number, the index data creation time, cache matching degree, the checksum of the executable unit, and the executable unit name prefix. The compiler version number reflects the compiler's version information. Since different compiler versions often have functional differences, to ensure the accuracy of the cached data, the cached data (including the index file and cached binary files) needs to be recollected whenever the compiler version is switched. The index data creation time is the same as the index file creation time. The cache matching degree represents the proportion of executable units marked as reusable in the index file out of the total number of executable units in the index file. To avoid name conflicts, executable units are renamed during incremental compilation, and the checksum is included as part of the new name (suffix). Optionally, a prefix is added to distinguish different types of executable units to improve the readability of the generated code and logs. For example, if the original name of the executable unit is as_2_35_FCG, and during the incremental compilation stage, the checksum corresponding to its content is calculated to be 20444874312210245, then its name needs to be renamed to pre_func20444874312210245, where pre_func is the prefix of the executable unit name.
[0022] In this embodiment, if the cache matching degree in the index file is 100%, the checksum of the executable unit recorded in the index file only includes the checksum of each executable unit in the intermediate representation generated by compiling the source code of the previous version corresponding to the current design; if the cache matching degree is less than 100%, the checksum of the executable unit recorded in the index file, in addition to including the checksum of each executable unit in the intermediate representation generated by compiling the source code of the previous version corresponding to the current design, also includes the checksum of at least one executable unit introduced by compiling the source code of an earlier historical version of the current design and still stored in the cache binary file.
[0023] The executable unit whose checksum comes from an earlier historical version does not match the intermediate representation generated by compiling the source code of the previous version corresponding to the current design. Furthermore, the functional entity in the corresponding cache binary file is not triggered during the execution of the source code of the previous version corresponding to the current design, which is a redundant part in the cache binary file. The redundant part is caused by the cache binary file being generated by appending during multiple incremental compilation processes without deleting the historically existing executable units.
[0024] In a preferred embodiment, prior to S300, the method further includes the following steps: first, determining whether the compiler version has changed; if a compiler version change is detected, then determining that the cached binary file and index file are invalid, clearing the cached binary file and the corresponding index file, and performing a full compilation of all executable units in the intermediate representation of the target source code to regenerate the cached binary file and the corresponding index file; otherwise, proceeding to S300-S400. Thus, in the case of a compiler version change, the calculation of cache matching degree based on checksum and the determination of executable unit reuse are avoided, and all executable units are directly recompiled, thereby reducing unnecessary computational overhead.
[0025] S400, compare the cache matching degree with a preset threshold to determine whether to recompile all executable units in the intermediate representation of the target source code; if not, perform incremental compilation processing, including: performing matching judgment based on the executable units in the intermediate representation of the target source code, dividing the executable units in the intermediate representation of the target source code into successfully matched executable units and unmatched executable units; reusing the corresponding functional entities in the cache binary file for the successfully matched executable units; performing compilation for the unmatched executable units, appending the generated functional entities to the cache binary file, and appending the corresponding checksum to the index file.
[0026] In this embodiment, the updated index file (i.e. the appended index file) may contain checksums that do not belong to the executable units of the current design. These checksums are redundant data from historical versions. The functional entities corresponding to the executable units of these checksums are not triggered during the execution of the current design. The functional entities corresponding to the executable units of these checksums belong to the redundant parts in the cached binary file.
[0027] As an optional specific implementation, the executable units in the intermediate representation of the target source code are not grouped. Instead, the checksum of each executable unit in the intermediate representation of the target source code is matched with the checksum in the index file, and the total number of all executable units in the intermediate representation of the target source code is counted. The cache matching degree is determined by the ratio of the total number of all executable units in the intermediate representation of the target source code to the total number of executable units in the updated index file. The total number of executable units in the updated index file includes the original number of executable units in the index file and the number of executable units that failed to match.
[0028] For example, if the intermediate representation of the target source code contains 80 executable units, and the index file contains 100 executable units, and 70 executable units in the intermediate representation of the target source code are identical to those in the index file, then the updated index file will contain a total of 110 executable units. In this specific implementation, the cache matching ratio is 80 / 110. It should be understood that two executable units matching means their checksums are the same, and two executable units not matching means their checksums are different.
[0029] Based on this optional specific implementation, determining whether to recompile all executable units in the intermediate representation of the target source code in S400 includes: if the cache matching degree is greater than or equal to a preset threshold, then determining that recompiling is not performed on all executable units in the intermediate representation of the target source code; otherwise, determining that recompiling is performed on all executable units in the intermediate representation of the target source code. The preset threshold can be set independently, for example, set to 0.9.
[0030] In a preferred embodiment, S300 includes: S310, group the executable units in the intermediate representation of the target source code into several groups.
[0031] S320: For any group, compare the checksums of the executable units included in the group with the checksums of the corresponding group in the index file to obtain the cache matching degree of the group.
[0032] In this preferred embodiment, the checksums of the executable units included in the intermediate representation of the target source code are matched with the checksums of the corresponding group in the index file, and the total number of executable units included in the intermediate representation of the target source code is counted. The cache matching degree of the group is determined by the ratio of the total number of executable units included in the intermediate representation of the target source code to the total number of executable units included in the corresponding group in the updated index file. The total number of executable units included in the corresponding group in the updated index file includes: the original number of executable units in the corresponding group in the index file, and the number of executable units in the intermediate representation of the target source code that failed to match. For example, if the number of executable units in a certain group in the intermediate representation of the target source code is 10, and the number of executable units in the corresponding group in the index file is 9, and 9 executable units in the intermediate representation of the target source code are the same as the executable units in the corresponding group in the index file, then the total number of executable units in the corresponding group in the updated index file is 10, and the cache matching degree of this group is 10 / 10.
[0033] Based on this preferred embodiment, S400 includes: for any group, if the cache matching degree of the group is greater than or equal to a preset threshold, then the executable units of the group in the intermediate representation of the target source code are not recompiled, and incremental compilation processing is performed, including: using the executable units of the group in the intermediate representation of the target source code as a benchmark for matching determination, dividing the executable units of the group in the intermediate representation of the target source code into successfully matched executable units and unmatched executable units; reusing the corresponding functional entities in the cache binary file of the corresponding group for the successfully matched executable units; compiling the unmatched executable units, and appending the generated functional entities to the cache binary file of the corresponding group, while appending the corresponding checksum to the corresponding group's part of the index file; otherwise, recompiling all executable units of the group in the intermediate representation of the target source code.
[0034] In one specific implementation, the executable units in the intermediate representation of the target source code are grouped according to their type or function. For example, in one specific implementation, S310 includes: grouping executable units of a preset type in the intermediate representation of the target source code into one group, and grouping executable units other than those of the preset type in the intermediate representation of the target source code into another group. The preset type of executable units includes internal general-purpose executable units and template-type executable units. Internal general-purpose executable units and template-type executable units are generally stable and do not change with specific design modifications, and can be reused; while other executable units are more likely to change with modifications to the target source code, requiring matching or recompilation. It should be understood that internal general-purpose executable units refer to public functions, tasks, etc., executable units with global visibility in the design, not dependent on a specific design unit, and can be called by multiple executable units in the design; template-type executable units refer to executable units designed using parameterization or generics, which only define general logical structures and operations, do not generate specific instances themselves, and only form executable entities when generating corresponding executable units based on actual call parameters.
[0035] By grouping executable units, each group contains zero or more executable units, and all executable units in each group are packaged into the same cache binary file. Simultaneously, a cache match score is calculated separately for each group of executable units. When the cache match score of a group falls below a preset threshold, only the corresponding executable unit data for that group needs to be recollected, without affecting the cache data of other groups. Grouping different executable units for caching has the following advantages: 1. Grouping executable units with similar functions into the same group facilitates maintenance. 2. Grouping executable units that are unlikely to change (e.g., internal general-purpose executable units, template classes, etc.) into the same group improves the reusability of their cache binary files. 3. Grouping defines boundaries for different types of executable units; recollecting the executable unit data of a group does not affect the reuse of cache data from other groups, improving the reusability of cache binary files. 4. For large designs containing a large number of executable units, creating a separate cache binary file for each executable unit would generate a large number of files, causing an I / O bottleneck. By grouping different executable units and generating the executable unit data in each group into the same cache binary file, the number of generated files can be reduced.
[0036] As a specific implementation method, part of the content in the index file is shown below: "preflycg": { "total": 2, "total_matched": 1, "total_unmatched": 1, "match_rate": 50.0, "matched_checksums": { "7635396864862797009": 1 }, "unmatched_checksums": { "17295374973219254171": 1 } }, "pretmpcg": { "total": 7, "total_matched": 7, "total_unmatched": 0, "match_rate": 100.0, "matched_checksums": { "1145882314765343631": 1, "2497494172672224947": 1, "3688372211206055841": 1, "5932787389050953226": 1, "9012494650720921443": 1, "13508409515391986311": 1, "15276231888102406538": 1 }, "unmatched_checksums": {} } The above content illustrates two distinct groups: `preflycg` and `pretmpcg`. The executable units in `pretmpcg` are those that are less prone to change. Here, `total` represents the number of executable units in the group, `total_matched` represents the number of successfully matched executable units, `total_unmatched` represents the number of unmatched executable units, `match_rate` represents the cache match rate of the group, `matched_checksums` represents the set of successfully matched checksums, `unmatched_checksums` represents the set of successfully matched checksums, and `{}` represents an empty string.
[0037] Furthermore, it should be understood that the index file and cache binary file in this embodiment are continuously updated. For example, before executing S100, the index file and cache binary file are the index file and cache binary file corresponding to the previous source code of the same design; after executing S400, the index file and cache binary file need to be updated to the index file and cache binary file corresponding to the target source code, so as to determine whether the next source code of the same design needs to recompile all executable units in the intermediate representation of the next source code of the same design. If it is determined that all executable units in the intermediate representation of the target source code do not need to be recompiled, then the updated index file and cache binary file, in addition to including the checksums and cache binary files of the executable units in the previous index file, also include the checksums and corresponding cache binary files of the executable units in the intermediate representation of the target source code that did not match the index file.
[0038] This embodiment first obtains an intermediate representation of the target source code, then acquires the checksum of each executable unit within it, and compares it with the checksum of each executable unit in the index file to obtain the cache matching degree. Based on a preset threshold and the relative magnitude of the cache matching degree, it determines whether to recompile all executable units in the intermediate representation of the target source code. If not recompiling, only the executable units that did not match the index file are compiled, and the corresponding functional entities in the cached binary file corresponding to the index file are reused. Therefore, this embodiment achieves incremental compilation by calculating a checksum for each executable unit at the intermediate representation layer, caching the previously compiled cached binary file of the same design, and reusing the corresponding functional entities in these cached binary files during the current compilation when the cache matching degree is satisfied, thereby improving compilation speed.
[0039] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims
1. An incremental compilation method based on intermediate representation, characterized in that, The method includes the following steps: S100: The compiler is used to scan and analyze the target source code to obtain an intermediate representation of the target source code. S200, obtain the checksum of each executable unit in the intermediate representation of the target source code; S300, compare the checksum of each executable unit in the intermediate representation of the target source code with the checksum in the index file to obtain the cache matching degree; the checksum in the index file includes the checksum of each executable unit in the intermediate representation of the previous source code of the same design. S400, compare the cache matching degree with a preset threshold to determine whether to recompile all executable units in the intermediate representation of the target source code; if the determination is no, perform incremental compilation processing, including: performing matching determination based on the executable units in the intermediate representation of the target source code, dividing the executable units in the intermediate representation of the target source code into successfully matched executable units and unmatched executable units; reusing the corresponding functional entities in the cache binary file for the successfully matched executable units; performing compilation for the unmatched executable units, appending the generated functional entities to the cache binary file, and appending the corresponding checksum to the index file; The S300 includes: S310, group the executable units in the intermediate representation of the target source code into several groups; S320, For any group, compare the checksums of the executable units included in the group with the checksums of the corresponding group in the index file to obtain the cache matching degree of the group; The process of obtaining the cache match degree of any group includes: counting the number of all executable units included in the intermediate representation of the target source code; and determining the cache match degree of the group as the proportion of the number of all executable units included in the intermediate representation of the target source code to the total number of executable units included in the corresponding group in the updated index file. S200 includes: for any executable unit, traversing the internal structure of the executable unit according to the intermediate representation of the executable unit, extracting elements belonging to a preset semantic type, performing deterministic transformation processing on the attribute information of the elements to construct a check code calculation context, and processing the context using a preset check code generation algorithm to obtain the check code of the executable unit.
2. The incremental compilation method based on intermediate representation according to claim 1, characterized in that, S400 includes: For any group, if the cache matching degree of the group is greater than or equal to a preset threshold, then the executable units of the group in the intermediate representation of the target source code are not recompiled, and incremental compilation processing is performed, including: using the executable units of the group in the intermediate representation of the target source code as a benchmark for matching determination, dividing the executable units of the group in the intermediate representation of the target source code into successfully matched executable units and unmatched executable units; reusing the corresponding functional entities in the cache binary file of the corresponding group for the successfully matched executable units; compiling the unmatched executable units, appending the generated functional entities to the cache binary file of the corresponding group, and appending the corresponding checksum to the corresponding group's part of the index file; otherwise, recompiling all executable units of the group in the intermediate representation of the target source code.
3. The incremental compilation method based on intermediate representation according to claim 1, characterized in that, Before S300, the method further includes the following steps: first, determine whether the compiler version has changed; if the compiler version has changed, determine that the cached binary file and index file are invalid, clear the cached binary file and the corresponding index file, and perform a full compilation on all executable units in the intermediate representation of the target source code to regenerate the cached binary file and the corresponding index file; if not, then execute S300-S400.
4. The incremental compilation method based on intermediate representation according to claim 1, characterized in that, The index file includes the compiler version number, the creation time of the index data, the cache match rate, the checksum of the executable unit, and the prefix of the executable unit name.
5. The incremental compilation method based on intermediate representation according to claim 1, characterized in that, S310 includes: grouping executable units of template classes in the intermediate representation of the target source code into one group, and grouping executable units of non-template classes in the intermediate representation of the target source code into one group.