Integrated high-performance software construction system for EDA simulation model compilation acceleration

Abstract

The application relates to the technical field of electronic design automation, in particular to an integrated high-performance software construction system for EDA simulation model compilation acceleration, which is characterized in that a scheduling control module, an integrated compilation thread pool and a shared memory area are placed in the same process space to save the creation and destruction time of a large number of processes in a single-process multi-thread architecture, thereby improving the construction efficiency, and the memory object pool is used to store pre-compiled module data to realize the cross-compilation task reuse of the intermediate representation of the header file. When multiple compilation tasks contain the same header file, only the first task performs parsing, and subsequent tasks directly obtain the pre-compiled result through a memory pointer, thereby avoiding repeated reading and parsing, greatly reducing the CPU computing amount and I / O overhead, and significantly improving the construction efficiency of the simulation model compilation.

Description

Technical Field

[0001] This invention relates to the field of electronic design automation (EDA) technology, and in particular to an integrated high-performance software building system for accelerating the compilation of EDA simulation models. Background Technology

[0002] In the field of Electronic Design Automation (EDA), especially in the development and operation of SystemVerilog / Verilog digital logic simulators, it is usually necessary to convert hardware description languages ​​(RTL) into massive amounts of C++ simulation model code, and then compile and link them into an executable simulation kernel. The traditional construction method adopts a multi-process model of "build tools (such as Make, Ninja) + external compilers (such as GCC, Clang)".

[0003] However, the process overhead of the multi-process mode is too large. During the chip verification stage, the number of generated C++ files often reaches tens of thousands or even hundreds of thousands. The build tool needs to frequently call fork / exec to start the compiler process, and the operating system consumes a lot of time in process scheduling, which seriously slows down the iterative efficiency of "modification-compilation-simulation".

[0004] Moreover, the C++ code generated by the simulator contains a large number of repeated header file references (such as the simulation kernel base class library, SystemVerilog data type definitions, etc.). In multi-process mode, each compiler process independently and repeatedly parses these header files, and redundant calculations account for more than 60% of the build time, resulting in a waste of resources due to repeated parsing.

[0005] Therefore, improving build efficiency has become an urgent problem to be solved. Summary of the Invention

[0006] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows: An integrated high-performance software building system for accelerating the compilation of EDA simulation models, the system includes a scheduling control module, an integrated compilation thread pool, and a shared memory area, wherein the shared memory area includes a memory object pool, and the scheduling control module, the integrated compilation thread pool, and the shared memory area are located in the same process space; The scheduling control module is used to send the current compilation task to the integrated compilation thread pool when the current compilation task meets the first preset condition; The integrated compilation thread pool is used to execute a acquired compilation task when a compilation task is obtained, and to obtain the target file corresponding to the compilation task. Executing the acquired compilation task includes: When executing header file inclusion instructions in the acquired compilation task, the memory object pool is queried; If the memory object pool contains precompiled module data of the target header file corresponding to the header file inclusion instruction, then the memory pointer corresponding to the precompiled module data of the target header file is directly obtained, and the reading and parsing of the target header file is skipped; If the memory object pool does not contain the precompiled module data of the target header file corresponding to the header file inclusion instruction, then the integration compilation thread pool reads the target header file, parses the target header file, and stores the precompiled module data corresponding to the parsed target header file into the memory object pool. The memory object pool is used to store the precompiled module data.

[0007] Compared with the prior art, the present invention has significant advantages. Through the above technical solution, the integrated high-performance software construction system for accelerating the compilation of EDA simulation models provided by the present invention achieves considerable technological progress and practicality, and has broad industrial application value. It has at least the following advantages: This invention improves build efficiency by placing the scheduling control module, integrated compilation thread pool, and shared memory area in the same process space, thereby saving the time spent on creating and destroying massive amounts of processes using a single-process, multi-threaded architecture. Furthermore, by utilizing a memory object pool to store pre-compiled module data, it enables the reuse of intermediate header file representations across compilation tasks. When multiple compilation tasks contain the same header file, only the first task performs parsing; subsequent tasks directly obtain the pre-compiled results through memory pointers, avoiding repeated reading and parsing, significantly reducing CPU computation and I / O overhead, and substantially improving the build efficiency of simulation model compilation. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of an integrated high-performance software building system for accelerating the compilation of EDA simulation models, provided as an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] This embodiment provides an integrated high-performance software building system for accelerating the compilation of EDA simulation models. The system includes a scheduling control module, an integrated compilation thread pool, and a shared memory area. The shared memory area includes a memory object pool. The scheduling control module, the integrated compilation thread pool, and the shared memory area are located in the same process space. The scheduling control module is used to send the current compilation task to the integrated compilation thread pool when the current compilation task meets the first preset condition; The integrated compilation thread pool is used to execute a acquired compilation task when a compilation task is obtained, and to obtain the target file corresponding to the compilation task. Executing the acquired compilation task includes: When executing header file inclusion instructions in the acquired compilation task, the memory object pool is queried; If the memory object pool contains precompiled module data of the target header file corresponding to the header file inclusion instruction, then the memory pointer corresponding to the precompiled module data of the target header file is directly obtained, and the reading and parsing of the target header file is skipped; If the memory object pool does not contain the precompiled module data of the target header file corresponding to the header file inclusion instruction, then the integration compilation thread pool reads the target header file, parses the target header file, and stores the precompiled module data corresponding to the parsed target header file into the memory object pool. The memory object pool is used to store the precompiled module data.

[0012] The scheduling and control module is responsible for routing and distributing compilation tasks. The integrated compilation thread pool is a thread pool containing multiple worker threads. Each worker thread loads an LLVM compiler instance and can directly execute the compilation of C++ source files within the process without starting a separate child process.

[0013] A shared memory area refers to a memory region accessible to all threads within the same process. In this embodiment, the shared memory area is used to store a memory object pool. The memory object pool adopts a global hash table structure to store the intermediate representation of the parsed header file. The memory object pool and the integrated compilation thread pool are bidirectionally connected to achieve data reuse. The precompiled module data (PCM) refers to the intermediate representation (AST, Abstract Syntax Tree) and precompilation results generated after the corresponding target header file is parsed by the LLVM frontend. The precompiled module data can be directly used for subsequent code generation.

[0014] A memory pointer is used to point to the direct address of the storage location of the corresponding pre-compiled module data in the memory object pool, thereby achieving zero-copy access to memory through the passing of the memory pointer.

[0015] Specifically, the scheduling control module acts as the entry point, allocating compilation tasks to the integrated compilation thread pool. When worker threads in the integrated compilation thread pool execute compilation tasks, if they encounter a header file inclusion directive (#include directive), they query the memory object pool. If a match is found, the memory pointer is directly obtained, and the precompiled module data is reused; if a match is not found, the current thread performs the parsing and stores the result in the memory object pool. Throughout this process, the scheduling control module, the thread pool, and the memory object pool reside in the same process space, achieving efficient collaboration through shared memory.

[0016] For example, suppose a simulation model contains 10,000 C++ source files, each containing the same header file include directive. In a traditional multi-process model, the header file in the include directive would be parsed 10,000 times. However, in this embodiment, after the first compilation task parses the header file, it stores the corresponding pre-compiled module data into a memory object pool. Subsequent 9,999 tasks, when encountering the same header file, directly access the corresponding pre-compiled module data through the memory pointer, without needing to read or parse the file again.

[0017] In one specific implementation, the system further includes a configuration parsing module and a DAG dependency engine; The configuration parsing module is used to read and parse the preset construction script, construct the full dependency directed acyclic graph corresponding to the simulation model file, and send the full dependency directed acyclic graph to the DAG dependency engine. The full dependency directed acyclic graph includes several graph nodes, and the directed edge between any two graph nodes is used to represent the dependency relationship between the two graph nodes. The DAG dependency engine is used to traverse the graph nodes in the received full dependency directed acyclic graph, determine the completion status of each graph node according to the hash value change of each graph node, and determine the compilation task sequence according to the completion status of each graph node and the dependency relationship between each graph node. The compilation task sequence contains several compilation tasks, and the first compilation task in the compilation task sequence is taken as the current compilation task. After the current compilation task is completed, update the completion status of the graph nodes corresponding to the current compilation task, and return to the step of determining the compilation task sequence based on the completion status of each graph node and the dependencies between each graph node.

[0018] The configuration parsing module is used to read the build script and parse the compilation rules, source file list, dependency relationships and other information in it, and convert it into a full dependency directed acyclic graph (DAG).

[0019] The DAG dependency engine is a module that maintains and manages the entire directed acyclic graph (DAG) of dependencies, and is responsible for graph traversal, node state tracking, and task sequence generation.

[0020] The full dependency DAG uses graph nodes to represent compilation units (such as source files and library targets) and directed edges to represent dependencies (such as A depending on B, then B must be compiled before A). The acyclic nature of the graph ensures the feasibility of the compilation order. Graph nodes are vertices in the DAG, and each graph node corresponds to a compilation unit. A single graph node contains attributes such as file path, hash value, and completion status.

[0021] The completion status is a marker used to indicate whether a graph node has been successfully compiled. It is used during DAG scheduling to determine which graph node dependencies have been satisfied. For any graph node, if the completion status of all graph nodes that the graph node depends on is completed, then the compilation task corresponding to the graph node is added to the compilation task sequence.

[0022] Specifically, the configuration parsing module generates a Directed Acyclic Graph (DAG) based on the build script. The DAG dependency engine traverses the DAG nodes, identifies the graph nodes that need to be compiled based on hash value changes, and dynamically generates a compilation task sequence based on dependencies. Upon completion of each compilation task, the completion status of the corresponding graph node is updated, and the DAG engine recalculates the compilation task sequence, forming a scheduling loop.

[0023] In one specific implementation, determining the completion status of each graph node based on the changes in the hash values ​​of each graph node includes: For any graph node, when traversing to that graph node, calculate the hash value corresponding to that graph node. If the hash value calculated for that graph node is different from the hash value stored in that graph node, then determine that the completion status of that graph node is completed. If the hash value calculated by the graph node is the same as the hash value stored by the graph node, then the completion status of the graph node is determined to be incomplete.

[0024] Calculating the hash value corresponding to the graph node refers to the verification value calculated using a hash algorithm based on the file content corresponding to the file path within the graph node. The hash algorithm can be MD5, SHA1, etc., and there are no restrictions here.

[0025] Specifically, when the hash value calculated by the graph node is different from the hash value stored in the graph node, it means that the file content corresponding to the graph node has been compiled and updated, so the completion status of the graph node is determined to be completed. When the hash value calculated by the graph node is the same as the hash value stored in the graph node, it means that the file content corresponding to the graph node has not been compiled and updated, so the completion status of the graph node is determined to be incomplete.

[0026] In one specific implementation, the first preset condition is that the task type of the current compilation task is an internal compilation type and the memory load required by the current compilation task is less than a preset memory load threshold.

[0027] The first preset condition refers to the decision rule by which the scheduling control module routes tasks to the integrated compilation thread pool, which is determined by both the task type and the system resource status.

[0028] When the task type of a compilation task is internal compilation, it means that the source file corresponding to the compilation task can be directly processed by the LLVM instance in the integrated compilation thread pool, which is usually a C++ source file compilation task.

[0029] Memory load refers to the estimated memory required for this compilation task. The preset memory load threshold is the upper limit of memory usage for a single compilation task configured by the system. If this value is exceeded, the compilation task will not be allocated to the integrated compilation thread pool to prevent memory overflow.

[0030] In one embodiment, the first preset condition may further include meeting CPU load requirements, such as the number of currently running threads being less than the preset maximum number of threads, and / or the CPU utilization being less than the preset maximum CPU utilization, thereby achieving more refined task scheduling.

[0031] In one specific implementation, the system further includes an external process manager and external compilation tools; The scheduling control module is used to send the current compilation task to the external process manager when the current compilation task meets the second preset condition; The external process manager is used to send the received compilation tasks to the external compilation tool; The external compilation tool is used to compile the received compilation task to obtain the target file corresponding to the compilation task.

[0032] The external process manager is a subsystem responsible for creating, monitoring, and recycling external child processes, and managing the execution lifecycle of external compilation tools.

[0033] External compilation tools refer to third-party executable programs that are independent of this system, such as linkers, archiving tools, and script interpreters.

[0034] The second preset condition refers to the decision rule by which the scheduling control module routes tasks to an external process manager.

[0035] Specifically, the scheduling control module sends tasks that meet the second preset condition to the external process manager. The external process manager starts an independent child process for each task, calls the corresponding external compilation tool to execute it, and collects the return results. After execution, the external process manager notifies the scheduling control module to update the task status.

[0036] In one specific implementation, the second preset condition is that the task type of the current compilation task is an external command type or the memory load required by the current compilation task is greater than or equal to a preset memory load threshold.

[0037] External command types refer to tasks that cannot be directly handled by the integrated compilation thread pool, such as calling custom scripts, running third-party code generators, and performing linking.

[0038] When the memory load is greater than or equal to the preset threshold, it indicates that the compilation task may put pressure on the system memory. Therefore, it is switched to external process execution to distribute the pressure using the process-isolated memory space.

[0039] Specifically, the scheduling control module makes routing decisions based on the task type and memory load of the compilation task. If the task type is an external command type, or if it is an internal compilation type but the memory load is greater than the memory load threshold, the external execution path is selected and the task is handed over to the external process manager.

[0040] In one specific implementation, after the integrated compilation thread pool reads the target header file, parses the target header file, and stores the pre-compiled module data corresponding to the parsed target header file into the memory object pool, the method further includes: Obtain the memory pointer corresponding to the precompiled module data of the target header file; After obtaining the memory pointer corresponding to the precompiled module data of the target header file by directly obtaining the memory pointer, or by storing the precompiled module data of the target header file generated by parsing into the memory object pool to obtain the memory pointer corresponding to the precompiled module data of the target header file, code generation is performed to generate the target file corresponding to the compilation task.

[0041] Code generation refers to the back-end stage of the compilation process, which converts the intermediate representation into target machine code or object files.

[0042] An object file is a binary file output by a compilation task, which typically contains relocatable machine code and data.

[0043] Specifically, when the integrated compilation thread pool executes compilation tasks, it first processes header file inclusion instructions, obtaining pre-compiled module data for all dependent header files through hit reuse or initial parsing. After obtaining the memory pointer, it enters the code generation stage, using the LLVM backend to combine the source files and pre-compiled data to compile them into object files.

[0044] See Figure 1 This is a schematic diagram of an integrated high-performance software building system for accelerating the compilation of EDA simulation models, provided by an embodiment of the present invention.

[0045] In one specific implementation, the system further includes a link module; The linking module is used to link all the compiled object files with the preset simulation kernel library after all compilation tasks are completed, so as to generate a simulation executable file.

[0046] The linker module collects the target files generated by all compilation tasks, calls the linker to merge them with the simulation kernel library, and generates an executable binary file.

[0047] The simulation kernel library refers to the pre-compiled basic library required for the EDA simulator to run, which usually includes core functions such as the simulation engine's main control logic, event scheduler, and waveform recording interface.

[0048] A simulation executable file refers to the final output executable program used to perform RTL simulation.

[0049] Specifically, the linking module can trigger the linking process based on the completion status of the DAG in the DAG dependency engine; that is, the linking process is triggered when all graph nodes are marked as completed. The linking module takes the list of object files and the simulation kernel library as input and calls the linker to generate the simulation executable file.

[0050] In this embodiment, by placing the scheduling control module, the integrated compilation thread pool, and the shared memory area in the same process space, a single-process multi-threaded architecture is used to save the time spent on creating and destroying massive amounts of processes, thereby improving build efficiency. Furthermore, by utilizing a memory object pool to store pre-compiled module data, the reuse of intermediate header file representations across compilation tasks is achieved. When multiple compilation tasks contain the same header file, only the first task performs parsing; subsequent tasks directly obtain the pre-compiled results through memory pointers, avoiding repeated reading and parsing, significantly reducing CPU computation and I / O overhead, and substantially improving the build efficiency of the simulation model compilation.

[0051] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of this invention is defined by the appended claims.

Claims

1. An integrated high-performance software building system for accelerating the compilation of EDA simulation models, characterized in that, The system includes a scheduling control module, an integrated compilation thread pool, and a shared memory area, wherein the shared memory area includes a memory object pool, and the scheduling control module, the integrated compilation thread pool, and the shared memory area are located in the same process space; The scheduling control module is used to send the current compilation task to the integrated compilation thread pool when the current compilation task meets the first preset condition; The integrated compilation thread pool is used to execute a acquired compilation task when a compilation task is obtained, and to obtain the target file corresponding to the compilation task. Executing the acquired compilation task includes: When executing header file inclusion instructions in the acquired compilation task, the memory object pool is queried; If the memory object pool contains precompiled module data of the target header file corresponding to the header file inclusion instruction, then the memory pointer corresponding to the precompiled module data of the target header file is directly obtained, and the reading and parsing of the target header file is skipped; If the memory object pool does not contain the precompiled module data of the target header file corresponding to the header file inclusion instruction, then the integration compilation thread pool reads the target header file, parses the target header file, and stores the precompiled module data corresponding to the parsed target header file into the memory object pool. The memory object pool is used to store the precompiled module data.

2. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 1, characterized in that, The system also includes a configuration parsing module and a DAG dependency engine; The configuration parsing module is used to read and parse the preset construction script, construct the full dependency directed acyclic graph corresponding to the simulation model file, and send the full dependency directed acyclic graph to the DAG dependency engine. The full dependency directed acyclic graph includes several graph nodes, and the directed edge between any two graph nodes is used to represent the dependency relationship between the two graph nodes. The DAG dependency engine is used to traverse the graph nodes in the received full dependency directed acyclic graph, determine the completion status of each graph node according to the hash value change of each graph node, and determine the compilation task sequence according to the completion status of each graph node and the dependency relationship between each graph node. The compilation task sequence contains several compilation tasks, and the first compilation task in the compilation task sequence is taken as the current compilation task. After the current compilation task is completed, update the completion status of the graph nodes corresponding to the current compilation task, and return to the step of determining the compilation task sequence based on the completion status of each graph node and the dependencies between each graph node.

3. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 1, characterized in that, The step of determining the completion status of each graph node based on the changes in the hash values ​​of each graph node includes: For any graph node, when traversing to that graph node, calculate the hash value corresponding to that graph node. If the hash value calculated for that graph node is different from the hash value stored in that graph node, then determine that the completion status of that graph node is completed. If the hash value calculated by the graph node is the same as the hash value stored by the graph node, then the completion status of the graph node is determined to be incomplete.

4. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 1, characterized in that, The first preset condition is that the task type of the current compilation task is an internal compilation type and the memory load required by the current compilation task is less than the preset memory load threshold.

5. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 1, characterized in that, The system also includes an external process manager and external compilation tools; The scheduling control module is used to send the current compilation task to the external process manager when the current compilation task meets the second preset condition; The external process manager is used to send the received compilation tasks to the external compilation tool; The external compilation tool is used to compile the received compilation task to obtain the target file corresponding to the compilation task.

6. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 5, characterized in that, The second preset condition is that the task type of the current compilation task is an external command type or the memory load required by the current compilation task is greater than or equal to the preset memory load threshold.

7. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 1, characterized in that, After the integrated compilation thread pool reads the target header file, parses the target header file, and stores the pre-compiled module data corresponding to the parsed target header file into the memory object pool, the process further includes: Obtain the memory pointer corresponding to the precompiled module data of the target header file; After obtaining the memory pointer corresponding to the precompiled module data of the target header file by directly obtaining the memory pointer, or by storing the precompiled module data of the target header file generated by parsing into the memory object pool to obtain the memory pointer corresponding to the precompiled module data of the target header file, code generation is performed to generate the target file corresponding to the compilation task.

8. The integrated high-performance software construction system for accelerating the compilation of EDA simulation models according to claim 1, characterized in that, The system also includes a link module; The linking module is used to link all the compiled object files with the preset simulation kernel library after all compilation tasks are completed, so as to generate a simulation executable file.

Images