Performance evaluation method of CPU micro-architecture, bare computer performance evaluation tool, electronic equipment and computer program product

Abstract

The invention is suitable for the technical field of computers, and provides a performance evaluation method of a CPU micro-architecture, a bare computer performance evaluation tool, electronic equipment and a computer program product, the method is applied to the bare computer performance evaluation tool, and the method comprises the following steps: based on a performance evaluation demand of a to-be-evaluated CPU micro-architecture, evaluating the performance of the to-be-evaluated CPU micro-architecture; configuring at least one to-be-monitored event and a target test code; a virtual CPU is controlled to execute the target test code, and event operation data corresponding to all the to-be-monitored events in the code execution process are collected; the virtual CPU is created on the basis of the to-be-evaluated CPU micro-architecture; and determining performance evaluation information of the CPU micro-architecture based on the event operation data. According to the scheme, the blank that the CPU micro-architecture performance evaluation method is lacked in the pre-silicon stage is filled, and the CPU micro-architecture performance is evaluated on the premise that tape-out is not needed.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a CPU microarchitecture performance evaluation method, a bare-metal performance evaluation tool, electronic devices, and computer program products. Background Technology

[0002] During the chip design phase (pre-silicon stage), a physical CPU with corresponding performance characteristics can be manufactured based on the Central Processing Unit (CPU) microarchitecture. In other words, the CPU microarchitecture design directly determines the performance ceiling of the future physical CPU. Current performance evaluation technologies primarily target physical CPUs that have already been taped out and manufactured; that is, a CPU must be manufactured before performance evaluation can be conducted. There is an urgent need for a solution that can directly evaluate the performance potential of the CPU microarchitecture, thereby predicting the future performance of physical CPUs in advance, assisting in optimizing design parameters, and deciding whether to put them into production. Summary of the Invention

[0003] This application provides a CPU microarchitecture performance evaluation method, a bare-metal performance evaluation tool, an electronic device, and a computer program product to address the problem of the lack of a CPU microarchitecture performance evaluation method in the prior art.

[0004] A first aspect of this application provides a CPU microarchitecture performance evaluation method, applied to a bare-metal performance evaluation tool, the method comprising: Based on the performance evaluation requirements of the CPU microarchitecture to be evaluated, configure at least one event to be monitored and target test code; The virtual CPU is controlled to execute the target test code, and event execution data corresponding to all the events to be monitored during the code execution process is collected; the virtual CPU is created based on the microarchitecture of the CPU to be evaluated. Based on the event execution data, the performance evaluation information of the CPU microarchitecture is determined.

[0005] A second aspect of this application provides a bare-metal performance evaluation tool for implementing the steps of the method described in the first aspect.

[0006] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0007] A fourth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0008] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] As can be seen from the above, the bare-metal performance evaluation tool in this application combines the performance evaluation requirements of the CPU microarchitecture to be evaluated, configures the events to be monitored and the target test code, controls the virtual CPU to run the target test code, and collects the event execution data corresponding to the events to be monitored, thereby determining the performance evaluation information of the CPU microarchitecture. This fills the gap in the prior art where there is no CPU microarchitecture performance evaluation method in the pre-silicon stage, and evaluates the performance of the CPU microarchitecture without the need for tape-out. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the architecture of a bare-metal performance evaluation tool provided in an embodiment of this application; Figure 2 This is a flowchart of a CPU microarchitecture performance evaluation method provided in an embodiment of this application; Figure 3 This application provides an example of custom test code for testing the Add instruction of IXU in dependent and non-dependent scenarios. Figure 4 This application provides an example of parameter definition and function description for an automated execution script of L5perf. Figure 5 This is an example command description of an automated execution script for L5perf provided in an embodiment of this application; Figure 6 This is example code for building a micro-benchmark provided in an embodiment of this application; Figure 7 This is example code for constructing a RISC-V test set and performing Spike simulation, provided in an embodiment of this application. Figure 8 This is an example code for VCS hardware-level simulation execution provided in an embodiment of this application; Figure 9 This is an example code for function definition and environment variable configuration provided in an embodiment of this application; Figure 10 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0014] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0015] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0016] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0017] In specific implementations, the terminals described in the embodiments of this application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0018] The following discussion describes terminals that include displays and touch-sensitive surfaces. However, it should be understood that terminals may include one or more other physical user interface devices such as physical keyboards, mice, and / or joysticks.

[0019] The terminal supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disc burning applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, digital camera applications, digital camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0020] Various applications that can run on the terminal can use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the terminal can be adjusted and / or changed between and / or within applications. In this way, the terminal's common physical architecture (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0021] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0022] In the field of CPU performance evaluation, Intel has proposed a performance analysis method based on CPU Performance Monitor Unit (PMU) events, namely a top-down performance analysis method and its required Performance Monitor Unit architecture, applicable to Linux systems. This method is used to assess and locate CPU performance bottlenecks during application execution, specifically through the performance (Perf) evaluation tools built into the Linux system.

[0023] This method and the Perf evaluation tool have clear limitations in their applicability: both are for post-silicon stage performance evaluation, mainly serving software engineers or performance evaluation engineers, and rely on Linux system startup to run, which cannot adapt to the performance evaluation needs of CPU microarchitecture in the chip design stage (i.e., the pre-silicon stage).

[0024] Existing technologies have two major limitations: First, the Perf evaluation tool requires Linux system support and cannot be applied to the pre-silicon chip design and parameter decision-making stages. Chip design engineers find it difficult to use it to evaluate the core performance of CPU microarchitectures and can only rely on experience-based judgments without actual testing. Second, the performance evaluation of CPU microarchitectures in the current pre-silicon bare-metal environment is mostly based on rough estimates by chip design engineers according to experience. The lack of performance evaluation tools in the pre-silicon bare-metal environment makes it impossible to achieve quantitative performance analysis and accurately support design parameter optimization and performance risk prediction.

[0025] To address the aforementioned issues, this application provides a bare-metal performance evaluation tool for evaluating the performance of CPU microarchitectures at the pre-silicon stage, without relying on a Linux system. This bare-metal performance evaluation tool may also be referred to as the LeapFive Performance Tool for Bare-metal (L5perf).

[0026] Figure 1 This is a schematic diagram of the architecture of a bare-metal performance evaluation tool provided in an embodiment of this application.

[0027] L5perf is a performance evaluation tool for pre-silicon bare-metal environments built using simulators such as Verilog Compiled Simulator (VCS) or Verilator. It includes several main modules: AutomaticScript, Framework, PMU Counter, CustomBenchmark, and Common Benchmark.

[0028] Automatic Script acts as the overall commander, responsible for the entire evaluation process's scheduling; the Framework is the frontline executor, responsible for launching test code according to instructions, connecting to event execution data, and processing the data; the PMU Counter is the data collector, responsible for collecting event execution data monitored by the PMU within the virtual CPU and transmitting it to the Framework; CustomBenchmark and Common Benchmark are the test vehicles, responsible for simulating actual loads to trigger hardware events. These modules have clearly defined roles and responsibilities, working together to achieve automated and accurate CPU microarchitecture performance evaluation.

[0029] The core of the Framework is running test code and processing event data. The core of Automatic Script is the end-to-end scheduling of scheduling, compilation, deep processing, and output.

[0030] The Framework enables automated configuration and data processing under the overall scheduling control of Automatic Script, requiring no manual intervention and adapting to scenarios such as batch evaluation and scheduled evaluation.

[0031] L5perf boasts excellent scalability while balancing specialization and versatility, precisely matching the diverse evaluation needs of the chip design stage.

[0032] Its specialization is reflected in its support for adding custom benchmarks, which can be used to specifically test the core performance of specific units or functions within the CPU microarchitecture. For example, it can test buffer depth, resource utilization, resource preemption, instruction fetch / memory access bandwidth and latency, single instruction cycle of execution units, and extreme value tests of specific units (such as the computing power of vector units for integer or floating-point data), enabling in-depth performance verification of local CPU microarchitecture.

[0033] Its versatility is reflected in its support for flexibly adding Common Benchmarks, which can cover a wide range of workload scenarios, with the number of instructions ranging from tens of thousands to tens of millions, enabling a comprehensive evaluation of the overall performance of the entire microarchitecture pipeline.

[0034] Custom benchmarks can be written in programming languages ​​such as Assembly, C, or C++, and are small in size and highly targeted. They can be used to test events of units such as the bandwidth and latency of the Instruction Fetch Unit (IFU), the branch prediction accuracy of the Branch Prediction Unit (BPU), the single instruction execution cycle of the Integer Execution Unit (IXU) / Float Execution Unit (FXU), and the memory access bandwidth of the Load-Store Unit (LSU).

[0035] General-purpose test code, such as CoreMark-Pro and SPEC CPU 2K17 test, cannot be too large when used in a bare-metal simulation environment due to the limitations of the environment's simulation efficiency. To obtain effective event execution data from these general-purpose test codes during the chip design phase, they need to be specifically adapted. The adaptation method can be flexibly chosen based on the structure of the general-purpose test code: one is to directly embed the general-purpose test code into the L5perf framework; the other is to compile the general-purpose test code into a static library and directly link and integrate it during the L5perf compilation process.

[0036] Both types of benchmarks support flexible expansion, accurately adapting to the dual needs of local in-depth testing and global overall evaluation during the design phase. They enable quantitative evaluation, balancing the depth and breadth of the assessment.

[0037] Custom Benchmark and Common Benchmark are independent and do not affect each other.

[0038] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0039] See Figure 2 , Figure 2 This is a flowchart illustrating a performance evaluation method for a CPU microarchitecture provided in an embodiment of this application. Figure 2 As shown, a performance evaluation method for CPU microarchitecture is applied to a bare-metal performance evaluation tool. The method includes the following steps: Step 101: Based on the performance evaluation requirements of the CPU microarchitecture to be evaluated, configure at least one event to be monitored and target test code.

[0040] This step is the preparation phase for CPU microarchitecture performance evaluation. Its core is to prepare the monitoring objects (events to be monitored) and test loads (target test code) for subsequent test execution based on the performance evaluation requirements.

[0041] Performance evaluation requirements are technical specifications that guide the entire CPU microarchitecture performance evaluation process, and are structured requests configured for the user end.

[0042] Monitoring events refer to low-level hardware behaviors defined by the PMU and directly countable. Monitoring events directly reflect the internal state of the CPU microarchitecture during runtime, such as the number of retired instructions, the number of clock cycles consumed, the number of cache misses, and the number of branch prediction errors. Configuring monitoring events essentially involves selecting the set of performance metrics that the PMU needs to track and record.

[0043] The PMU is a component within the virtual CPU. The virtual CPU is built based on the microarchitecture of the CPU being evaluated.

[0044] In some embodiments, a virtual CPU refers to a virtual processor that simulates the behavior of a real CPU by simulating the hardware description code using a simulator such as VCS or Verilator; it is a software model. The virtual CPU runs on a testbench based on a simulator such as VCS or Verilator, such as a virtual simulation platform.

[0045] In some embodiments, hardware description code is input into a simulator, which compiles and links the hardware description code to generate a callable virtual CPU. This virtual CPU can run in a bare-metal simulation environment such as a test platform, and its behavior is highly consistent with that of a future manufactured physical CPU, ensuring the accuracy of performance evaluation information.

[0046] Hardware description code refers to code written in a hardware description language to accurately depict the core functions (such as instruction execution, data processing, and logic control), internal structure (such as execution units, cache units, and branch prediction units), and interaction logic between internal structures (such as data transfer paths and instruction scheduling flows) of the CPU microarchitecture being evaluated. It is a digital and structured description of the CPU microarchitecture and directly determines whether a virtual CPU can accurately reproduce the hardware behavior and operational characteristics of that microarchitecture. The core form of hardware description code is Register Transfer Level (RTL) code.

[0047] The core component of the virtual CPU, the PMU, is closely related to the design of the hardware description code of the CPU microarchitecture being evaluated. The PMU's register information, such as the registers corresponding to the number of retired instructions and cycles, is directly determined by the hardware description code and forms the hardware foundation for data acquisition.

[0048] The performance evaluation environment in this application includes Testbench running virtual CPUs and L5perf.

[0049] Target test code refers to a program compiled to meet performance evaluation requirements and executable by a virtual CPU. Configuring target test code allows for the targeted triggering of events to be monitored, enabling the PMU to collect specific event execution data.

[0050] L5perf receives performance evaluation requirements from the user and configures the events to be monitored and the corresponding test programs accordingly.

[0051] This step transforms abstract performance evaluation requirements into concrete, executable test plans, laying the foundation for accurate data collection and analysis in the bare-metal simulation environment.

[0052] In some embodiments, configuring at least one event to be monitored and target test code based on the performance evaluation requirements of the CPU microarchitecture to be evaluated includes: determining at least one event to be monitored corresponding to the item to be evaluated in the performance evaluation requirements; writing the event to be monitored into a performance monitoring unit inside the virtual CPU; and selecting local test code and / or global test code corresponding to the item to be evaluated; compiling the local test code and / or the global test code into target test code adapted to the operation of the virtual CPU, and loading the target test code into the memory of the virtual CPU.

[0053] The items to be evaluated are the specific requirements for performance evaluation, such as testing the latency of addition instructions in the IXU unit and evaluating the branch prediction accuracy of the BPU.

[0054] Local test code, also known as custom test code, is a program specifically written to test the performance of a specific unit of the CPU microarchitecture. It is usually small in size and highly targeted.

[0055] Global test code, also known as common benchmark, is a standardized test program selected to evaluate the overall performance of the CPU microarchitecture.

[0056] In some embodiments, performance evaluation requirements are parsed to identify the items to be evaluated and the corresponding events to be monitored. For example, if the item to be evaluated is the efficiency of the IXU, then events such as the number of execution cycles and the number of retired instructions need to be monitored, and these events to be monitored are configured (i.e. written) into the PMU of the virtual CPU to achieve targeted monitoring by the PMU.

[0057] In some embodiments, the Framework in L5perf explicitly identifies the events to be monitored and configures them into the PMU inside the virtual CPU.

[0058] In some embodiments, the Automatic Script execution code in L5perf compiles to generate target test code that can run in a virtual CPU. The Framework loads the compiled target test code into the memory of the virtual CPU.

[0059] In some embodiments, test code is selected based on the item to be evaluated: if local in-depth testing is to be performed, local test code is selected or written; if global performance evaluation is to be performed, a general benchmark program, i.e., global test code, is selected; if both global performance evaluation and targeted evaluation of a specific item are required, both local and global test code are selected. This code is compiled into executable binary files for the virtual CPU using Automatic Script, such as Executable and Linkable Format (ELF), and loaded into memory for execution by the virtual CPU.

[0060] By transforming abstract requirements into concrete event configurations and executable test code, the evaluation process becomes modular and precise. Simultaneously, it supports various test scenarios, including local, global, and combinations of local and global scenarios, making the evaluation both in-depth and broad, and more flexible.

[0061] Step 102: Control the virtual CPU to execute the target test code and collect event execution data corresponding to all the events to be monitored during the code execution process; the virtual CPU is created based on the microarchitecture of the CPU to be evaluated.

[0062] This step is the data collection phase for performance evaluation, and its core is to run the test code and record relevant data.

[0063] Event execution data refers to the raw values ​​of each monitored event obtained by the PMU during the execution of the target test code, such as cycles=5000 and instructions retired=10000 for a single execution.

[0064] In a bare-metal simulation environment, targeted testing is performed on a virtual CPU created based on the CPU microarchitecture to be evaluated, thereby achieving a quantitative and accurate evaluation of the CPU microarchitecture performance.

[0065] The data acquisition process is completed collaboratively by PMU, PMU Counter, Framework, and Automatic Script.

[0066] In some embodiments, the Automatic Script in L5perf issues a test start command, and the Framework controls the virtual CPU to execute the loaded target test code, i.e., start the target test code, while instructing the PMU Counter to continuously collect event execution data monitored by the PMU.

[0067] During execution, the PMU inside the virtual CPU automatically monitors the specified events based on the previously configured events to be monitored, and stores the monitored event execution data in the PMU's registers. Specifically, for each type of event to be monitored, its corresponding event execution data is collected using a data acquisition method that is appropriate for it.

[0068] By directly acquiring event execution data reflecting the CPU microarchitecture's operating status in a bare-metal simulation environment, interference from operating system layers such as Linux is avoided, ensuring the purity and accuracy of the data source.

[0069] The Framework uses the PMU Counter interface to trigger the PMU Counter to read the raw data in the PMU registers, thus avoiding compatibility issues caused by directly manipulating the virtual CPU.

[0070] Custom Benchmark and Common Benchmark, as specific test programs for L5perf, will share the PMU's register storage function and corresponding performance monitoring capabilities.

[0071] In some embodiments, controlling the virtual CPU to execute the target test code includes: controlling the virtual CPU to execute the target test code according to preset execution parameters; the preset execution parameters include a preset number of execution times, single execution duration, and load intensity.

[0072] Preset execution parameters refer to a series of control parameters that are set in advance to ensure the stability and comparability of test results. These parameters include a preset number of executions, the duration of each execution, and the load intensity.

[0073] To avoid the randomness of a single test, an automated script is used to control the virtual CPU to run the same test code multiple times, such as a preset number of 100 times.

[0074] By adjusting the code logic (such as the number of loop unrolls) to control the length of instructions executed each time (ensuring it is long enough, such as tens of thousands to tens of millions of instructions), the load intensity of the test can be precisely adjusted, ensuring the effectiveness of data collection.

[0075] By executing the program multiple times, a large number of data samples can be obtained, providing a basis for subsequent statistical analysis and helping to eliminate noise caused by random factors (such as interruptions and anomalies in the simulation environment).

[0076] During each test code execution, the PMU continuously counts. The L5perf Framework obtains event execution data from the PMU registers through the PMU Counter, thereby achieving data acquisition.

[0077] This application collects data from a bare-metal simulation environment, without any Linux system consuming resources during the process, resulting in high data purity.

[0078] In this application, differentiated PMU reading timings are adopted for different test focuses of local test code and global test code.

[0079] In some embodiments, the data collection timing for the performance monitoring unit to collect data is determined based on the inclusion of the local test code and the global test code in the target test code; the data collection timing includes a first collection timing corresponding to the local test code and / or a second collection timing corresponding to the global test code.

[0080] The inclusion scenario specifically refers to the composition of the target test code, which may include only local test code, only global test code, or both types of test code that need to be executed separately.

[0081] Data collection timing includes a first collection timing specific to local test code and a second collection timing specific to global test code. Through the logic of code structure → timing matching, the collection timing is precisely bound to the test requirements.

[0082] The PMU's data acquisition start time is related to the code inclusion information of the target test code.

[0083] In addition to the data acquisition start time, the PMU also has a corresponding data acquisition end time.

[0084] The start and end times of data collection constitute the timing of data collection.

[0085] In some embodiments, the evaluation process is divided into the following phases in terms of timing: initialization (init), warmup (Warmup), PMU read start (PMU read start), PMU read end (PMU read end), and test code execution (Test Code Execution).

[0086] The initialization phase (init) completes basic preparatory work such as L5perf startup, PMU register configuration (loading events to be monitored), virtual CPU memory initialization, and test code loading, clearing environmental obstacles for subsequent processes.

[0087] The warmup phase involves pre-filling the target test code into various levels of caches, such as cache (including data cache or instruction cache) and translation lookaside buffer (TLB), so that the virtual CPU reaches a stable and efficient operating state before the formal test. This allows for the collection of performance data that can truly and accurately reflect the program's performance under stable load, eliminating or reducing measurement bias caused by the cold start effect.

[0088] The PMU read start phase initiates the PMU's event counting function, and the PMU begins recording the raw data of the events to be monitored in subsequent phases (test code execution), thus defining the starting point for data collection.

[0089] The PMU read end phase stops the PMU's event counting function, clearly defining the end point of data collection, avoiding the collection of irrelevant data after the test code has finished executing, and ensuring targeted data collection.

[0090] During the test code execution phase, the virtual CPU runs the target test code, and the PMU continuously counts the events to be monitored, which is the core phase of data acquisition.

[0091] In some embodiments, the timing sequence for reading local test code is: init → Warmup → PMU read start → test code execution → PMU read end, meaning the data collection starts after the warm-up phase and before the test code execution, and ends after the test code execution. The timing sequence for reading global test code is: init → PMU read start → Warmup → test code execution → PMU read end, enabling targeted data collection; that is, the data collection starts after the initialization phase and before the warm-up phase, and ends after the test code execution.

[0092] In some embodiments, if the target code contains local test code and global test code, the local test code can be run first (enabling its dedicated timing and collecting corresponding event execution data), and then the global test code can be run (switching to its dedicated timing and collecting corresponding event execution data). The two tests are performed independently, the data is stored separately, and finally summarized and analyzed to avoid timing conflicts and data chaos.

[0093] Step 103: Based on the event execution data, determine the performance evaluation information of the CPU microarchitecture.

[0094] This step is the analysis phase of performance evaluation, the core of which is to process and analyze the collected raw data and generate performance evaluation information.

[0095] Performance evaluation information refers to the metrics and / or conclusions obtained after processing event execution data that can directly characterize the performance of the CPU microarchitecture, such as instructions per cycle (IPC), cache miss rate, and direct design compliance judgment.

[0096] In some embodiments, the event execution data for some of the events to be monitored is obtained through initial calculations and analysis by L5perf. That is, data calculations are performed based on the original event execution data to obtain the event execution data used for subsequent comparison. For example, IPC is calculated based on the number of instructions and cycles; cache miss rate is calculated based on the number of cache misses and cache accesses. By performing targeted calculations and analyses on the original event execution data, intuitive performance indicators such as IPC and cache miss rate are generated, providing accurate and direct basis for CPU microarchitecture performance evaluation and design decisions.

[0097] In some embodiments, each execution of the target test code yields a set of event execution data for each event to be monitored. Determining the performance evaluation information of the CPU microarchitecture based on the event execution data includes: performing an average operation on a preset number of sets of event execution data corresponding to each event to be monitored, obtaining a preset number of averages corresponding to the event to be monitored; selecting a target extreme value from the preset number of averages corresponding to each event to be monitored; and determining the performance evaluation information of the CPU microarchitecture based on the target extreme value.

[0098] The target extreme value is the maximum or minimum value.

[0099] When testing performance metrics such as single instruction cycle, latency, and miss rate, where smaller values ​​are preferred, the target extreme value is the minimum. When testing performance metrics such as bandwidth, computing power, and throughput, where larger values ​​are preferred, the target extreme value is the maximum.

[0100] The framework performs calculations and processing on the event execution data.

[0101] Automatic Script collects this processed data from the Framework for more complex analysis (such as comparison with performance reference data) and finally generates performance evaluation information (such as performance achievement conclusions).

[0102] During each code execution, a set of event execution data for each monitored event can be obtained. The code is executed a preset number of times, resulting in a preset number of sets of event execution data for each monitored event. The mean of each preset number of sets of event execution data for each monitored event is calculated to obtain the mean of each set of event execution data for each monitored event. In other words, under a preset number of code execution scenarios, a preset number of mean values ​​are calculated for each monitored event. For example, if the target test code is executed 100 times, 100 sets of event execution data for a certain monitored event are obtained, and the mean of each set is calculated to obtain the mean of 100 cycles for that monitored event.

[0103] Select the target extreme value from a preset number of averages for each event to be monitored to determine performance evaluation information.

[0104] A two-layer processing logic of mean and extreme values ​​is adopted to filter out random interference such as interruptions and anomalies in the simulation environment.

[0105] This mean-to-extreme-value data processing flow is a method designed specifically for unstable simulation environments to improve data accuracy. It can effectively filter random interference and more accurately determine whether the design has achieved theoretical expectations (e.g., whether the minimum number of cycles dependent on the addition instruction chain is close to 1 cycle).

[0106] In some embodiments, determining the performance evaluation information of the CPU microarchitecture based on the target extreme value includes: comparing the target extreme value of each of the events to be monitored with the corresponding performance reference data; and determining that the performance of the CPU microarchitecture meets the design expectations when all the target extreme values ​​reach the corresponding performance reference data.

[0107] Performance reference data refers to the theoretical performance values ​​expected during the CPU microarchitecture design phase, such as the theoretical number of execution cycles for a certain instruction.

[0108] The target extreme values ​​(such as the actual measured minimum number of cycles) of each monitored event are compared with their corresponding theoretical performance values ​​expected by the design. If all target extreme values ​​are equal to or better than the performance reference data, the CPU microarchitecture performance is determined to have met the design expectations; otherwise, a potential design bottleneck is indicated.

[0109] If the measured value of the target extreme value of any monitored event is better than or equal to the expected value of the corresponding performance reference data, then the module design is deemed to meet the standard.

[0110] Transform raw, difficult-to-understand counting data into performance metrics and conclusions that design engineers can directly use for decision-making.

[0111] This step automates the closed-loop performance evaluation process and provides objective judgment, offering a direct and clear basis for design decisions and improving chip design efficiency.

[0112] In some embodiments, after determining the performance evaluation information of the CPU microarchitecture based on the event runtime data, the method further includes: visualizing the performance evaluation information according to the information display format configured in the performance evaluation requirements; and sending the visualized performance evaluation information to the user terminal.

[0113] In some embodiments, the performance evaluation information includes evaluation conclusions on the merits of the CPU microarchitecture design, bottleneck analysis reports, and information such as event execution data, corresponding averages, and / or corresponding target extreme values ​​for the events to be monitored.

[0114] Based on the information display format configured in the performance evaluation requirements, the evaluation information is visualized to generate one or more visualization results (i.e., visualized performance evaluation information), including bar charts of performance indicators, pie charts of bottleneck percentages, tables of compliance status, etc. The visualization results are sent to the user's device, allowing the user to quickly view them through the visualization interface.

[0115] This step improves the user-friendliness of the tool and the readability of the results, enabling performance data to be interpreted efficiently and guiding subsequent design optimizations.

[0116] In some embodiments, when analyzing the event execution data collected during Common Benchmark execution, in addition to the basic statistics of conventional performance instructions (such as IPC, L3 cache miss and hit rate, and branch miss and hit rate), Intel's Top-Down Micro-architecture Analysis Method (TMAM) is also used to perform hierarchical data decomposition, classify and statistically analyze the data, and then determine bottleneck points and other information based on the classified data. This achieves in-depth data analysis.

[0117] The TMAM hierarchical statistical rules are as follows: Level 1 (top-level classification) divides performance data into four core dimensions: frontend, backend, instruction retirement, and bad-speculation, forming a basic statistical framework. Level 2-Level 4 (Layered Decomposition): For the three dimensions of Frontend, Backend, and Bad-Speculation, the data is gradually refined in a hierarchical manner, and finally decomposed to the smallest indivisible statistical unit (for example, Frontend can be decomposed into specific sub-items such as insufficient IFU bandwidth and prefetch failure), so as to achieve fine-grained traceability of performance data. Core performance metrics calculation: Based on the above hierarchical statistical data, key performance metrics are further calculated, including IPC, Cycles per Instruction (CPI), Misses per KiloInstructions (MPKI), Tera Operations Per Second (TOPS), and cache / tap table hit rate, to intuitively quantify CPU microarchitecture performance.

[0118] By analyzing the proportion characteristics of TMAM hierarchical data, performance bottlenecks in global testing scenarios can be accurately located. For example, an excessively high proportion of the Backend dimension indicates insufficient execution unit resources; an excessively high proportion of the Bad-Speculation dimension indicates that there is room for optimization in the branch prediction algorithm, providing clear guidance for the design and adjustment of the CPU microarchitecture.

[0119] In some embodiments, the visualized performance evaluation information of this application includes hierarchical bottleneck information, providing a clear direction for design optimization.

[0120] In some embodiments, the single-instruction execution time (in cycles) of IXU's Add instruction in dependent and independent scenarios is tested using a Custom Benchmark. See the example code below. Figure 3 As shown, Figure 3 This is an example of custom test code provided in this application embodiment for testing the Add instruction of IXU in dependent and non-dependent scenarios.

[0121] Figure 3 Both functions in the code are L5perf's Custom Benchmark, designed specifically for evaluating the performance of the virtual CPU IXU, and belong to a dedicated test scenario.

[0122] dep_add_chain (dependency-based addition chain): Forces IXU to perform operations sequentially through 128 consecutively dependent addition instructions. The core test is the sequential operation latency of IXU (time cost of single instruction execution).

[0123] indep_add_chain (independent addition chain): Maximizes the parallel computing load of IXU through 50 groups of 8 addition instructions each with no data dependency. It is the core test of IXU's parallel computing capabilities, including microarchitectural features such as multi-issue unit width and out-of-order execution efficiency, and reflects the performance limit of IXU's parallel processing.

[0124] In some embodiments, two test functions are embedded in the L5perf Framework and then loaded into the memory of the virtual CPU, controlling the virtual CPU to run multiple times (to avoid the randomness of a single test). During each run, the PMU (Performance Monitoring Unit) records core data such as the number of instructions and cycles in real time.

[0125] The data processing adopts the mean + target extreme value (minimum value in this case) strategy: first calculate the mean of each run, and then select the minimum value from the mean to avoid random interference such as simulation environment interruption and anomaly (such interference will cause the test results to be too large) and ensure that the final data is close to the real performance of IXU.

[0126] The theoretical execution time, or performance reference data, of the two functions can be predefined by the chip design engineer. The theoretical execution time of dependent addition is 1 cycle, and the theoretical execution time of independent addition is about 0.5 cycles (less than 1 cycle).

[0127] If the measured minimum value is equal to or less than the theoretical value, it means that the IXU design meets expectations; if the measured minimum value is consistently higher than the theoretical value (e.g., 1.5 cycles for dependent addition and 1 cycle for independent addition), it indicates that the IXU has a performance bottleneck, such as the operation unit resources being occupied or insufficient optimization of the parallel scheduling logic.

[0128] This test logic is also applicable to single-instruction performance testing of other units such as LSU and FXU, and has strong universality.

[0129] Different CPU microarchitecture modules have different testing focuses. For example, IFU focuses on testing bandwidth and latency, while BPU / Return Address Stack (RAS) focuses on testing branch prediction accuracy. Each module needs to have its own custom benchmark designed according to its characteristics.

[0130] Unlike the localized in-depth testing of Custom Benchmark, Common Benchmark uses programs that are closer to real-world applications (adapted to bare-metal simulation efficiency, with the number of instructions controlled to tens of thousands to tens of millions), and uses TMAM to break down performance data into layers, ultimately locating the performance bottleneck of the overall microarchitecture and forming a complete evaluation system that combines local and global aspects.

[0131] In some embodiments, run_sim.py is an automated execution script for L5perf. Its core function is to automate the entire process from test code building and simulation execution to performance data analysis. It is the core carrier for automating the entire process of CPU microarchitecture performance testing. run_sim.py can be written in Python.

[0132] Part of the code in run_sim.py is as follows: Figure 4 As shown, Figure 4 This is an example of parameter definition and function description for an automated execution script of L5perf provided in an embodiment of this application. Figure 4 The function, value, and technical significance of the positional parameters and optional parameters (such as steps, insts, fsdb, etc.) of the run_sim.py script are defined in detail.

[0133] The location parameter `case` is used to specify the test platform name (default `testbench`), and can switch between different CPU microarchitecture units such as IXU and LSU for specific test scenarios. Optional parameters (such as `steps`, `insts`, `fsdb`, `seed`, `simVlog`, etc.) control the execution steps of the simulation process, instruction tracing switch, waveform file generation, random number seed, and log path, respectively. It is the core configuration entry point for achieving precise control of the test process, reproducible simulation results, and traceable problems.

[0134] Figure 5 This is an example command description of an automated execution script for L5perf provided in an embodiment of this application. Figure 5 This section lists typical use cases for the `run_sim.py` script, visually demonstrating how to select execution steps based on testing needs. Running `python run_sim.py --help` will display a complete explanation of the parameters.

[0135] This script covers the entire process from code compilation to logic verification, hardware simulation, and bottleneck identification, and is suitable for performance verification and optimization in the pre-silicon design phase of CPU microarchitecture.

[0136] By default, executing `python run_sim.py` automatically completes the test library construction, simulation execution, and TMAM analysis. This can be done in single-step execution, such as steps 1 (building only the micro-benchmark library) and steps 3 (running only the hardware simulation); or in multi-step combinations, such as steps 1 and 2 (completing test library construction and instruction logic verification); or in a dedicated bottleneck analysis section, steps 4 combine `seed` or `simvlog` to process TMAM data and pinpoint micro-architecture performance bottlenecks. Figure 5 The example commands explain how to use them in different scenarios such as test code debugging, logic verification, and performance analysis, lowering the barrier to entry.

[0137] Step 1: Build Microbench → Clean up old projects → Compile to generate a test library adapted for the VCS environment (depending on the MICROBENCH_HOME environment variable). Example code is shown below. Figure 6 As shown, Figure 6 This is sample code for building a micro-benchmark provided in an embodiment of this application.

[0138] Step 2: Integrate riscv-tests with micro-benchmarks → Compile to generate ELF file → Verify instruction logic using the Spike simulator (depends on the RISCV_TESTS_HOME environment variable). Example code is shown below. Figure 7 As shown, Figure 7 This is an example code for constructing a RISC-V test set and performing Spike simulation, provided in an embodiment of this application.

[0139] Step 3: Load the VCS simulation environment → Construct simulation commands based on seed (random seed), fsdb (waveform tracing), and insts (instruction tracing) → Execute the SIMV simulation and collect performance data (depending on the VCS_SIMV_HOME environment variable). Example code is shown below. Figure 8 As shown, Figure 8 This is an example code for VCS hardware-level simulation execution provided in this application embodiment.

[0140] Step 1 and Step 2 compile the selected test code into a binary file (ELF format) adapted to the virtual CPU. During the compilation process, the instruction set, register layout and virtual CPU running interface of the microarchitecture to be evaluated are adapted.

[0141] Step 3 records data to simv.log to ensure no data loss. It also supports a switch for generating a Fast Signal Database (FSDB) from waveform files, facilitating troubleshooting. Step 3 automates test execution and data acquisition, supports seed specification, and ensures test reproducibility.

[0142] To avoid code errors, you need to set the three environment variables MICROBENCH_HOME, RISCV_TESTS_HOME, and VCS_SIMV_HOME before Steps1. Figure 9 This is an example code for function definition and environment variable configuration provided in an embodiment of this application.

[0143] In some embodiments, based on the performance evaluation method of this application, a CPU microarchitecture suitable for production can be selected to generate a physical CPU. Performance testing is performed on the physical CPU, and the performance of the physical CPU is substantially consistent with the performance of the virtual CPU constructed based on its corresponding CPU microarchitecture. That is, the performance data obtained by the CPU microarchitecture performance evaluation method of this application is highly accurate.

[0144] The performance evaluation method in this application is based on an innovative combination of virtual CPU in a pre-silicon bare-metal environment and the modular tool L5perf, achieving the following technological breakthroughs: (1) Fill the gap in performance evaluation in the silicon pre-stage. Quantitative evaluation can be carried out in the chip design and parameter decision-making stages without tape-out, which can identify performance bottlenecks in advance, facilitate performance optimization, and reduce the risk of over-design or underperformance. (2) Balancing accuracy and efficiency: The pre-silicon bare metal environment ensures data purity, mean + extreme value processing filters interference, the TMAM method achieves accurate bottleneck location, and the instruction number adaptation design balances evaluation efficiency and accuracy. (3) High ease of use: Through automated scripts (such as run_sim.py), it can achieve "compiling, execution, acquisition, analysis and output with one command", and supports flexible configuration such as waveform tracking and seed specification. Chip design engineers can use it directly, which is practical and convenient. (4) Excellent scalability: It supports the addition of custom test code and adaptation to general test code, which can meet the evaluation needs of different microarchitectures and different design stages.

[0145] In this embodiment, the bare-metal performance evaluation tool combines the performance evaluation requirements of the CPU microarchitecture to be evaluated, configures the events to be monitored and the target test code, controls the virtual CPU to run the target test code, and collects the event execution data corresponding to the events to be monitored, thereby determining the performance evaluation information of the CPU microarchitecture. This fills the gap in the prior art where there is a lack of CPU microarchitecture performance evaluation methods in the pre-silicon stage, and evaluates the performance of the CPU microarchitecture without the need for tape-out.

[0146] The bare-metal performance evaluation tool provided in this application can implement all the processes of the above-described CPU microarchitecture performance evaluation method and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0147] Figure 10 This is a structural diagram of an electronic device provided in an embodiment of this application. As shown in the figure, the electronic device 2 of this embodiment includes: at least one processor 20 ( Figure 10(Only one is shown in the diagram), memory 21, and computer program 22 stored in said memory 21 and executable on said at least one processor 20, wherein said processor 20 executes said computer program 22 to implement the steps in any of the above method embodiments.

[0148] The electronic device 2 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that... Figure 10 This is merely an example of electronic device 2 and does not constitute a limitation on electronic device 2. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0149] The processor 20 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0150] The memory 21 can be an internal storage unit of the electronic device 2, such as a hard disk or memory. The memory 21 can also be an external storage device of the electronic device 2, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 21 can include both internal and external storage units of the electronic device 2. The memory 21 is used to store the computer program and other programs and data required by the electronic device. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0154] In the embodiments provided in this application, it should be understood that the disclosed systems / electronic devices and methods can be implemented in other ways. For example, the system / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of systems or units may be electrical, mechanical, or other forms.

[0155] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0157] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0158] The processes in the above-described embodiments can be implemented by a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps in the above-described method embodiments.

[0159] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A performance evaluation method for CPU microarchitecture, characterized in that, The method, applied to bare-metal performance evaluation tools, includes: Based on the performance evaluation requirements of the CPU microarchitecture to be evaluated, configure at least one event to be monitored and target test code; The virtual CPU is controlled to execute the target test code, and event execution data corresponding to all the events to be monitored during the code execution process is collected; the virtual CPU is created based on the microarchitecture of the CPU to be evaluated. Based on the event execution data, the performance evaluation information of the CPU microarchitecture is determined.

2. The method according to claim 1, characterized in that, The performance evaluation requirements based on the CPU microarchitecture to be evaluated include configuring at least one event to be monitored and target test code, including: Identify at least one of the monitored events corresponding to the items to be evaluated in the performance evaluation requirements; The event to be monitored is written into the performance monitoring unit inside the virtual CPU; and... Select the local test code and / or global test code corresponding to the item to be evaluated; The local test code and / or the global test code are compiled into the target test code adapted to run on the virtual CPU, and the target test code is loaded into the memory of the virtual CPU.

3. The method according to claim 2, characterized in that, The method further includes: Based on the inclusion of the local test code and the global test code in the target test code, the data collection timing for the performance monitoring unit to collect data is determined; the data collection timing includes a first collection timing corresponding to the local test code and / or a second collection timing corresponding to the global test code.

4. The method according to claim 1, characterized in that, The process of controlling the virtual CPU to execute the target test code includes: The virtual CPU is controlled to execute the target test code according to preset execution parameters; the preset execution parameters include a preset number of execution times, single execution duration, and load intensity.

5. The method according to claim 4, characterized in that, Each execution of the target test code yields a set of event execution data for each event to be monitored. The determination of performance evaluation information for the CPU microarchitecture based on this event execution data includes: For each of the preset number of event operation data corresponding to the event to be monitored, the mean value is calculated to obtain the preset number of mean values ​​corresponding to the event to be monitored. Select the target extreme value from a preset number of averages corresponding to each of the monitored events; Based on the target extreme value, the performance evaluation information of the CPU microarchitecture is determined.

6. The method according to claim 5, characterized in that, The process of determining the performance evaluation information of the CPU microarchitecture based on the target extreme value includes: The target extreme value of each of the events to be monitored is compared with the corresponding performance reference data; If all the target extreme values ​​are reached, the corresponding performance reference data is used to determine that the performance of the CPU microarchitecture has met the design expectations.

7. The method according to claim 1, characterized in that, After determining the performance evaluation information of the CPU microarchitecture based on the event runtime data, the process further includes: The performance evaluation information is visualized according to the information display format configured in the performance evaluation requirements. The visualized performance evaluation information is sent to the user's device.

8. A bare-metal performance evaluation tool, characterized in that, The bare-metal performance evaluation tool is used to perform the method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device performs the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes a computer program, which, when run, causes the method as described in any one of claims 1 to 7 to be performed.

Images