Code verification methods, devices, equipment and systems

Abstract

This disclosure provides a code verification method, apparatus, device, and system, relating to the field of computer technology. The method includes: running runtime data of a benchmark test program based on a reference model to determine a target program slice in the benchmark test program; wherein the reference model is a software model that simulates hardware description code, and the first runtime data corresponding to the target program slice satisfies preset runtime characteristics; running the target program slice through the hardware description code to obtain second runtime data; and determining the verification result of the hardware description code based on the first and second runtime data. This method can improve verification efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a code verification method, apparatus, device and system. Background Technology

[0002] Traditional code verification schemes use register transfer level (RTL) code for co-verification with software models used for hardware functional software simulation. However, this co-verification process suffers from low verification efficiency. Summary of the Invention

[0003] This disclosure provides a code verification method, apparatus, device, system, chip, storage medium, and program product.

[0004] In a first aspect, embodiments of this disclosure propose a code verification method, comprising: running benchmark test program execution data based on a reference model to determine a target program slice in the benchmark test program; wherein the reference model is a software model for simulating hardware description code, and the first execution data corresponding to the target program slice satisfies preset execution characteristics; running the target program slice through hardware description code to obtain second execution data; and determining the verification result of the hardware description code based on the first execution data and the second execution data.

[0005] In one possible implementation, the target program slice includes a pre-abnormal code segment, an abnormal code segment, and a recovery code segment, which are executed in a sequential order; wherein, the pre-abnormal code segment is the code segment in which the running data does not meet the preset running characteristics, the abnormal code segment is the code segment in which the running data meets the preset running characteristics, and the recovery code segment is the code segment in which the running data recovers from a state that meets the preset running characteristics to a state that does not meet the preset running characteristics.

[0006] In one possible implementation, the above-mentioned determination of the target program slice in the benchmark program based on the runtime data of the benchmark program running based on the reference model includes: determining the abnormal code segment and the recovery code segment based on the runtime data of the benchmark program running based on the reference model; determining the target program slice from the benchmark program according to a first quantity, the abnormal code segment and the recovery code segment; wherein, the first quantity is the number of instructions included in the preset code segment before the exception.

[0007] In one possible implementation, determining abnormal code segments and recovery code segments based on runtime data from running a benchmark program using a reference model includes: running the benchmark program using the reference model to generate basic block vectors; clustering the code in the benchmark program into one or more first code intervals based on the basic block vectors; running the one or more first code intervals using the reference model to obtain runtime data corresponding to the first code intervals; and determining abnormal code segments and recovery code segments based on the runtime data corresponding to the first code intervals.

[0008] In one possible implementation, the method further includes: updating the hardware description code based on the target program slice, the first running data, and the second running data if the verification result is unsuccessful.

[0009] In one possible implementation, updating the hardware description code based on the target program slice, the first running data, and the second running data includes: iteratively updating the hardware description code based on the target program slice, the first running data, and the second running data until a hardware description code that satisfies a first condition is obtained; wherein, the first condition includes: the difference between the running data of the updated hardware description code running the target program slice and the running data of the reference model corresponding to the updated hardware description code running the target program slice is less than or equal to a difference threshold.

[0010] In one possible implementation, the above-mentioned iterative update of the hardware description code based on the target program slice, the first running data, and the second running data until a hardware description code satisfying the first condition is obtained includes: iteratively updating the reference model based on the first running data to obtain an updated reference model; wherein the running data obtained by running the target program slice on the updated reference model does not satisfy the preset running characteristics; updating the hardware description code based on the parameters of the updated reference model and the second running data to obtain an updated hardware description code; iteratively updating the hardware description code if the updated hardware description code does not satisfy the first condition; and stopping the updating of the hardware description code if the updated hardware description code satisfies the first condition.

[0011] In one possible implementation, the above-mentioned iterative update of the reference model based on the first running data to obtain the updated reference model includes: determining a first update parameter for updating the reference model based on the first running data; wherein the first update parameter includes a configuration parameter corresponding to the first running data, and the configuration parameter corresponding to the first running data is used to update the reference model; and determining the updated reference model based on the reference model and the configuration parameter corresponding to the first running data.

[0012] In one possible implementation, the above-mentioned updating of the hardware description code based on the updated reference model parameters and the second running data to obtain the updated hardware description code includes: when the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics, determining a second update parameter for updating the hardware description code based on the updated reference model parameters and the second running data; and obtaining the updated hardware description code based on the second update parameter and the hardware description code.

[0013] In one possible implementation, the preset operating characteristics are determined based on a second condition, which includes one or more of the following: The front-end performance data is less than or equal to the first threshold; A memory miss occurs for M consecutive clock cycles; where M is an integer greater than or equal to 1. The duration of the stop receiving command is greater than or equal to N clock cycles; where N is an integer greater than or equal to 1; and, The proportion of pipeline recovery cycles caused by branch prediction errors to the total execution cycles is greater than or equal to the second threshold.

[0014] In one possible implementation, the runtime data includes one or more of the following: front-end runtime performance data, back-end runtime performance data, speculative execution phase performance data, and instruction retirement phase runtime performance data.

[0015] Secondly, this disclosure provides a code verification device, including: a slice determination module, a data acquisition module, and a result verification module. The slice determination module is configured to determine a target program slice in the benchmark test program based on runtime data from a benchmark test program running using a reference model; wherein the reference model is a software model that simulates hardware description code, and the first runtime data corresponding to the target program slice satisfies preset runtime characteristics. The data acquisition module is configured to obtain second runtime data by running the target program slice using hardware description code. The result verification module is configured to determine the verification result of the hardware description code based on the first runtime data and the second runtime data.

[0016] In one possible implementation, the target program slice includes a pre-abnormal code segment, an abnormal code segment, and a recovery code segment, which are executed in a sequential order; wherein, the pre-abnormal code segment is the code segment in which the running data does not meet the preset running characteristics, the abnormal code segment is the code segment in which the running data meets the preset running characteristics, and the recovery code segment is the code segment in which the running data recovers from a state that meets the preset running characteristics to a state that does not meet the preset running characteristics.

[0017] In one possible implementation, the slice determination module is further configured to: determine the abnormal code segment and the recovery code segment based on the running data of the benchmark test program running the reference model; and determine the target program slice from the benchmark test program according to a first quantity, the abnormal code segment and the recovery code segment; wherein the first quantity is the number of instructions included in the preset code segment before the exception.

[0018] In one possible implementation, the slice determination module is further configured to: run a benchmark test program through a reference model to generate basic block vectors; cluster the code in the benchmark test program into one or more first code intervals based on the basic block vectors; run one or more first code intervals through the reference model to obtain running data corresponding to the first code intervals; and determine abnormal code segments and recovery code segments based on the running data corresponding to the first code intervals.

[0019] In one possible implementation, the code verification apparatus further includes an update module. The update module is configured to update the hardware description code based on the target program slice, first runtime data, and second runtime data if the verification result is unsuccessful.

[0020] In one possible implementation, the update module is further configured to: iteratively update the hardware description code based on the target program slice, the first running data, and the second running data, until a hardware description code that satisfies a first condition is obtained; wherein the first condition includes: the difference between the running data of the updated hardware description code running the target program slice and the running data of the reference model corresponding to the updated hardware description code running the target program slice is less than or equal to a difference threshold.

[0021] In one possible implementation, the update module is further configured to: iteratively update the reference model based on the first running data to obtain an updated reference model; wherein the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics; update the hardware description code based on the parameters of the updated reference model and the second running data to obtain an updated hardware description code; iteratively update the hardware description code if the updated hardware description code does not meet the first condition; and stop updating the hardware description code if the updated hardware description code meets the first condition.

[0022] In one possible implementation, the update module is further configured to: determine first update parameters for updating the reference model based on first running data; wherein the first update parameters include configuration parameters corresponding to the first running data, and the configuration parameters corresponding to the first running data are used to update the reference model; and determine the updated reference model based on the reference model and the configuration parameters corresponding to the first running data.

[0023] In one possible implementation, the update module is further configured to: determine a second update parameter for updating the hardware description code based on the parameters of the updated reference model and the second running data when the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics; and obtain the updated hardware description code based on the second update parameter and the hardware description code.

[0024] In one possible implementation, the preset operating characteristics are determined based on a second condition, which includes one or more of the following: The front-end performance data is less than or equal to the first threshold; A memory miss occurs for M consecutive clock cycles; where M is an integer greater than or equal to 1. The duration of the stop receiving command is greater than or equal to N clock cycles; where N is an integer greater than or equal to 1; and, The proportion of pipeline recovery cycles caused by branch prediction errors to the total execution cycles is greater than or equal to the second threshold.

[0025] In one possible implementation, the runtime data includes one or more of the following: front-end runtime performance data, back-end runtime performance data, speculative execution phase performance data, and instruction retirement phase runtime performance data.

[0026] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the methods of the first aspect and any possible implementation thereof when executing the instructions stored in the memory.

[0027] Fourthly, embodiments of this disclosure provide a code verification system, including: a first electronic device and a second electronic device.

[0028] A first electronic device is configured to: run benchmark test program based on reference model running data to determine a target program slice in the benchmark test program; wherein the reference model is a software model for simulating hardware description code, and the first running data corresponding to the target program slice meets preset running characteristics; run the target program slice through hardware description code to obtain second running data; and determine the verification result of hardware description code based on the first running data and the second running data.

[0029] The second electronic device is configured to update the hardware description code based on the target program slice, the first running data, and the second running data if the verification result is unsuccessful.

[0030] Fifthly, embodiments of this disclosure provide a chip including a processor that can be used to implement the methods of the first aspect and any possible implementation thereof.

[0031] In a sixth aspect, embodiments of this disclosure provide a non-volatile computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the methods described in the first aspect and any possible implementation thereof.

[0032] In a seventh aspect, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods described in the first aspect and any possible implementation of the first aspect.

[0033] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description.

[0034] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0035] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture provided in the embodiments of this disclosure; Figure 2 A flowchart of a code verification method provided in this embodiment of the disclosure; Figure 3 A flowchart of another code verification method provided in this disclosure embodiment; Figure 4 A structural block diagram of a code verification device provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for executing a code verification method, provided as an embodiment of this disclosure. Detailed Implementation

[0036] To better understand this application, various aspects of this application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are merely illustrative of exemplary embodiments of this application and are not intended to limit the scope of this application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0037] It should be understood that expressions such as "comprising," "including," "having," "containing," and / or "comprising" are open-ended rather than closed-ended expressions in this specification, indicating the presence of the stated features, elements, and / or components, but not excluding the presence of one or more other features, elements, components, and / or combinations thereof. Furthermore, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features, not just individual elements in the list. Additionally, when describing embodiments of this application, the word "may" is used to mean "one or more embodiments of this application." And the term "exemplary" is intended to refer to examples or illustrations.

[0038] Unless otherwise specified, all terms used herein (including engineering and technical terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that, unless expressly stated herein, terms defined in common dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or overly formalized meaning.

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. Furthermore, unless explicitly limited or contradicted by the context, the specific steps included in the methods described in this application are not limited to the order in which they are described, but can be performed in any order or in parallel. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the code verification methods, apparatus, chips, electronic devices, computer-readable storage media, and computer program products of this disclosure can be applied.

[0041] like Figure 1 As shown, system architecture 100 may include a first electronic device 101 and a second electronic device 102, which can be connected via a network. The network serves as a medium to provide a communication link between the first electronic device 101 and the second electronic device 102. The network may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0042] The first electronic device 101 and the second electronic device 102 can be either hardware or software. When the first electronic device 101 and the second electronic device 102 are hardware, the first electronic device 101 can be various electronic devices, components on electronic devices, etc., and the second electronic device 102 can be various electronic devices, components on electronic devices, etc. When the first electronic device 101 and the second electronic device 102 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitations are made here.

[0043] The first electronic device 101 can be used to perform steps related to determining the verification result of the hardware description code. The second electronic device 102 can be used to perform steps related to updating the hardware description code. The first electronic device 101 and the second electronic device 102 can also be the same device. The steps related to updating the hardware description code and the steps related to determining the verification result of the hardware description code can be performed by the same device.

[0044] It should be understood that Figure 1 The number of first and second electronic devices shown is merely illustrative. Any number of first and second electronic devices can be used depending on implementation requirements.

[0045] In some embodiments, in the verification environment, the hardware description code is loaded and run by a simulator to execute the complete test set. The entire process typically takes about a week to complete the collection of log files for each test set. This process not only has a long execution cycle, but also consumes a large amount of hardware computing resources and manpower, resulting in low verification iteration efficiency.

[0046] Please refer to Figure 2 , Figure 2 The flowchart of a code verification method provided in this embodiment of the disclosure is shown. The execution subject is a first electronic device 101 or a second electronic device 102, wherein process 200 includes the following steps.

[0047] Step 201: Based on the running data of the benchmark program running the reference model, determine the target program slice in the benchmark program.

[0048] For example, hardware description code may include RTL code, etc.

[0049] For example, a reference model is a software model that simulates hardware description code. For instance, a reference model can simulate the behavior of hardware circuits corresponding to register-transfer level code, and can recreate the logical processes of hardware circuits executing instructions and processing data through software computation.

[0050] For example, a benchmark program is used to test the performance of a processor, which may be a CPU or the like. A benchmark program may also be called a standard program or a standard test program; the name is not limited.

[0051] For example, the first running data corresponding to the target program slice meets preset running characteristics. For instance, the first running data corresponding to the target program slice is abnormal.

[0052] For example, preset operating characteristics are used to distinguish between normal and abnormal states of operating data. For instance, if the preset operating characteristics are met, the operating data is in an abnormal state, and if the preset operating characteristics are not met, the operating data is in a normal state.

[0053] For example, a benchmark program is run using a reference model, generating corresponding runtime data during the process. Based on this runtime data, segments with abnormal runtime data are selected from all code snippets of the benchmark program; these are called target program slices. Runtime data can also be referred to as runtime performance data.

[0054] In some embodiments, runtime data includes, but is not limited to, one or more of the following: front-end runtime performance data, back-end runtime performance data, performance data of the speculative execution phase, and runtime performance data of the instruction retirement phase.

[0055] For example, front-end performance data can indicate the operational status data of the processor instruction fetching and decoding stages, including instruction prefetching efficiency, decoding throughput, and / or instruction cache hit rate.

[0056] For example, backend performance data can indicate the operational status data of the processor instruction execution and data processing phases, including memory misses.

[0057] For example, performance data during the speculative execution phase can indicate the processor's runtime status data during the instruction prediction phase based on predictive logic, and may include performance losses caused by branch prediction errors.

[0058] For example, performance data during the instruction retirement phase can represent the operational status data of the processor during the phase of completing instruction execution and releasing related resources, and may include the throughput of retired instructions, etc.

[0059] In this way, by collecting multi-dimensional operational data from different stages of the processor, the performance characteristics during program execution can be comprehensively and accurately reflected, thereby improving the accuracy of abnormal segment identification.

[0060] Step 202: Run the target program slice through the hardware description code to obtain the second running data.

[0061] For example, hardware description code may include register transfer level code, etc.

[0062] For example, the target program slice is input into the verification environment corresponding to the register-transfer level code, and the register-transfer level code is driven to execute the target program slice. During the execution of the target program slice by the register-transfer level code, the runtime performance data generated during the execution process is collected to obtain the second runtime data.

[0063] Using hardware description code at the hardware logic level as the execution carrier, the second execution data, which is different from the first execution data generated by running the target program slice in the reference model (software model), can reflect the real performance status of the hardware logic executing the target program slice and can provide hardware-side data support for subsequent comparative verification.

[0064] Step 203: Based on the first running data and the second running data, determine the verification result of the hardware description code.

[0065] For example, the first running data generated by running the target program slice on the reference model is compared and analyzed with the second running data generated by running the hardware description code on the same target program slice to determine the verification result.

[0066] For example, by combining the matching degree, differences, and anomaly characteristics of the first and second running data, the logical correctness and performance compliance of the hardware description code executing the target program slice are determined. This determination result is the verification result of the hardware description code. The verification result includes pass or fail.

[0067] The method provided in this disclosure locates a target program slice that meets preset operating characteristics in a benchmark test program based on the runtime data of a reference model. It performs hardware description code verification and analysis only on the target program slice, avoiding verification and analysis of all hardware description code. This can improve the overall hardware description code verification efficiency while ensuring the accuracy of performance anomaly verification.

[0068] In some embodiments, the preset operating characteristics are determined based on a second condition, which may include, but is not limited to, one or more of the following: front-end operating performance data is less than or equal to a first threshold; memory misses occur for M consecutive clock cycles; the duration of stopping instruction reception is greater than or equal to N clock cycles; and the proportion of pipeline recovery cycles caused by branch prediction errors to the total execution cycles is greater than or equal to a second threshold.

[0069] For example, whether an operational data anomaly has occurred can be determined based on a second condition.

[0070] For example, if the instruction prefetch efficiency, decoding throughput, and / or instruction cache hit rate in the front-end runtime performance data are less than or equal to the corresponding thresholds, it is determined that the runtime data is abnormal.

[0071] For example, if cache misses occur frequently within M consecutive clock cycles, it is determined that the running data is abnormal. M is an integer greater than or equal to 1. For example, the cache may include a Level 2 (L2) cache and a Last Level Cache (LLC).

[0072] For example, if the reorder buffer (ROB) in the processor stops receiving instructions for a duration greater than or equal to N clock cycles, where N is an integer greater than or equal to 1, it is determined that there is an anomaly in the running data.

[0073] The Operating Controller (ROB) is responsible for managing out-of-order instructions and retiring them according to the program sequence. If the ROB enters a paused state for N clock cycles, it cannot update the instruction status internally, nor can it pass retiring instructions to the instruction retirement stage. The instruction retirement stage cannot proceed due to a lack of pending instructions, creating a blockage in the instruction retirement process.

[0074] For example, it can be determined whether the performance loss caused by branch prediction errors exceeds the normal range. If it does, then the runtime data is considered abnormal. For instance, if the proportion of pipeline recovery cycles caused by branch prediction errors in the total number of program execution cycles is greater than or equal to a preset second threshold, then the runtime data is considered abnormal.

[0075] The pipeline recovery cycle refers to the clock cycles consumed by the processor to clear the erroneously executed instructions and rebuild the pipeline state after a branch prediction error; the total execution cycle is the total clock cycles of the entire program execution process.

[0076] Optionally, log file printing and output functions can be set at key nodes of the software model to record runtime data of various performance events, providing data support for performance bottleneck identification. Key nodes may include front-end instruction extraction, back-end instruction execution, data storage, instruction retirement, etc., ensuring that the runtime status of each type of performance event can be completely recorded.

[0077] Optionally, runtime data can be obtained by monitoring top-down performance events during the benchmark run.

[0078] For example, top-down performance events include, but are not limited to, one or more of the following: front-end bottleneck events, back-end bottleneck events, speculative execution events, and instruction retirement events.

[0079] For example, front-end bottleneck events can be used to analyze potential bottleneck scenarios during instruction fetching. Instruction fetching is a prerequisite for processor program execution; bottlenecks in this stage directly prevent subsequent instruction execution stages from obtaining valid instructions in a timely manner, thus affecting the overall program execution efficiency. Front-end bottleneck events can include scenarios such as instruction cache misses and branch prediction failures.

[0080] Instruction cache miss means that the processor cannot find the target instruction in the instruction cache when fetching instructions, and must read the instruction from memory, increasing instruction fetching latency. Branch prediction failure means that the processor's prediction of the program branch jump direction is inconsistent with the actual execution direction, causing the fetched instructions to fail to execute correctly, and the correct instructions must be fetched again.

[0081] For example, back-end bottleneck events are used to analyze bottleneck scenarios in execution units and storage units. Execution units are the modules in a processor that execute instructions, while storage units are the modules in a processor that store instructions and data. Back-end bottleneck events can include scenarios such as memory misses and execution unit latency.

[0082] Memory miss refers to a situation where, during instruction execution, the processor needs to access data that is not found in the cache and must read the data from memory, increasing data access latency. Execution unit latency refers to a situation where the time required for an execution unit to complete instruction computation exceeds a preset range, causing subsequent instructions to fail to enter the execution phase in a timely manner.

[0083] For example, speculative execution events can be used to analyze performance losses caused by branch prediction errors during program execution. Speculative execution is a technique used by processors to improve execution efficiency. The processor executes subsequent instructions in advance based on branch prediction results. If a branch prediction error occurs, the pre-executed instructions are invalidated, and the correct instructions must be re-executed. This process consumes additional clock cycles, resulting in performance loss. Speculative execution events primarily monitor data such as the duration of invalid execution caused by branch prediction errors and the number of invalid instructions, thereby quantifying the performance loss caused by branch prediction errors.

[0084] For example, instruction retirement events can be used to analyze performance losses caused by instruction retirement during program execution. Instruction retirement is the final stage of instruction execution, referring to the process by which the processor releases the hardware resources occupied by an instruction after execution and records the execution result. If the instruction retirement process malfunctions, hardware resources may not be released in a timely manner, preventing subsequent instructions from properly utilizing those resources, thus causing performance loss. Instruction retirement events primarily monitor data such as instruction retirement throughput and the backlog of unretired instructions.

[0085] Optionally, during the execution of the benchmark program, top-down performance events are monitored in real time. When any of the conditions in the second condition are met, it is determined that the running data is abnormal, the target program slice is determined, and a slice interrupt can be triggered to capture the current program execution segment and save the running context for subsequent hardware description code simulation and performance alignment.

[0086] In some embodiments, the target program slice includes a pre-exception code segment, an exception code segment, and a recovery code segment that are executed in a sequential order.

[0087] Optionally, the code segment before the exception is the code segment where the running data does not meet the preset running characteristics, the code segment after the exception is the code segment where the running data meets the preset running characteristics, and the code segment after the exception is the code segment where the running data is restored from a state that meets the preset running characteristics to a state that does not meet the preset running characteristics.

[0088] For example, if the running data meets preset running characteristics, it indicates that the running data is abnormal. The abnormal code segment is the code segment where the running data is abnormal, and the recovery code segment is the code segment where the running data recovers from the abnormal state to the normal state. The running data in the code segment before the abnormality did not show any abnormality and did not meet the preset running characteristics.

[0089] For example, an exception code segment refers to a code fragment in which abnormal data occurs during program execution.

[0090] For example, a recovery code segment refers to a code snippet during program execution that restores running data from an abnormal state to a normal state. The execution of this code snippet can terminate performance exceptions and restore the program to normal operation.

[0091] The code segment before the exception, the exception code segment, and the recovery code segment are arranged consecutively in the order of program execution, fully covering the entire process of performance exception from preparation before occurrence to exception occurrence and exception recovery.

[0092] For example, the execution order of code segments in a target program slice might be: pre-exception code segment, exception code segment, and recovery code segment. These segments execute sequentially, covering the entire exception handling cycle. The pre-exception code segment executes before the exception code segment, and the recovery code segment executes after it; generally, there are no redundant code segments between adjacent segments. Thus, by explicitly including consecutive pre-exception, exception, and recovery code segments in the target program slice, the entire process of performance anomalies can be captured, accurately covering the code segments involved in exception generation, persistence, and recovery, avoiding omissions of critical performance data. This provides comprehensive and accurate slice data support for subsequent microarchitecture alignment and performance verification of register-transfer level hardware and software models, improving the accuracy of microarchitecture alignment, further shortening the verification cycle, and reducing verification resource consumption. It can solve the problem of low verification iteration efficiency caused by focusing only on program behavior similarity without considering the code's sensitivity to microarchitecture.

[0093] In some embodiments, step 201, which determines the target program slice in the benchmark program based on the running data of the benchmark program running the reference model, may include steps A1-A2.

[0094] Step A1: Based on the running data of the benchmark program running the reference model, identify the abnormal code segment and the recovery code segment.

[0095] For example, during the benchmark test program running on the reference model, running data is recorded. When the running data is detected to be abnormal for the first time, the corresponding code to be executed is the abnormal code segment; when the abnormal running data is detected to be restored to the normal range, the corresponding code to be executed is the recovery code segment, ensuring the accuracy of slice positioning.

[0096] Step A2: Determine the target program slice from the benchmark program based on the first quantity, the exception code segment, and the recovery code segment.

[0097] Optionally, the first quantity is the number of instructions included in the preset exception pre-exception code segment.

[0098] For example, using the abnormal code segment as a baseline, K instructions are automatically traced back to the code segment before the abnormality, and execution continues forward until the performance bottleneck disappears, thus determining the window period of the target program slice. The number of instruction execution cycles and bottleneck events within this window period are statistically analyzed, and the metric density within the window period is calculated based on the ratio of bottleneck events to instruction cycles. When the metric density falls back to a normal threshold, the bottleneck is considered to have disappeared, thus determining the termination boundary of the target program slice. The length and position of the target program slice obtained in this way are no longer fixed, but dynamically determined by the actual operating behavior of the processor microarchitecture.

[0099] In this way, by dynamically and adaptively generating target program slices, the target program slices are no longer fixed and rigid, but are more in line with the actual hardware operating characteristics, making bottleneck identification more reliable and improving code verification efficiency.

[0100] Optionally, a target program slice can be obtained through a preset boot file. This boot file may include the conditions for triggering the capture of the target program slice, interrupt handling locations, and state saving strategies. Corresponding interrupt handling logic is set at the start and end points of the target program slice. When the capture of the target program slice is triggered, program execution is paused via an interrupt mechanism, and the current program running state is saved to memory using hardware checkpoints or memory dumping techniques. This allows the program to restore its running state and reproduce the program execution scenario during subsequent simulations.

[0101] Optionally, code stubs can be pre-embedded at the beginning of the target program slice to trigger the slice process, such as interrupting or pausing, or initiating state capture. Code stubs can also be pre-embedded at the end of the target program slice to terminate the slice process, such as resuming program execution or completing state saving, which can improve the accuracy of slice boundaries.

[0102] In some embodiments, step A1 determines the abnormal code segment and the recovery code segment based on the running data of the benchmark program running on the reference model, including steps A11-A14.

[0103] Step A11: Run the benchmark program using the reference model to generate basic block vectors.

[0104] For example, a Basic Block Vector (BBV) may include: a Basic Block ID, an execution frequency, and a fixed window.

[0105] For example, a basic block identifier is a unique identifier assigned to each basic block in a program, used to accurately locate the basic block's position in the program.

[0106] For example, the execution frequency is the number of times the basic block corresponding to the basic block identifier is executed within a preset fixed window, or the proportion of the total number of executions.

[0107] For example, a fixed window is a pre-defined statistical interval for instruction execution, which can be defined by the number of instructions, the number of CPU cycles, or the length of time. It serves as the statistical boundary for the execution frequency, ensuring that the statistical range of BBV is controllable and reproducible.

[0108] For example, the benchmark program is loaded into the reference model and started to execute. The reference model monitors the execution process of the benchmark program, focusing on monitoring the execution status of each instruction, and uniquely marks and analyzes the execution characteristics of each basic block in the program. It records all execution information in real time and uses this execution information as the original data source to generate the basic block vector required for subsequent clustering.

[0109] Step A12: Based on the basic block vectors, cluster the code in the benchmark program into one or more first code regions.

[0110] For example, the BBV (Browser Values) corresponding to different code snippets in the benchmark program are extracted, and the similarity between each BBV is calculated. Algorithms such as cosine similarity and Euclidean distance are used to group code snippets whose BBV feature similarity meets a preset threshold into one category. The code snippets corresponding to each category form a first code interval.

[0111] For example, if the BBV features of all code in the benchmark program are highly consistent, then they are clustered into a single first code interval. If there are multiple sets of BBVs with significantly different features, then they are clustered into multiple first code intervals.

[0112] For example, the basic block vectors are imported into the SimPoint clustering analysis script to perform data cleaning and normalization, outputting a standardized BBV matrix. The SimPoint clustering analysis script calls the K-means algorithm, inputs the standardized BBV matrix, and sets the number of clusters K, which is determined according to program complexity / elbow rule. The program execution window is clustered into K clusters based on BBV similarity, with each cluster corresponding to a typical behavioral stage. The centroid BBV or the most representative BBV within each cluster is extracted and mapped to the physical execution interval of the program. The physical interval includes the start / end PC address and instruction range; these intervals are the simulation points or the first code interval. This can reduce the time and computational resources required for subsequent analysis.

[0113] Step A13: Run one or more first code intervals using the reference model to obtain the running data corresponding to the first code intervals.

[0114] For example, one or more first code intervals obtained by clustering basic block vectors are input into the reference model for execution, and the corresponding running data generated by each first code interval during the execution is obtained.

[0115] Step A14: Based on the runtime data corresponding to the first code segment, determine the abnormal code segment and the recovery code segment.

[0116] For example, based on the runtime data generated by running the first code segment using the reference model, code segments in the runtime data that meet the second condition are identified as abnormal code segments. Code segments in which the runtime data is restored to the normal range after locating the abnormal code segments are identified as restored code segments.

[0117] Thus, by using Basic Block Vector (BBV) clustering, coarse-grained behavioral phases of the program execution process are divided, determining the first code segment. Based on runtime performance data, fine-grained slices are triggered and generated on the basis of the first code segment. This achieves hierarchical decoupling between coarse-grained phase division and fine-grained performance slicing, effectively improving the flexibility and accuracy of program slicing.

[0118] In some embodiments, the method provided in this disclosure may further include step B1.

[0119] Step B1: If the verification result is unsuccessful, update the hardware description code based on the target program slice, the first running data, and the second running data.

[0120] For example, the verification result includes pass or fail. When the verification result is fail, the target program slice is used as the analysis object. By combining the differences between the first running data and the second running data, the logical problem in the hardware description code that caused the verification failure is located. Based on the analysis result, the hardware description code is updated and optimized, which can improve the optimization efficiency of the hardware description code and the verification convergence speed.

[0121] In some embodiments, step B1, which updates the hardware description code based on the target program slice, the first running data, and the second running data, may include step B11.

[0122] Step B11: Based on the target program slice, the first running data, and the second running data, iteratively update the hardware description code until a hardware description code that satisfies the first condition is obtained.

[0123] Optionally, the first condition includes: the difference between the running data in the target program slice process of the updated hardware description code and the running data in the target program slice process of the reference model corresponding to the updated hardware description code is less than or equal to the difference threshold.

[0124] For example, the runtime data during the execution of the target program slice from the updated hardware description code is largely consistent with the runtime data during the execution of the target program slice from the reference model corresponding to the updated hardware description code, verifying that the hardware description code meets the performance standards. Similarly, the time taken to obtain runtime data based on the updated hardware description code is largely consistent with the time taken to obtain runtime data based on the reference model corresponding to the updated hardware description code, verifying that the hardware description code meets the performance standards.

[0125] In this way, by using target program slicing and two sets of runtime data to drive the iterative update of hardware description code, it is possible to accurately converge to hardware description code that is largely consistent with the runtime data and reference model, which can significantly improve the optimization efficiency of hardware logic verification.

[0126] In some embodiments, step B11 iteratively updates the hardware description code based on the target program slice, the first running data, and the second running data until a hardware description code that satisfies the first condition is obtained, and may include steps B111-B114.

[0127] Step B111: Based on the first running data, iteratively update the reference model to obtain the updated reference model.

[0128] For example, the runtime data obtained by running the target program slice using the updated reference model does not meet the preset runtime characteristics. For instance, the runtime data obtained by running the target program slice using the updated reference model does not show any anomalies.

[0129] For example, based on the first running data, the logical parameters, execution rules, etc. of the original reference model are continuously adjusted and updated until the running data obtained by running the target program slice of the updated reference model does not show any abnormalities, and finally the updated reference model is obtained.

[0130] Step B112: Based on the updated reference model parameters and the second running data, update the hardware description code to obtain the updated hardware description code.

[0131] For example, using the parameters of the updated reference model as a benchmark, and combining the second running data to analyze the differences between the two, the hardware logic problems in the hardware description code that cause the differences are located, and the hardware description code is updated accordingly, ultimately resulting in the updated hardware description code.

[0132] Step B113: If the updated hardware description code does not meet the first condition, iteratively update the hardware description code.

[0133] For example, if the updated hardware description code fails to meet the preset first condition after verification, the hardware logic problem is located based on the difference between the hardware description code's running data and the first condition, and the RTL code is iteratively adjusted and updated. The above update verification process is repeated until the hardware description code meets the first condition.

[0134] Step B114: If the updated hardware description code meets the first condition, stop updating the hardware description code.

[0135] For example, if the updated hardware description code is verified to meet the preset first condition, the iterative update process of the hardware description code is terminated, and the hardware description code that currently meets the condition is taken as the final hardware description code.

[0136] In this way, by iteratively calibrating the reference model using the first running data and ensuring that its running data is free of anomalies, and then combining the parameters of the calibrated reference model with the second running data to update the hardware description code until the first condition is met, the accuracy and efficiency of hardware description code optimization can be improved.

[0137] In some embodiments, step B111 iteratively updates the reference model based on the first running data to obtain the updated reference model, and may include steps B1111-B1112.

[0138] Step B1111: Based on the first running data, determine the first update parameters for updating the reference model.

[0139] Optionally, the first update parameters include configuration parameters corresponding to the first running data. The configuration parameters corresponding to the first running data are used to update the reference model.

[0140] For example, quantitative analysis and difference analysis are performed on the first running data to extract the core parameters that need to be adjusted in the reference model and their corresponding target values, which are then determined as the first update parameters for updating the reference model.

[0141] For example, the first runtime data includes, but is not limited to, one or more of the following: front-end runtime performance data, back-end runtime performance data, performance data of the speculative execution phase, and runtime performance data of the instruction retirement phase. Taking the instruction prefetching efficiency, which includes front-end runtime performance data, as an example where the instruction prefetching efficiency is abnormal, before updating the reference model, if the monitored instruction prefetching efficiency is less than a preset efficiency threshold, it is determined that the instruction prefetching efficiency is abnormal. Based on this abnormal instruction prefetching efficiency data, the corresponding configuration parameters of the reference model are determined. These configuration parameters can be used to improve the instruction prefetching efficiency of the reference model. The reference model is updated using these configuration parameters to improve the instruction prefetching efficiency of the reference model. After the model update is completed, the new instruction prefetching efficiency output based on the updated reference model is greater than or equal to the preset efficiency threshold, thus restoring the instruction prefetching efficiency to a normal operating state.

[0142] Step B1112: Determine the updated reference model based on the reference model and the configuration parameters corresponding to the first running data.

[0143] Thus, by determining the first update parameter based on the first running data and applying the first update parameter to the reference model to complete parameter calibration, the accuracy of the running data of the reference model can be improved.

[0144] In some embodiments, step B112 updates the hardware description code based on the parameters of the updated reference model and the second operating data to obtain the updated hardware description code, including steps B1121-B1122.

[0145] Step B1121: If the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics, determine the second update parameters for updating the hardware description code based on the parameters of the updated reference model and the second running data.

[0146] Step B1122: Obtain the updated hardware description code based on the second update parameters and the hardware description code.

[0147] For example, after confirming that the running data of the target program slice in the updated reference model is normal, the parameters of the updated reference model are used as the optimization benchmark. Combined with the second running data, second update parameters are extracted and determined to adjust the hardware description code. Substituting the second update parameters into the original hardware description code completes the hardware logic parameter update, resulting in the updated hardware description code, which can improve the accuracy of RTL updates.

[0148] In some embodiments, during the execution of benchmark programs or target program slices using hardware description code, performance counters can be used to monitor and statistically analyze the occurrence, duration, and frequency of various performance-related events during hardware execution in real time, such as the number, duration, and frequency of running data failing to meet the second condition. Optionally, the counting results can be used for software and hardware performance comparison and hardware design parameter adjustment.

[0149] The hardware description code is monitored to obtain runtime performance data. This data is compared with the simulation performance data of the software model to identify alignment differences between the hardware and software models. The hardware design is then progressively adjusted until the hardware model's performance data matches that of the reference model. Performance metrics obtained from RTL simulations are fed back into the model, allowing for targeted adjustments to design parameters (such as cache size and branch prediction mechanisms). This hardware-software performance comparison accelerates the iteration process, progressively optimizing the hardware design, and validating the effectiveness of these optimizations in simulation. This approach allows for the correction of model alignment through hardware-software performance data comparison, and uses the reference model's performance data to guide RTL design, thus accelerating hardware iteration.

[0150] For example, the count result of the hardware description code includes "number of branch prediction errors = 10 times"; the count result of the reference model includes "number of branch prediction errors = 2 times". By comparing the difference between the two counts, the alignment difference between the hardware model and the software model can be accurately identified. For example, if the number of branch prediction errors differs by 8 times, the design parameters of the branch prediction mechanism can be adjusted accordingly until the count results of the two are roughly consistent.

[0151] Figure 3 This is a flowchart of another code verification method provided in an embodiment of this disclosure. The executing entity is a first electronic device or a second electronic device, wherein process 300 includes the following steps.

[0152] Step 301: Input the benchmark test program.

[0153] For example, input the benchmark program into the reference model.

[0154] Step 302: Run the benchmark test program using the reference model to generate the basic fast vector.

[0155] Step 303: Perform clustering based on the basic block vectors to obtain the clustering results.

[0156] For example, based on basic block vectors, the code in the benchmark program is clustered into one or more first code regions, and the clustering result includes one or more first code regions. Representative first code regions can also be selected from the clustering results.

[0157] Optionally, the corresponding slice intervals can be cut out based on the clustering results to obtain one or more first code intervals. An interruption is triggered at the start of the first code interval, and the breakpoint is saved.

[0158] Step 304: Top-down performance analysis.

[0159] For example, a top-down performance analysis can be performed on a selected representative first code segment to obtain a slice of the target program. An interrupt can be triggered at the start of the target program slice to save the breakpoint.

[0160] For example, it can also output log files. Log files include various types of runtime performance data.

[0161] Step 305: Run the target program slice using the hardware description code.

[0162] For example, the target program slice is run through the hardware description code and hardware-side performance data is collected by comparing it with the performance acquisition points.

[0163] Step 306: Comparison and iteration of software and hardware performance.

[0164] For example, performance data collected through hardware description code can be compared with software-side performance data in log files to complete hardware and software performance comparison, supporting subsequent model calibration and RTL iterative optimization.

[0165] The method provided in this disclosure is based on performance event-driven adaptive slicing, which efficiently selects target program slices and avoids full simulation. An interrupt mechanism is introduced to make the simulation environment controllable, and the top-down performance analysis method outputs fast and accurate feedback.

[0166] The method provided in this disclosure reduces the iteration cycle from weekly to hourly by slicing the target program, which can effectively alleviate the problem of cycle extension caused by the increase of the test set and significantly shorten the iteration cycle.

[0167] The method provided in this disclosure uses a top-down analysis to quickly identify key features and performance indicators. It features efficient and comprehensive feature extraction, outputs static logs to support software and hardware comparison, accurately locates gaps, and reduces the need for full-scale simulation.

[0168] The method provided in this disclosure introduces an interrupt, supporting rerunning the simulation from a specified location without requiring a full execution. Combined with a top-down analysis mechanism, test segments can be reasonably divided. Each rerun does not require a full execution, but only reloads from the interrupted state, which can significantly shorten the verification and development cycles. This avoids the situation where, when verifying and iterating on batches of long test segments, if key feature segments have been identified and corresponding optimizations and improvements have been completed, subsequent iterations still require a full test from the beginning, resulting in low architecture iteration efficiency.

[0169] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a code verification device, which corresponds to the above method embodiments, and the device can be specifically applied to various electronic devices.

[0170] like Figure 4 As shown, the code verification device 400 of this embodiment may include: a slice determination module 401, a data acquisition module 402, and a result verification module 403. The slice determination module 401 is configured to determine a target program slice in the benchmark test program based on the runtime data of the benchmark test program running from a reference model; wherein the reference model is a software model that simulates the hardware description code, and the first runtime data corresponding to the target program slice satisfies preset runtime characteristics. The data acquisition module 402 is configured to run the target program slice through the hardware description code to obtain second runtime data. The result verification module 403 is configured to determine the verification result of the hardware description code based on the first runtime data and the second runtime data.

[0171] In this embodiment, the specific processing of the slice determination module 401, the data acquisition module 402, and the result verification module 403 in the code verification device 400 and the resulting technical effects can be referred to the relevant descriptions of the above method embodiments, and will not be repeated here.

[0172] In one possible implementation, the target program slice includes a pre-exception code segment, an exception code segment, and a recovery code segment, which are executed in a sequential order. The exception code segment is the code segment in which the running data meets preset running characteristics. The execution order of the pre-exception code segment is before that of the exception code segment. The recovery code segment is the code segment in which the running data is restored from a state that meets the preset running characteristics to a state that does not meet the preset running characteristics.

[0173] In one possible implementation, the slice determination module 401 is further configured to: determine the abnormal code segment and the recovery code segment based on the running data of the benchmark test program running the reference model; and determine the target program slice from the benchmark test program according to a first quantity, the abnormal code segment and the recovery code segment; wherein the first quantity is the number of instructions included in the preset code segment before the exception.

[0174] In one possible implementation, the slice determination module 401 is further configured to: run a benchmark test program through a reference model to generate basic block vectors; cluster the code in the benchmark test program into one or more first code intervals based on the basic block vectors; run one or more first code intervals through the reference model to obtain running data corresponding to the first code intervals; and determine abnormal code segments and recovery code segments based on the running data corresponding to the first code intervals.

[0175] In one possible implementation, the code verification device 400 further includes an update module. The update module is configured to update the hardware description code based on the target program slice, the first runtime data, and the second runtime data if the verification result is unsuccessful.

[0176] In one possible implementation, the update module is further configured to: iteratively update the hardware description code based on the target program slice, the first running data, and the second running data, until a hardware description code that satisfies a first condition is obtained; wherein the first condition includes: the difference between the running data of the updated hardware description code running the target program slice and the running data of the reference model corresponding to the updated hardware description code running the target program slice is less than or equal to a difference threshold.

[0177] In one possible implementation, the update module is further configured to: iteratively update the reference model based on the first running data to obtain an updated reference model; wherein the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics; update the hardware description code based on the parameters of the updated reference model and the second running data to obtain an updated hardware description code; iteratively update the hardware description code if the updated hardware description code does not meet the first condition; and stop updating the hardware description code if the updated hardware description code meets the first condition.

[0178] In one possible implementation, the update module is further configured to: determine first update parameters for updating the reference model based on first running data; wherein the first update parameters include configuration parameters corresponding to the first running data, and the configuration parameters corresponding to the first running data are used to update the reference model; and determine the updated reference model based on the reference model and the configuration parameters corresponding to the first running data.

[0179] In one possible implementation, the update module is further configured to: determine a second update parameter for updating the hardware description code based on the parameters of the updated reference model and the second running data when the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics; and obtain the updated hardware description code based on the second update parameter and the hardware description code.

[0180] In one possible implementation, the preset operating characteristics are determined based on a second condition, which includes one or more of the following: The front-end performance data is less than or equal to the first threshold; A memory miss occurs for M consecutive clock cycles; where M is an integer greater than or equal to 1. The duration of the stop receiving command is greater than or equal to N clock cycles; where N is an integer greater than or equal to 1; and, The proportion of pipeline recovery cycles caused by branch prediction errors to the total execution cycles is greater than or equal to the second threshold.

[0181] In one possible implementation, the runtime data includes one or more of the following: front-end runtime performance data, back-end runtime performance data, speculative execution phase performance data, and instruction retirement phase runtime performance data.

[0182] This disclosure proposes a code verification system, including a first electronic device and a second electronic device.

[0183] The first electronic device is configured to: run benchmark test program based on reference model running data to determine target program slices in the benchmark test program; wherein, the reference model is a software model for simulating hardware description code, and the first running data corresponding to the target program slice meets preset running characteristics; run the target program slice through hardware description code to obtain second running data; and determine the verification result of hardware description code based on the first running data and the second running data.

[0184] The second electronic device is configured to update the hardware description code based on the target program slice, the first running data, and the second running data if the verification result is unsuccessful.

[0185] Optionally, the first electronic device is configured to perform any relevant steps in the above method embodiments for determining the verification result of the hardware description code. The second electronic device is configured to perform any relevant steps in the above method embodiments for updating the hardware description code.

[0186] This disclosure also provides a chip including a processor that can be used to implement any of the possible implementations of the methods provided in this disclosure.

[0187] This disclosure also proposes an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing instructions stored in the memory.

[0188] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the described method. The computer-readable storage medium can be volatile or non-volatile.

[0189] This disclosure also provides a computer program product that, when executed by a processor, can implement the methods described in any of the above embodiments.

[0190] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0191] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. For example... Figure 5 As shown, the electronic device 500 of this embodiment includes a processor 501 and may further include a memory 502; wherein the memory 502 is used to store computer execution instructions; and the processor 501 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0192] Alternatively, the memory 502 can be either standalone or integrated with the processor 501.

[0193] When the memory 502 is set up independently, the electronic device also includes a bus 503 for connecting the memory 502 and the processor 501.

[0194] It should be understood that the processor 501 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules within the processor.

[0195] The memory 502 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0196] Bus 503 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0197] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0198] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0199] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0200] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0201] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0202] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0203] According to the technical solution of this disclosure, the target program slice that meets the preset running characteristics in the benchmark test program is located based on the running data of the reference model. The hardware description code is verified and analyzed only for the target program slice, avoiding the verification and analysis of all hardware description code. This can improve the overall hardware description code verification efficiency while ensuring the accuracy of performance anomaly verification.

[0204] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0205] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A code verification method, comprising: Based on the running data of the benchmark test program running the reference model, the target program slice in the benchmark test program is determined; wherein, the reference model is a software model for simulating hardware description code, and the first running data corresponding to the target program slice satisfies preset running characteristics; The target program slice is run by the hardware description code to obtain the second running data; Based on the first running data and the second running data, the verification result of the hardware description code is determined.

2. The method according to claim 1, wherein, The target program slice includes a code segment before the exception, an exception code segment, and a recovery code segment, all executed in a sequential order. Wherein, the pre-abnormal code segment is the code segment in which the running data does not meet the preset running characteristics, the abnormal code segment is the code segment in which the running data meets the preset running characteristics, and the recovery code segment is the code segment in which the running data recovers from a state that meets the preset running characteristics to a state that does not meet the preset running characteristics.

3. The method according to claim 2, wherein, The determination of the target program slice in the benchmark program based on the runtime data of the benchmark program running based on the reference model includes: Based on the runtime data of the benchmark program run on the reference model, the abnormal code segment and the recovery code segment are determined; The target program slice is determined from the benchmark test program based on the first quantity, the abnormal code segment, and the recovery code segment; wherein, the first quantity is the preset number of instructions included in the code segment before the abnormality.

4. The method according to claim 3, wherein, The process of determining the abnormal code segment and the recovery code segment based on the runtime data of the benchmark program run on the reference model includes: The benchmark program is run using the reference model to generate basic block vectors; Based on the basic block vector, the code in the benchmark program is clustered into one or more first code intervals; By running the reference model through the one or more first code segments, running data corresponding to the first code segments is obtained; Based on the runtime data corresponding to the first code interval, the abnormal code segment and the recovery code segment are determined.

5. The method according to any one of claims 1-4, wherein, The method further includes: If the verification result is unsuccessful, the hardware description code is updated based on the target program slice, the first running data, and the second running data.

6. The method according to claim 5, wherein, The step of updating the hardware description code based on the target program slice, the first running data, and the second running data includes: Based on the target program slice, the first running data, and the second running data, the hardware description code is iteratively updated until a hardware description code that satisfies a first condition is obtained; wherein, the first condition includes: the difference between the running data of the updated hardware description code running the target program slice and the running data of the reference model corresponding to the updated hardware description code running the target program slice is less than or equal to a difference threshold.

7. The method according to claim 6, wherein, The step of iteratively updating the hardware description code based on the target program slice, the first running data, and the second running data until hardware description code satisfying the first condition is obtained includes: Based on the first running data, the reference model is iteratively updated to obtain an updated reference model; wherein, the running data obtained by running the target program slice using the updated reference model does not meet the preset running characteristics; Based on the parameters of the updated reference model and the second running data, the hardware description code is updated to obtain the updated hardware description code; If the updated hardware description code does not meet the first condition, the hardware description code is iteratively updated. If the updated hardware description code satisfies the first condition, the updating of the hardware description code shall be stopped.

8. The method according to claim 7, wherein, The step of iteratively updating the reference model based on the first running data to obtain the updated reference model includes: Based on the first running data, a first update parameter for updating the reference model is determined; wherein, the first update parameter includes a configuration parameter corresponding to the first running data; the configuration parameter corresponding to the first running data is used to update the reference model; The updated reference model is determined based on the reference model and the configuration parameters corresponding to the first running data.

9. The method according to claim 7 or 8, wherein, The step of updating the hardware description code based on the parameters of the updated reference model and the second operating data to obtain the updated hardware description code includes: If the running data obtained by running the target program slice on the updated reference model does not meet the preset running characteristics, a second update parameter for updating the hardware description code is determined based on the parameters of the updated reference model and the second running data. Based on the second update parameter and the hardware description code, the updated hardware description code is obtained.

10. The method according to any one of claims 1-4, wherein, The preset operating characteristics are determined based on a second condition, which includes one or more of the following: The front-end performance data is less than or equal to the first threshold; A memory miss occurs for M consecutive clock cycles; where M is an integer greater than or equal to 1. The duration of the stop receiving command is greater than or equal to N clock cycles; where N is an integer greater than or equal to 1; and, The proportion of pipeline recovery cycles caused by branch prediction errors to the total execution cycles is greater than or equal to the second threshold.

11. The method according to any one of claims 1-4, wherein, The operational data includes one or more of the following: front-end operational performance data, back-end operational performance data, performance data of the speculative execution phase, and operational performance data of the instruction retirement phase.

12. A code verification device, comprising: The slice determination module is configured to determine the target program slice in the benchmark test program based on the running data of the benchmark test program running a reference model; wherein, the reference model is a software model for simulating hardware description code, and the first running data corresponding to the target program slice satisfies preset running characteristics; The data acquisition module is configured to run the target program slice through the hardware description code to obtain the second running data; The result verification module is configured to determine the verification result of the hardware description code based on the first running data and the second running data.

13. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 11 when executing instructions stored in the memory.

14. A code verification system, comprising: A first electronic device is configured to: run benchmark test program based on reference model running data to determine a target program slice in the benchmark test program; wherein the reference model is a software model for simulating hardware description code, and the first running data corresponding to the target program slice satisfies preset running characteristics; run the target program slice through the hardware description code to obtain second running data; and determine the verification result of the hardware description code based on the first running data and the second running data. The second electronic device is configured to update the hardware description code based on the target program slice, the first running data, and the second running data if the verification result is unsuccessful.

15. A chip comprising a processor for performing the method of any one of claims 1 to 11.

16. A non-volatile computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method of any one of claims 1 to 11.

17. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images