Performance test system based on hardware simulation, hardware simulator and test method

Abstract

The invention discloses a hardware simulation-based performance test system, a hardware simulator and a test method, and relates to the technical field of chip design. The system comprises a request generation module, a protocol core module, a data stream sending module and a to-be-tested equipment simulation module, the request generation module is used for generating a test request of the data bus system according to the loaded executable file, and the test request is used for testing the data bus system; the protocol core module is used for converting the test request into a test message and converting the test message into a transmission data stream, and the test message conforms to a bus protocol corresponding to the data bus system; and the data stream sending module is used for sending the transmission data stream to the to-be-tested equipment simulation module, and the to-be-tested equipment simulation module is used for simulating equipment using the data bus system. Real physical equipment is simulated through the performance test system based on hardware simulation so as to test the data bus system, and the test accuracy of the data bus system can be improved.

Description

Technical Field

[0001] This application relates to the field of chip design technology, and in particular to a performance testing system, hardware simulator and testing method based on hardware simulation. Background Technology

[0002] Emulation (hardware simulation) is a chip verification technology based on dedicated hardware platforms (such as FPGA (Field Programmable Gate Array), custom processors, etc.). It achieves a speedup efficiency of thousands of times higher than software simulation by compiling the RTL (Register Transfer Level) code of the device under test (DUT) to physical hardware for execution.

[0003] Currently, PCIe (Peripheral Component Interconnect Express, a high-speed serial point-to-point bus standard) systems can be verified in an Emulation environment. For example, related technologies run the RTL code of the PCIe controller inside the Emulator (hardware emulator) and connect the Emulator and PCIe devices (i.e., physical devices that use PCIe system communication) through a dedicated hardware adapter (such as Cadence SpeedBridge), thereby enabling interactive verification between the RTL code of the PCIe controller and the PCIe devices.

[0004] However, the dedicated hardware adapters in related technologies cannot simulate a real PCIe interface environment, which leads to distortion in the performance test of PCIe systems and inaccurate performance test results. Summary of the Invention

[0005] This application provides a hardware simulation-based performance testing system, hardware simulator, and testing method. The technical solution provided by this application includes the following:

[0006] According to one aspect of the embodiments of this application, a performance testing system based on hardware simulation is provided, the system comprising: a request generation module, a protocol core module, a data stream sending module, and a device under test simulation module; The request generation module is used to generate a test request for the data bus system based on the loaded executable file, and the test request is used to test the data bus system. The core module of the protocol is used to convert the test request into a test message, and the test message into a transmission data stream. The test message conforms to the bus protocol corresponding to the data bus system. The data stream sending module is used to send the transmitted data stream to the device under test simulation module, and the device under test simulation module is used to simulate a device using the data bus system.

[0007] According to one aspect of the embodiments of this application, a hardware emulator is provided, on which a hardware emulation-based performance testing system as described above is run.

[0008] According to one aspect of the embodiments of this application, a performance testing method is provided, the method being implemented by a hardware simulation-based performance testing system; the method includes: Based on the loaded executable file, a test request for the data bus system is generated, the test request being used to test the data bus system; The test request is converted into a test message, and the test message is converted into a transmission data stream. The test message conforms to the bus protocol corresponding to the data bus system. The transmitted data stream is sent to the device under test (DUT) simulation module, which is used to simulate devices using the data bus system.

[0009] The technical solutions provided in this application embodiment may include the following beneficial effects.

[0010] By using a hardware simulation-based performance testing system, a real data bus system and devices using the data bus system (hereinafter referred to as bus devices) are simulated to realize interactive verification simulation between the data bus system and the devices using the data bus system, without the need to connect real physical devices such as dedicated hardware adapters or bus devices. This avoids performance test distortion caused by the introduction of dedicated hardware adapters, thereby improving the testing accuracy of the data bus system. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of a computer system provided in one possible implementation of this application; Figure 2This is a schematic diagram of a hardware simulation-based performance testing system provided in one possible implementation of this application; Figure 3 This is a flowchart of a performance testing system test data bus system provided in one possible implementation of this application; Figure 4 This is a flowchart of a performance testing method provided in one possible implementation of this application; Figure 5 This is a simplified structural block diagram of a computer device provided in one possible implementation of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0014] Emulation (hardware simulation) is a chip verification technology based on dedicated hardware platforms (such as FPGAs, custom processors, etc.). Chip verification refers to the process of verifying the chip structure characterized by the chip design document before fabricating the physical chip. The chip design document can be code written in a hardware description language, which can be synthesized into the actual chip structure, i.e., the hardware circuit.

[0015] An emulation environment can be implemented as an emulator (hardware simulator). The emulator can run chip design files, such as the RTL code of a chip design. For example, an emulator can be a simulation device based on a dedicated hardware platform such as an FPGA or a custom processor. It is a real physical device and can be used to perform hardware-level simulations, such as functional, performance, and power consumption testing before chip tape-out. For instance, hardware-level simulation of the design under test (such as a chip) can be achieved by compiling the RTL code of the design under test (such as a chip) and running it on the emulator.

[0016] In related technologies, an Emulator is used to verify PCIe systems. A PCIe system refers to a system built on PCIe, which is a real physical device (i.e., hardware device). A PCIe system can include: a Transaction Layer, a Data Link Layer, and a Physical Layer. The Transaction Layer handles end-to-end data routing and is responsible for generating and processing Transaction Layer Packets (TLPs). The Data Link Layer ensures reliability through Ack (Acknowledge Character) or Nak (Negative Acknowledgement) mechanisms and is responsible for error detection and correction of TLPs. The Physical Layer uses differential signal transmission (such as 8B / 10B or 128B / 130B encoding) and is responsible for converting the data packets from the Data Link Layer into actual electrical signals for transmission over the link.

[0017] For example, related technologies run the RTL code of the PCIe controller inside the emulator to simulate the PCIe controller, which integrates the transaction layer and data link layer of the PCIe system. These technologies connect the emulator and the PCIe device via SpeedBridge to simulate the transaction layer of the PCIe system. The emulator interacts with the PCIe device through SpeedBridge, enabling interactive verification between the PCIe system and the PCIe device.

[0018] However, the relevant technologies have at least the following technical problems.

[0019] (1) The ability to inject errors is limited.

[0020] SpeedBridge and external hosts (i.e., PCIe devices) cannot accurately control protocol layer error scenarios in the PCIe system, such as injecting erroneous TLP packets or simulating AER (Advanced Error Reporting) mechanism triggering. This makes it difficult to perform critical protocol robustness tests of the PCIe system (such as ECRC checksum errors and link layer retry timeouts). For example, related technologies cannot simulate the response of PCIe devices to unexpected CA (Completer Abort, a type of error in the PCIe protocol) and UR (Unsupported Request, a type of error in the PCIe protocol) states, which affects the integrity of protocol consistency verification.

[0021] (2) Rate distortion and bandwidth compression.

[0022] The emulator operates in a low-frequency (≤5MHz) SpeedBridge environment, meaning SpeedBridge simulates PCIe system behavior at a clock frequency of ≤5MHz. This cannot simulate a real PCIe interface environment, leading to a mismatch between the actual speeds on both ends of the speed-down bridge, thus distorting PCIe system performance tests. Furthermore, this speed mismatch can increase the latency of MSI (Message Signaled Interrupt) triggers in the CPU (Central Processing Unit), resulting in misjudged interrupt loss. CPU DMA writes to the real system may also cause cache coherency delays due to the speed mismatch.

[0023] For the SpeedBridge environment, metrics such as big data read / write and DMA (Direct Memory Access) transfer latency cannot reflect the true high-speed scenarios of PCIe systems.

[0024] (3) The true performance boundary cannot be verified.

[0025] In the emulator, both the SpeedBridge environment and external PCIe devices must comply with the PCIe protocol and consider the Credit flow control mechanism. The Credit flow control mechanism of the PCIe system under test relies on precise Credit calculation and RTT (Round-Trip time) latency. The presence of SpeedBridge and its forced insertion of a fixed delay (typically >100ns) in the transmission path decouples the flow control Credit feedback of the PCIe device from the actual link. This results in the inability to update the RX Buffer (the receiver's buffer area) state in a timely manner when the sender's Credit is exhausted, triggering false Credit flow control blocking and significantly increasing the risk of insufficient Credit under high load scenarios. Furthermore, since the Credit configuration of external PCIe devices is fixed, it cannot meet the needs of different DUT iteration designs. The performance testing interaction of the PCIe system is easily limited by the external PCIe devices, making it impossible to verify the true performance boundaries of the PCIe system.

[0026] (4) Lack of dynamic delay control.

[0027] SpeedBridge's electrical signal conversion path is fixed, making it impossible to dynamically adjust the transmission delay between devices (which is usually fixed at several hundred ns). This makes it impossible for related technologies to simulate performance test environments under different inter-device delay states, thus limiting the acquisition of performance data for PCIe systems.

[0028] (5) Collaborative verification is inefficient and cannot be tested in parallel.

[0029] For external PCIe devices, the relevant technologies need to start a complex upper-layer PCIe driver protocol stack and rely on the operating system to complete the stimulus distribution. When parallel execution of protocol-level verification, it is easy to cause software crashes and system disconnection, which affects the problem location and collection. In addition, this also makes it impossible for the relevant technologies to perform performance verification and protocol-level verification in parallel, which is not conducive to improving verification efficiency.

[0030] To address the aforementioned issues, this application provides a hardware simulation-based performance testing system. This system runs on the aforementioned hardware emulator. Through this hardware simulation-based system (such as a protocol core module and a device-under-test simulation module), it simulates a real data bus system (such as a PCIe system) and bus devices (such as PCIe devices). This enables hardware-level interactive verification simulation between the data bus system and bus devices without requiring the connection of real physical devices such as dedicated hardware adapters or bus devices. This avoids performance testing distortion caused by introducing dedicated hardware adapters, thereby improving the accuracy of data bus system testing.

[0031] The hardware simulation-based performance testing system provided in the embodiments of this application will be described in detail below.

[0032] Please refer to Figure 1 This is a schematic diagram of a computer system provided in one possible implementation of this application. For example... Figure 1 As shown, the computer system includes a computer device 110 and a hardware emulator 120.

[0033] Computer device 110 can be an electronic device such as a mobile phone, tablet computer, vehicle terminal (vehicle system), or PC (Personal Computer). Hardware emulator 120 can be implemented as an emulator in emulation technology. Computer device 110 can be used to control hardware emulator 120, such as allowing users to use computer device 110 to call hardware emulator 120 to simulate and test the design under test.

[0034] In this embodiment, the hardware emulator 120 runs a performance testing system, which can be used for simulating and testing the data bus system. For example, the data bus system can refer to a system built based on a data bus protocol, such as a PCIe system built based on the PCIe protocol. This embodiment does not limit the data bus protocol corresponding to the data bus system. For instance, a user can use computer device 110 to call the performance testing system running on the hardware emulator 120 to test the PCIe system. The aforementioned data bus system is a real physical device.

[0035] Optionally, a client configured for the performance testing system is installed and runs on the computer device 110. The client provides a user interface through which the user can use the performance testing system running on the hardware emulator 120. For example, the client generates control commands based on user interaction information, such as Emulation commands for the Emulator or commands that the performance testing system can process. The client sends the control commands to the performance testing system, which then tests the data bus system according to the commands and obtains the test results. The performance testing system returns the test results to the client, which then displays the results to the user in the user interface.

[0036] A connection is established between the computer device 110 and the hardware emulator 120, which includes at least one of a wired connection and a wireless connection. For example, the computer device 110 and the hardware emulator 120 can be connected via a Universal Serial Bus (USB).

[0037] In this embodiment, the performance testing system can be a software system compiled using a hardware description language and a high-level scripting language for a data bus system, supporting the processing of hardware executable instructions. The hardware description language can include at least one of the following: Verilog, Python, or System Verilog. The high-level scripting language can include TCL (Tool Command Language).

[0038] The aforementioned performance testing system is implemented based on hardware simulation, such as using a hardware emulator. The hardware emulator 120 supports the operation of the performance testing system through its hardware components. For example, the hardware emulator 120 maps a corresponding logic circuit structure to the performance testing system. This logic circuit structure can be composed of hardware components planned for the performance testing system to support its hardware-level speed operation and functional implementation. For instance, the performance testing system can be implemented as a software model, such as RTL code, which is pre-compiled and run on the hardware emulator 120. The hardware emulator 120 provides hardware resources for the performance testing system, which users can then call to test the data bus system.

[0039] Optionally, the performance testing system can use software models to simulate the data bus system to simulate its behavior, or it can use software models to simulate the devices using the data bus system (i.e., bus devices) to simulate their behavior. This allows users to simulate the interaction between the data bus system and bus devices through the performance testing system, thereby enabling the testing of the data bus system. Optionally, the performance testing system can be used for full-stack performance testing of the data bus system. Full-stack performance testing refers to the process of verifying the complete functionality and performance of a data bus system (such as a PCIe system) from the physical layer to the application layer. It covers testing processes such as signal integrity, link stability, protocol robustness, and data transmission performance (such as correctness and efficiency). This application embodiment does not limit this aspect.

[0040] The system architecture of the performance testing system will be described below through an embodiment of the architecture of this application. For details not described in the embodiments of this application, please refer to the above embodiments, which will not be repeated here.

[0041] Please refer to Figure 2 This illustrates a schematic diagram of a hardware simulation-based performance testing system provided in one possible implementation of this application. Figure 2 As shown, the performance testing system 200 may include: a request generation module 202, a protocol core module 203, a data stream sending module 204, and a device under test simulation module 205.

[0042] The request generation module 202 and the protocol core module 203 have a data interaction relationship. For example, the request generation module 202 can provide test requests to the protocol core module 203, and the test requests are used to test the data bus system. The protocol core module 203 and the data stream sending module 204 also have a data interaction relationship. For example, the protocol core module 203 can send transmission data streams to the data stream sending module 204, and the transmission data streams are converted from test messages. The data stream sending module 204 and the device under test (DUT) simulation model 205 also have a data interaction relationship. For example, the data stream sending module 204 sends transmission data streams to the DUT simulation model 205. Therefore, the data interaction process between the request generation module 202 and the DUT simulation module 205 constitutes the interactive simulation process between the data bus system and the bus devices.

[0043] In one example, the performance testing system 200 described above may further include an adapter 201. The adapter 201 has a data interaction relationship with the request generation module 202, such as the adapter 201 being able to invoke the request generation module 202 via hardware-recognizable instructions. Hardware-recognizable instructions are instructions that can be processed by hardware. Optionally, the adapter 201 may also have data interaction relationships with the protocol core module 203, the data stream sending module 204, and the device under test (DUT) simulation module 205, respectively. For example, the adapter 201 may be able to collect event reports issued by the request generation module 202, the protocol core module 203, the data stream sending module 204, and the DUT simulation module 205, respectively, to understand and analyze the interactive simulation process between the data bus system and the bus devices.

[0044] The following section will provide a detailed explanation of the functions of each module in the performance testing system 200.

[0045] 1. Adapter 201, used for: receiving control commands triggered by the user; and converting the control commands into recognizable commands.

[0046] In this embodiment, adapter 201 is a software model used to convert control instructions into instructions that the request generation module 202 can recognize. The control instructions can be triggered by the user through a client, such as an Emulation instruction. This embodiment does not limit the triggering method of the control instructions; they can be entered by the user through a command input line provided by the client, such as entering at least one of the following instructions: a performance test instruction to trigger the start of performance testing, an error injection instruction to trigger the start of error injection, or a parallel test instruction to trigger parallel performance testing and error injection. Control instructions can also be generated by the user through controls provided by the client, such as controls to trigger the start of performance testing, controls to trigger the start of error injection, controls to trigger parallel performance testing and error injection, etc.

[0047] Users can trigger control commands according to actual testing needs. For example, by triggering the performance test command, the performance test system can be controlled to perform performance testing on the data bus system; by triggering the error injection command, the performance test system can be controlled to perform error injection at least at the protocol layer on the data bus system, where the transaction layer, data link layer, and transaction layer can be collectively referred to as the protocol layer; by triggering the parallel test command, the performance test system can be controlled to perform performance testing and error injection on the data bus system in parallel. This application embodiment does not limit the function of the control commands, which can be set and adjusted according to actual usage needs.

[0048] Optionally, after the user triggers the control command, the client sends the control command to the performance testing system 200, which is received and processed by the adapter 201 in the performance testing system 200.

[0049] In one example, the aforementioned control commands can be based on the Emulation environment, using TCL scripts and Emulation commands to simulate the Linux (an operating system) environment and obtain basic control commands. For example, in the command window of the Emulation environment, executing commands such as lspci can directly enumerate and scan the performance test system, such as the protocol core module 203 and the device under test (DUT) simulation module 205; through custom enumeration, BIOS loading, and other commands, the enumeration, Mem resource allocation, and interrupt allocation of the DUT simulation module 205 can be completed; through commands such as Reset and Rescan, the reset and rescanning of the DUT simulation module 205 can be completed. This application does not limit this aspect.

[0050] For example, the control command `lspci-vvvvs 00:0.0` is used to display detailed configuration space, functions, drivers, and other information of the specified PCIe device.

[0051] The control command `echo 1> / sys / bus / pci / devices / 0000:01:00.0 / reset` is used to send a reset signal to the specified PCIe device.

[0052] The aforementioned recognizable instruction can refer to an instruction that the request generation module 202 can recognize. This recognizable instruction can be recognized by hardware; for example, it can be implemented as a hardware signal-level control command or a hardware-recognizable instruction.

[0053] The adapter 201 controls the request generation module 202 to generate test requests by converting control commands into recognizable commands and sending the recognizable commands to the request generation module 202.

[0054] This application embodiment deploys an adapter in the performance testing system to achieve the connection between the client and the performance testing system, thereby enabling the performance testing system to be invoked. This allows users to control the performance testing system to test the data bus system according to actual testing needs, thereby improving the testing convenience, flexibility and customization of the data bus system.

[0055] 2. Request generation module 202 is used to generate test requests for the data bus system based on the loaded executable file. The test requests are used to test the data bus system.

[0056] In this embodiment, the protocol core module 203 is used to simulate a data bus system, and the request generation module 202 is used to control how the protocol core module 203 simulates the behavior of the data bus system according to the user's test requirements. The request generation module 202 can be a software model compiled using hardware description languages ​​and high-level scripting languages, which can process the aforementioned recognizable instructions to load executable files.

[0057] For example, the request generation module 202 is configured to load an executable file according to a recognizable instruction received from the adapter 201. The recognizable instruction may include identification information of the executable file, so that the request generation module 202 can accurately load the executable file. Optionally, the executable file may be stored in the hardware emulator 120 for loading by the request generation module 202, or the executable file may be stored in the computer device 110 for loading by the request generation module 202; this embodiment of the application does not limit this.

[0058] The aforementioned executable file can refer to a file that can be executed by the request generation module 202. The request generation module 202 obtains a test request by executing the executable file. The aforementioned test request is used to request the protocol core module 203 to generate a test message to test the data bus system. After generating the test request, the request generation module 202 sends the test request to the protocol core module 203.

[0059] In one example, the executable file described above is generated from user-edited data. This edited data can be data that the user edits on the client side. For instance, this edited data could be test cases, which are collections of test scenarios designed for a test data bus system. Test cases can include elements such as input data, execution steps, and expected results. For example, the user edits the test cases in an Excel file provided by the client according to predefined rules, and then Python converts this into an executable file.

[0060] Optionally, for performance testing of the data bus system, such as data transmission performance, the user can edit performance test cases. These performance test cases are used to test the performance of the data bus system. For example, the performance test case is used to instruct the performance testing system to perform at least one of the following performance test tasks (i.e., data processing tasks): reading a specific data size, writing a specific data size, mixed reading and writing of a specific data size, extreme scenarios of data reading or writing, and extreme scenarios of mixed data reading and writing. This application embodiment does not limit the performance test cases; they can be set and adjusted according to actual usage requirements.

[0061] For protocol testing (i.e., protocol layer testing) of data bus systems, such as testing the stability of the protocol layer and the robustness of the bus protocol, users can edit error test cases. These error test cases are test cases used to test the protocol of the data bus system. Optionally, error test cases can be used to trigger the injection of signal-level errors. For example, the error test case is used to instruct the performance testing system to perform at least one of the following error injection tasks: ECRC (Extended CRC) error, LCRC (Link CRC) error, DLLP Error (Data Link Layer Protocol Error, an error occurring during the execution of the data link layer protocol), and TLP Error (Transaction Layer Protocol Error, an error occurring during the execution of the transaction layer protocol). This application embodiment does not limit the error test cases; they can be set and adjusted according to actual usage requirements. The above error test cases conform to the bus protocol, such as errors corresponding to error test cases conforming to the PCIe protocol.

[0062] For configuring a data bus system or bus device, the user can edit configuration test cases. These test cases are used to instruct the target module on its configuration information. The target module may include at least one of the following: a request generation module 202, and a device under test (DUT) simulation module 205. The target module may also include a protocol core module 203 or a data stream sending module 204. For example, this configuration test case is used to instruct the performance testing system to perform at least one of the following configuration tasks: configuring the request generation module 202 according to configuration information, configuring the DUT simulation module 205 according to configuration information, configuring the protocol core module 203 according to configuration information, and configuring the data stream sending module 204 according to configuration information. This application embodiment does not limit the configuration test cases; they can be set and adjusted according to actual usage requirements.

[0063] The aforementioned test request is used to request the protocol core module 203 to generate test messages for testing the data bus system. Different types of test requests can be used to request the protocol core module 203 to generate different types of test messages for different tests of the data bus system. For example, the test request can be implemented as including a data read / write request, which is a test request for testing the performance of the data bus system, such as requesting the injection of unprocessed data generated according to performance test cases; the test request can be implemented as including an error injection request, which is a test request for testing the protocol of the data bus system, such as requesting the injection of errors generated according to error test cases, such as signal-level errors for the data bus system; the test request can also be implemented as including a configuration request, which is a test request for configuring the target module; the test request can also be implemented as a combination of any two of the data read / write request, error injection request, and configuration request, that is, including at least two of the data read / write request, error injection request, and configuration request. This application embodiment does not limit this.

[0064] In one example, the request generation module 202 may include at least one generation engine, with different functional types of generation engines used to generate different types of test requests. For example, the request generation module 202 may include at least one of the following generation engines: configuration engine 2021, performance testing engine 2022, and error injection engine 2023.

[0065] Specifically, Configuration Engine 2021 is the generation engine for generating configuration requests, which can be compiled from both Python and Verilog. Performance Testing Engine 2022 is the generation engine for generating data read / write requests, which can be compiled from Python, System Verilog, Verilog, and TCL. Error Injection Engine 2023 is the generation engine for generating error injection requests, which can be compiled from both Verilog and TCL and is controlled at the RTL level.

[0066] The aforementioned identifiable instructions may include at least one instruction, and different types of instructions can be used to invoke generation engines of different functional types. For example, a first type of instruction can be used to invoke the performance testing engine 2022, a second type of instruction can be used to invoke the error injection engine 2023, and a third type of instruction can be used to invoke the configuration engine 2021. That is, the adapter 201 can parse control instructions and convert them into at least one instruction. This at least one instruction may include at least one of the following: a first type of instruction, a second type of instruction, and a third type of instruction.

[0067] Optionally, adapter 201 is also used to send identifiable instructions to the target engine in request generation module 202 according to the type of identifiable instructions; wherein the function type of the target engine matches the type of the identifiable instructions, and the identifiable instructions are used to trigger the target engine to load the executable file corresponding to the function type of the target engine.

[0068] The type of identifiable instruction can refer to the type of instruction included in the identifiable instruction, and the target engine includes at least one of the configuration engine 2021, the performance testing engine 2022, and the error injection engine 2023.

[0069] Adapter 201 can send different types of instructions to the corresponding function type of the generation engine. Instructions for triggering the generation of data read / write requests are applied to the performance testing engine 2022, which generates data read / write requests; instructions for triggering the generation of error injection requests are applied to the error injection engine 2023, which generates error injection requests; and instructions for triggering the generation of configuration requests are applied to the configuration engine 2021, which generates configuration requests. For example, adapter 201 sends instructions of the first type to the performance testing engine 2022, instructions of the second type to the error injection engine 2023, and instructions of the third type to the configuration engine 2021.

[0070] For different functional types of generation engines, the types of files they load are different. The executable file mentioned above may include at least one sub-file, and different sub-files are converted from different test cases. Different test cases are used to test different characteristics of the data bus system. For example, the executable file mentioned above may include at least one of the following types of sub-files: a first sub-file converted from performance test cases, a second sub-file converted from error test cases, and a third sub-file converted from configuration test cases. The request generation module 202 mentioned above is used to execute at least one sub-file to generate a test request. For example, executing at least one sub-file will obtain sub-test requests corresponding to at least one sub-file, thereby forming the test request mentioned above.

[0071] For example, in the case where the request generation module 202 includes a performance testing engine 2022 and the executable file includes a first sub-file converted from performance test cases, the performance testing engine 2022 is configured to load the first sub-file and execute the first sub-file to generate a data read / write request upon receiving a first type of instruction from the adapter 201. The test request includes a data read / write request.

[0072] Optionally, the aforementioned first type of instruction may include at least one of the following information: identification information of at least one first sub-file, the first number of test requests generated, and the type of performance test scenario. For example, the performance test engine 2022 loads at least one first sub-file based on the identification information of at least one first sub-file, and generates data read / write requests according to the first number of data requests generated based on at least one first sub-file. Alternatively, the performance test engine 2022 may also randomly generate different types and quantities of data read / write requests.

[0073] Optionally, the first type of instruction can be parsed from the aforementioned performance test instruction or parallel test instruction.

[0074] Optionally, the performance testing engine 2022 may include: GBM (Global Bus Model), GSM (Global Slave Model), and GPM (Global Perf Model). GBM is used to send data read / write requests, GSM is used to load the first sub-file, and GPM is used to test the performance of the performance testing engine 2022.

[0075] This application embodiment deploys a performance testing engine in the performance testing system, enabling the system to perform performance testing on data bus devices. Furthermore, the performance testing engine allows for flexible configuration of performance testing scenarios for data bus devices, thus expanding the applicability of the performance testing system.

[0076] For example, when the request generation module 202 includes an error injection engine 2023 and the executable file includes a second sub-file converted from error test cases, the error injection engine 2023 is used to load the second sub-file and execute the second sub-file to generate an error injection request upon receiving a second type of instruction from the adapter 201. The aforementioned test request includes an error injection request.

[0077] The errors in the error injection request conform to the bus protocol, such as the PCIe protocol.

[0078] Optionally, the instructions of the second type described above may include at least one of the following information: identification information of at least one second sub-file, a second generation quantity of error injection requests, the type of error, and the number of errors. For example, error injection engine 2023 loads at least one second sub-file based on the identification information of at least one second sub-file, generates errors according to the second generation quantity based on at least one second sub-file, and generates error injection requests based on the generated errors. Alternatively, error injection engine 2023 may also randomly generate different numbers and types of errors to construct error injection requests.

[0079] Optionally, the second type of instruction can be parsed from the above-mentioned error injection instruction or parallel test instruction.

[0080] Optionally, the error injection engine 2023 further includes: a selection sub-module and a calculation sub-module. The selection sub-module is used to select the type of error, such as selecting the type of error according to the second sub-file. The calculation sub-module is used to count the number of errors. The calculation sub-module can be implemented as a counter.

[0081] By deploying an error injection engine in the performance test system in the embodiments of the present application, the performance test system has the precise error injection ability for data bus devices. And the flexible invocation of the error injection engine can be used to flexibly set the error test scenarios for data bus devices, which is beneficial to improving the applicable range of the performance test system.

[0082] In addition, since the error injection engine can support error injection for at least one protocol layer, the error coverage test of multi-protocol layer linkage can be realized, and the full-stack performance test of the protocol layer of the data bus system can be supported, so as to improve the protocol robustness test of the data bus system. And error injection with nanosecond-level precision can be completed (such as setting nanosecond-level error test cases), such as ECRC check errors, link layer retry timeouts, etc.

[0083] Exemplarily, when the request generation module 202 includes a configuration engine 2021 and the executable file includes a third sub-file obtained by converting a configuration test case, the configuration engine 2021 is used to load the third sub-file when receiving a third type of instruction from the adapter 201; execute the third sub-file to generate a configuration request. The above test request includes a configuration request.

[0084] Optionally, the above-mentioned third type of instruction may include at least one of the following information: identification information of at least one third sub-file, identification information of the target module. For example, the configuration engine 2021 loads at least one third sub-file according to the identification information of at least one third sub-file, and generates a configuration request for the target model according to at least one third sub-file. When the target module is the request generation module 202, the target module may be at least one of the following: the performance test engine 2022, the error injection engine 2023.

[0085] Optionally, the third type of instruction can be parsed from the above-mentioned performance test instruction, error injection instruction or parallel test instruction, or can be parsed from a separate configuration instruction. The embodiments of the present application do not limit this.

[0086] This application embodiment, by deploying a configuration engine in the performance testing system, enables the system to have precise configuration capabilities for data bus devices and bus devices. Furthermore, the configuration engine allows for flexible configuration of the testing environment for data bus devices (such as different types of data bus devices, bus devices, etc.), which improves the applicability of the performance testing system.

[0087] In one example, the request generation module 202 may include both a performance testing engine 2022 and an error injection engine 2023, which enables the performance testing engine 2022 to perform performance testing and error injection in parallel for the data bus system.

[0088] For example, adapter 201 is configured to: send instructions of the first type to performance test engine 2022 in request generation module 202 when the identifiable instructions include instructions of the first type, performance test engine 2022 loads a first sub-file corresponding to the function type of performance test engine 2022 based on instructions of the first type, and constructs a data processing task for the data bus system based on the first sub-file to generate a data read / write request.

[0089] If the identifiable instruction includes a second type of instruction, the second type of instruction is sent to the error injection engine 2023 in the request generation module 202. The error injection engine 2023 loads a second sub-file corresponding to the function type of the error injection engine 2023 according to the second type of instruction, and constructs an error for the data bus system according to the second sub-file to generate an error injection request.

[0090] After generating a data read / write request based on the first sub-file, the performance testing engine 2022 sends the data read / write request to the protocol core module 203. Similarly, after generating an error injection request based on the second sub-file, the error injection engine 2023 sends the error injection request to the protocol core module 203, enabling the performance testing system 200 to execute performance testing and error injection in parallel. In this scenario, the aforementioned test request may include both data read / write requests and error injection requests.

[0091] The test request sent by the request generation module 202 to the protocol core module 203 may include sub-test requests generated by each generation engine in the request generation module 202. The executable file loaded by the request generation module 202 includes the sub-files loaded by each generation engine in the request generation module 202.

[0092] For example, if the executable file includes a first sub-file converted from performance test cases and a second sub-file converted from error test cases, the above test requests include data read / write requests and error injection requests.

[0093] This application embodiment supports the parallel implementation of performance testing and error injection through the request generation module 202, which solves the problem that a single simulation can only execute a single verification scenario, which is beneficial to improving the verification efficiency of the data bus system.

[0094] In addition, embodiments of this application support the parallel generation of data read requests and error injection requests triggered by the same instruction (such as a parallel test instruction). This is beneficial to improving the convenience of triggering parallel execution of performance verification and protocol verification, and can greatly improve the verification efficiency of the data bus system, thereby accelerating verification convergence.

[0095] Optionally, adapter 201 is further configured to: if the identifiable instructions include instructions of a third type, send the third type of instructions to the configuration engine 2021 in the request generation module 202; the configuration engine 2021 loads a third sub-file corresponding to the function type of the configuration engine 2021 based on the third type of instructions, and constructs configuration parameters for the data bus system or the device under test (DUT) simulation module based on the third sub-file to generate a configuration request. The configuration parameters indicate the method of configuring the data bus system or the DUT simulation module.

[0096] 3. Protocol core module 203 is used to convert test requests into test messages and test messages into transmission data streams.

[0097] In this embodiment, the protocol core module 203 is a software model for simulating a data bus system, which can be synthesized into a real data bus system. Optionally, the protocol core module 203 is a software model for simulating the controller of the data bus system, to simulate the behavior of the controller of the data bus system. The protocol core module 203 can be compiled from Verilog, and it has been thoroughly verified by silicon and has high reliability.

[0098] For example, the protocol core module 203 is a module compiled from the circuit design code of the data bus system, which characterizes the hardware architecture and bus protocol of the data bus system. For example, for a PCIe system, the circuit design code can be implemented as RTL code. The hardware architecture of the data bus system indicates the protocol layers of the data bus system, such as the transaction layer and the data link layer.

[0099] Since the protocol core module 203 is compiled from the circuit design code of the data bus system, it has high-fidelity protocol interaction capability and physical layer LTSSM (Link Training and Status State Machine) negotiation capability.

[0100] The above test message is used to test the data bus system. The test message conforms to the bus protocol corresponding to the data bus system, such as the PCIe protocol. For example, the protocol core module 203 collects sub-test requests generated by each generation engine and generates at least one of the following messages as needed: TLP message, DLLP message, and Message message, as test messages. For example, for a PCIe system, the test message is a PCIe message, which can be implemented as at least one of the following: PCIe TLP message, PCIe DLLP message, and PCIeMessage message.

[0101] Optionally, the protocol core module 203 can identify test requests. It can generate data based on data read / write requests and perform protocol-level error injection based on error injection requests, thereby obtaining test messages. When a test request includes both data read / write requests and error injection requests, the protocol core module 203 generates test messages that are used for both performance testing and protocol testing. When a test request includes only data read / write requests or error injection requests, the protocol core module 203 generates test messages for either performance testing or protocol testing.

[0102] After generating the test message, the core protocol module 203 converts the test message into a serial data stream, i.e., the aforementioned transmission data stream, through parallel-to-serial conversion. The core protocol module 203 then sends the transmission data stream to the data stream sending module 204.

[0103] In one example, the performance testing system 200 described above also includes a wrapper used to simulate the interface standards of a data bus system, such as a software module that simulates the connection protocol of the data bus system. For example, for a PCIe system, the wrapper is implemented as a PIPE (Physical Interface for PCI Express) wrapper for PCIe systems, used to simulate the physical layer interface specification. The protocol core module 203 can generate the aforementioned transmission data stream through the wrapper.

[0104] In this embodiment, the encapsulator and data stream sending module 204 can be used to simulate the physical layer of a data bus system.

[0105] 4. Data stream sending module 204 is used to send the transmitted data stream to the device under test simulation module 205. The device under test simulation module 205 is used to simulate devices using a data bus system.

[0106] In this embodiment, the data stream transmission module 204 can be implemented as a software model for simulating data transmission delay caused by the physical layer and physical traces in an environment where two bus devices are connected. The data stream transmission module 204 can be compiled from SystemVerilog, Verilog, and TCL, with TCL dynamically controlling the transmission delay duration.

[0107] The device under test simulation module 205 can be implemented as a software model for simulating real bus devices, and the device under test simulation module 205 conforms to the bus protocol of the data bus system.

[0108] For example, the device under test (DUT) simulation module 205 is a module compiled from the circuit design code of a bus device, which is used to characterize the circuit design of the bus device. For instance, the DUT simulation module 205 is a module compiled from the circuit design code of a PCIe device, which is used to simulate the behavior of the PCIe device.

[0109] In one example, the data stream sending module 204 is further configured to: acquire a delay signal of the transmitted data stream, the delay signal being used to indicate the transmission delay duration of the transmitted data stream; and after receiving the transmitted data stream, send the transmitted data stream to the simulation module 205 of the device under test after the transmission delay duration.

[0110] The transmission delay duration is used to simulate the data transmission delay caused by the physical layer and physical traces in an environment where two bus devices are connected. The aforementioned delay signal can be generated based on user-defined delay settings, which are used to determine the transmission delay duration. For example, the user can edit the delay settings into a control command, which is then parsed by the adapter 201 to obtain the delay signal. The adapter 201 then sends the measured delay signal to the data stream transmission module 204, which extracts the transmission delay duration from the delay signal. This application embodiment does not limit the method for setting the transmission delay duration.

[0111] This application's embodiments support simulating data transmission latency caused by the physical layer and physical wiring in a scenario where two bus devices are interconnected via a data stream sending module. This enables dynamic control of the actual link interconnection latency, avoiding the drawbacks of fixed latency in real physical devices that are affected by hardware operating speed, thereby improving the accuracy of performance testing for the data bus system. Furthermore, it can simulate data transmission latency under different operating conditions and with different real physical devices, thus expanding the applicability of the performance testing system.

[0112] Optionally, the device under test (DUT) simulation module 205 also includes a data bus interface, through which the DUT simulation module 205 receives the transmission data stream sent by the data stream sending module 204. The data bus interface conforms to the bus protocol of the data bus system. For example, the data bus interface is a software model used to simulate a PCIe interface.

[0113] In summary, the technical solution provided in this application uses a hardware simulation-based performance testing system to simulate a real data bus system and devices using the data bus system (hereinafter referred to as bus devices). This enables interactive verification simulation between the data bus system and the devices using the data bus system without the need to connect real physical devices such as dedicated hardware adapters or bus devices. This avoids performance testing distortion caused by the introduction of dedicated hardware adapters and thus helps improve the testing accuracy of the data bus system.

[0114] Furthermore, since this embodiment does not introduce actual physical devices, it avoids the problem of system crashes caused by software hangs that cannot be located, thereby ensuring the integrity of protocol consistency verification for the data bus system. It also avoids the fixed path latency introduced by actual physical devices (such as SpeedBridge and bus devices) at the architectural level, thus avoiding performance distortion issues and limitations on rate and bandwidth compression and stress testing in performance verification scenarios. Additionally, it ensures end-to-end credit interaction between the bus device and the data bus system at the architectural level, thereby avoiding issues such as false credit flow control blocking and limitations on extreme performance testing (i.e., testing the true performance boundaries of the data bus system).

[0115] In this embodiment of the application, the device under test simulation module 205 is further configured to: receive the transmitted data stream; perform error checking operations on the transmitted data stream, and obtain error checking results.

[0116] Optionally, if an error is injected into the test message, the device under test (DUT) simulation module 205 performs error checking on the transmitted data stream according to the bus protocol to obtain error checking results. For example, the DUT simulation module 205 can locate the problem through error reporting, error forwarding, and error messages based on the detection mechanism in the PCIe protocol to obtain error checking results, such as the type and number of errors, and the protocol layers involved. The error checking results can reflect the data bus system's ability to detect, report, and handle errors. Based on the error checking results, the reliability and stability of the data bus system, as well as the robustness of the protocol or protocol layer, can be determined.

[0117] The embodiments of this application support the acquisition of error checking results, which is beneficial for realizing protocol robustness verification for data bus systems.

[0118] In one example, during the global operation of the performance processing system 200, various modules within the performance processing system 200 may generate event reports, such as signal-level event reports. Adapter 201 performs the collection and aggregation of these event reports.

[0119] For example, the event report generated by the configuration engine 2021 includes at least one of the following: configuration address, configuration type, and configuration action. The event report generated by the performance test engine 2022 includes at least one of the following: the number of data read / write requests, the type of data read / write requests, and the performance status of the performance test engine 2022 (e.g., bandwidth). The event report generated by the error injection engine 2023 includes at least one of the following: the number of errors and the error type. The event reports generated by the device under test simulation module 205 and the protocol core module 203 include at least one of the following: the number of test requests and the type of test requests. The event report generated by the data stream sending module 204 may include the transmission delay duration of the transmitted data stream.

[0120] Optionally, adapter 201 can collect and summarize event reports generated by each module in real time, or it can collect and summarize event reports generated by each module upon receiving a report collection instruction. The report collection instruction is used to instruct the collection of event reports from the performance verification system 200. This application embodiment does not limit this.

[0121] For example, adapter 201 is further configured to: acquire an event report set, the event report set including at least one of the following: event reports from request generation module 202, event reports from protocol core module 203, event reports from data stream sending module 204, and event reports from device under test simulation module 205; and generate test results based on the event report set, the test results including at least one of the following: performance test results of the data bus system, protocol test results of the data bus system, and error checking results of the data bus system.

[0122] The event reports from the self-request generation module 202 are event reports from each generation engine. Event reports from the device under test (DUT) simulation module 205 may also include error checking results. Performance test results indicate the performance of the data bus system, such as the stability, accuracy, efficiency, and reliability of data transmission. Protocol test results indicate the protocol test results of the data bus system, such as the robustness, reliability, and stability of the protocol.

[0123] The adapter 201 generates test results by analyzing various event reports in the event report set. For example, based on event reports from the data stream sending module 204 and the event reports from the device under test (DUT) simulation module 205, it determines the performance test results and protocol test results of the data bus system. The error checking results of the data bus system are determined based on the event reports from the DUT simulation module 205; however, this embodiment of the application does not limit this process.

[0124] Optionally, after generating the test results, the adapter 201 returns the test results to the client, and the client displays the test results to the user through the user interface.

[0125] This application embodiment uses the event reports of each module in the performance testing system to generate test results by using an adapter to statistically analyze the data. This gives the performance testing system a deep observability capability, which is beneficial to improving the user experience.

[0126] In some embodiments, the data bus system described above is a PCIe system, and the device under test simulation module 205 is used to simulate a PCIe device.

[0127] In other words, the performance testing system 200 described above is used to simulate PCIe systems and PCIe devices to perform tests on PCIe systems. For example, the performance testing system 200 runs on a hardware emulator 120, which may be pre-compiled into the hardware emulator 120. The protocol core module 203 in the performance testing system 200 is used to simulate the controller of the PCIe system, the data stream sending module 204 in the performance testing system 200 is used to simulate the physical layer of the PCIe system, and the device-under-test (DUT) simulation module 205 in the performance testing system 200 is used to simulate PCIe devices.

[0128] For example, refer to Figure 3 The testing process for a PCIe system may include the following:

[0129] 1. Adapter 201, used to acquire control commands.

[0130] Optionally, users can input control commands through the command window provided by the Emulation environment. For example, users can input performance test commands, which may include the identification information of a first sub-file, which is converted from performance test cases. As another example, users can add a suffix to the performance test command to adjust it into a parallel test command. This suffix format is Einj_Type Einj_Count, which contains the type and number of errors to be injected, as well as the identification information of a second sub-file, which is converted from error test cases. In the case of random error injection, the suffix may only include the error type and the number of errors; in the case of specified error injection, the suffix may include the identification information of the second sub-file. This embodiment of the application does not limit this.

[0131] In response to a user-triggered control command, the client sends the control command to adapter 201, which receives the control command. Optionally, the control command may also include user-configured delay settings.

[0132] Optionally, the client may also provide a user interface, including various options such as those related to performance testing and error injection. The user interface also includes a trigger control. After the user selects an option, in response to the user's triggering action on the trigger control, the client generates a control command and sends the control command to the client. For example, if the performance testing-related option is selected, the client generates a performance testing command; if the user selects both the performance testing-related option and the error injection-related option, the client generates a parallel testing command. This application does not limit the method for generating control commands.

[0133] 2. Adapter 201 is used to convert control commands into hardware-recognizable commands.

[0134] Adapter 201 translates control instructions into hardware signal-level control commands to invoke the target engine in request generation module 202.

[0135] Optionally, adapter 201 is used to convert the control instruction into a first type of instruction when the control instruction is detected to be a performance test instruction; and to convert the control instruction into a first type of instruction and a second type of instruction when the control instruction is detected to be a parallel test instruction (i.e., parallel error injection).

[0136] 2.1 Adapter 201, used to send the first type of instructions to the performance test engine 2022.

[0137] If parallel error injection is determined, step 2 also includes step 2.2.

[0138] 2.2 Adapter 201, used to send second type of instructions to error injection engine 2023.

[0139] 3. Request generation module 202 is used to generate test requests based on hardware-recognizable instructions.

[0140] 3.1 Performance testing engine 2022 is used to load the first sub-file and execute the first sub-file according to the first type of instructions, and generate data read and write requests.

[0141] Optionally, user-edited performance test cases can form a testlist. Python tools can convert these test cases into a first sub-file, creating a file list, such as a performance test executable bin file. The Performance Test Engine 2022 is used to load the corresponding first sub-file from the file list based on the first type of instructions. The second and third sub-files can be generated separately by Python tools based on error test cases and configuration test cases, respectively.

[0142] The performance testing engine 2022 is also used to send data read and write requests to the protocol core module 203.

[0143] Optionally, if parallel error injection is determined, step 2 may also include step 3.2.

[0144] 3.2 Error Injection Engine 2023 is used to load and execute the second sub-file based on the second type of instructions, and generate error injection requests.

[0145] The selection and calculation submodules in Error Injection Engine 2023 generate error injection requests of the corresponding type and number based on the second subfile. Optionally, Error Injection Engine 2023 can also randomly generate error injection requests of the corresponding type and number.

[0146] Error injection engine 2023 is also used to send error injection requests to protocol core module 203.

[0147] 4. Protocol core module 203, used to generate test messages.

[0148] The protocol core module 203 is used to convert received test requests into PCIe packets. For example, if only a data read / write request is received, it is converted into a PCIe packet; if both a data read / write request and an error injection request are received simultaneously, both are converted into PCIe packets.

[0149] The protocol core module 203 is also used to convert PCIe messages into serial data streams, i.e., the aforementioned transmission data streams, through parallel-to-serial conversion, and to send the transmission data streams to the data stream sending module 204.

[0150] 5. Data stream sending module 204, used to send the transmission data stream according to the transmission delay duration.

[0151] The data stream sending module 204 is used to send the transmitted data stream to the simulation module 205 of the device under test after receiving the transmitted data stream and after a transmission delay period.

[0152] Optionally, the data stream transmission module 204 receives a delay signal from the adapter 201 and determines the transmission delay duration of the data stream based on the delay signal. The adapter 201 generates the delay signal based on the delay setting information.

[0153] 6. The device under test simulation module 205 is used to receive the transmitted data stream.

[0154] The device under test simulation module 205 is also used to perform error checking operations on the transmitted data stream and obtain error checking results when parallel error injection is determined.

[0155] 7. The client displays the test results.

[0156] During the global operation of the performance testing system, the performance testing engine 2022, the error injection engine 2023, the protocol core module 203, the data stream sending module 204, and the device under test simulation module 205 will all generate signal-level event reports.

[0157] Adapter 201 is also used to collect and summarize event reports generated by various modules to obtain an event report set; generate test results based on the event report set, including: performance test results of the PCIe system, protocol test results of the PCIe system, and error checking results of the PCIe system; and return the test results to the client. The client displays the test results through a user interface.

[0158] In summary, the technical solution provided in this application uses a hardware simulation-based performance testing system to simulate a real PCIe system and PCIe devices, thereby achieving interactive verification simulation between the PCIe system and PCIe devices without needing to connect real physical devices such as dedicated hardware adapters or PCIe devices. This avoids performance test distortion caused by the introduction of dedicated hardware adapters, and thus helps to improve the testing accuracy of the PCIe system.

[0159] Furthermore, since this application's embodiments do not introduce actual physical devices, they can avoid the problem of system crashes caused by software hangs that cannot be located, thereby ensuring the integrity of PCIe protocol conformance verification. They also avoid the fixed path latency introduced by actual physical devices (such as SpeedBridge and PCIe devices) at the architectural level, thus avoiding performance distortion issues and limitations on rate and bandwidth compression and stress testing in performance verification scenarios. Additionally, they can ensure end-to-end credit interaction between PCIe devices and the PCIe system at the architectural level, thereby avoiding issues such as false credit flow control blocking and limitations on extreme performance testing (i.e., testing the true performance boundaries of the data bus system).

[0160] The following are embodiments of the method of this application. For details not disclosed in the embodiments of the method of this application, please refer to the above embodiments, which will not be repeated here.

[0161] Please refer to Figure 4 The diagram illustrates a flowchart of a performance testing method provided in one possible implementation of this application. This method is implemented by a hardware simulation-based performance testing system. Figure 4 As shown, the method may include the following steps: Step 401: Generate a test request for the data bus system based on the loaded executable file. The test request is used to test the data bus system.

[0162] The performance testing system described above includes: a request generation module, a protocol core module, a data stream sending module, and a device-under-test (DUT) simulation module. The performance testing system may also include an adapter. Step 401 can be implemented by calling the request generation module.

[0163] In one example, a control command triggered by the user is received; the control command is converted into a recognizable command; and the recognizable command is sent to the target engine in the request generation module according to the type of the recognizable command. The function type of the target engine matches the type of the recognizable command, and the recognizable command is used to trigger the target engine to load an executable file corresponding to the function type of the target engine. The executable file is generated from the user's edit data.

[0164] In one example, if the identifiable instructions include instructions of the first type, the instructions of the first type are sent to the performance testing engine in the request generation module. The performance testing engine loads a first sub-file corresponding to the function type of the performance testing engine based on the instructions of the first type, and constructs a data processing task for the data bus system based on the first sub-file to generate a data read / write request. If the identifiable instructions include instructions of the second type, the instructions of the second type are sent to the error injection engine in the request generation module. The error injection engine loads a second sub-file corresponding to the function type of the error injection engine based on the instructions of the second type, and constructs an error for the data bus system based on the second sub-file to generate an error injection request.

[0165] Step 402: The core module of the protocol converts the test request into a test message, and then converts the test message into a transmission data stream. The test message conforms to the bus protocol corresponding to the data bus system.

[0166] Step 402 can be achieved by calling the protocol core module.

[0167] In one example, the executable file includes at least one sub-file, and different sub-files are derived from different test cases. The different test cases are used to test different characteristics of the data bus system; the request generation module executes at least one sub-file to generate test requests.

[0168] In one example, the request generation module includes a performance testing engine, and the executable file includes a first sub-file converted from performance test cases, which are test cases used to test the performance of the data bus system. The performance testing engine executes the first sub-file, generating data read / write requests. These data read / write requests are test requests used to test the performance of the data bus system.

[0169] In one example, the request generation module includes an error injection engine, and the executable file includes a second sub-file converted from error test cases, which are test cases used to test the protocol of the data bus system. The error injection engine executes the second sub-file to generate an error injection request. The error injection request is a test request used to test the protocol of the data bus system, and the errors in the error injection request conform to the bus protocol.

[0170] In one example, the request generation module includes a configuration engine, the executable file includes a third sub-file converted from configuration test cases, the configuration test cases are test cases used to indicate the configuration information of the target module, and the target module includes at least one of the following: the request generation module, the device under test simulation module; The configuration engine executes the third sub-file to generate a configuration request, which is a test request used to configure the target module.

[0171] In one example, where the executable file includes a first sub-file derived from performance test cases and a second sub-file derived from error test cases, the test requests include data read / write requests and error injection requests. The data read / write requests are test requests used to test the performance of the data bus system, and the errors in the error injection requests conform to the bus protocol.

[0172] In one example, the protocol core module is a module compiled from the circuit design code of the data bus system, which is used to characterize the hardware structure and bus protocol of the data bus system.

[0173] Step 403: Send the transmitted data stream to the device under test simulation module. The device under test simulation module is used to simulate devices using a data bus system.

[0174] Step 403 can be achieved by calling the data stream sending module.

[0175] In one example, the data stream sending module acquires the delay signal of the transmitted data stream, which indicates the transmission delay duration of the transmitted data stream; after receiving the transmitted data stream, it sends the transmitted data stream to the simulation module of the device under test after the transmission delay duration.

[0176] In one example, the device under test (DUT) simulation module receives the transmitted data stream; performs error checking on the transmitted data stream, and obtains the error checking results.

[0177] In one example, the adapter acquires an event report set, which includes at least one of the following: event reports from the request generation module, event reports from the protocol core module, event reports from the data stream sending module, and event reports from the device under test simulation module; based on the event report set, test results are generated, which include at least one of the following: performance test results of the data bus system, protocol test results of the data bus system, and error checking results of the data bus system.

[0178] In one example, the data bus system is a PCIe system, and the device under test (DUT) simulation module is used to simulate PCIe devices.

[0179] In summary, the technical solution provided in this application uses a hardware simulation-based performance testing system to simulate a real data bus system and devices using the data bus system (hereinafter referred to as bus devices). This enables interactive verification simulation between the data bus system and the devices using the data bus system without the need to connect real physical devices such as dedicated hardware adapters or bus devices. This avoids performance testing distortion caused by the introduction of dedicated hardware adapters and thus helps improve the testing accuracy of the data bus system.

[0180] Furthermore, since this embodiment does not introduce actual physical devices, it avoids the problem of system crashes caused by software hangs that cannot be located, thereby ensuring the integrity of protocol consistency verification for the data bus system. It also avoids the fixed path latency introduced by actual physical devices (such as SpeedBridge and bus devices) at the architectural level, thus avoiding performance distortion issues and limitations on rate and bandwidth compression and stress testing in performance verification scenarios. Additionally, it ensures end-to-end credit interaction between the bus device and the data bus system at the architectural level, thereby avoiding issues such as false credit flow control blocking and limitations on extreme performance testing (i.e., testing the true performance boundaries of the data bus system).

[0181] Please refer to Figure 5 This diagram illustrates a simplified structural block diagram of a computer device provided in one embodiment of this application. The computer device 500 can be any electronic device capable of data calculation, processing, and storage.

[0182] Computer device 500 includes: processor 501 and memory 502.

[0183] Processor 501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 501 may also include an AI processor for handling computational operations related to machine learning.

[0184] Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 502 is used to store a computer program configured to be executed by one or more processors.

[0185] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the computer device 500, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0186] In some embodiments, a hardware emulator is also provided, on which the hardware emulation-based performance testing system described above is run.

[0187] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0188] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A performance testing system based on hardware simulation, characterized in that, The system includes: a request generation module, a protocol core module, a data stream sending module, and a device under test simulation module; The request generation module is used to generate a test request for the data bus system based on the loaded executable file, and the test request is used to test the data bus system. The core module of the protocol is used to convert the test request into a test message, and the test message into a transmission data stream. The test message conforms to the bus protocol corresponding to the data bus system. The data stream sending module is used to send the transmitted data stream to the device under test simulation module, and the device under test simulation module is used to simulate a device using the data bus system.

2. The system according to claim 1, characterized in that, The executable file includes at least one sub-file, and different sub-files are derived from different test cases. Different test cases are used to test different characteristics of the data bus system. The request generation module is used to execute the at least one sub-file to generate the test request.

3. The system according to claim 2, characterized in that, The request generation module includes a performance testing engine, and the executable file includes a first sub-file converted from performance test cases, wherein the performance test cases are test cases used to test the performance of the data bus system. The performance testing engine is used to execute the first sub-file and generate data read / write requests, which are test requests used to test the performance of the data bus system.

4. The system according to claim 2, characterized in that, The request generation module includes an error injection engine, and the executable file includes a second sub-file converted from error test cases, wherein the error test cases are test cases used to test the protocol of the data bus system; The error injection engine is used to execute the second sub-file and generate an error injection request. The error injection request is a test request for testing the protocol of the data bus system, and the error in the error injection request conforms to the bus protocol.

5. The system according to claim 2, characterized in that, The request generation module includes a configuration engine, and the executable file includes a third sub-file converted from configuration test cases. The configuration test cases are test cases used to indicate the configuration information of the target module. The target module includes at least one of the following: the request generation module and the device under test simulation module. The configuration engine is used to execute the third sub-file and generate a configuration request, which is a test request for configuring the target module.

6. The system according to any one of claims 1 to 5, characterized in that, In the case where the executable file includes a first sub-file converted from performance test cases and a second sub-file converted from error test cases, the test request includes a data read / write request and an error injection request. The data read / write request is a test request used to test the performance of the data bus system, and the error in the error injection request conforms to the bus protocol.

7. The system according to any one of claims 1 to 5, characterized in that, The system further includes an adapter; the adapter is used for: Receive control commands triggered by the user; Convert the control commands into recognizable commands; Based on the type of the identifiable instruction, the identifiable instruction is sent to the target engine in the request generation module; The target engine's function type matches the type of the identifiable instruction. The identifiable instruction is used to trigger the target engine to load an executable file corresponding to the target engine's function type. The executable file is generated from the user's edit data.

8. The system according to claim 7, characterized in that, The adapter is used for: When the identifiable instruction includes an instruction of the first type, the instruction of the first type is sent to the performance testing engine in the request generation module. The performance testing engine loads a first sub-file corresponding to the function type of the performance testing engine based on the instruction of the first type, and constructs a data processing task for the data bus system based on the first sub-file to generate a data read / write request. If the identifiable instruction includes a second type of instruction, the second type of instruction is sent to the error injection engine in the request generation module. The error injection engine loads a second sub-file corresponding to the function type of the error injection engine based on the second type of instruction, and constructs an error for the data bus system based on the second sub-file to generate an error injection request.

9. The system according to claim 1, characterized in that, The data stream sending module is used for: Acquire the delay signal of the transmitted data stream, the delay signal being used to indicate the transmission delay duration of the transmitted data stream; After receiving the transmitted data stream, the transmitted data stream is sent to the simulation module of the device under test after the transmission delay period.

10. The system according to any one of claims 1 to 5, characterized in that, The simulation module for the device under test is also used for: Receive the transmitted data stream; An error checking operation is performed on the transmitted data stream to obtain the error checking results.

11. The system according to any one of claims 1 to 5, characterized in that, The system further includes an adapter; the adapter is used for: Obtain an event report set, which includes at least one of the following: an event report from the request generation module, an event report from the protocol core module, an event report from the data stream sending module, and an event report from the device under test simulation module; Based on the event report set, test results are generated, and the test results include at least one of the following: performance test results of the data bus system, protocol test results of the data bus system, and error checking results of the data bus system.

12. The system according to any one of claims 1 to 5, characterized in that, The core module of the protocol is a module compiled from the circuit design code of the data bus system. The circuit design code is used to characterize the hardware structure and bus protocol of the data bus system.

13. The system according to any one of claims 1 to 5, characterized in that, The data bus system is a PCIe system, and the device under test simulation module is used to simulate PCIe devices.

14. A hardware emulator, characterized in that, The hardware emulator runs a hardware simulation-based performance testing system as described in any one of claims 1 to 13.

15. A performance testing method, characterized in that, The method is implemented by a hardware simulation-based performance testing system; the method includes: Based on the loaded executable file, a test request for the data bus system is generated, the test request being used to test the data bus system; The test request is converted into a test message, and the test message is converted into a transmission data stream. The test message conforms to the bus protocol corresponding to the data bus system. The transmitted data stream is sent to the device under test (DUT) simulation module, which is used to simulate devices using the data bus system.

Images