Test verification method and system for storage controller

Abstract

The invention provides a test verification method and system of a storage controller. The method comprises the following steps executed by a first simulation program: recording a plurality of commands under a test case as a plurality of records, and recording the earliest command for two or more commands with the same command identifier; serially sending a plurality of commands to a storage controller; the result notifications fed back by the storage controller are serially received and checked one by one, the storage controller checks whether command identification conflicts exist or not for each command, and if the command identification conflicts do not exist, the commands are executed, and the result notifications which do not contain the command identification conflicts are fed back; deleting a corresponding record from a plurality of records for the result notice which does not contain the command identifier conflict; and when the number of records under the test case is zero, determining that the test case is completed. And the record number of zero is adopted to replace traversal to serve as a mark for ending the test case, so that the simulation verification time is shortened.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a test verification method and system of a storage controller. BACKGROUND

[0002] NVMe (Non-Volatile Memory Express) is a connection and access protocol of a non-volatile storage medium. The NVMe protocol is directly connected to a processor of a host through a PCI Express (PCIe) bus, thereby greatly reducing the delay of data transmission.

[0003] The interaction process between a host and a storage controller based on the NVMe protocol is roughly as follows: the host issues a command to the storage controller, the storage controller executes the command, and after the execution of the command is completed, the host is informed that the execution of the command is completed. In simulation test verification, a test bench (TB) simulates the host to issue a command and store the command information, and after receiving the feedback result of the storage controller, the recorded command information is checked and compared to determine the execution of the command. However, a test case often includes multiple commands, so the execution of all commands needs to be confirmed before the test case can be ended. In order to determine whether the test case is ended, the test bench needs to constantly traverse the command information of all commands to determine the number of completed and uncompleted commands. If the number of commands in the test case is large, the repeated traversal work will consume a large amount of simulation resources, thereby slowing down the entire simulation process and reducing the processing efficiency. SUMMARY

[0004] Therefore, the present application provides a test verification method and system of a storage controller to solve the problems in the prior art.

[0005] In a first aspect, the present application provides a test verification method of a storage controller, the storage controller adopts an NVMe protocol, and the test verification method is executed by a first simulation program and includes the following steps:

[0006] Record multiple commands under a test case as multiple records, wherein each command has a command identifier, and for two or more commands with the same command identifier, record the earliest command;

[0007] Serially send the multiple commands to the storage controller;

[0008] serially receiving and checking the result notifications of the multiple commands fed back by the storage controller, the storage controller checking, for each command, whether there is a command identifier conflict, if there is no command identifier conflict, executing the command and feeding back a result notification not containing the command identifier conflict, and if there is a command identifier conflict, not executing the command and feeding back a result notification containing the command identifier conflict;

[0009] deleting, for the result notification not containing the command identifier conflict, a corresponding record from the multiple records;

[0010] when the number of records under the test case is zero, determining that the test case is completed.

[0011] In some embodiments, a second simulation program is used to replace the storage controller to perform corresponding operations.

[0012] In some embodiments, the first simulation program performs the following steps:

[0013] counting the number of commands with the same command identifier to obtain a first count value;

[0014] counting the result notifications containing the command identifier conflict to obtain a second count value;

[0015] determining whether the second simulation program identifies all command identifier conflicts in the test case according to a comparison result of the first count value and the second count value.

[0016] In some embodiments, the second simulation program further comprises: reporting a command requiring firmware modification to the first simulation program and receiving a modified command from the first simulation program after checking that there is no command identifier conflict and before executing the command, so that the second simulation program executes the modified command.

[0017] In some embodiments, the first simulation program stores the multiple records in a memory or a database table.

[0018] In some embodiments, the first simulation program further comprises: checking data in a target address according to the result notification not containing the command identifier conflict to verify whether there is a design loophole in the storage controller or the second simulation program.

[0019] In a second aspect, the embodiments of the present disclosure provide a test verification system for a storage controller adopting an NVMe protocol, the test verification system comprising a first simulation program simulating host behavior and a second simulation program simulating the storage controller, the first simulation program performing the following operations:

[0020] The multiple commands under the test case are recorded as multiple records, wherein each command has a command identifier, but two or more commands with the same command identifier are recorded as the earliest command;

[0021] The multiple commands are serially sent to the storage controller;

[0022] The multiple result notifications of the multiple commands fed back by the storage controller are serially received and checked one by one;

[0023] For the result notification not containing the command identifier conflict, a corresponding record is deleted from the multiple records;

[0024] When the number of records under the test case is zero, it is determined that the test case is completed;

[0025] The second simulation program performs the following operations:

[0026] For each command, it is checked whether there is a command identifier conflict, if not, the command is executed and the result notification not containing the command identifier conflict is fed back, if yes, the command is not executed and the result notification containing the command identifier conflict is fed back.

[0027] In some embodiments, the first simulation program further performs the following operations:

[0028] The number of commands with the same command identifier is counted to obtain a first count value;

[0029] The result notification containing the command identifier conflict is counted to obtain a second count value;

[0030] According to the comparison result of the first count value and the second count value, it is determined whether the second simulation program identifies all command identifier conflicts in the test case.

[0031] In some embodiments, the second simulation program reports the command needing firmware modification to the first simulation program after checking that there is no command identifier conflict and before executing the command, and receives the modified command from the first simulation program, and the second simulation program executes the modified command.

[0032] In some embodiments, the first simulation program further comprises: for the result notification not containing the command identifier conflict, checking the data in the target address to determine whether the data carrying of the second simulation program is successful, so as to verify whether the second simulation program has a design loophole.

[0033] The test verification method and system of the storage controller provided by the embodiments of the present disclosure replace the traversal as a test case end flag with the record number of zero, so that the simulation verification process consumes less time. Further, whether the second simulation program or the storage controller finds all the command identification conflicts is determined through comparison between the first count value and the second count value. If the second count value is less than the first count value, it indicates that the second simulation program or the storage controller does not find all the command identification conflicts, and the vulnerability of the design code of the second simulation program or the storage controller can be located. BRIEF DESCRIPTION OF DRAWINGS

[0034] The above and other objects, features and advantages of the present application will become more apparent from the following description of the embodiments of the present application taken with reference to the accompanying drawings, in which:

[0035] Figure 1 is a structural schematic diagram of a storage system;

[0036] Figure 2 shows an interaction flowchart of a first simulation program and a second simulation program or a storage controller according to a first embodiment of the present disclosure;

[0037] Figure 3 shows an interaction flowchart of a first simulation program and a second simulation program or a storage controller according to a second embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The present application is described below based on the embodiments, but the present application is not limited to only these embodiments. In the following detailed description of the present application, some specific details are described in detail. The present application can also be understood without the description of these details by those skilled in the art. In order to avoid confusion of the essence of the present application, the well-known methods, processes, and flows are not described in detail. In addition, the drawings are not necessarily drawn to scale.

[0039] The flowcharts and block diagrams in the drawings illustrate the possible system, method, and device of the embodiments of the present disclosure, the blocks on the flowcharts and block diagrams can represent a module, a program segment, or only a piece of code, and the module, the program segment, and the code are executable instructions for implementing the specified logic function. It should also be noted that the executable instructions for implementing the specified logic function can be recombined to generate new modules and program segments. Therefore, the blocks of the drawings and the block sequence are only used to better illustrate the processes and steps of the embodiments, and should not be regarded as a limitation on the present application itself.

[0040] Figure 1is a configuration diagram of a storage system. The storage 200 is a storage device electrically connected to the host 100. Communication between the storage 200 and the host 100 is achieved via the bus 107. The bus 107 is a transmission line connecting the host 100 and the storage 200. The bus 107 is, for example, a PCIe bus. The PCIe bus is a full-duplex transmission line including a transmission line for transmitting data from the host 100 to the storage 200 and a transmission line for transmitting data from the storage 200 to the host 100.

[0041] The host 100 includes a processor 110 and a memory 120. The processor 110 is, for example, a central processing unit (CPU). The processor 110 executes software loaded into the memory 120 from the storage 200 or other storage device connected to the host 100, such as an operating system, a file system, an application program, and the like. A part of a storage area of the memory 120 is used to hold a submission queue / completion queue pair (SQ / CQ pair).

[0042] The storage 200 includes a storage controller 120 and a storage medium 130. The storage controller 120 is electrically connected to the storage medium 130. The storage medium 130 is, for example, a NAND-type flash memory. The storage controller 120 is, for example, a control circuit such as a system on a chip (SoC). Parts of the storage controller 120 can be implemented by dedicated hardware, a processor executing a program, or a combination thereof.

[0043] The storage controller 120 performs a data readout operation or a data write operation on the storage medium 130 by processing a host command received from the host 100. The host command is, for example, a read command or a write command. The read command is a command for the storage medium 130 to read out data. The write command is a command for the storage medium 130 to write data.

[0044] The storage controller 120 includes a host interface 121, a microprocessor 123, a cache unit 125, and a storage medium interface 128 electrically connected by internal lines.

[0045] The microprocessor 123 is used to execute program instructions from a software program stored as firmware in the cache unit 125 and the storage medium 130, thereby implementing management of data stored in the storage medium 130 and management of blocks included in the storage medium 130. The management of data includes management of mapping information indicating a correspondence between each logical address and each physical address. The microprocessor 123 manages the mapping relationship between each logical address and each physical address using an L2P table. The management of blocks included in the storage medium 130 includes operations such as wear leveling and garbage collection. The microprocessor 123 can also process various host commands received via the host interface 121, and notify the results to the host 100 via the host interface after execution.

[0046] The cache unit 125 is, for example, a Dynamic Random Access Memory (DRAM) or a Static Random-Access Memory (SRAM). The cache unit 125 can be used for the mapping relationship between the logical address and each physical address described above. The storage medium interface 128 is a circuit that controls the storage medium 130. In the case where the storage medium 130 is configured of a plurality of NAND-type flash memory dies, the storage medium interface 128 is connected to each flash memory chip of the NAND-type flash memory via a plurality of channels (for example, CH0 in the drawing).

[0047] The host interface 121 complies with the NVMe protocol. According to the NVMe protocol, in the communication between the host 100 and the host interface 121, a queue pair is used for issuing a command from the host 100 to the storage controller 120 and sending a completion response from the storage controller 120 to the host 100. Each queue pair includes at least one submission queue (SQ) and one completion queue (CQ) associated with the at least one submission queue (SQ), and such a pair of queues is usually referred to as a submission queue / completion queue pair (SQ / CQ pair). The submission queue (SQ) is a queue used for issuing a command to the storage 200. The completion queue (CQ) is a queue used for receiving a completion response indicating the command from the storage 200. The completion response includes information indicating the status of the completed command. The completion response is also referred to as a command completion notification. The transmission of a write command, write data, read data, and a completion response between the host 100 and the storage 200 is performed via the bus 107. According to the NVMe protocol, the host interface 121 first extracts each host command from the submission queue (SQ) and transmits it to the microprocessor 123 for execution to complete the work corresponding to the host command, and when the microprocessor 123 completes the corresponding work, sends the completion response to the completion queue (CQ) via the host interface 121. According to the NVMe protocol, the host interface 121 checks the command identifier of the host command before passing the host command to the microprocessor 123 for processing, and if there is a command identifier conflict, does not execute the command and sends a completion response with error information to the host, and if there is no command identifier conflict, passes the command to the microprocessor 123 for processing. The NVMe protocol identifies a command identifier conflict as the command identifier of one command being the same as the command identifier of another command being executed, and identifying a command identifier conflict usually also needs to be limited in scope, for example, if the scope is limited to a single submission queue, it means that in the same submission queue, if the command identifier of one command is the same as the command identifier of another command being executed, it is identified as a command identifier conflict, and for another example, if the scope is limited to a queue pair consisting of a submission queue and a completion queue, it means that in the multiple submission queues of the same queue pair, if the command identifier of one command is the same as the command identifier of another command being executed, it is identified as a command identifier conflict. However, it should be understood that the specific definition of the command identifier conflict does not affect the various embodiments of the present disclosure. After the microprocessor 123 receives the command from the host interface 121, it usually performs some data transfer work, such as transferring write data from a specified location in the memory 120 to the storage medium 130 or reading data from the storage medium 130 and storing it to a specified address in the memory 120.

[0048] When a product of a storage controller is prepared, it is necessary to build a simulation test environment to test and verify the processing logic of the storage controller (including the processing logic of the NVMe protocol). In the simulation test environment, a first simulation program can be run to simulate the behavior of a host, while a second simulation program is run to simulate the behavior of the storage controller, or only the first simulation program is run to simulate the behavior of the host, and the first simulation program interacts with the storage controller (chip product to be tested and verified).

[0049] Figure 2 An interaction flowchart of the first simulation program and the second simulation program or the storage controller according to the first embodiment of the present disclosure is shown. As shown in the figure, the following steps are included.

[0050] In step S1, the first simulation program records a plurality of commands under a test case as a plurality of records.

[0051] In order to perform simulation test verification, a tester usually prepares a large number of test cases, each of which simulates the sending of thousands of commands. Therefore, regarding this step, the first simulation program can include the following operations: reading the test cases one by one, generating a plurality of commands according to each test case, each command having a command identifier, but the plurality of command identifiers corresponding to the plurality of commands can be repeated, and therefore when recording the plurality of commands as a plurality of records in the storage unit, two or more commands having the same command identifier are identified, and only the earliest command is recorded for the two or more commands having the same command identifier.

[0052] In addition, the first simulation program can also apply a plurality of structure arrays in the memory to store a plurality of records corresponding to the plurality of commands, or can use a database table to store the plurality of records. Each record can include a command identifier, command details (such as an operation code, a target address, and a data length), and the like.

[0053] In step S2, the first simulation program serially sends the plurality of commands.

[0054] In step S3, the second simulation program or the storage controller checks whether each command has a command identifier conflict, and if so, steps S4 and S5 are performed, and if not, steps S6 and S7 are performed.

[0055] In step S4, the second simulation program or the storage controller does not execute the command.

[0056] In step S5, the second simulation program or the storage controller feeds back a result notification containing the command identifier conflict.

[0057] In step S6, the second simulation program or the storage controller executes the command.

[0058] In step S7, the second simulation program or the memory controller feeds back a result notification not containing the command identification conflict. Meanwhile, the result notification can contain the command execution result of successful command execution or failed command execution, and if the command execution fails, the result notification can also include error code and error information indicating specific errors.

[0059] It should be understood that the first simulation program and the second simulation program can simulate the submission queue / completion queue of the NVMe protocol for data transmission, i.e., the first simulation program stores multiple commands into the submission queue one by one, and the second simulation program takes out the commands from the submission queue one by one, the second simulation program stores the result notification of each command into the completion queue, and the first simulation program takes out the result notification from the completion queue one by one and checks the result notification to determine whether there is a command identification conflict, successful command execution or failed command execution. Of course, the first simulation program and the second simulation program can also use other ways (such as shared memory, Socket communication) for data transmission.

[0060] It should be understood that steps S2 to S7 are a loop body, i.e., the second simulation program receives one command each time, and then executes steps S2 to S7.

[0061] In step S8, the first simulation program determines whether each received result notification is a result notification not containing a command identification conflict, and if so, step S9 is executed.

[0062] In step S9, the first simulation program deletes a corresponding record from the multiple records.

[0063] It should be understood that steps S8 to S9 are a loop body and will be repeatedly executed multiple times until the number of records is zero.

[0064] In step S10, the first simulation program determines that the test case is completed when the number of records under the test case is zero.

[0065] The embodiment proposes a brand-new test verification method. In the method, a first simulation program simulating host behavior stores multiple commands under a test case as multiple records in a storage unit respectively, and when storing, for commands with the same command identifier, only one record is stored in the storage unit. Then the first simulation program deletes a corresponding record from the multiple records whenever a result notification not containing command identifier conflict is received. When the first simulation program deletes the number of the multiple records to zero, it means that all the commands under the test case except the commands with command identifier conflict return the result notification, and thus it can be determined that the test case is completed. The method does not need to constantly traverse the records in the storage unit to determine whether the test case is completed, and since the number of records in the storage unit is constantly reduced, the first simulation program spends less time retrieving the required record each time, thereby shortening the total execution time of the first simulation program.

[0066] Figure 3 An interaction flowchart of a first simulation program and a second simulation program or a memory controller according to a second embodiment of the disclosure is shown. The following steps are included.

[0067] In step S1, the first simulation program records multiple commands under a test case as multiple records. Each command has a command identifier, and two or more commands with the same command identifier only record the earliest command.

[0068] Step S1_1: The first simulation program counts the number of commands with the same command identifier to obtain a first count value.

[0069] In step S2, the first simulation program sends the multiple commands in series.

[0070] In step S3, the second simulation program or the memory controller checks whether each command has command identifier conflict, and if so, steps S4 and S5 are performed, otherwise, steps S6 and S7 are performed.

[0071] In step S4, the second simulation program or the memory controller does not execute the command.

[0072] In step S5, the second simulation program or the memory controller feeds back a result notification containing command identifier conflict.

[0073] In step S6, the second simulation program or the memory controller executes the command.

[0074] In step S7, the second simulation program or the memory controller feeds back a result notification not containing the command identification conflict. Meanwhile, the result notification can contain a command execution result of successful command execution or failed command execution, and if the command execution fails, the result notification can also contain an error code and error information indicating specific errors.

[0075] It should be understood that steps S2 to S7 are a loop body, i.e., the second simulation program executes steps S2 to S7 once for each received command.

[0076] In step S8, the first simulation program determines whether each received result notification contains a command identification conflict. If the result notification does not contain a command identification conflict, step S9 is executed, and if the result notification contains a command result notification, S8-1 is executed.

[0077] In step S8_1, the first simulation program counts the result notifications to obtain a second count value.

[0078] In step S9, the first simulation program deletes a corresponding record from the plurality of records.

[0079] It should be understood that steps S8, S8-1 and S9 constitute a loop body, i.e., the first simulation program executes steps S8, S8-1 and S9 once for each received command.

[0080] In step S10, the first simulation program determines that the test case is completed when the number of records under the test case is zero.

[0081] In step S11, the first simulation program determines whether the second simulation program identifies all command identification conflicts in the test case according to a comparison result of the first count value and the second count value.

[0082] Compared with the first embodiment, the embodiment adds steps S1_1, S8_1 and S11. Through these steps, the first simulation program counts the number of commands with the same command identification to obtain a first count value before issuing the commands, counts the result notifications containing the command identification conflicts to obtain a second count value after receiving the result notifications, and when the second count value is equal to the first count value, it indicates that the second simulation program or the memory controller finds all the command identification conflicts. If the second count value is less than the first count value, it indicates that the second simulation program or the memory controller does not find all the command identification conflicts, and accordingly, the design logic vulnerability of the second simulation program or the memory controller can be located.

[0083] The following analyzes whether the above embodiments cover all possible command identification conflicts specified in the NVMe protocol. According to the NVMe protocol, the command identification of a command issued to a certain submission queue (SQ) is not allowed to be the same as the command identification of a currently executing command, otherwise a specific error message (i.e., command identification conflict) needs to be reported to the host on the NVMe interface, which is a functional point that needs to be implemented in the design code and is a functional point that needs to be supported by the verification environment for scenario verification. For the verification scenario of command identification conflict, the above embodiments can support it, for example, when the host issues two commands with duplicate command identification to a certain submission queue (SQ), according to the protocol, the former is executed normally, and the latter should report an error to the host. In the verification process, before storing the record in the storage unit, it is checked whether there is already a command with the same command identification, and if it is found, it will not be stored. Although the command with the later command identification is not saved, it does not affect the verification of the functional point. Whether the command with the later command identification is reported to the host can be verified by comparing the first count value and the second count value of the error command identification, thereby verifying this functional point. If the design code behaves correctly, this command will be immediately returned to the host after being taken out, informing the host of the error information of command identification conflict through the command identification conflict checking mechanism, and the controller will not perform data transmission and other operations, which will not affect the execution of the remaining commands. When the first simulation program receives this specified command identification conflict notification, it does not need to retrieve and delete the corresponding record, which avoids affecting the execution of normal commands with the same command identification, and only needs to increase the count of command identification conflicts. If the design code behaves incorrectly, the later duplicate command is not reported as an error of command identification conflict, but is executed normally. Then when the command is executed, the first simulation program will receive multiple result notifications with the same command identification. Since the first simulation program retrieves the command record in the storage unit through the command identification, when the result notification of the later command is received, the data has been compared and the corresponding record has been deleted. At this time, the retrieval cannot find the corresponding record, resulting in an error. Accordingly, it can be found that the design code cannot handle the command identification conflict scenario and there is a design vulnerability. Therefore, the embodiments of the present disclosure support various possible command identification conflicts specified in the NVMe protocol.

[0084] In addition, the NNMe protocol specifies that some commands need to be modified by the firmware (FW) in the storage controller, so when the second simulation program is used to simulate the behavior of the storage controller, the commands that need to be modified by the firmware (FW) need to be considered.

[0085] If yes, the command is reported to the first simulation program, modified by the first simulation program, and sent to the second simulation program, and the second simulation program executes the modified command. Since the only command type that needs to be modified by the firmware in the current NVMe protocol is ZONE APPENDED CMD, and the only modification content is the logical address in the command, the command identifier cannot be modified, so as long as a step is added between S3 and S6 in the method of the prior art: "if the received command needs to be modified by the firmware (FW), then the command is reported to the first simulation program, and the modified command of the first simulation program is received", the problem can be solved. Figure 3

[0086] Correspondingly, the disclosure also provides a test verification system for a storage controller that adopts an NVMe protocol, the test verification system comprising the first simulation program and the second simulation program described above. Since the aspects of the first simulation program and the second simulation program have been described in detail above, they will not be repeated here.

[0087] Those skilled in the art can understand that the various modules or units of the test verification system according to the present application can be implemented by hardware, firmware or software. The software includes, for example, coded programs formed by various programming languages such as JAVA, C / C++ / C#, SQL, etc. Although the steps and the order of the steps of the present disclosure are given in the method and the method legend, the executable instructions that implement the logical functions of the steps can be recombined to generate new steps. The order of the steps should not be limited to the order of the steps in the method and the method legend, and can be adjusted at any time according to the needs of the functions. For example, some of the steps can be executed in parallel or in reverse order.

[0088] The above description is only the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A testing and verification method for a storage controller, wherein the storage controller adopts the NVMe protocol, the testing and verification method is executed by a first simulation program, and includes the following steps: Multiple commands under a test case are recorded as multiple records. Each command has a command identifier. For two or more commands with the same command identifier, the earliest command is recorded. The multiple commands are sent serially to the storage controller; The system serially receives and checks the result notifications of the multiple commands fed back by the storage controller one by one. For each command, the storage controller checks whether there is a command identifier conflict. If there is no command identifier conflict, the system executes the command and feeds back a result notification that does not contain a command identifier conflict. If there is a command identifier conflict, the system does not execute the command and feeds back a result notification that contains a command identifier conflict. For the result notification that does not contain a command identifier conflict, delete one corresponding record from the multiple records; The test case is considered complete when the number of records under the test case is zero.

2. The test verification method according to claim 1, wherein, A second simulation program is used to perform the corresponding operations in place of the storage controller.

3. The test verification method according to claim 2, wherein, The first simulation program performs the following steps: Count the number of commands with the same command identifier to obtain a first count value; The result notification containing command identifier conflicts is counted to obtain a second count value; Based on the comparison result between the first count value and the second count value, it is determined whether the second simulation program has identified all command identifier conflicts in the test case.

4. The test verification method according to claim 2, wherein, The second simulation program further includes: after determining that there is no command identifier conflict and before executing the command, reporting the command that requires firmware modification to the first simulation program, and receiving the modified command from the first simulation program so that the second simulation program can execute the modified command.

5. The test and verification method according to claim 1, wherein, The first simulation program stores the multiple records in memory or in a database table.

6. The test verification method according to claim 2, wherein the first simulation program further includes: Based on the result that does not contain command identifier conflicts, the system notifies the inspection of data in the target address to verify whether the storage controller or the second emulation program has any design flaws.

7. A test and verification system for a storage controller, the storage controller employing the NVMe protocol, the test and verification system comprising a first simulation program simulating host behavior and a second simulation program simulating the storage controller, the first simulation program performing the following operations: Record multiple commands under the test case as multiple records, where, Each command has a command identifier, but if two or more commands have the same command identifier, the earliest command is recorded. The multiple commands are sent serially to the storage controller; The results notifications of the multiple commands fed back by the second simulation program are received serially and checked one by one. For result notifications that do not contain command identifier conflicts, delete one corresponding record from the multiple records; When the number of records under the test case is zero, the test case is considered complete. The second simulation program performs the following operations: For each command, check for command identifier conflicts. If no command identifier conflict exists, execute the command and provide a notification indicating that no command identifier conflict exists. If a command identifier conflict exists, do not execute the command and provide a notification indicating that a command identifier conflict exists.

8. The test and verification system according to claim 7, wherein, The first simulation program also performs the following operations: Count the number of commands with the same command identifier to obtain a first count value; The result notification containing command identifier conflicts is counted to obtain a second count value; Based on the comparison result between the first count value and the second count value, it is determined whether the second simulation program has identified all command identifier conflicts in the test case.

9. In the test and verification system according to claim 7, after checking that there is no command identifier conflict and before executing the command, the second simulation program reports the command that needs firmware modification to the first simulation program, and receives the modified command from the first simulation program, and the second simulation program executes the modified command.

10. The test and verification system according to claim 7, wherein the first simulation program further comprises: For the result notification that does not contain command identifier conflicts, check the data in the target address to determine whether the data transfer of the second simulation program was successful, thereby verifying whether the second simulation program has any design flaws.

Images