Systems and methods for testing virtual functions of a device under test

Abstract

Embodiments of the present invention may provide an extended NVMe driver that supports the execution of virtual functions (and related physical functions) of a DUT without using a VM or hypervisor. In this way, the amount of memory and processing resources used to test NVMe SSDs can be significantly reduced, and multiple DUTs (e.g., up to 16 DUTs) can be tested independently in parallel. That is, each DUT is tested separately as if it were the only device being tested, and there are no contention conditions or resource contention between workloads during testing.

Description

Technology Field

[0001] The present invention claims the priority and benefit of U.S. Provisional Application No. 63 / 440,003, filed January 19, 2023, titled “System and method for testing virtual functions of a device under test” and U.S. Provisional Application No. 63 / 429,056, filed November 30, 2022, titled “Testing of an NVME device with virtualization capabilities, i.e., low overhead testing of a virtualized DUT without a virtual machine (VM) in a host system,” the entire contents of which are incorporated herein by reference for all purposes.

[0002] Embodiments of the present invention generally relate to the field of device testing. More specifically, embodiments of the present invention relate to a method and system for testing a virtual function of a device under test (DUT). Background Technology

[0003] A device or equipment under test is typically tested to determine the device's performance and consistency before it is sold. For example, the DUT can be tested using various test cases, and the results of the test cases can be compared to expected output results. If the results of the test cases do not match a satisfactory value or range of values, the device may be considered a failed device or an outlier, and the device may be binned based on performance, etc.

[0004] The DUT is typically tested by automated or automated test equipment (ATE) that can be used to perform complex tests using software and automation to improve test efficiency. The DUT can be any type of semiconductor device, wafer, or component intended to be integrated into a final product, such as a computer, network interface, or solid-state drive (SSD). By using ATE to remove defective or unsatisfactory chips during manufacturing, the quality of the yield can be significantly improved.

[0005] Recently, new SSD protocols such as Non-Volatile Memory Express (NVMe) have advanced the capabilities of modern storage devices. For example, some NVMe SSDs support virtual functions, providing an interface to virtual machines (VMs) managed by a hypervisor, which allows for the separation of operating systems and resources from virtual machines and enables the creation and management of VMs. Unfortunately, existing approaches to testing NVMe SSDs cannot test a large number of virtual functions simultaneously due to resource limitations. Running a virtual machine to test virtual functions typically requires significant overhead (e.g., memory and CPU requirements) to simulate a virtual environment for DUT testing. For this reason, testing virtual functions of a DUT using a VM is a relatively slow and inefficient process. An improved approach to testing virtual functions of devices (e.g., NVMe SSDs) is needed. An example of relevant background technology is U.S. Patent Application Publication No. 2015 / 0095712 (publication date April 2, 2015).

[0006] Therefore, an approach to device testing is required that can test multiple virtual functions in parallel across multiple devices. Embodiments of the present invention may provide an extended NVMe driver that supports the execution of virtual functions (and related physical functions) of a DUT without using a VM or hypervisor. In this way, the amount of memory and processing resources used to test NVMe SSDs can be significantly reduced, and multiple DUTs (e.g., up to 16 DUTs) can be tested independently in parallel. That is, each DUT is tested separately as if it were the only device being tested, and there are no contention conditions or resource contention between workloads during testing.

[0007] According to one embodiment, a tester system for testing a plurality of DUT virtual functions of a plurality of DUTs is disclosed. The tester system includes a host computer system including memory, and the memory includes a plurality of test program processes operable to execute simultaneously. Each of the plurality of test program processes is associated with each of the plurality of DUTs, and each test program process includes a plurality of threads that generate a workload for testing the DUT virtual functions of the associated DUT. The tester system further includes a plurality of driver instances instantiated in memory, and each driver of the plurality of driver instances is operable to enable testing of the associated DUT virtual functions on the plurality of DUTs, and each driver instance includes a data structure for each DUT to enable testing of the DUT virtual functions on the plurality of DUTs. The driver instances are operable to test the DUT virtual functions of the plurality of DUTs individually, independently, and simultaneously by simulating a virtualization environment for the plurality of test program processes on the host computer system, and any given test program process among the plurality of test program processes appears to be the only user operating on the host computer system at any given time.

[0008] According to some embodiments, each data structure of each driver instance of a plurality of driver instances finds and enables an associated DUT virtual function on an associated DUT, presents access to the DUT virtual function on the associated DUT to a host computer system, and operates to be used by a test program process of the host computer system to access and test the associated DUT virtual function on the associated DUT.

[0009] According to some embodiments, for a given test program process and an associated DUT and a given DUT virtual function of the associated DUT, the workload of the given test program process accesses and tests the given DUT virtual function through a driver instance associated with the given DUT virtual function among a plurality of driver instances.

[0010] According to some embodiments, each driver instance of a host computer system further comprises a controller handler, an interrupt handler, a queue for sending a command to a specific virtual function of a specific DUT, a queue for receiving a response back from a specific virtual function of a specific DUT, and computer resources for implementing the queues.

[0011] According to some embodiments, a plurality of DUTs are solid-state drives (SSDs).

[0012] According to some embodiments, a plurality of driver instances are NVMe device driver instances.

[0013] According to some embodiments, a driver instance of a host computer system simulates the functions of a hypervisor.

[0014] According to some embodiments, the host computer system does not have virtual machine software.

[0015] According to another embodiment, a tester system for testing a plurality of virtual functions of different DUTs is disclosed. The tester system comprises a host computer system having memory having first and second test program processes operable to execute simultaneously, wherein the first test program process is operable to test a virtual function of the first DUT. The second test program process is operable to test a virtual function of the second DUT, and the first test program process includes a plurality of first threads that generate a workload for testing a virtual function of the first DUT. The second test program process includes a plurality of second threads that generate a workload for testing a virtual function of the second DUT. The tester system further comprises a plurality of driver instances instantiated in memory, and each driver instance is operable to enable testing of associated virtual functions on the first and second DUTs. Additionally, each driver instance includes a first data structure for testing an associated virtual function on the first DUT and a second data structure for testing an associated DUT virtual function on the second DUT. The driver instance is capable of simulating a virtualization environment on the host computer system for the first and second test program processes to test the virtual functions of the first and second DUTs individually, independently, and simultaneously, and it is shown that each of the first and second test program processes is the only test program process operating on the host computer system.

[0016] According to some embodiments, a first data structure of each driver instance of a plurality of driver instances finds and enables an associated DUT virtual function on a first DUT, presents access to the associated virtual function on the first DUT to a host computer system, and operates to be used by a first test program process of the host computer system to access and test the associated DUT virtual function on the first DUT.

[0017] According to some embodiments, a second data structure of each driver instance of a plurality of driver instances finds and enables an associated DUT virtual function on the second DUT, presents access to the associated virtual function on the second DUT to a host computer system, and operates to be used by a second test program process of the host computer system to access and test the associated DUT virtual function on the second DUT.

[0018] According to some embodiments, each driver instance of a host computer system further comprises a controller handler, an interrupt handler, a first queue for sending a command to a specific virtual function of a specific DUT, a second queue for receiving a response back from a specific virtual function of a specific DUT, and computer resources for implementing the first and second queues.

[0019] According to some embodiments, the first and second DUTs include solid-state drives (SSDs).

[0020] According to some embodiments, a plurality of driver instances include NVMe device driver instances.

[0021] According to some embodiments, multiple driver instances of a host computer system simulate the functions of a hypervisor.

[0022] According to some embodiments, the host computer system does not have virtual machine software.

[0023] According to different embodiments, a method for testing multiple DUT virtual functions of multiple DUTs using a tester system is disclosed. The method comprises the steps of: simultaneously executing multiple test program processes in a host computer system including memory—each of the multiple test program processes is associated with each of the multiple DUTs, and each of the multiple test program processes includes multiple threads that generate a workload for testing the DUT virtual functions of the associated DUTs—and simulating a virtualization environment for the multiple test program processes on the host computer system using multiple device driver instances instantiated in memory—where the multiple DUT virtual functions are operable to be tested individually, independently, and simultaneously for the multiple DUTs within the virtualization environment, and any given test program process among the multiple test program processes appears to be the only user operating on the host computer system at any given time. Each of the multiple driver instances is operable to enable testing of associated DUT virtual functions on the multiple DUTs, and each of the multiple driver instances includes multiple data structures, and each data structure for each of the multiple DUTs enables testing of each of the DUT virtual functions on the multiple DUTs.

[0024] According to some embodiments, the method further comprises the steps of using the respective data structures of each of the plurality of driver instances to locate and enable associated DUT virtual functions on the associated DUT, presenting access to the DUT virtual functions on the associated DUT to a host computer system, and using the test program process of the host computer system to access and test the associated DUT virtual functions on the associated DUT.

[0025] According to some embodiments, for a given test program process and an associated DUT and a given DUT virtual function of the associated DUT, the workload of the given test program process accesses and tests the given DUT virtual function through a driver instance associated with the given DUT virtual function among a plurality of driver instances.

[0026] According to some embodiments, each driver instance of a plurality of driver instances of a host computer system further comprises a controller handler, an interrupt handler, a queue for sending a command to a specific virtual function of a specific DUT, a queue for receiving a response back from a specific virtual function of a specific DUT, and computer resources for implementing the queues.

[0027] According to some embodiments, multiple drivers of a host computer system simulate the functions of a hypervisor. Brief explanation of the drawing

[0028] The accompanying drawings included in and constituting part of this specification serve to illustrate embodiments of the present invention and to explain the principles of the present invention together with the description. FIG. 1 is a block diagram of an exemplary test system communicating with a plurality of DUTs for parallel testing of virtual and physical functions of DUTs using an extended NVMe driver that executes commands of a workload generated by a thread of a test program process executed on a test system according to an embodiment of the present invention. FIG. 2 is a block diagram of an exemplary test system communicating with a DUT to parallel test virtual functions associated with the namespace of the DUT using an extended NVMe driver according to an embodiment of the present invention. FIG. 3 is a block diagram of an exemplary test system communicating with a plurality of DUTs to test a virtual function in parallel using a data structure instantiated by an extended NVMe driver according to an embodiment of the present invention. FIG. 4 is a block diagram and data flow diagram for testing virtual and physical functions of a DUT using an extended NVMe driver according to an embodiment of the present invention. FIG. 5 is a block diagram of an exemplary extended NVMe driver according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating an exemplary sequence of computer implementation steps for testing a virtual function of a DUT using an extended NVMe driver according to an embodiment of the present invention. FIG. 7 illustrates an exemplary test computer system platform in which an embodiment of the present invention can be implemented. Specific details for implementing the invention

[0029] We will now refer to various embodiments of the present invention in detail. While the subject matter will be described in relation to other embodiments, it will be understood that the claimed subject matter is not intended to be limited to these embodiments. On the contrary, the claimed subject matter is intended to encompass alternatives, modifications, and equivalents that may be included within the spirit and scope of the claimed subject matter as defined by the appended claims.

[0030] Additionally, in the following detailed description, various specific details are presented to provide a thorough understanding of the claimed subject matter. However, it will be recognized by those skilled in the art that embodiments may be practiced without these specific details or with equivalents thereof. In other cases, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects and features of the subject matter of this application.

[0031] Parts of the following detailed description are presented and discussed in terms of the method. Steps and their sequences are disclosed in the drawings of this invention (e.g., FIG. 6) illustrating the operation of the method, but such steps and sequences are exemplary. The embodiments are suitable for performing various other steps or variations of steps disclosed in the flowcharts of the drawings in this specification, and in a different order than that shown and described herein.

[0032] Parts of the detailed description are presented in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations on data bits that can be performed in computer memory. These descriptions and representations are the means used by a person of ordinary skill in the field of data processing technology to most effectively convey the content of their work to other persons of ordinary skill in the same field. Procedures, computer execution steps, logic blocks, and processes are generally regarded here as a coherent sequence of steps or instructions that lead to a desired result. A step is a step that requires the physical manipulation of a physical quantity. Although not strictly necessary, these quantities generally take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated in a computer system. Due to general usage, it has sometimes proven convenient to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, parameters, etc.

[0033] However, it should be kept in mind that all these and similar terms are associated with appropriate physical quantities and are merely convenient labels applied to such quantities. As will be evident from the following discussion, unless otherwise specifically stated, discussions throughout the invention using terms such as “access,” “record,” “include,” “store,” “transmit,” “connect,” “identify,” “encode,” “label,” etc. refer to the operations and processes of a computer system or similar electronic computing device that manipulate and convert data expressed as physical (electronic) quantities within the registers and memory of a computer system into other data similarly expressed as physical quantities within the computer system memory, registers, or other information storage, transmission, or display devices.

[0034] Some embodiments may be described in the general context of computer-executable instructions, such as program modules executed by one or more computers or other devices. Generally, a program module includes routines, algorithms, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. Typically, the functions of a program module may be combined or distributed as desired in various embodiments.

[0035] Parallel testing of virtual functions across multiple DUTs

[0036] Embodiments of the present invention may provide an extended NVMe driver that supports the execution of virtual functions (and related physical functions) of a DUT without using a VM or hypervisor. In this way, the amount of memory and processing resources used to test NVMe SSDs can be significantly reduced, and a large number of DUTs (e.g., up to 16 DUTs) can be tested independently in parallel. That is, each DUT is tested separately as if it were the only device being tested, and there are no contention conditions or competition for resources between workloads during testing.

[0037] FIG. 1 illustrates an exemplary host test system (100) (e.g., ATE) coupled to DUTs (105, 110, 115, and 120). According to the embodiment, a smaller number or additional DUTs (e.g., 16 DUTs) may be tested in parallel. The host system (100) executes a test program that starts multiple test program processes (125, 130, and 135) using multiple threads to execute virtual functions of the DUTs (105, 110, 115, and 120) through NVMe drivers (140, 145, 150, and 155). The NVMe drivers (140, 145, 150, and 155) are extended drivers that can advantageously test virtual functions of multiple DUTs in parallel without running a VM or hypervisor, thereby reducing the overhead and complexity of the test. In addition, testing virtual functions of the DUT in parallel using extended NVMe drivers (140, 145, 150, and 155) can significantly improve performance during testing compared to conventional approaches.

[0038] A thread of the test program is a user-level thread that generates a workload to test the DUT using an NVMe driver. For example, thread 1.1 of test program process 1 (125) can generate a workload to execute virtual function 1 of DUT 1 (105) using NVMe driver 1 (140), thread 1.2 can generate a workload to execute virtual function 2 of DUT 1 (105) using NVMe driver 2 (145), and so on. The workloads are executed independently and separately without competition between devices for resources (e.g., memory, storage, etc.), and each driver can test the virtual function of each DUT using allocated data structures and other resources (e.g., interrupts and queues).

[0039] Testing a virtual function may involve performing a corresponding physical function of the DUT, and the test results may be returned to a corresponding test program process for evaluation. The physical function of the DUT is executed using a dedicated local register within each DUT that is accessible by the host (100). Typically, a virtual function can be thought of as a smaller, lighter version (e.g., with fewer attributes) of the physical function being tested by the host (100).

[0040] The extended NVMe driver typically includes instantiated functions / routines and data structures for each virtual function to be tested, as well as physical resources allocated for the test (e.g., queues, interrupts, memory, etc.). According to some embodiments, the NVMe driver includes a controller handler for interfacing with the NVMe device, an interrupt handler, a queue for sending commands to the virtual functions of the DUT, a queue for receiving responses back from the DUT, and optionally other computer resources for implementing the queue. Thus, the NVMe driver can execute the functions / routines necessary to perform the workload based on the instantiated data structures. Additionally, the test program process executed by the test program appears to be the only user operating on the host computer system at any given time. In this way, the test system (100) can simulate a virtualization environment for multiple test program processes on the host computer system to test the virtual functions of the DUT individually, independently, and simultaneously, so there is no contention for resources or need to run a hypervisor or virtual machine.

[0041] Data structures can be used by the NVMe driver to locate and enable virtual functions of the DUT for testing. For example, each data structure may present access to a corresponding virtual function accessed by the NVMe driver to perform a workload generated by a thread. Typically, a separate data structure is used to test the virtual functions of each DUT. For example, to test 10 virtual functions of 16 DUTs in parallel, 160 data structures are instantiated. Additionally, data structures can be instantiated by the NVMe driver to directly test the physical functions of the DUT (e.g., without testing virtual functions).

[0042] A thread runs on a host (100) in user space and generates a specific workload that typically includes a combination of NVMe commands (e.g., read, write, and manage commands). An NVMe driver services the thread's request and issues commands to the device. According to some embodiments, multiple threads may be used to test virtual functions, for example, using different workloads or commands.

[0043] The NVMe driver may also include one or more queues (e.g., NVMe queue, I / O queue), as well as controller handlers and namespaces (notified by the DUT) for accessing / modifying data stored on the DUT during the execution of the workload. The namespaces notified by the DUT provide access points through which threads can access the corresponding user-space data via the NVMe driver. In this way, 64 or 128 virtual functions of the DUT can be tested by independently testing 8 or 16 DUTs in parallel without any contention (e.g., race conditions) between resources.

[0044] FIG. 2 is a block diagram of an exemplary host test system (200) capable of testing multiple virtual functions in parallel across multiple DUTs without using a hypervisor or virtual machine according to an embodiment of the present invention. In the example of FIG. 2, a single DUT (205) is shown for clarity. It should be understood that typically, multiple DUTs are coupled to the host system (200) to test in parallel.

[0045] In the example of FIG. 2, the host (200) executes a test program that initiates a plurality of test program processes (210, 215, and 220). The test program processes (210, 215, and 220) execute a plurality of threads, each thread typically associated with a different virtual function. An extended NVMe driver is instantiated for testing each namespace assigned to a virtual function. Threads (e.g., threads 1.1, 2.1, 8.1, 8.n) generate workloads for the corresponding namespaces to test the virtual functions of the DUT (e.g., virtual function 1, virtual function 2, virtual function 3, virtual function 4). Using test program processes (210, 215, and 220) to execute threads to generate commands executed by the extended NVMe driver eliminates the need for a hypervisor or virtual machine to significantly reduce test complexity and overhead. According to some embodiments, multiple threads are used to test a virtual function using, for example, different commands or workloads.

[0046] As illustrated in FIG. 2, virtual function 1 is assigned NVMe namespace 1, virtual function 2 is assigned NVMe namespace 2, and so on. Testing the virtual functions of the DUT (e.g., DUT (205)) using the extended NVMe driver (225, 230, 235, and 240) typically involves executing the corresponding NVMe physical functions (265). Advantageously, the test program processes executed by the test program processes appear to be the only users operating on the host computer system at any given time. In this way, the test system (200) can simulate a virtualization environment for multiple test program processes on the host computer system to test the virtual functions of the DUT separately, independently, and simultaneously, so there is no contention for resources or need to run a hypervisor or virtual machine.

[0047] FIG. 3 is a block diagram of an exemplary test system (300) communicating with a plurality of DUTs (305, 310, 315 and 320) to test virtual functions in parallel using data structures instantiated by an extended NVMe driver according to an embodiment of the present invention. As shown in FIG. 3, a plurality of NVMe drivers (1-n) are instantiated to test a corresponding number of virtual functions of the DUTs, and each extended NVMe driver includes a different data structure for performing the test.

[0048] Testing virtual functions typically involves executing the corresponding physical functions of the DUT based on a workload generated by a test program process (e.g., test program processes (325 and 330)) executed by a test program running on a host test system (300). Importantly, each DUT instantiates multiple data structures corresponding to different virtual functions. For example, to test 10 virtual functions of 16 DUTs (n=16) in parallel, 160 data structures may be instantiated.

[0049] In the example of FIG. 3, the NVMe drivers (335 and 340) each instantiate four data structures to test the corresponding virtual functions of the DUT (305, 310, 315 and 320). Specifically, the NVMe driver 1 (335) tests virtual function 1 of the DUT (305, 310, 315 and 320) using data structures 1, 2, 3 and 4, respectively, and the NVMe driver n (330) tests virtual function n of the DUT (305, 310, 315 and 320) using data structures 5, 6, 7 and 8, respectively. By using different data structures to test workloads generated by test program processes, the host test system (300) can simulate a virtualization environment for multiple test program processes on a host computer system and test virtual functions of the DUT individually, independently, and simultaneously, so there is no competition for resources or need to run a hypervisor or virtual machine.

[0050] FIG. 4 is a block diagram and data flow diagram (400) for testing virtual and physical functions of a DUT using an extended NVMe driver according to an embodiment of the present invention. In step (405) of the data flow diagram (400), a test program process 1 executes a plurality of threads that generate a workload to test the virtual functions of the DUT. In step (410), the NVMe driver 1 receives the workload generated by thread 1.1 of the test program process 1 and tests the virtual function 1 of the NVMe device. As shown in FIG. 4, the namespace (1) of the user space of the NVMe device is assigned to the virtual function 1.

[0051] NVMe driver 1 instantiates a data structure for testing namespace 1, and the driver may include a controller handler for interfacing with the NVMe device, one or more queues (e.g., an I / O queue), an interrupt handler for controlling the operation of the NVMe driver by the CPU, and optionally other computing resources used to perform the test. In step (415), the DUT executes virtual function 1 according to an NVMe command received from the instantiated NVMe driver 1 to test virtual function 1. Executing virtual function 1 typically involves performing one or more physical functions that access memory registers of the NVMe device. In this way, test results can be generated by executing various functions of the DUT without using a hypervisor or virtual machine, and the results can be returned to a host computer system running test program process 1, thereby significantly reducing the overhead and complexity of the test. Additionally, the steps of the data flow diagram (400) are performed simultaneously by multiple test program processes and multiple threads, so that multiple DUTs can be tested in parallel using a single host computer test system.

[0052] FIG. 5 is a block diagram of an exemplary NVMe driver (510) instantiated in the memory of a host computer (500) to test virtual and physical functions of a DUT (535) without using a virtual machine or hypervisor according to an embodiment of the present invention. The DUT (535) may be placed in a socket of a test interface board together with other DUTs for parallel testing, for example. Multiple NVMe drivers (510) may be executed by the host computer (500) and may test multiple virtual functions of the DUT in parallel using threads executed by the CPU (505). The CPU (505) executes a test program including multiple test program processes that generate threads for generating different workloads to be performed by the DUT (535) according to commands transmitted by the controller handler (530) of the NVMe driver (510). As illustrated in FIG. 5, the NVMe driver (510) includes an interrupt handler (515) for receiving an interrupt from a CPU (505), one or more queues (520) such as a queue for sending and receiving data and NVMe commands to / from a DUT (535), and a data structure (525) for executing a workload within a virtualization environment.

[0053] FIG. 6 is a flowchart illustrating an exemplary process (600) for testing virtual and physical functions of an NVMe device using an extended NVMe driver according to an embodiment of the present invention. The process (600) may be performed without running a hypervisor or virtual machine to test virtual functions.

[0054] In step (605), the host system executes a test program to test multiple connected DUTs in parallel. The host system typically includes memory and a CPU. The test program executes multiple test program processes to test the DUTs by starting threads to test different virtual functions of the associated DUTs. Each thread can be used, for example, to test different virtual functions (or physical functions) of the DUTs.

[0055] In step (610), the host system simulates a virtual environment (similar to a virtual machine created and managed using a hypervisor) for testing virtual functions of the DUT using an NVMe device driver instantiated in the host system's memory. The NVMe device driver instantiated in the host system's memory includes different data structures created for different virtual functions. Typically, one data structure exists for each virtual function (or namespace) of the DUT to be tested, and the virtual functions are tested individually, independently, and simultaneously for the DUT within the virtualization environment.

[0056] In step (615), the host system executes various functions of the DUT in parallel according to the test program within the virtualization environment.

[0057] Exemplary test system

[0058] Embodiments of the present invention relate to an electronic system for testing virtual functions of an NVMe device without using a hypervisor or virtual machine, thereby reducing complexity and overhead during testing. A single test system can execute a test program that initiates multiple threads to test virtual functions of a DUT using an extended NVMe driver. Each DUT can be tested independently of other DUTs without competing for time or resources according to the workload generated by the threads, using data structures instantiated by the NVMe driver for the corresponding virtual functions. Virtual functions access user-space data of the DUT corresponding to namespaces accessible by the NVMe driver.

[0059] In the example of FIG. 7, an exemplary test computer system (712) includes a central processing unit (CPU) (701) for executing software applications and an operating system. Random access memory (702) and read-only memory (703) store applications and data to be used by the CPU (701). The CPU (701) can instantiate an NVMe driver (Fig. 5) and data structures for testing virtual and physical functions of the DUT in memory (702), and can communicate with the DUT by sending commands to the DUT (or test interface board, etc.) using a controller handler assigned to the NVMe driver.

[0060] The data storage device (704) provides non-volatile storage for applications and data and may include a fixed disk drive, a removable disk drive, a flash memory device, and a CD-ROM, DVD-ROM, or other optical storage device. The data storage device (704) or memory (702 / 703) may store historical and real-time test data (e.g., test results, limits, calculations, etc.).

[0061] Optional user input units (706 and 707) include a device (e.g., mouse, joystick, camera, touch screen, keyboard and / or microphone) that communicates input from one or more users to the computer system (712). A communication or network interface (708) enables the computer system (712) to communicate with other computer systems, networks, or devices through an electronic communication network that includes wired and / or wireless communication and includes an intranet or the Internet.

[0062] The optional display device (710) may be any device capable of displaying visual information, e.g., a final scan report, in response to a signal from the computer system (712), e.g., a flat touch-sensing display. Components of the computer system (712), including a CPU (701), memory (702 / 703), data storage device (704), user input device (706), and graphics subsystem (705), may be combined via one or more data buses (700).

[0063] As such, embodiments of the present invention have been described. Although the present invention has been described with respect to specific embodiments, it should be understood that the present invention should not be interpreted as being limited by such embodiments, but rather should be interpreted according to the following claims.

Claims

Claim 1 A tester system for testing multiple virtual functions of multiple DUTs, wherein the tester system comprises: a host computer system including memory—the memory includes multiple test program processes operable to execute simultaneously, wherein each of the multiple test program processes is associated with each DUT of the multiple DUTs, and each of the multiple test program processes includes multiple threads that generate a workload for testing the virtual functions of the associated DUTs—and multiple driver instances instantiated within the memory—each driver of the multiple driver instances is operable to enable testing of the associated virtual functions of the multiple DUTs, and each driver instance includes multiple data structures, and each data structure for each DUT of the multiple DUTs enables testing of the virtual functions of the respective DUTs on the multiple DUTs, and furthermore, the multiple driver instances are operable to test the virtual functions of the multiple DUTs individually, independently, and simultaneously by simulating a virtualization environment for the multiple test program processes on the host computer system, and any given test program process among the multiple test program processes at any given time on the host computer system Appears to be the only operating user - including, the tester system. Claim 2 A tester system according to claim 1, wherein each data structure of each of the plurality of drivers operates to find and enable an associated DUT virtual function on an associated DUT, present access to the DUT virtual function on the associated DUT to a host computer system, and be used to access and test the associated DUT virtual function on the associated DUT by a test program process of the host computer system. Claim 3 In paragraph 2, for a given test program process and an associated DUT and a given DUT virtual function of said associated DUT, the workload of said test program process accesses and tests said given DUT virtual function through a driver associated with said given DUT virtual function among said plurality of drivers, a tester system. Claim 4 A tester system according to claim 1, wherein each of the plurality of drivers of the host computer system further comprises a controller handler, an interrupt handler, a queue for transmitting a command to a specific virtual function of a specific DUT, a queue for receiving a response again from the specific virtual function of the specific DUT, and computer resources for implementing the queues. Claim 5 In claim 1, the tester system wherein the plurality of DUTs are solid-state drives (SSDs). Claim 6 In claim 1, the plurality of drivers is a tester system that is an NVMe device driver. Claim 7 A tester system according to claim 1, wherein the plurality of drivers of the host computer system simulate the functions of a hypervisor. Claim 8 In paragraph 7, the host computer system is a tester system without virtual machine software. Claim 9 A tester system for testing multiple virtual functions of a first and second DUT, wherein the tester system comprises a host computer system including a memory—the memory comprises first and second test program processes operable to execute simultaneously, wherein the first test program process is operable to test a virtual function of the first DUT and the second test program process is operable to test a virtual function of the second DUT, and further wherein the first test program process comprises a plurality of first threads for generating a workload for testing the virtual function of the first DUT and the second test program process comprises a plurality of second threads for generating a workload for testing the virtual function of the second DUT—and a plurality of drivers instantiated within the memory—each of the plurality of drivers is operable to enable testing of an associated virtual function on the first and second DUTs, and further wherein each driver comprises a first data structure for testing the associated virtual function on the first DUT and a second data structure for testing the associated DUT virtual function on the second DUT, and the plurality of drivers are the first and second test A tester system comprising: a program process capable of simulating a virtualization environment on the host computer system to test virtual functions of the first and second DUTs individually, independently, and simultaneously, wherein each of the first and second test program processes is found to be the only test program process operating on the host computer system. Claim 10 In claim 9, the first data structure of each of the plurality of drivers operates to find and enable an associated DUT virtual function on the first DUT, present access to the associated virtual function on the first DUT to the host computer system, and be used to access and test the associated DUT virtual function on the first DUT by the first test program process of the host computer system. Claim 11 In paragraph 10, the second data structure of each of the plurality of drivers operates to find and enable an associated DUT virtual function on the second DUT, present access to the associated virtual function on the second DUT to the host computer system, and be used to access and test the associated DUT virtual function on the second DUT by the second test program process of the host computer system. Claim 12 In claim 9, each driver of the plurality of drivers of the host computer system further comprises a controller handler, an interrupt handler, a first queue for transmitting a command to a specific virtual function of a specific DUT, a second queue for receiving a response again from the specific virtual function of the specific DUT, and computer resources for implementing the first and second queues, a tester system. Claim 13 In claim 9, the tester system wherein the first and second DUTs include solid-state drives (SSDs). Claim 14 In claim 9, the tester system, wherein the plurality of drivers includes NVMe device drivers. Claim 15 In claim 9, a tester system in which the plurality of drivers of the host computer system simulate the functions of a hypervisor. Claim 16 In paragraph 15, the above host computer system is a tester system without virtual machine software. Claim 17 A method for testing multiple virtual functions of multiple DUTs using a tester system, comprising: a step of simultaneously executing multiple test program processes in a host computer system including memory, wherein each of the multiple test program processes is associated with each DUT of the multiple DUTs, and each of the multiple test program processes includes multiple threads that generate a workload for testing the virtual functions of the associated DUTs; and a step of simulating a virtualization environment for the multiple test program processes on the host computer system using multiple device drivers instantiated in memory, wherein the multiple virtual functions of the DUTs are operable to be tested individually, independently, and simultaneously for the multiple DUTs within the virtualization environment, and any given test program process among the multiple test program processes appears to be the only user operating on the host computer system at any given time; wherein each of the multiple drivers is operable to enable testing of the associated virtual functions of the multiple DUTs, and each of the multiple drivers includes multiple data structures, and each data structure for each DUT of the multiple DUTs is for each DUT on the multiple DUTs Method to enable testing of virtual functions. Claim 18 A method according to claim 17, further comprising the steps of using the respective data structures of each of the plurality of drivers to find and enable an associated DUT virtual function on an associated DUT, presenting access to the DUT virtual function on the associated DUT to a host computer system, and using a test program process of the host computer system to access and test the associated DUT virtual function on the associated DUT. Claim 19 In paragraph 18, for a given test program process and an associated DUT and a given DUT virtual function of said associated DUT, the workload of said test program process accesses and tests said given DUT virtual function through a driver associated with said given DUT virtual function among said plurality of drivers. Claim 20 In claim 17, each of the plurality of drivers of the host computer system further comprises a controller handler, an interrupt handler, a queue for transmitting a command to a specific virtual function of a specific DUT, a queue for receiving a response again from the specific virtual function of the specific DUT, and computer resources for implementing the queues. Claim 21 In claim 17, a method in which the plurality of drivers of the host computer system simulate the functions of a hypervisor.

Images