Live update of kernel device modules

Abstract

Disclosed are systems, apparatus, and methods for implementing real-time device driver updates. When a processor loads a given version of a device driver, the given version is registered with a proxy module instead of with an operating system. If a previous version of the device driver is already running, the proxy module provides the given version with a pointer to the previous version. The given version uses the pointer to retrieve static data from the previous version. After silencing the previous version, the given version retrieves transient data from the previous version and then takes over as the running version of the device driver. A subsequent version of the device driver can replace the previous version in a similar manner. Similarly, in the case of a downgrade, the previous version of the device driver can replace the subsequent version in a similar manner.

Description

Background Art

[0001] Related technical notes

[0002] Bug fixes or security patches are regularly applied to live production kernel module code bases. In some computing environments, these patches will cause disruptions in the operating environment. For example, cloud service providers (CSPs) run various software modules in live production environments with thousands of hosts. It is impractical for large CSPs to kick users off hosts in order to install updated versions of kernel device modules (i.e., device drivers). Waiting for all users to log off on their own is also not feasible.

[0003] Additionally, attempting to replace a running kernel device module (or simply module) while the corresponding device is running can be a challenge. The act of unloading and reloading a module can result in a lost state scenario. One problem is that the new module will not know about the state left behind by the old module. For a period of time after the old module has been unloaded and the new module has begun to load, there will be no driver managing the device. Any activity or state changes to the device will be lost. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0005] Figure 1 is a block diagram of one implementation of a computing system.

[0006] Figure 2 It is a block diagram of an implementation method of a server virtual environment.

[0007] Figure 3 A block diagram of one implementation of a computing system is shown.

[0008] Figure 4 A block diagram of one implementation of a computing system is shown.

[0009] Figure 5 A block diagram of one implementation of a computing system is shown.

[0010] Figure 6 A block diagram of one implementation of a computing system is shown.

[0011] Figure 7 A sequence diagram illustrating one implementation of the interaction between the OS, agent module, and kernel device module.

[0012] Figure 8 A sequence diagram illustrating one implementation of interactions when loading a new kernel device module.

[0013] Fig. 9A sequence diagram illustrating one implementation of interactions when loading a new kernel device module.

[0014] Fig.10 A sequence diagram illustrating one implementation of quiescing an existing kernel device module version.

[0015] Fig.11 is a generalized flow chart illustrating one implementation of a method for loading a kernel device module.

[0016] Fig.12 is a generalized flow chart illustrating one implementation of a method for quiescing a first version of a device driver to transfer control of a device from the first version of the device driver to a second version of the device driver. DETAILED DESCRIPTION

[0017] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, it will be appreciated by those of ordinary skill in the art that various implementations may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid confusion with the methods described herein. It should be understood that for simplicity and clarity of illustration, the elements shown in the accompanying drawings are not necessarily drawn to scale. For example, the size of some elements may be enlarged relative to other elements.

[0018] Various systems, devices and methods for performing real-time updates of kernel device modules are disclosed herein. In one implementation, a proxy module is loaded onto a system, wherein the proxy module provides registration and communication points for two different versions of kernel device modules. In one implementation, the proxy module and the kernel device module are loadable kernel modules. When control is transferred from the kernel device module of the first version to the kernel device module of the second version, the proxy module performs driver activities that cannot be copied or released when the loadable kernel modules of the two versions are activated simultaneously. In one implementation, the activities performed by the proxy module include, but are not limited to, device registration, input-output control (IOCTL) calls, sysfs calls and other activities.

[0019] When the first version of the kernel device module is loaded, it will register with the proxy module using the application programming interface (API) exported from the proxy module. The proxy module will then register with the operating system (OS), pretending to be the first version of the kernel device module. The proxy module will maintain all required open sysfs attribute files and IOCTL file attributes. When the second version of the kernel device module is loaded, the second version of the kernel device module will also register with the proxy module. Since the proxy module has already been registered with the OS, the proxy module does not need to repeat the registration step. Since each version of the kernel device module is registered with the proxy module, the proxy module knows how to find and communicate with each version of the kernel device module. The new version of the kernel device module will query the proxy module to find other versions so that the two versions can communicate directly. In one implementation, once the two versions of the kernel device modules are able to communicate directly with each other, the two versions manage the transfer of device control between them without further help from the proxy module.

[0020] Reference now Figure 1 , a block diagram of an implementation of a computing system 100 is shown. In one implementation, the computing system 100 includes at least processors 105A to 105N, an input / output (I / O) interface 120, a bus 125, one or more memory controllers 130, a network interface controller (NIC) 135, and one or more memory devices 140. In other implementations, the computing system 100 includes other components and / or the computing system 100 is arranged in a different manner. Processors 105A to 105N represent any number of processors included in the system 100. In one implementation, the processor 105A is a general-purpose processor, such as a central processing unit (CPU). In one implementation, the processor 105N is a data parallel processor with a highly parallel architecture. Data parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and the like. In some implementations, the processors 105A to 105N include multiple data parallel processors.

[0021] The one or more memory controllers 130 represent any number and type of memory controllers accessible by the processors 105A to 105N and an I / O device (not shown) coupled to the I / O interface 120. The one or more memory controllers 130 are coupled to any number and type of one or more memory devices 140. The one or more memory devices 140 represent any number and type of memory devices. For example, the type of memory in the memory device 140 includes dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), or other memory.

[0022] The I / O interface 120 represents any number and type of I / O interfaces (e.g., a peripheral component interconnect (PCI) bus, a PCI extension (PCI-X), a PCIE (PCI Express) bus, a Gigabit Ethernet (GBE) bus, a universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to the I / O interface 120. Such peripheral devices include (but are not limited to) a display, a keyboard, a mouse, a printer, a scanner, a joystick or other type of game controller, a media recording device, an external storage device, a network interface card, etc. The network interface controller (NIC) 135 receives and sends network messages across the network 145.

[0023] The network 145 represents any type of network or combination of networks, including wireless connections, direct local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), intranets, the Internet, wired networks, packet switching networks, fiber optic networks, routers, storage area networks, or other types of networks. Examples of LANs include Ethernet networks, fiber distributed data interface (FDDI) networks, and token ring networks. In various implementations, the network 145 also includes remote direct memory access (RDMA) hardware and / or software, transmission control protocol / Internet protocol (TCP / IP) hardware and / or software, routers, repeaters, switches, power grids, and / or other components.

[0024] In various implementations, computing system 100 is any of a computer, a laptop, a mobile device, a game console, a server, a streaming device, a wearable device, or various other types of computing systems or devices. It should be noted that the number of components of computing system 100 varies from implementation to implementation. For example, in other implementations, there are Figure 1 It should also be noted that in other implementations, the computing system 100 includes Figure 1 In addition, in other implementations, the computing system 100 is configured to Figure 1The different ways of structuring are shown in .

[0025] Now go to Figure 2 , a block diagram of one implementation of a virtual environment of a server 200 is shown. In one implementation, the server 200 includes clients 205A to 205N, virtual machines (VMs) 215A to 215N, a virtual machine monitor (VMM) 220, host hardware 230, memory 240, bus / fabric 250, and device 255. In one implementation, the server 200 is part of a cloud service provider environment. It should be understood that the server 200 is only an illustration of a server architecture used in one implementation. In other implementations, the server includes other numbers and / or types of components arranged in other suitable ways.

[0026] Clients 205A to 205N represent any number of clients that are coupled to server 200 and / or use resources on the server. Clients 205A to 205N include any number and type of applications 207 and 208, respectively. One of the applications within each client 205A to 205N may be, for example, or 205A to 205N. Each client 205A to 205N may also include a guest driver 210A to 210N, respectively, to interact with the device 255. In one implementation, a virtual machine (VM) 215A to 215N is instantiated on the server 200 for each client 205A to 205N. In one implementation, the VMM 220 schedules the VMs 215A to 215N, and the operating system inside each VM manages the scheduling of the applications 207 to 208 on the device 255. The host hardware 230 includes any number and type of processing resources. For example, in one implementation, the host hardware 230 includes multiple CPUs (e.g., multi-core CPUs). The host hardware 230 also includes memory controller logic, one or more cache subsystems, fabric and / or interconnect resources, network interface circuits, power management circuits, voltage regulators, and the like. The memory 240 represents any number and type of memory devices included in the server 200. The bus / fabric 250 includes any type of interconnect logic for connecting the various components of the server 200. Although a single device 255 is shown connected to the bus / fabric 250, it should be understood that multiple other devices can also be connected to the bus / fabric 250. In one implementation, the device 255 is a GPU. In other implementations, the device 255 is any of a variety of other types of devices. In various implementations, the clients 205A to 205N may include applications that are executed at least in part using the computing resources of the device 255. The guest drivers 210A to 210N of the clients 205A to 205N using the device 255 are respectively designed to manage the interaction between the applications 207 and 208 and the resources of the device 255, respectively.

[0027] In order to manage the interaction between OS 270 and device 255, memory 240 stores the Nth version of kernel device module 260A, wherein kernel device module 260A is a device driver of device 255. When the Nth version of kernel device module 260A is loaded into memory 240 and installed on server 200, the Nth version of kernel device module 260A registers with agent module 265 instead of registering with operating system (OS) 270. In one implementation, OS 270 is based on In other implementations, OS 270 is various other types of OS (e.g., )

[0028] When the Nth version of the kernel device module 260A is loaded into the memory 240 and installed on the server 200, the Nth version of the kernel device module 260A registers with the agent module 265 instead of registering with the operating system (OS) 270. The Nth version of the kernel device module 260A does not know whether the previous version of the device driver has been installed on the server 200 of the device 255 and is running. The agent module 265 returns an indication during the registration process to notify the Nth version of the kernel device module 260A whether it is the first device driver or the previous version of the device driver already exists. During the life cycle of the server 200, it may be expected to update the device driver of the device 255 to add new functions, fix errors and / or make other types of changes. However, it is also expected that the update of the device driver causes minimal interference with the operation of the client 205A to 205N. The (N+1)th version of the kernel device module 260B represents a newer version of the device driver for the device 255, which is intended to replace the Nth version of the kernel device driver 260A. For example, in one implementation, the (N+1)th version of the kernel device module 260B uses one or more data structures that are different from the Nth version of the kernel device driver 260A.

[0029] When the (N+1)th version of the kernel device module 260B is loaded into the memory 240 and installed on the server 200, the (N+1)th version of the kernel device module 260B registers with the agent module 265 instead of registering with the operating system (OS) 270. When the (N+1)th version of the kernel device module 260B registers with the agent module 265, the (N+1)th version of the kernel device module 260B does not know whether the previous version of the device driver has been installed on the server 200 of the device 255 and is running. In the case where the previous version (Nth version) is already running, the agent module 265 returns an indication during the registration process to notify the (N+1)th version of the kernel device module 260B that the existing device driver is already running. The agent module 265 also provides the (N+1)th version of the kernel device module 260B with a pointer to the Nth version of the kernel device module 260A. This allows the (N+1)th version of the kernel device module 260B to communicate with the Nth version of the kernel device module 260A to manage the transfer of control to the (N+1)th version of the kernel device module 260B.

[0030] In some scenarios, this transfer of control from the Nth version of the kernel device module 260A to the (N+1)th version of the kernel device module 260B must occur without the clients 205A to 205N being aware of the transfer. As part of the process of transferring control between the modules 260A to 260B, static data associated with the device 255 is copied from the Nth version of the kernel device module 260A to the (N+1)th version of the kernel device module 260B. While the static data is being copied, the Nth version of the kernel device module 260A continues to manage the device 255, and the clients 205A to 205N continue to run and use the device 255 without any interruption. In one implementation, the static data includes a plurality of VFs, the size of the frame buffer, data from a configuration file, a video basic input output system (BIOS) image, and / or other data. After the static data is copied, the Nth version of the kernel device module 260A is silenced. In one implementation, quiescing the Nth version of the kernel device module 260A involves stopping all input from entering the Nth version of the kernel device module 260A. However, stopping all input from entering the Nth version of the kernel device module 260A does not necessarily interrupt the clients 205A to 205N. For example, in one implementation, the device 255 is a GPU, and when the quiescing operation is initiated, the given client 205 may have a rendering operation running on the device 255. While the Nth version of the kernel device module 260A is being quiesced, the rendering operation can continue to run on the device 255 without interruption. Only when the given client 205 is waiting for a response from the Nth version of the kernel device module 260A will the given client 205 experience a small delay. If an external stimulus to the Nth version of the kernel device module 260A occurs during the time when the Nth version is quiesced and the (N+1)th version of the kernel device module 260B has not yet been started, a small delay may be caused to the system. After the Nth version of kernel device module 260A is quiesced, transient data associated with device 255 is transferred from the Nth version of kernel device module 260A to the (N+1)th version of kernel device module 260B. The (N+1)th version of kernel device module 260B then takes over as the new device driver for device 255. Clients 205A to 205N will not notice this change in the device driver for device 255.

[0031] Reference now Figure 3, a block diagram of one implementation of a computing system 300 is shown. In one implementation, a kernel device module 305A registers with an agent module 315A to serve as a device driver for a given device (not shown). In one implementation, the given device is a GPU. In other implementations, the given device is any of a variety of other types of devices. It should be noted that the terms "kernel device module" and "device driver" are used interchangeably herein.

[0032] VM 320A, 330A and 340A represent any number of VMs that utilize the processing resources of a given device. VM 320A, 330A and 340A correspond to virtual functions (VFs) 325A, 335A and 345A, respectively, that run on a given device in turn. In another implementation, various clients that are not VFs can run on a given device in turn. After the kernel device module 305A registers with the agent module 315A, the agent module 315A registers with the operating system (OS) (not shown), pretending to be the kernel device module 305A. In other words, the agent module 315A identifies itself as the kernel device module 305A of the OS so that the kernel device module 305A is registered as the device driver of the given device. Then, after the agent module 315A registers with the OS for the kernel device module 305A, the kernel device module 305A can begin to manage the given device. In one implementation, managing a given device involves processing IOCTL and sysfs calls, processing interrupts (e.g., mailbox interrupts, world switch interrupts), and processing debug requests. In other implementations, managing a given device involves other types of actions.

[0033] Now go to Figure 4 , a block diagram of one implementation of a computing system 400 is shown. Computing system 400 is intended to represent a computing system at a later point in time ( Figure 3 Thus, kernel device module 305B and agent module 315B are intended to represent kernel device module 305A and agent module 315A, respectively, at a later point in time. Figure 4 At the time point shown, the kernel device module 310B has been loaded onto the system 400. Loading onto the system 400 means that the kernel device module 310B has been stored in the memory of the system 400, and the execution of the kernel device module 310B has been initiated on one or more processors of the system 400.

[0034] For the purpose of this discussion, it is assumed that kernel device module 310B is a subsequent version of kernel device module 305B. For example, if kernel device module 305B is a device driver of the first version, kernel device module 310B is a device driver of the second or later version (or an earlier version in the case of downgrade). Device drivers are usually updated from time to time to merge new functions, fix errors, provide security updates and / or implement other changes. Therefore, it is expected that during the entire life cycle of system 400, the original device driver of a given device will not be the only device driver of a given device. Therefore, the update to kernel device module 305B will be loaded onto system 400 at various times. Kernel device module 310B is intended to represent a complete replacement of kernel device module 305B. When performing a complete replacement of kernel device module 305B, the transfer of control from kernel device module 305B to kernel device module 310B should not interrupt VM 320B, 330B and 340B. Or, in other words, it is desirable to minimize disruption to VMs 320B, 330B, and 340B during the transfer of control from kernel device module 305B to kernel device module 310B. Figure 4 As shown, VMs 320B, 330B, and 340B correspond to VFs 325B, 335B, and 345B, respectively, which in turn run on a given device. In another implementation, various clients that are not VFs may run on a given device.

[0035] As used herein, the term "kernel device module" is defined as a device driver that handles the docking between a given device of system 400 and the OS or other software application of system 400. In one implementation, the given device is a GPU. In other implementations, the given device is any of a variety of other types of devices. For the purposes of this discussion, it is assumed that both kernel device module 310B and kernel device module 310A are designed to dock with the same device of system 400. Therefore, if both kernel device module 310B and kernel device module 310A simultaneously attempt to handle docking with the same given device, this will result in unpredictable behavior of system 400. Therefore, kernel device module 305B and kernel device module 310B will coordinate access to a given device to manage the transfer of control.

[0036] When the kernel device module 310B is loaded onto the system 400, the kernel device module 310B registers with the proxy module 315B instead of registering with the OS. The OS has already registered an existing driver for a given device, and therefore the OS will not allow the kernel device module 310B to register the same driver name for this given device. The existing registration is performed by the proxy module 315B on behalf of the kernel device module 305B. Therefore, since the kernel device module 310B cannot register with the OS to become the device driver for a given device, the kernel device module 310B registers with the proxy module 315B. In response to receiving the registration attempt, the proxy module 315B will notify the kernel device module 310B that another kernel device module (that is, the kernel device module 305B) has been registered as the device driver for the given device. The proxy module 315B also provides a pointer to the kernel device module 310B that allows the kernel device module 310B to communicate with the kernel device module 305B.

[0037] In this implementation, the kernel device module 310B uses a pointer to communicate with the kernel device module 305B to facilitate the transfer of control of a given device. In the first step, the kernel device module 310B requests any static data associated with the given device. In response to the request, the kernel device module 305B sends the static data associated with the given device to the kernel device module 310B. In one implementation, the static data includes multiple VFs, the size of the frame buffer, data from a configuration file, a video basic input and output system (BIOS) image, and / or other data. When these events occur, the kernel device module 305B continues to act as a device driver for the given device. By continuing to act as a device driver for the given device, the kernel device module 305B continues to process IOCTLs and sysfs calls for the given device. The kernel device module 305B also continues to process mailbox interrupts, world switch interrupts, and debug signals for the given device.

[0038] Reference now Figure 5 , a block diagram of one implementation of a computing system 500 is shown. Figure 5 express Figure 4 In other words, computing system 500 is intended to represent computing system 400 at a later point in time. Thus, kernel device module 305C, kernel device module 310C, and agent module 315C are intended to represent, respectively, ( Figure 4 The kernel device module 305B, the kernel device module 310B and the agent module 315B are also represented by VMs 320C, 330C and 340C and VFs 325C, 335C and 345C at a later time point. Figure 4 )VMs 320B, 330B and 340B and VFs 325B, 335B and 345B.

[0039] exist Figure 5 At the time point shown, the processing of mailbox interrupts, world switch interrupts and debug signals by kernel device module 305C has been suspended. Any interrupt or debug signal generated for a given device during this time and sent to kernel device module 305C will be queued. However, kernel device module 305C will continue to process IOCTL and sysfs calls at this time point. In this implementation, if one of VM 320B, 330B or 340B happens to be utilizing a given device, they can continue to do so. Any interrupt or operation sent to VM can continue to be processed unhindered. Kernel device module 305C and kernel device module 310C will communicate with each other to coordinate the suspension of the processing of interrupts and debug signals and the silent timing of kernel device module 305C. Once kernel device module 305C has been silent, kernel device module 305C sends transient data to kernel device module 310C. In one implementation, the transient data includes the currently running VF, the status of the currently running VF, the order of the VF on the run list, event buckets, cumulative data, and / or other data. During the period of time when the kernel device module 305C has been quiesced, any interrupts or operations sent to the VM can continue to be processed unimpeded.

[0040] Now go to Figure 6 , a block diagram of one implementation of a computing system 600 is shown. Computing system 600 is intended to represent computing system 500 at a later point in time. Thus, kernel device module 305D, kernel device module 310D, and agent module 315D are intended to represent kernel device module 305C, kernel device module 310C, and agent module 315C, respectively, at a later point in time. Also, VMs 320D, 330D, and 340D and VFs 325D, 335D, and VF 345D represent VMs 320C, 330C, and 340C and VFs 325C, 335C, and VF 345C, respectively, at a later point in time.

[0041] exist Figure 6 At the time point shown, the processing of mailbox interrupts, world switch interrupts, and debug signals has been restarted by the core device module 310D. Any interrupts or debug signals generated during the time between the core device 305D stopping processing interrupts and debug signals and the core device 310D starting processing interrupts and debug signals will be processed out of the corresponding queue or status register where they were recorded. Figure 6At the time indicated, kernel device module 310D will begin processing IOCTLs and sysfs calls. Kernel device module 310D and kernel device module 305D communicate with each other to coordinate the transfer of control for handling the processing of IOCTLs and sysfs calls.

[0042] Reference now Figure 7 , a sequence diagram of one implementation of the interaction between the OS, the proxy module, and the kernel device module is shown. In one implementation, when a modprobe call is made to the kernel device module 706, the OS 702 resolves the dependencies of the kernel device module 706. Depending on the implementation, the modprobe is performed manually or the modprobe is automated as part of the startup script. When the OS 702 determines that the kernel device module 706 depends on the proxy module 704, the OS 702 will load and initialize the proxy module 704 in step 710. The proxy module 704 performs internal initialization in step 712 to prepare for loading the kernel device module 706 in step 714. Then, after loading the kernel device module 706 in step 714, the kernel device module 706 calls the function "register_driver" in step 716 to register with the proxy module 704. For the purpose of this discussion, it is assumed that the kernel device module 706 is the initial device driver to be loaded for a specific device of the system. Therefore, since kernel device module 706 is the initial device driver to be loaded for the particular device, in step 718, agent module 704 marks its internal table associated with this instance of kernel device module 706 as "active" and performs the first initialization of the hardware for the particular device.

[0043] The kernel device module 706 will not register as a device driver with the OS 702, but will register as a device driver with the proxy module 704. The proxy module 704 will act as an alternative OS for the kernel device module 706. The bus (e.g., peripheral component interconnect (PCI)) device identifier (ID) is transparent to the proxy module 704. The proxy module 704 obtains the device ID from the kernel device module 706. This adds another level of independence between the proxy module 704 of each version and the kernel device module 706 of each version. The proxy module 704 will use the device ID provided by the kernel device module 706 to register its own driver with the OS 702 on behalf of the kernel device module 706. The kernel device module 706 passes a function pointer table to the proxy module 704, and the proxy module 704 will use these function pointers as callbacks to the kernel device module 706. For security reasons, the proxy module 704 makes a copy of the function pointer table. This can prevent the kernel device module 706 from intentionally maliciously or accidentally (e.g., due to an error) changing the table.

[0044] In loop 721, OS 702 searches for devices that match a given device ID in step 722, and calls the .probe function of proxy module 704 for each matching device ID. OS 702 will perform this operation for all devices that match the device ID in the device ID table of kernel device module 706. After receiving the call to the .probe function, proxy module 704 will not perform any hardware device initialization. Instead, device initialization will be performed by kernel device module 706. Proxy module 704 will call the .probe callback function for each matching device ID of kernel device module 706. In one implementation, proxy module 704 calls the .probe function of kernel device module 706 with additional parameters in addition to OS parameters. The additional parameters indicate whether the load is the initial load of the driver or the replacement of the previous driver. For the implementation shown in sequence diagram 700, proxy module 704 will use the additional parameters to notify kernel device module 706 that this is the initial load of the driver. In response to receiving the notification that this is the initial load of the driver, kernel device module 706 will perform complete hardware initialization.

[0045] Once all devices have been detected and initialized, the kernel device module 706 will enter its normal operating mode. During runtime, examples of stimulations that can affect the state of the device and the kernel device module 706 can include mailbox interrupts, world switch timer interrupts, and notifications from the hardware emulator. The guest driver in the guest OS will use the mailbox interrupt to generate notifications for the kernel device module 706. When the kernel device module 706 receives the mailbox interrupt, the kernel device module 706 will schedule the work items on the mailbox work queue for later processing. When OS 702 is available, OS 702 will perform the work on the work queue. In one implementation, the kernel device module 706 processes the world switch timer interrupt at programmable intervals. In one implementation, the programmable interval is in the range of 3 milliseconds (ms) to 5ms. At each interval, the kernel device module will schedule the world switch request in the world switch work queue.

[0046] In various implementations, the hardware emulator and the kernel device module 706 communicate with each other through IOCTL calls (hardware emulator→kernel device module 706) and sysfs calls (kernel device module 706→hardware emulator). In one implementation, the hardware emulator is a fast emulator (QEMU). When the hardware emulator needs the kernel device module 706 to handle some work, the hardware emulator will turn on the IOCTL attribute, send a request, and then turn off the attribute. When the kernel device module 706 returns from the IOCTL call, the work is completed. In response to the call on the caller's thread, the work is done directly in the kernel device module 706. Just like the case of world switching and mailbox interrupts, there is no work queue involved. When the kernel device module 706 has work for the hardware emulator to do, the kernel device module 706 will place a command in the buffer and send a signal to notify the hardware emulator instance of the attribute it is waiting for. When the hardware emulator has completed the work, the hardware emulator will read the buffer and respond with an IOCTL call.

[0047] The kernel device module 706 will remain in loop 731 to detect and process stimuli until the kernel device module 706 is replaced or uninstalled. In one implementation, during normal operation of the kernel device module 706, the kernel device module 706 operates independently of the agent module 704. In other words, in this implementation, the agent module 704 is not involved in the single root input / output virtualization (SRIOV) operation of the device. In one implementation, the agent module 704 is responsible for updating or replacing the kernel device module 706. In this implementation, if the kernel device module 706 attempts to uninstall from the system, or if a newer kernel device module attempts to replace the kernel device module 706, the agent module 704 is activated.

[0048] Now go to Figure 8 , a sequence diagram showing one implementation of the interaction when loading a new kernel device module 810. When the new kernel device module 810 is loaded onto the system, the new kernel device module 810 will register with the proxy module 805 instead of registering with the OS. The new kernel device module version 810 does not know that the existing kernel device module 815 is already running. The new kernel device module version 810 will attempt to register with the proxy module 805 in the same manner as registering with the proxy module 805 by calling the register_driver() given function to register with the proxy module 805 and registering with the proxy module 805 the existing kernel device module version 815. In step 820, the new kernel device module version 810 passes both the function pointer table and the device ID table to the proxy module 805.

[0049] In one implementation, the loading of new kernel device module 810 will trigger the real-time kernel device module update process, and further user interaction is not required. In one implementation, remove existing kernel device module version 815 when successful, or remove new kernel device module version 810 if error is detected. A goal of real-time kernel device module version update is to be able to update kernel device module executable code when the client session running is not interrupted. This includes the forward and backward update of code. In order to be able to carry out complete update, it is expected that all real-time client sessions existing before the update will also be in active state after the update. If new kernel device module 810 supports the same device as the device supported by existing kernel device module 815, then new kernel device module 810 is registered as the new device driver of these devices. Alternatively, if new kernel device module 810 can not support the same device as the device supported by existing kernel device module 815, then stop updating, and record error description.

[0050] Reference now Fig. 9 , shows a sequence diagram of one implementation of the interactions when loading a new kernel device module 810. Fig. 9 Intent to continue Figure 8 When the control of the device is transferred from the existing kernel device module 815 to the new kernel device module 810, static data is sent from the existing kernel device module 815 to the new kernel device module 810. After the static data is transmitted, the existing kernel device module 815 is silenced so that transient data is transmitted from the existing kernel device module 815 to the new kernel device module 810.

[0051] In one implementation, in order to transmit sensitive transient data when transitioning to a new device driver, the state of the device should be stable. In this implementation, the device can still run, but its state known to the kernel device module cannot be changed. In one implementation, stimuli that can change the state of the software and / or device are identified. In one implementation, these stimuli include mailbox interrupts, world switch interrupts, and notifications from hardware emulators. In other implementations, other stimuli are identified and stopped to prevent device state changes. In box 910, the existing kernel device module 815 is silenced by stopping these identified stimuli to prevent any state changes from occurring during the transmission of sensitive transient data.

[0052] Now go to Fig.10, a sequence diagram of one implementation of quiescing an existing kernel device module version 1010 is shown. For the purposes of this discussion, it is assumed that a new kernel device module version 1005 takes over as a device driver for a given device by replacing an existing kernel device module version 1010. In one implementation, the given device is a GPU, while in other implementations, the given device is any of a variety of other devices (e.g., a network controller, a storage device). To replace the existing kernel device module version 1010, the existing kernel device module version 1010 will be quiesced. In one implementation, the quiescing of the existing kernel device module version 1010 is initiated by the new kernel device module version 1005, but it is the existing kernel device module version 1010 that performs various actions to achieve the quiesced state. In one implementation, the existing kernel device module version 1010 is quiesced by stopping multiple threads executing on the host computing system. In one implementation, the stopped threads include the scheduler, mailbox, and IOCTL threads. In one implementation, each thread has an associated flag (for each adapter) that indicates whether the thread can accept new work.

[0053] In one implementation, the first thread to be stopped is the scheduler thread in step 1015. Setting the scheduler thread's flag will indicate to the world switch scheduler in the kernel device module that the scheduler thread can no longer initiate new scheduler requests (i.e., prevent further world switches from occurring). The scheduler thread is stopped first because the mailbox thread may attempt to generate a request to restart the scheduler. Note that the scheduler thread can still continue to process in-progress requests, but once the scheduler thread has been stopped, the scheduler thread cannot initiate new requests. If a call is made to restart the scheduler thread, the call will be ignored.

[0054] In one implementation, the scheduler thread is stopped in step 1015 before the mailbox thread is stopped in step 1020. When the guest driver maintains the exclusive access mode, the guest driver will also maintain the world switch mutex. The world switch mutex prevents the world from switching to any other virtual function (VF). When the guest driver releases the exclusive mode (via a mailbox message) or times out, the world switch mutex will be released. The mailbox thread will prioritize the request to release the exclusive mode over the request to grant the exclusive mode. If the mailbox thread is running and there is a request to release the exclusive mode, the request will be in progress. Before waiting for the scheduler thread to stop, the mailbox thread is allowed to complete the release of the exclusive mode. Note that if the exclusive mode is maintained and there is no pending exclusive mode release on the work queue, the scheduler thread can be stopped during the transfer of the transient state and the mutex state can be transferred to the kernel device module version 1005. In other words, the real-time update can complete the real-time update without waiting for the exclusive mode to complete.

[0055] The mailbox thread is stopped in step 1020 to prevent the mailbox thread from accepting any new work. New work for the mailbox thread results from a request by the guest driver to enter or exit exclusive mode. In one implementation, the mailbox interrupt is disabled instead of stopping the guest driver. The guest driver can still attempt to trigger the mailbox interrupt, but the mailbox interrupt will not be processed. The interrupt request will be saved in the mailbox status register and can be polled later to determine whether any work requests were generated while the interrupt was disabled. The request to stop the mailbox thread in step 1020 may occur while the mailbox thread is processing work items. There may also be work items queued for the mailbox thread. Setting the flag does not indicate that the mailbox thread will be idle. The next step is to wait for the mailbox thread to stop in step 1025.

[0056] After requesting to stop the mailbox thread in step 1020, the work item in progress will be processed to completion, but no new work item will be started. Any work item on the queue will be transferred to the new kernel device module version 1005 to be processed during the transmission of transient data. In one implementation, at step 1025, the existing kernel device module version 1010 waits for the mailbox thread to stop, and the scheduler thread stops, and then in step 1035, the IOCTL thread is stopped. Usually, the mailbox work item will also need a response from the VMM. In order to receive this response without deadlock, the IOCTL thread is allowed to continue running until the mailbox thread stops in step 1025. After requesting the IOCTL thread to stop in step 1035, the existing kernel device module version 1010 waits for the IOCTL thread to stop in step 1040. The existing kernel device module version 1010 waits until any unfinished work is completed, and then the successful response is returned to the new kernel device module version 1005. When any outstanding work has been completed, transient data is transferred from the existing kernel device module version 1010 to the new kernel device module version 1005, and control of the device is then passed to the new kernel device module version 1005.

[0057] Now go to Fig.11 , shows an implementation of a method 1100 for loading a kernel device module. For discussion purposes, the steps and Fig.12 However, it should be noted that in various implementations of the described method, one or more of the described elements are performed simultaneously, in a different order than shown, or omitted entirely. Other additional elements are also performed when necessary. Any of the various systems or devices described herein is configured to implement method 1100.

[0058] A given version of a device driver is registered with an agent module, wherein the given version of the device driver is configured to manage the interaction between a software application and a given device (box 1105). In one implementation, the device is a GPU. In other implementations, the device is any of various other types of processing or peripheral devices. Next, the agent module determines whether a previous version of the device driver has been loaded on the system of the given device and is being executed (conditional box 1110). If the previous version of the device driver has been loaded on the system and is being executed (conditional box 1110, "yes" branch), the agent module provides a first indication to the given version of the device driver, wherein the first indication specifies that the previous version of the device driver has been loaded (box 1115). The agent module also provides a pointer to the previous version of the device driver to the given version of the device driver (box 1120). In response to receiving the first indication, the given version of the device driver abandons the hardware initialization of the device (box 1125). The given version of the device driver uses the pointer to communicate with the previous version of the device driver to handle the transfer of control of the device from the previous version of the device driver to the given version of the device driver (box 1130). After control of the device has been transferred from the previous version of the device driver to the given version of the device driver, the previous version of the device driver is deactivated (block 1135). After block 1135, method 1100 ends.

[0059] If the previous version of the device driver has not been loaded onto the system and is being executed (conditional block 1110, "no" branch), the agent module provides a second indication to the given version of the device driver, wherein the second indication is different from the first indication (block 1140). Moreover, the agent module registers with the OS as the given version of the device driver (block 1145). In other words, the agent module emulates the given version of the device driver to the OS. Also in response to receiving the second indication, the given version of the device driver causes the device to execute a hardware initialization routine (block 1150). Also in response to receiving the second indication, the given version of the device driver begins managing the device (block 1155). After block 1155, method 1100 ends. It should be noted that in various implementations, method 1100 is performed for each new version of the device driver installed on the system.

[0060] Reference now Fig.12, shows an implementation of a method 1200 for quiescing a first version of a device driver in order to transfer control of a device from the first version of the device driver to a second version of the device driver. The second version of the device driver receives an indication that the first version of the device driver is already managing a given device of the computing system (block 1205). In one implementation, the second version of the device driver receives the indication in response to registering with the proxy module. In response to receiving the indication, a request to stop a scheduler thread is generated (block 1210). In one implementation, the scheduler thread is stopped by setting a corresponding flag that indicates to the guest driver that the scheduler thread can no longer initiate new scheduler requests. Next, a request to stop a mailbox thread is generated (block 1215). The request to stop the mailbox thread is generated in block 1215 to prevent the mailbox thread from accepting any new work. Then, the first version of the device driver waits for the mailbox thread to stop (block 1220). Next, the first version of the device driver waits for the scheduler thread to stop (block 1225). Then, a request to stop the IOCTL thread is generated (block 1230). Next, the system waits for the IOCTL thread to stop (block 1235). Then, the transient data is transferred from the first version of the device driver to the second version of the device driver (block 1240). Next, control of the device is transferred from the first version of the device driver to the second version of the device driver (block 1245). After block 1245, the method 1200 ends.

[0061] In various implementations, the method and / or mechanism described herein is implemented using program instructions of a software application. For example, program instructions that can be executed by a general or special processor are contemplated. In various implementations, such program instructions are represented by a high-level programming language. In other implementations, program instructions are compiled from a high-level programming language into a binary form, an intermediate form, or other forms. Alternatively, program instructions describing the behavior or design of hardware are written. Such program instructions are represented by high-level programming languages ​​such as C. Alternatively, hardware design languages ​​(HDLs) such as Verilog are used. In various implementations, program instructions are stored on any of a variety of non-temporary computer-readable storage media. During use, a computing system can access a storage medium to provide program instructions to a computing system for program execution. Generally speaking, such a computing system includes at least one or more memories and one or more processors configured to execute program instructions.

[0062] It should be emphasized that the above implementations are only non-limiting examples of implementations. Once the above disclosure is fully understood, numerous changes and modifications will become apparent to those skilled in the art. The attached claims are intended to be interpreted as including all such changes and modifications.

Claims

1. A computer system comprising: a memory storing program instructions for an operating system (OS), an agent module, and a given version of a device driver configured to manage interactions between one or more software applications and a device; as well as A processor, the processor being configured to: registering the given version of the device driver with the proxy module instead of the operating system; and The program instructions of the proxy module are executed to: In response to a device driver of a previous version of the given version already existing on the system: providing a first indication from the agent module to the given version of the device driver, the first indication indicating that the previous version of the device driver has been loaded onto the system and is managing the device; as well as transferring control of the device from the previous version of the device driver to the given version of the device driver without performing hardware initialization of the device; In response to a previous version of the device driver not already existing on the system for the given version: providing a second indication different from the first indication to the given version of the device driver (1140); Registering, by the agent module, with the operating system as the device driver of the given version; In response to the second indication, initializing hardware of the device; and The device is managed by the given version of the device driver.

2. The system of claim 1, wherein the processor is further configured to: In response to the previous version of the given version of the device driver already existing on the system: The proxy module provides the given version of the device driver with a pointer to the previous version of the device driver; and The given version of the device driver requests status information from the previous version of the device driver via the pointer.

3. The system of claim 2, wherein the processor is further configured to: generating one or more requests to stop one or more external threads; Waiting for the one or more external threads to stop; and In response to the one or more external threads being stopped, control of the device is transferred from the previous version of the device driver to the given version of the device driver.

4. The system of claim 2, wherein the processor is further configured to: transferring state information from the previous version of the device driver to the given version of the device driver; transferring control of the device from the previous version of the device driver to the given version of the device driver; and The previous version of the device driver is disabled.

5. The system of claim 4, wherein the processor is configured to execute the agent module to notify the given version of the device driver to skip hardware initialization of the device in response to determining that the previous version of the device driver is loaded on the system and is managing the device.

6. The system of claim 4, wherein the processor is further configured to: transferring static data from the previous version of the device driver to the given version of the device driver; quiescing the previous version of the device driver; and After quiescing the previous version of the device driver, transient data is transferred from the previous version of the device driver to the given version of the device driver.

7. The system of claim 1, wherein the processor is further configured to: providing a first indication from the agent module to the given version of the device driver if no other device driver is currently managing the device; and If another device driver is currently managing the device, providing a second indication from the agent module to the given version of the device driver, wherein the second indication is different from the first indication.

8. A method for real-time updating of a kernel device module, comprising: executing program instructions of a given version of a device driver to register the given version of the device driver with an agent module instead of an operating system, wherein the given version of the device driver is configured to manage interactions between a device and one or more software applications; registering the given version of the device driver with the agent module; The program instructions of the proxy module are executed to: In response to a device driver of a previous version of the given version already existing on the system: providing a first indication from the agent module to the given version of the device driver, the first indication indicating that the previous version of the device driver has been loaded onto the system and is managing the device; as well as transferring control of the device from the previous version of the device driver to the given version of the device driver without performing hardware initialization of the device; In response to a previous version of the device driver not already existing on the system for the given version: providing a second indication different from the first indication to the given version of the device driver; Registering, by the agent module, with the operating system as the device driver of the given version; Initializing the hardware of the device; and The device is managed by the given version of the device driver.

9. The method of claim 8, further comprising: In response to the previous version of the given version of the device driver already existing on the system: The proxy module provides the given version of the device driver with a pointer to the previous version of the device driver; and The given version of the device driver requests status information from the previous version of the device driver via the pointer.

10. The method of claim 9, further comprising: generating one or more requests to stop one or more external threads; Waiting for the one or more external threads to stop; as well as In response to the one or more external threads being stopped, control of the device is transferred from the previous version of the device driver to the given version of the device driver.

11. The method of claim 9, further comprising: transferring state information from the previous version of the device driver to the given version of the device driver; transferring control of the device from the previous version of the device driver to the given version of the device driver; as well as The previous version of the device driver is disabled.

12. The method of claim 11, further comprising: In response to determining that the previous version of the device driver is loaded on the system and is managing the device, the agent module is executed to notify the given version of the device driver to skip hardware initialization of the device.

13. The method of claim 11, further comprising: transferring static data from the previous version of the device driver to the given version of the device driver; quiescing the previous version of the device driver; as well as After quiescing the previous version of the device driver, transient data is transferred from the previous version of the device driver to the given version of the device driver.

14. The method of claim 8, wherein the processor is further configured to: providing a first indication from the agent module to the given version of the device driver if no other device driver is currently managing the device; and If another device driver is currently managing the device, providing a second indication from the agent module to the given version of the device driver, wherein the second indication is different from the first indication.

15. A non-transitory computer-readable storage medium comprising program instructions executable by a processor to perform the method of any one of claims 8-14.

Images