Lifecycle management of in-vehicle hosted machines

Abstract

The invention relates to lifecycle management of in-vehicle hosted machines. Some embodiments provide a novel approach for efficiently updating software of a virtual machine (VM) executing on a device in a vehicle such that updating of the software does not interrupt operations performed by the VM. While the vehicle is operating, the method collects configuration state information for a first VM executing on a device and performing a set of operations. The method uses the configuration state information to instantiate a second VM in an isolated network environment isolated from the first VM. The second VM is the same as the first VM at a specific time in which the configuration state information is collected. The method updates the second VM and detects a calculated downtime of the vehicle. During the compute downtime, the method replaces the first VM with the updated second VM on the device such that once the compute downtime ends, the updated second VM performs the set of operations.

Description

[0001] CROSS REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 529,139, filed on July 26, 2023, and entitled “LIFECYCLE MANAGEMENT OF IN-VEHICLE HOSTED MACHINES,” the entire teachings of which are hereby incorporated by reference for all purposes. Technical Field

[0003] The present disclosure relates to lifecycle management of onboard managed machines. Background Art

[0004] As vehicles modernize and implement advanced computing application platforms (e.g., ESXi provided by VMware Inc.) into embedded in-vehicle computing hardware systems, in some embodiments, applications increase due to the integration opportunities provided by the in-vehicle hypervisor. However, as new virtual machines (VMs) become part of the critical functions of commercial vehicles (e.g., first responder vehicles), the need to update, protect, and manage VMs that are critical to the use of the vehicle and its occupants arises. For example, in some embodiments, a police car with such an in-vehicle system executes multiple virtual machines that perform various functions such as (1) license and registration verification, (2) video streaming from in-vehicle cameras and body cameras, (3) forensic data collection (e.g., collection of phone images of crime victims and associated analysis), (4) logging and safety functions of the vehicle itself (along with logistical information conveyed to the in-vehicle operator about his or her immediate mission), and (5) data synchronization from in-vehicle sources to the cloud. Over time, many additional functions may emerge that are performed by the vehicle's VMs.

[0005] In a typical data center, some embodiments schedule updates and other generally intensive functions (e.g., anti-virus scanning activities) at any time that is convenient to the vehicle or data center operator. However, in such embodiments, there is minimal consideration for interruption of services provided by the vehicle's VMs. For in-vehicle systems, interruption of service of the VMs has greater significance and must be managed more carefully. Therefore, there is a need for methods and systems for updating VMs executing in a vehicle that minimize VM downtime. Summary of the invention

[0006] Some embodiments provide a novel method for efficiently updating software of a virtual machine (VM) executing on one or more devices in a vehicle so that the update to the software does not interrupt operations performed by the VM. While the vehicle is operating, the method collects configuration state information of a first VM that is executing on a specific device and performing a specific set of operations. The method uses the configuration state information to instantiate a second VM in an isolated network environment isolated from the first VM. The second VM is the same as the first VM at the specific time when the configuration state information was collected. The method updates the second VM. The method detects computing downtime of the vehicle. During the computing downtime, the method replaces the first VM on the device with the updated second VM so that once the computing downtime ends, the updated second VM performs the set of operations.

[0007] In some embodiments, the method is performed by a management agent (MA) executed on a specific device. The MA is an application, program, or module executed on the specific device. In some embodiments, the isolated network environment is an isolated network, so that when a second VM is instantiated in the isolated network, the second VM does not interfere with the set of specific operations performed by the first VM. In various embodiments, the isolated environment is referred to as an isolated network, a sandbox, and a staging area, and allows the creation and updating of the second VM without interrupting the first VM and the operations performed by it. In some embodiments, the specific device is a computer.

[0008] In some embodiments, the isolated network environment is an isolated network associated with a dedicated virtual local area network (VLAN) such that when a second VM is instantiated in the isolated network and prior to computing downtime of the vehicle, the second VM does not interfere with the specific set of operations performed by the first VM. In such embodiments, the isolated network is a dedicated network segment within a virtual switch of a hypervisor of a specific device using a dedicated VLAN. In some embodiments, the dedicated VLAN is associated with a unique port group to ensure that traffic sent to and from the second VM does not conflict with traffic sent to and from the first VM in a different VLAN because they have the same Internet Protocol (IP) address.

[0009] The MA of the specific device performs collection, use, update, and detection after receiving an instruction to update the first VM from a management plane (MP), and the MP is implemented by a set of one or more management servers connected to the vehicle through a software-defined wide area network (SD-WAN). In such embodiments, the MP guides the MA to update the first VM, and the MA performs collection, use, update, and detection in order to update the first VM.

[0010] In some embodiments, the set of managers implementing the MP operates at a different location than the vehicle. For example, in some embodiments, the vehicle moves around in different locations, and the set of management servers executes at an edge site of the SD-WAN. Alternatively, the set of management servers executes at a data center site of the SD-WAN. Additionally, in other embodiments, the set of management servers executes at two or more sites of the SD-WAN in a distributed manner.

[0011] In some embodiments, the vehicle includes an edge router connected to the SD-WAN using a set of one or more wireless network connection links. The edge router of some embodiments is one of: an edge router device, an edge router executed on a computer operating in the vehicle, or an edge router executed on a machine executing on the computer. In some embodiments, the wireless network connection link is provided by one or more different telecommunication link providers (e.g., different cellular providers).

[0012] In some embodiments, the MA updates the second VM by instructing the hypervisor of the specific device to instruct the update agent associated with the second VM to download a set of one or more update files associated with a specific update of the first VM and update the second VM. In such embodiments, each VM instantiated on the specific device and each copy of each VM executes an update agent, which performs a software update of its corresponding VM. In some embodiments, the update agent downloads the set of update files from the MP using one or more of the set of wireless network connection links. In other embodiments, the update agent downloads the set of update files from a set of one or more other locations (e.g., an external repository) specified by the MP using the set of wireless network connection links in some embodiments.

[0013] In some embodiments, the configuration state information includes an operating system (OS) disk of the first VM specifying one or more OS files and application software associated with the first VM at a specific time when the configuration state information is collected. In such embodiments, the OS disk is copied at a specific time to freeze the state of the first VM and create a second VM using the OS disk. In some embodiments, the second VM is referred to as a replica of the configuration file of the first VM. In some embodiments, the configuration file of the first VM and the copy of the configuration file of the first VM are referred to as VMX files. In some embodiments, after copying the OS disk, the first VM writes subsequent OS disk operations to a log.

[0014] In some embodiments, the MA collects configuration state information by directing the hypervisor of a specific device to collect the OS disk at a specific time. In such embodiments, the MA receives a notification from the MP to update the first VM, which causes the MA to direct the hypervisor to copy the OS disk of the first VM. Then, the MA uses the configuration state information to instantiate the second VM by directing the hypervisor to use the OS disk to instantiate the second VM. In such embodiments, the OS disk is copied to create the second VM, but the independent data disk is not copied. In some embodiments, the data disk specifies application data and does not specify software. In some embodiments, the independent data disk is a data disk marked as independent, meaning that it will not be copied by the hypervisor.

[0015] Instead, when the first VM is to be replaced with the second VM, the hypervisor will instead transfer the first VM's independent data disks to the second VM. In some embodiments, this is referred to as a zero-copy transfer, which enables the MA to update the first VM more quickly than if its boot hypervisor also copied the first VM's data disks. Performing a zero-copy transfer of the first VM's data disks to the second VM is very fast because none of this data is copied within the hypervisor system, and because no heavy storage operations are required to detach the data disks from the first VM and reattach them to the second VM.

[0016] In some embodiments, the MA directs the hypervisor to replace the first VM with the updated second VM by directing the hypervisor to: (1) detach a set of one or more data disks containing application data from the first VM; and (2) attach the set of data disks to the updated second VM. This transfers the set of data disks required to replace the first VM with the updated second VM, rather than copying them and adding the copies to the second VM. In some embodiments, the MA also directs the hypervisor to replace the first VM with the updated second VM by directing the hypervisor to: (1) archive the first VM to the hypervisor's data store; and (2) migrate the updated second VM from the isolated network environment to a production network environment with a production network port group, the production network environment enabling the updated second VM to perform the specific set of operations once the vehicle's computing downtime is over. The production network environment is associated with a VLAN used by other VMs of a particular device, enabling the second VM to communicate with these VMs and any other VMs executing on other devices.

[0017] In some embodiments, computing downtime is detected by detecting a power outage of the vehicle. In other embodiments, computing downtime is detected by detecting a power outage of the vehicle and the locking of the vehicle's doors. In some embodiments, the hypervisor of a particular device is configured to operate during a specific time period after the computing downtime is detected. In such embodiments, the hypervisor is operable during a specific time period using the vehicle's battery and is inoperable after the specific time period. By directing the hypervisor to replace the first VM with an updated second VM during the computing downtime, and by not performing collection, use, or updates during the computing downtime, the method uses the least battery of the vehicle and completes the VM replacement in the smallest amount of time possible. If the duration of the vehicle's computing downtime is short, this avoids affecting vehicle operation.

[0018] In some embodiments, the computing downtime ends at the beginning of the next vehicle computing cycle. In some embodiments, the next vehicle computing cycle is a power-on of the vehicle. Once the vehicle is undergoing the next computing cycle, the updated second VM is used to perform the set of specific operations so that it operates as if the first VM was directly updated.

[0019] The foregoing summary is intended to serve as a brief introduction to some embodiments of the present invention. It is not meant to be an introduction or overview of all the subject matter of the invention disclosed in this document. The following detailed description and the drawings referenced in the detailed description will further describe the embodiments described in the summary as well as other embodiments. Therefore, in order to understand all the embodiments described by this document, a comprehensive reading of the summary, the detailed description, the drawings and the claims is required. In addition, the claimed subject matter is not limited by the illustrative details in the summary, the detailed description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The novel features of the invention are set forth in the appended claims. However, for purposes of illustration, several embodiments of the invention are set forth in the following figures.

[0021] Figure 1 Conceptually illustrates a process of some embodiments for efficiently updating software of a VM executing on one or more devices in a vehicle such that the update to the software does not interrupt operations performed by the VM.

[0022] Figure 2 An example vehicle including components managed and configured by a set of managers and controllers is described.

[0023] Figure 3 An example illustrating a vehicle communicating with various components of an SD-WAN.

[0024] Figure 4AB illustrates an example hypervisor of a device that uses a file system, an isolated network, and an archival data store to update a VM without interrupting operations performed by the VM.

[0025] Figure 5 Three time periods are described that occur when the MA directs the hypervisor of the vehicle device to update the first VM by creating and updating the second VM.

[0026] Figure 6 Describes an instance host computer that executes a VM and updates the VM without disrupting its operation.

[0027] Figure 7 An electronic system is conceptually described with which some embodiments of the present invention are implemented. DETAILED DESCRIPTION

[0028] In the following detailed description of the invention, many details, examples and embodiments of the invention are set forth and described. However, it will be clear and apparent to those skilled in the art that the invention is not limited to the embodiments set forth, and the invention can be practiced without some of the specific details and examples discussed.

[0029] Some embodiments provide a novel method for efficiently updating software of a virtual machine (VM) executing on one or more devices in a vehicle so that the update to the software does not interrupt operations performed by the VM. While the vehicle is operating, the method collects configuration state information of a first VM that is executing on a specific device and performing a specific set of operations. The method uses the configuration state information to instantiate a second VM in an isolated network environment isolated from the first VM. The second VM is the same as the first VM at the specific time when the configuration state information was collected. The method updates the second VM. The method detects computing downtime of the vehicle. During the computing downtime, the method replaces the first VM on the device with the updated second VM so that once the computing downtime ends, the updated second VM performs the set of operations.

[0030] In some embodiments, the method is performed by a management agent (MA) executed on a specific device. The MA is an application, program, or module executed on the specific device. In some embodiments, the isolated network environment is an isolated network, so that when a second VM is instantiated in the isolated network, the second VM does not interfere with the set of specific operations performed by the first VM. In various embodiments, the isolated environment is referred to as an isolated network, a sandbox, and a staging area, and allows the creation and updating of the second VM without interrupting the first VM and the operations performed by it. In some embodiments, the specific device is a computer.

[0031] In some embodiments, the isolated network environment is an isolated network associated with a dedicated virtual local area network (VLAN) such that when a second VM is instantiated in the isolated network and prior to computing downtime of the vehicle, the second VM does not interfere with the specific set of operations performed by the first VM. In such embodiments, the isolated network is a dedicated network segment within a virtual switch of a hypervisor of a specific device using a dedicated VLAN. In some embodiments, the dedicated VLAN is associated with a unique port group to ensure that traffic sent to and from the second VM does not conflict with traffic sent to and from the first VM in a different VLAN because they have the same Internet Protocol (IP) address.

[0032] The MA of the specific device performs collection, use, update, and detection after receiving an instruction to update the first VM from a management plane (MP), and the MP is implemented by a set of one or more management servers connected to the vehicle through a software-defined wide area network (SD-WAN). In such embodiments, the MP guides the MA to update the first VM, and the MA performs collection, use, update, and detection in order to update the first VM.

[0033] In some embodiments, the set of managers implementing the MP operates at a different location than the vehicle. For example, in some embodiments, the vehicle moves around in different locations, and the set of management servers executes at an edge site of the SD-WAN. Alternatively, the set of management servers executes at a data center site of the SD-WAN. Additionally, in other embodiments, the set of management servers executes at two or more sites of the SD-WAN in a distributed manner.

[0034] In some embodiments, the vehicle includes an edge router connected to the SD-WAN using a set of one or more wireless network connection links. The edge router of some embodiments is one of: an edge router device, an edge router executed on a computer operating in the vehicle, or an edge router executed on a machine executing on the computer. In some embodiments, the wireless network connection link is provided by one or more different telecommunication link providers (e.g., different cellular providers).

[0035] In some embodiments, the MA updates the second VM by instructing the hypervisor of the specific device to instruct the update agent associated with the second VM to download a set of one or more update files associated with a specific update of the first VM and update the second VM. In such embodiments, each VM instantiated on the specific device and each copy of each VM executes an update agent, which performs a software update of its corresponding VM. In some embodiments, the update agent downloads the set of update files from the MP using one or more of the set of wireless network connection links. In other embodiments, the update agent downloads the set of update files from a set of one or more other locations (e.g., an external repository) specified by the MP using the set of wireless network connection links in some embodiments. In some embodiments, the update operation performed by the update agent is referred to as a patching operation or a patching cycle.

[0036] In some embodiments, the configuration state information includes an operating system (OS) disk of the first VM specifying one or more OS files and application software associated with the first VM at a specific time when the configuration state information is collected. In such embodiments, the OS disk is copied at a specific time to freeze the state of the first VM and create a second VM using the OS disk. In some embodiments, the second VM is referred to as a replica of the configuration file of the first VM. In some embodiments, the configuration file of the first VM and the copy of the configuration file of the first VM are referred to as VMX files. In some embodiments, after copying the OS disk, the first VM writes subsequent OS disk operations to a log.

[0037] In some embodiments, the MA collects configuration state information by directing the hypervisor of a specific device to collect the OS disk at a specific time. In such embodiments, the MA receives a notification from the MP to update the first VM, which causes the MA to direct the hypervisor to copy the OS disk of the first VM. Then, the MA uses the configuration state information to instantiate the second VM by directing the hypervisor to use the OS disk to instantiate the second VM. In such embodiments, the OS disk is copied to create the second VM, but the independent data disk is not copied. In some embodiments, the data disk specifies application data and does not specify software. In some embodiments, the independent data disk is a data disk marked as independent, meaning that it will not be copied by the hypervisor.

[0038] Instead, when the first VM is to be replaced with the second VM, the hypervisor will instead transfer the first VM's independent data disks to the second VM. In some embodiments, this is referred to as a zero-copy transfer, which enables the MA to update the first VM more quickly than if its boot hypervisor also copied the first VM's data disks. Performing a zero-copy transfer of the first VM's data disks to the second VM is very fast because none of this data is copied within the hypervisor system, and because no heavy storage operations are required to detach the data disks from the first VM and reattach them to the second VM.

[0039] In some embodiments, the MA directs the hypervisor to replace the first VM with the updated second VM by directing the hypervisor to: (1) detach a set of one or more data disks containing application data from the first VM; and (2) attach the set of data disks to the updated second VM. This transfers the set of data disks required to replace the first VM with the updated second VM, rather than copying them and adding the copies to the second VM. In some embodiments, the MA also directs the hypervisor to replace the first VM with the updated second VM by directing the hypervisor to: (1) archive the first VM to the hypervisor's data store; and (2) migrate the updated second VM from the isolated network environment to the production network environment with the production network port group, which enables the updated second VM to perform the specific set of operations once the vehicle's computing downtime is over. The production network environment is associated with a VLAN used by other VMs of a particular device, enabling the second VM to communicate with these VMs and any other VMs executing on other devices.

[0040] In some embodiments, computing downtime is detected by detecting a power outage of the vehicle. In other embodiments, computing downtime is detected by detecting a power outage of the vehicle and the locking of the vehicle's doors. In some embodiments, the hypervisor of a particular device is configured to operate during a specific time period after the computing downtime is detected. In such embodiments, the hypervisor is operable during a specific time period using the vehicle's battery and is inoperable after the specific time period. By directing the hypervisor to replace the first VM with an updated second VM during the computing downtime, and by not performing collection, use, or updates during the computing downtime, the method uses the least battery of the vehicle and completes the VM replacement in the smallest amount of time possible (or referred to as a pivot period in some embodiments). If the duration of the vehicle's computing downtime is short, this avoids affecting vehicle operation.

[0041] In some embodiments, the computing downtime ends at the beginning of the next vehicle computing cycle. In some embodiments, the next vehicle computing cycle is a power-on of the vehicle. Once the vehicle is undergoing the next computing cycle, the updated second VM is used to perform the set of specific operations so that it operates as if the first VM was directly updated.

[0042] By updating a VM executing on a device in a vehicle by upgrading the VM, updating a clone, and replacing the original VM with the updated clone, the update minimizes disruption of operations performed by the original VM. This allows for reduced maintenance downtime of VMs, which is useful for lifecycle management of critical VMs (i.e., VMs whose operation is critical to the vehicle and / or the user). For example, police vehicles and first responder vehicles execute VMs that perform various critical operations. By updating these VMs using clones rather than updating them directly, disruption to their operation is avoided.

[0043] Figure 1 A process 100 for efficiently updating software of VMs executed on one or more devices in a vehicle is conceptually illustrated so that updates to the software do not interrupt operations performed by the VMs. In some embodiments, process 100 is performed by an MA that is executed in conjunction with a hypervisor on a specific device operating in the vehicle. In other embodiments, the MA is executed as a microservice within the hypervisor. As used herein, the term hypervisor may generally refer to a software layer or component that supports the execution of multiple VMs, including system-level software that supports namespace containers. In some embodiments, a specific device is a computer that executes several VMs that perform various operations. In some embodiments, the hypervisor includes applications, programs, and / or modules that perform operations directed by the MA. In some embodiments, the following steps 105 to 155 are performed in the order in which they are described. In other embodiments, they are performed in different orders. In addition, in other embodiments, two or more steps are performed simultaneously.

[0044] Process 100 begins by receiving (at 105) from the MP an indication to update a first VM executing on a particular device and performing a particular set of operations. The MA receives from the MP a data message directing the MA to update (or, in some embodiments, patch) the first VM. In some embodiments, patching the VM refers to making changes, updating, or repairing the software of the VM. In some embodiments, a vehicle is connected to an SD-WAN managed by a set of management servers implementing the MP for the SD-WAN. In such embodiments, the vehicle includes an edge router that is connected to the SD-WAN using one or more wireless network connection links (e.g., a mobile hotspot link) and receives data messages along one of the wireless network connection links. The edge router is one of: (1) an edge router device, (2) an edge router executed on a computer operating in the vehicle, or (3) an edge router executed on a machine (e.g., a VM) executed on a computer. In some of these embodiments, the management server resides in a particular location remote from the vehicle. In other embodiments, at least two management servers reside in at least two locations remote from the vehicle (i.e., the MP is implemented in a distributed manner). In such embodiments, the MA is implemented on a particular device to perform management operations directed by the MP.

[0045] Next, the process (100) collects (at 110) current configuration state information associated with the first VM. In some embodiments, the MA directs the hypervisor to collect current configuration state information of the first VM related to the configuration of the first VM. In some embodiments, the current configuration state information includes an OS disk specifying OS files and application software associated with the first VM. In some embodiments, the current configuration state information collected for a VM is referred to as a snapshot. A snapshot of a VM is a point-in-time copy of the VM in which its current OS disk is placed in a frozen state, and as the VM continues to operate, subsequent OS disk operations are written to a log (e.g., a data store local to the hypervisor). By copying and storing the OS disk at a specific time, at any time in the future, the VM can be restored back to the exact state at the specific time when the snapshot was taken.

[0046] In some embodiments, a snapshot of the first VM is taken while the first VM is running. When a snapshot is taken while the VM is running, in some embodiments, the snapshot also contains a copy of the active memory of the VM. In such embodiments, this is done so that the restoration of the snapshot of the VM can contain the running memory state. In addition to restoring a VM using its snapshots, in some embodiments, these snapshots of other VMs can also be used to instantiate new VMs so that the new VM is a clone of the other VMs (i.e., is identical to the other VMs).

[0047] In some embodiments, the collected configuration state information (i.e., the snapshot) does not include any disks associated with the first VM that are marked as independent in the definition of the first VM (i.e., in a file defined by the hypervisor that is stored by the hypervisor). In such embodiments, the first VM is associated with a set of one or more data disks that store critical data required to run the first VM. These data disks are marked as independent and are therefore not replicated by the hypervisor (e.g., directed by the MA) to minimize the amount of data that needs to be replicated to create a replica of the first VM. Instead, any data disks that are not marked as independent in the definition of the first VM are replicated in the snapshot.

[0048] In some embodiments, the OS disk (not marked as independent) includes the C drive for the first VM, and the independent data disk includes the D drive and the E drive. In such embodiments, the C drive is replicated in the configuration state information, and the D and E drives are not replicated. As will be described below, in some embodiments, the D and E drives will instead be transferred rather than replicated.

[0049] At 115, process 100 uses the collected current configuration state information to create a second VM in an isolated network environment isolated from the first VM. The MA boots the hypervisor to instantiate a second VM as a clone of the first VM using the snapshot collected for the first VM (i.e., the OS disk of the first VM). This clone is a clone of the first VM at the exact time of collecting the configuration state information (i.e., the snapshot). In some embodiments, the second VM is instantiated in an isolated network environment (also referred to as an isolated network, a temporary storage area, or a sandbox in some embodiments) so that it does not interfere with any operations performed by the first VM. In some embodiments, the second VM is instantiated in an isolated network environment by providing (i.e., adding) a separate virtual switch to a specific device to connect to the second VM. In other embodiments, the second VM is instantiated in an isolated network environment by instantiating the second VM in a dedicated network segment within a virtual switch of a hypervisor using a dedicated virtual local area network (VLAN). In such embodiments, the dedicated VLAN is associated with a unique port group to ensure that the business sent to and from the second VM does not conflict with the business sent to and from the first VM in a different VLAN because they have the same Internet Protocol (IP) address.

[0050] Next, process 100 downloads (at 120) one or more update files associated with the update of the first VM. In some embodiments, the MA directs an update agent associated with the second VM to download the update file from the MP. In such embodiments, the update agent operates within the second VM. In other embodiments, the MA directs the update agent to download the update file from one or more locations (e.g., one or more external repositories) specified by the MP. In addition, in other embodiments, the MA itself downloads the update file from the MP and provides them to the update agent. The update file is downloaded from the MP using one or more wireless network connection links of the edge router of the vehicle. In some embodiments, the update file is downloaded by requesting the update file from the MP (e.g., in an application programming interface (API) request) and receiving the update file in a response from the MP (e.g., in an API response).

[0051] Using the downloaded update file, process 100 updates (at 125) a second file of the second VM in the isolated environment. In order not to interrupt the operation of the first VM, the MA directs the update agent to update the second VM in the isolated network environment because it is a clone of the first VM when collecting configuration state information. In some embodiments, the second VM is updated by using the update file to make application and / or OS changes to the OS disk of the second VM. In some embodiments, the update operation performed by the update agent is referred to as a patching operation or patching cycle. In some embodiments, patching a VM refers to making changes, updates, or repairs to the software of the VM.

[0052] At 130, process 100 waits for the vehicle to experience computing downtime. In some embodiments, instead of directly updating the first VM (which causes its operation to be interrupted), when the first VM is not currently running, the MA directs updating the cloned second VM and replacing the first VM with the updated second VM. In such embodiments, when the vehicle is experiencing computing downtime (e.g., when the vehicle is turned off), the MA knows that the first VM is not currently running. In order for the MA to direct the replacement of the first VM with the updated second VM without interrupting the operation of the first VM, the MA waits for the vehicle to experience computing downtime. In some embodiments, the second VM is updated before the vehicle experiences computing downtime (i.e., while the vehicle is running) so as to minimize the use of the vehicle battery during the computing downtime.

[0053] At 135, process 100 determines whether a calculated downtime for the vehicle has been detected. In some embodiments, the MA detects a calculated downtime when the MA detects that the vehicle has experienced an ignition event. In some embodiments, the ignition event is when the vehicle has been powered off (i.e., turned off). In other embodiments, the ignition event is when the vehicle has been powered off and the vehicle's doors have been locked.

[0054] If the process 100 determines that computing downtime has not been detected, the process 100 returns to step 130 to continue waiting for the vehicle to experience computing downtime. The MA will not guide the replacement (or in some embodiments, conversion) of the first VM with the second VM until the computing downtime of the vehicle has begun. If the process 100 determines that computing downtime has been detected, the process 100 determines (at 140) whether there are any data disks associated with the first VM. As previously discussed, in some embodiments, the collected configuration state information (i.e., snapshot) of the first VM includes the OS disk of the first VM and does not include any independent data disks. In such embodiments, the MA needs to transfer the independent data disks from the first VM to the second VM because they have not been previously copied to create the second VM. In some embodiments, this is referred to as zero-copy transfer of data because the independent data disks are unmounted from the first VM and attached to the second VM, rather than copying them and attaching the copied data disks to the second VM. In other embodiments, the collected configuration state information includes an OS disk and at least one data disk that is not marked as independent (and therefore copied to create the second VM). In such embodiments, the MA determines which data disks, if any, are marked as independent and need to be transferred to the second VM.

[0055] If the process 100 determines that no independent data disks are associated with the first VM, then the process 100 continues to step 150, which will be described below. If the process 100 determines that there are one or more independent data disks associated with the first VM, then the process 100 transfers (at 145) the independent data disk(s) from the first VM to the updated second VM. In some embodiments, the MA directs the hypervisor to detach the independent data disks from the first VM and attach them to the updated second VM. By doing so, the MA and the hypervisor successfully transfer the information needed for the updated second VM to run in place of the first VM without having to directly copy the information (i.e., saving computing time and power).

[0056] As previously discussed, when collecting configuration state information, the first VM will then write subsequent OS disk operations to the log. In such embodiments, the MA will not direct the copying of these OS disk files or the transfer of these OS disk files to the updated second VM, i.e., the second VM will not receive these OS disk files. In some embodiments, this is done because this data is not significant for running the updated second VM in place of the first VM. In other embodiments, the MA directs the hypervisor to extract subsequent OS disk files from the log and provide them to the updated second VM.

[0057] Next, process 100 archives (at 150) the first VM in the hypervisor's data store. In some embodiments, the MA directs the hypervisor to archive the first VM to its local data store so that the first VM can be reinstantiated in the future. For example, after replacing the first VM with the updated second VM (i.e., after a VM conversion cycle), in some embodiments, (e.g., if the update has defects) the MP directs the MA to restore the first VM back to its state before the update occurred. Since the first VM is archived, the MA can simply direct the hypervisor to power off the updated second VM, transfer the data disk back to the first VM, and power on the first VM. In some embodiments, after this execution, the updated second VM is deleted. In other embodiments, after this execution, the updated second VM is archived. In some embodiments, the first VM is archived for a specific time period (e.g., five days), and after the specific time period has passed, the first VM is deleted from the archive. In other embodiments, the first VM is archived indefinitely.

[0058] At 155, process 100 migrates the updated second VM from the isolated network environment to the production network environment. To instantiate the updated second VM in place of the first VM, the MA directs the hypervisor to migrate the updated second VM to the production folder and production network port group, which allows it to communicate with other VMs (e.g., executing on a specific device, executing on other devices, etc.). Thereafter, the updated second VM executes in the same VLAN as the originally executed first VM, and the updated second VM will execute the set of specific operations previously performed by the first VM on the specific device. After migrating the updated second VM from the isolated network environment to the production network environment, process 100 ends.

[0059] In some embodiments, steps 140 to 155 of process 100 are performed during a specific time period after the vehicle's computing downtime is detected (i.e., during a specific time period after the vehicle is powered off). In such embodiments, during the specific time period, the hypervisor operates using the vehicle's battery. In embodiments where the specific device includes an OS (i.e., when the hypervisor is not the OS itself), in some embodiments, the OS also operates using the battery during the specific time period. After performing steps 140 to 155 and after the specific time period ends, the hypervisor (and in some embodiments, the OS) is powered off until the vehicle's computing downtime ends at the beginning of the next vehicle computing cycle (i.e., until the vehicle is powered back on).

[0060] When the next vehicle computing cycle begins (e.g., the next time the vehicle is turned on), the updated second VM will perform the specific set of operations previously performed by the first VM. In some embodiments, the archive of the first VM is stored in the data store for a specific period of time (e.g., seven days) and deleted from the data store after the specific period of time ends.

[0061] By performing the VM update while the vehicle is operating and by performing the conversion of the first VM with the updated second VM while the vehicle is not operating, the MA minimizes the hypervisor's use of the vehicle's battery, since in some embodiments, VM updates are time consuming. In view of this, the cloned VM is updated in advance, and the transfer of the independent data disk, the archiving of the first VM, and the migration of the updated second VM are performed using only the vehicle's battery.

[0062] Figure 2 An example vehicle 200 is illustrated that includes components managed and configured by a set of managers and controllers 205. In this example, the vehicle 200 includes a set of one or more devices 210 and an edge router 220 connected to the manager and controller set 205 using a set of one or more wireless network connection links 230. The edge router 220 can connect to the manager and controller set 205 using any number of wireless network connection links 230 provided by a cellular provider.

[0063] In some embodiments, the manager and controller group 205 is a group of management servers implementing the MP and a group of controllers implementing the control plane (CP) of the SD-WAN connected to the vehicle 200. The manager and controller group 205 operates in a location different from the vehicle 200, such as at a public or private cloud data center. Other information about the manager, controller, and SD-WAN connected to the vehicle will be described below.

[0064] The vehicle 200 includes a group of one or more devices 210. In some embodiments, the device 210 is a group of computers. In some embodiments, each device 210 includes a group of one or more machines 240, a hypervisor 250, an MA 260, and a group of one or more data stores 270. The device 210 can execute any number of machines 240. In some embodiments, the machine 240 is at least one of a VM, a container, and a Pod. In some embodiments, each VM 240 performs a set of operations for a set of one or more applications. In some embodiments, these operations are directed to be performed by a user of the vehicle 200, for example, through a network interface provided by a network server running in the machine 240. Although the MA 260 is illustrated in this figure as a module separated from the hypervisor 250, in other embodiments, the MA 260 is executed within the hypervisor 250 (for example, as a microservice).

[0065] For example, in some embodiments, a police car includes a device that executes a VM to perform various functions, such as: 1) license and registration verification, 2) video streaming from on-board cameras and body cameras, 3) forensic data collection (e.g., collection of crime victim phone images and associated analysis), 4) logging and safety functions of the vehicle itself (along with logistical information communicated to the on-board operator about his or her immediate mission), and 5) data synchronization from on-board sources to the cloud.

[0066] In some embodiments, one or more of the VMs 240 perform critical operations that cannot experience significant downtime during the vehicle's computing cycle in order to be updated. To minimize downtime experienced by one or more of the VMs 240 executing in the vehicle 200, in some embodiments, the manager and controller group 205 (i.e., MP) directs the MA 260 to update the VMs 240 (e.g., in the event of a shutdown) by directing the hypervisor 250 to clone the VMs, initiate an update of the cloned VMs, and replace (i.e., convert) the original VMs with the updated cloned VMs. Figure 1 100).

[0067] As directed by MA 260 (i.e., after MA 260 is notified by manager and controller group 205 of an update of one of VMs 240), hypervisor 250 executing on the same device 210 collects configuration state information of the VM (i.e., OS disk or C drive) to create cloned files in an isolated network environment, updates the cloned files (e.g., using an update agent of the cloned VM), transfers any independent data disks (e.g., D drive, E drive) from the VM to the updated cloned VM, archives the original VM, and migrates the updated cloned VM to the production network environment. In some embodiments, the files of the updated cloned VM (i.e., OS disk, one or more dependent data disks) are stored in the set of data stores 270, so that the updated cloned VM is stored in the set of data stores 270 along with other VMs executing on the device 210. In other embodiments, the files of the updated cloned VM are stored in different data stores local to the hypervisor 250.

[0068] In some embodiments, the files of the original VM are archived to a data store that is distinct from the data store 270 but local to the hypervisor 250. In other embodiments, the files of the original VM are archived to the set of data stores 270. In this example, the hypervisor 250 is the OS of the device 210 (i.e., a type 1 hypervisor). However, in other embodiments, a separate device OS executes on the device 210 along with the hypervisor 250 (i.e., a type 2 hypervisor).

[0069] As previously discussed, in some embodiments, the vehicle is connected to the SD-WAN. Figure 3 An example of vehicles 300 and 310 communicating with various components of an SD-WAN is illustrated. Any number of vehicles 1 to N may be used. Vehicles 300 and 310 include groups of one or more computing machines 302 and 312, SD-WAN routers 304 and 314, and network links 305 to 307 and 315 to 317, respectively. The SD-WAN connects a cloud gateway (CGW) 320, a data center site 330, and an edge site 340.

[0070] In some embodiments, the CGW 320 is a forwarding element in a private or public data center 325. In some embodiments, the CGW 320 has a secure connection link (e.g., tunnel) with an edge forwarding element (e.g., SD-WAN edge FE 342) at a multi-machine site (e.g., SD-WAN edge site 340 with multiple machines 345) such as a branch office, data center, etc. These multi-machine sites are typically at different physical locations (e.g., different buildings, different cities, different states, etc.) and are referred to as multi-machine sites or nodes in some implementations. Two multi-machine sites 330 and 340 are illustrated in this figure, one of which is an edge site 340 (e.g., a branch site) and one is a data center site 330. The edge site is shown as including an edge forwarding node 342, while the data center site 330 is shown as including a hub forwarding node 332. The edge forwarding element (e.g., SD-WAN edge FE 342) exchanges data messages with one or more CGWs 320 via one or more connection links 350 (e.g., multiple connection links available at the edge forwarding element).

[0071] When multiple such links are defined between the edge node and the gateway, in some embodiments, each secure connection link is associated with a different physical network link between the edge node and the external network. For example, in order to access the external network, in some embodiments, the edge node has one or more commercial broadband Internet links (e.g., cable modems, fiber optic links) for accessing the Internet, MPLS (Multi-Protocol Label Switching) links for accessing the external network through the MPLS provider's network, wireless cellular links (e.g., 5G LTE networks). In some embodiments, the different physical links between the edge node 342 and the CGW 320 are links of the same type (e.g., different MPLS links).

[0072] In some embodiments, the edge forwarding node 342 also has multiple direct links 350 (e.g., secure connection links established through multiple physical links) to the data center hub node 332. Similarly, in some embodiments, different links may use different types of physical links or the same type of physical links. In addition, in some embodiments, the edge forwarding node 342 at the branch site can be connected to the vehicle's SD-WAN router by: (1) directly through one or more links 350, or (2) through a CGW or data center hub to which the edge forwarding node is connected via two or more links 350. Therefore, in some embodiments, the edge forwarding node 342 at the branch site 340 can use multiple SD-WAN links 350 to reach the SD-WAN router (e.g., 304) of the vehicle (e.g., 300) or the hub forwarding node 332 of the data center site 330.

[0073] In some embodiments, the cloud gateway 320 is used to connect the vehicle's SD-WAN router (e.g., the SD-WAN router 314 of the vehicle 310) to the SD-WAN forwarding node (e.g., the edge forwarding element 342) through at least two secure connection links 350 between the CGW 320 and the SD-WAN router and between the CGW 320 and the forwarding element at the SD-WAN site (e.g., the branch site 340 or the data center site 330). In some embodiments, the CGW 320 also provides network data from the vehicle to the multi-machine site, or from one multi-machine site to another multi-machine site (e.g., providing an accessible subnet of one site to another site). Like the CGW 320, in some embodiments, the hub forwarding element 332 of the data center 330 can be used to connect the SD-WAN forwarding node 342 of the branch site to the vehicle's SD-WAN router through at least two secure connection links 350 between the hub 332 and the SD-WAN router and between the hub 332 and the forwarding element at the branch site 340.

[0074] In some embodiments, each edge forwarding element, hub, and CGW in the SD-WAN (e.g., edge forwarding element 342, data center hub forwarding element 332, and CGW 320) includes a router that performs data message forwarding operations of the edge forwarding element, hub forwarding element, or CGW. In such embodiments, the next-hop forwarding records of these edge forwarding elements, hub forwarding elements, and CGWs are routing records used by the routers to forward data messages through the SD-WAN.

[0075] In some embodiments, each edge forwarding element at an edge site (e.g., edge forwarding element 342 at edge site 340) is connected to an external network through two or more forwarding devices (e.g., MPLS devices, cable modem routers, 5G routers) of two or more communications service providers (e.g., telephone company providers for MPLS networks, cable modem providers for ISPs (Internet service providers), wireless providers for 5G connectivity). In some of these embodiments, each edge forwarding element is connected to a forwarding device of a service provider through two or more physical ports of the edge forwarding element.

[0076] In some embodiments, each secure connection link between two SD-WAN forwarding nodes (i.e., CGW 320 and edge forwarding node 342) is formed as a VPN (virtual private network) tunnel between the two forwarding nodes. In this example, a collection of SD-WAN forwarding nodes (e.g., forwarding element 342 and CGW 320) and secure connections 350 between forwarding nodes form a virtual network for a specific entity that spans at least a public or private cloud data center 325 to connect the data center and edge sites 330 and 340.

[0077] In some embodiments, secure connection links are defined between gateways in different public cloud data centers to allow paths through virtual networks to traverse from one public cloud data center to another, while in other embodiments such links are not defined. In addition, in some embodiments, CGW 320 is a multi-tenant gateway for defining other virtual networks for other entities (e.g., other companies, organizations, etc.). Some such embodiments use a tenant identifier to create a tunnel between a gateway and an edge forwarding element of a particular entity, and then use the tunnel identifier of the created tunnel to allow the gateway to distinguish between the data message streams it receives from the edge forwarding element of an entity and the data information streams it receives along other tunnels of other entities. In other embodiments, the gateway is single-tenant and is deployed specifically for use by only one entity.

[0078] Figure 3A cluster 360 of managers and controllers is illustrated that serves as a central point for managing (e.g., defining and modifying) configuration data that is provided to edge nodes and / or gateways to configure some or all operations. In some embodiments, this manager and controller cluster 360 is in one or more public cloud data centers, while in other embodiments, it is in one or more private data centers. In some embodiments, the manager and controller cluster 360 has a set of manager servers that define and modify configuration data, and a set of controller servers that distribute configuration data to edge FEs, hubs, and / or gateways. In some embodiments, the manager and controller cluster 360 facilitates updates to machines 302 and 312 by providing configuration data to vehicles 300 and 310. The updated configuration data is used by the MA and management programs executed on the devices executing machines 302 and 312 to update machines 302 and 312.

[0079] In some embodiments, a cloned VM (ie, a copy of a running VM) is instantiated in an isolated network environment within the hypervisor before it replaces the original VM in the hypervisor file system in order to update the original VM without interrupting its operation. Figure 4A B illustrates an instance hypervisor 400 of a device (not shown) that uses a file system 410, an isolated network 420, and an archival data store 430 to update a VM executing in a vehicle without interrupting operations performed by the VM during a computing cycle.

[0080] Hypervisor 400 instantiates a copy of the VM (i.e., a clone of the VM) in isolated network 420. In some embodiments, isolated network 420 is referred to as an isolated network environment, a staging area, or a sandbox, and allows hypervisor 400 to instantiate copies of VMs without the copied VMs interfering with the VMs from which they were copied. In some embodiments, isolated network 420 includes table 425 that maintains files of the copied VMs. Hypervisor 400 is able to update the copied VMs using their associated files stored in table 425 by using an update agent that downloads update files from the MP and / or one or more other locations (e.g., external repositories) specified by the MP.

[0081] Hypervisor 400 configures VMs executed on the device using its file system 410. In some embodiments, file system 410 is a data storage area that stores table 415, which contains an entry for each instantiated VM. In some embodiments, each file of a VM contains a file name and an identifier (ID) of the VM of the file. In some embodiments, the ID is a universally unique ID (UUID). In other embodiments, it is a network address associated with the VM (e.g., a media access control (MAC) address, an IP address). In some embodiments, each VM file also includes an OS disk (e.g., a C drive) and one or more data disks associated with the VM (e.g., a D drive, an E drive).

[0082] Hypervisor 400 stores archived VM files in archival data storage area 430. In some embodiments, archival data storage area 430 includes a table storing archived VM files. In this example, the hypervisor is an OS of a device (i.e., a type 1 hypervisor) and stores archived VM files in archival data storage area 430. Alternatively, when a separate device OS is executed together with hypervisor 400 (i.e., a type 2 hypervisor), in some embodiments, the OS directs hypervisor 400 to store archived VM files in archival data storage area 430. In other embodiments, even when a separate device OS is executed together with hypervisor 400, hypervisor 400 still directly stores archived VM files in archival data storage area 430, rather than being directed to do so by the OS.

[0083] Figure 4A 4. Illustrated is a file system 410, an isolated network 420, and an archival data store 430 after a first VM (VM1) has been copied but before the first VM is replaced with a copy (VM4). As shown, hypervisor 400 stores files F1-VM1 of the original VM in file system 410. Hypervisor 400 has created a copy (F4-VM4) of the original VM in isolated network 420. It can be seen that the VM files stored in isolated network 420 are copies of the original VM1 because they share the same VM ID (IP address 10.2.2.2). Hypervisor 400 is able to make any updates to the copied VM4 without interfering with the operation of the original VM1 (e.g., using an update agent executing within the copied VM4). Archival data store 430 does not store any VM files, meaning that no VMs are archived from the device and archived to data store 430.

[0084] Figure 4BIllustrated is a file system 410, isolated network 420, and archival data store 430 after VM1 has been replaced with a copied (and, in some embodiments, updated) VM4. As shown, VM4's files have been removed from the isolated network 420 and added to the hypervisor file system 410 (i.e., the hypervisor's production data store). VM1's files have been removed from the hypervisor file system 410 and added to the archival data store 430. After VM1 has been archived and VM4 has been added to the hypervisor production data store 410, the cloned VM4 is now able to perform the same operations that VM1 used to perform on the device.

[0085] As previously discussed, a copy of the VM (ie, a copy of the VM's OS disk) is created and updated during the vehicle's computing cycles, and the copy of the VM is configured for operation during the vehicle's downtime cycles (ie, transferring independent data disks). Figure 5 Three time periods 510 to 530 are illustrated that occur when the hypervisor of a vehicle device updates a first VM by creating and updating a second VM (as directed by the MA of the vehicle device). The first time period 510 is a first computing cycle of the vehicle. In some embodiments, the first computing cycle 510 is when the vehicle, the devices in the vehicle, and the VMs executed on the devices are running. In other embodiments, the first computing cycle 510 is when the vehicle itself is not running, but the devices in the vehicle and the VMs executed on the devices are running (e.g., using the vehicle's battery).

[0086] During the first computing cycle 510, the first VM is executed on a device in the vehicle and performs a set of operations. For example, in some embodiments, a user in the vehicle directs the first VM to execute a set of specific services (e.g., data analysis services, data streaming services, messaging services, etc.). The hypervisor receives a notification (e.g., from the MA) that it needs to (1) create a second VM that is a copy of the first VM (i.e., a copy of the OS disk of the first VM) (e.g., in a network environment isolated from the first VM) and (2) update the second VM. After receiving an indication from the MP to update the first VM, the MA directs the hypervisor to do so. After the hypervisor performs these operations, it waits to receive an indication from the MA to replace the first VM with the second VM. The MA waits for the first computing cycle 510 to end before directing the hypervisor to replace the first VM with the second VM.

[0087] After the first computing cycle 510 ends, the vehicle begins its second time period, which is a downtime period 520. In some embodiments, the downtime period 520 begins when the vehicle loses power. In other embodiments, the downtime period 520 begins when the vehicle loses power and the vehicle's doors are locked (i.e., indicating that the user of the vehicle has closed and left the vehicle). During the downtime period 520, even though the vehicle loses power, the hypervisor of the devices in the vehicle still operates for at least some of the downtime period using the vehicle's battery. For example, after the downtime period 520 begins, the hypervisor operates on the vehicle's battery for five minutes.

[0088] During the downtime period 520 and while the hypervisor is running on the vehicle battery, the MA directs the hypervisor to use the updated second VM on the device instead of the first VM. In some embodiments, the hypervisor does this by transferring any independent data disks associated with the first VM to the second VM, archiving the first VM, and migrating the updated second VM from the isolated network environment to its production network environment. Also during the downtime period 520, and after the second VM has been configured by the hypervisor, the MA powers off any non-critical VMs for the remainder of the downtime period 520. After the specified amount of time that the hypervisor of the device uses the battery of the vehicle has passed, all VMs and hypervisors executed on the device are powered off. In some embodiments, the VMs and hypervisors are set to power off after a specific time period to avoid the need to completely power off and power on the vehicle's devices in the case of a short downtime period. Updating before the downtime period allows the hypervisor to quickly replace the first VM with the updated second VM when the first VM is not running, and does not use a large amount of vehicle battery or require extended downtime to complete the software update.

[0089] After the downtime period 520 ends, the vehicle undergoes its third time period, which is a second computing period 530. In some embodiments, the second computing period 530 begins when the vehicle, the devices in the vehicle, and the VMs executing on the devices are powered back on. In other embodiments, the second computing period 530 begins when the vehicle itself is not running, but the devices in the vehicle and the VMs executing on the devices are powered back on (e.g., using the vehicle's battery).

[0090] During the second computing cycle 530, since the first VM is replaced by the updated second VM, the updated second VM is now executing on the device and performing the set of operations. The updated second VM now performs the set of operations according to the update, just as if the first VM had performed these operations if it had been directly updated.

[0091] Figure 6An example host computer 600 is illustrated that executes VMs 608-1 through N and updates the VMs without interfering with their operation. In some embodiments, the host 600 executes in a vehicle and is connected to other computers executing in the same vehicle, other computers executing in other vehicles, and / or machines (e.g., VMs, containers, Pods) and forwarding elements (e.g., routers) executing at sites (e.g., branch sites, edge sites, data center sites, cloud sites) through an SD-WAN 610. In some embodiments, the host 600 performs processing of network traffic (e.g., data messages).

[0092] In some embodiments, the host 600 includes a hypervisor 606. In this example, the hypervisor 606 is executed together with the computer OS 616 (i.e., a type 2 hypervisor). However, in other embodiments, the hypervisor 606 is the OS of the computer 600 (i.e., a type 1 hypervisor). The host 600 provides resources to allow the VMs 608 to send and receive data messages to and from the SD-WAN 610. In some embodiments, the hypervisor 606 includes a network stack 612 for each VM 608 that processes data messages. In some embodiments, the network stack 612 performs operations on the data messages in software using the hypervisor 606. In some embodiments, the operations performed are defined by protocols such as the Transmission Control Protocol / Internet Protocol (TCP / IP). In some embodiments, the network stack 612 includes an input / output (I / O) chain that performs network operations related to the delivery and reception of data messages and other operations.

[0093] The host 600 also includes physical hardware 604, which includes hardware devices for the host 600, such as a central processing unit (CPU), storage devices, etc. The physical hardware 604 also includes a physical network interface card (PNIC) 602, which in some embodiments is a physical interface for the host 600 to connect to the SD-WAN 610. For example, the PNIC 602 is an interface for sending data messages from the VM 608 to the SD-WAN 610 (e.g., through an edge router executing in a vehicle) and also for receiving data messages destined for the VM 608 from the SD-WAN 610. In a receive scenario, the PNIC 602 includes a queue (not shown) that stores data messages received by the PNIC 602. The queue buffers incoming data messages received from the SD-WAN 610, but the queue can also perform other operations. Then, the driver 618 reads the data message corresponding to the queue for further processing by the network stack 612.

[0094] It should be noted that the above explanations, as well as other explanations herein, may reflect common explanations or abstractions of actual processing mechanisms. For example, data messages may not actually be stored in a queue, but instead descriptors or pointers pointing to data messages (actually stored in host memory) may be stored in the queue. These descriptors / pointers identifying data messages are conventionally referred to as data messages themselves to abstract away complexity and explain higher-level operations without burdening the reader with unnecessary technical details of well-known mechanisms. Such abstractions in the description herein should not be construed as limiting in any way.

[0095] In some embodiments, the hypervisor replicates, updates, and replaces the VM, such as directed by MA 614. In such embodiments, MA 614 receives notification to update the VM from a set of management servers (not shown) that implement the MP. Then, MA 614 directs hypervisor 606 to clone the VM (i.e., copy the OS disk and any data disks not marked as independent), instantiate the cloned VM in isolated network 620, update the cloned VM (e.g., using update agent 626 for the VM), and replace the VM on computer 600 with the updated cloned VM. Although MA 614 is illustrated in this figure as a separate module of computer 600 from hypervisor 606, in other embodiments, MA 614 executes within hypervisor 606.

[0096] As previously discussed, in some embodiments, the hypervisor of the device directs an update agent associated with the cloned VM to download one or more update files from the MP and / or one or more other locations specified by the MP. Each cloned VM 624 in the isolated network 620 includes an update agent 626 for retrieving upgrade files associated with its VM 624. For example, in some embodiments, the MA 614 receives a notification to update a particular VM 608-1. After receiving this notification, the MA 614 directs the hypervisor 606 to clone the VM 608-1. After the hypervisor 606 creates a clone of the VM 608-1 as the cloned VM 624-1, the MA 614 directs the hypervisor 606 to direct the update agent 626-1 to retrieve the update files required to update the VM 624-1 from the MP through the SD-WAN 610.

[0097] Agent 626-1 retrieves the necessary update files and uses them to update (or in some embodiments, patch) clone VM 624-1. Then, once MA 614 detects computing downtime of computer 600, MA 614 directs hypervisor 606 to replace original VM 608-1 with updated clone VM 624-1.

[0098] A VM executing on a device in a vehicle is updated by cloning the VM, updating the clone, and replacing the original VM with the updated clone, without interrupting operations performed by the original VM. This allows for reduced maintenance downtime for VMs, which is important for lifecycle management of critical VMs (e.g., VMs executing in first response vehicles or police cars). In some embodiments, it eliminates maintenance downtime due to rapid VM swap operations performed when the vehicle is in a downtime period (e.g., when the vehicle loses power). This results in zero downtime update operations that can be performed for critical VMs that implement critical applications for vehicle users.

[0099] The above embodiments are applicable to many different types of devices operating on different types of vehicles, such as police cars, fire trucks, and first response vehicles (e.g., ambulances). However, any other type of vehicle may be used. Any type of vehicle that includes a computer operating therein that is connected to the SD-WAN to access resources at another location may implement the above embodiments.

[0100] Many of the above features and applications are implemented as software processes specified as sets of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, cores of a processor, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or via wired connections.

[0101] In this specification, the term "software" means firmware residing in a read-only memory or an application stored in a magnetic storage device, which can be read into a memory for processing by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while still being different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together is within the scope of the present invention. In some embodiments, the software program, when installed to operate on one or more electronic systems, defines one or more specific machine implementations that execute and implement the operations of the software program.

[0102] Figure 7A computer system 700 is conceptually illustrated with which some embodiments of the present invention are implemented. The computer system 700 can be used to implement any of the computers and servers described above. Thus, it can be used to perform any of the processes described above. This computer system includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. The computer system 700 includes a bus 705, a processing unit 710, a system memory 725, a read-only memory 730, a permanent storage device 735, an input device 740, and an output device 745.

[0103] Bus 705 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of computer system 700. For example, bus 705 communicatively connects processing unit 710 with read-only memory 730, system memory 725, and permanent storage 735.

[0104] Processing unit 710 retrieves instructions to be executed and data to be processed from these various memory units in order to perform the processes of the present invention. In different embodiments, the processing unit may be a single processor or a multi-core processor. Read-only memory (ROM) 730 stores static data and instructions required by processing unit 710 and other modules of the computer system. On the other hand, permanent storage device 735 is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when computer system 700 is turned off. Some embodiments of the present invention use a mass storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 735.

[0105] Other embodiments use removable storage devices (e.g., flash drives, etc.) as permanent storage devices. Like permanent storage device 735, system memory 725 is a read-write memory device. However, unlike storage device 735, system memory is a volatile read-write memory, such as random access memory. System memory stores some instructions and data that the processor needs at run time. In some embodiments, the processes of the present invention are stored in system memory 725, permanent storage device 735, and / or read-only memory 730. Processing unit 710 retrieves instructions to be executed and data to be processed from these various memory units in order to perform the processes of some embodiments.

[0106] Bus 705 also connects to input and output devices 740 and 745. Input devices enable a user to communicate information and select commands to the computer system. Input devices 740 include an alphanumeric keyboard and a pointer device (also called a "cursor control device"). Output devices 745 display images generated by the computer system. Output devices include printers and display devices, such as a cathode ray tube (CRT) or a liquid crystal display (LCD). Some embodiments include devices that function as both input and output devices, such as a touch screen.

[0107] Finally, if Figure 7 705 also couples the computer system 700 to a network 765 through a network adapter (not shown). In this way, the computer may be part of a computer network, such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of networks, such as the Internet. Any or all components of the computer system 700 may be used in conjunction with the present invention.

[0108] Some embodiments include electronic components, such as microprocessors, storage devices, and memories, that store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), read-only digital versatile disks (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-ray discs, and the like. CDs, ultra-density CDs, and any other optical or magnetic media. Computer-readable media can store computer programs that are executable by at least one processing unit and contain sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0109] Although the above discussion mainly involves microprocessors or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves.

[0110] As used in this specification, the terms "computer", "server", "processor" and "memory" all refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or displaying mean displaying on an electronic device. As used in this specification, the terms "computer-readable medium", "computer-readable media" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other short-lived or transient signals.

[0111] Although the present invention has been described with reference to numerous specific details, it will be appreciated by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit of the present invention. Therefore, it will be appreciated by those skilled in the art that the present invention is not limited by the foregoing illustrative details, but is defined by the appended claims.

Claims

1. A method for efficiently updating software of a virtual machine VM executed on one or more devices in a vehicle so that the update of the software does not interrupt operations performed by the VM, the method comprising: While said vehicle is operating: collecting configuration state information of a first VM executing on a specific device, the first VM performing a specific set of operations; using the configuration state information to instantiate a second VM in an isolated network environment isolated from the first VM, the second VM being identical to the first VM at the specific time when the configuration state information was collected; and Updating the second VM; detecting a calculated downtime of the vehicle; and During the computing downtime of the vehicle, the first VM is replaced with the updated second VM on the particular device such that once the computing downtime of the vehicle ends, the updated second VM performs the particular set of operations.

2. The method of claim 1 , wherein the isolated network environment is an isolated network associated with a dedicated virtual local area network (VLAN) such that when the second VM is instantiated in the isolated network and prior to the computing downtime of the vehicle, the second VM does not interfere with the set of specific operations.

3. The method according to claim 1, wherein the collection, use, update and detection are performed by the management agent MA of the specific device after receiving an instruction to update the first VM from the management plane MP, and the MP is implemented by a group of one or more management servers connected to the vehicle via a software-defined wide area network SD-WAN. The method of claim 3 , wherein the set of management servers operates at a location different from the vehicle.

5. The method of claim 3, wherein the vehicle comprises an edge router connected to the SD-WAN using a set of one or more wireless network connection links.

6. The method of claim 5, wherein the edge router is one of: (i) an edge router device, (ii) an edge router executed on a computer operating in the vehicle, or (iii) an edge router executed on a machine executing on the computer.

7. The method of claim 5, wherein updating the second VM comprises directing a hypervisor of the specific device to direct an update agent associated with the second VM to download a set of one or more update files associated with a specific update of the first VM from the MP using one or more of the set of wireless network links and update the second VM.

8. The method according to claim 3, wherein: The configuration state information includes an operating system OS disk of the first VM specifying one or more OS files and application software related to the first VM at the specific time when the configuration state information is collected, collecting the configuration state information includes instructing a hypervisor of the specific device to collect the OS disk at the specific time, and Instantiating the second VM using the configuration state information includes directing the hypervisor to instantiate the second VM using the OS disk.

9. The method of claim 8, wherein replacing the first VM on the particular device with the updated second VM comprises booting the hypervisor: unmounting a set of one or more data disks including application data from the first VM; and The set of data disks is attached to the updated second VM such that the set of data disks required to replace the first VM with the updated second VM is transferred to the updated second VM.

10. The method of claim 8, wherein replacing the first VM with the updated second VM on the particular device further comprises booting the hypervisor: Archiving the first VM to a data store of the hypervisor; and The updated second VM is migrated from the isolated network environment to a production network environment, the production network environment enabling the updated second VM to perform the specific set of operations once the computing downtime of the vehicle ends. The method of claim 1 , wherein detecting the computing downtime comprises detecting a power outage of the vehicle. 12 . The method of claim 11 , wherein the first VM is replaced with the updated second VM within a specific time period after the computing downtime is detected.

13. The method of claim 12, wherein the computing downtime ends at the beginning of a next vehicle computing cycle. The method of claim 13 , wherein the start of the next vehicle computing cycle is a power-on of the vehicle.

15. A non-transitory machine-readable medium storing a program for execution by at least one processing unit to effectively update software of a virtual machine (VM) executing on one or more devices in a vehicle so that the update of the software does not interrupt operations performed by the VM, the program comprising a set of instructions for: While said vehicle is operating: collecting configuration state information of a first VM executing on a specific device, the first VM performing a specific set of operations; using the configuration state information to instantiate a second VM in an isolated network environment isolated from the first VM, the second VM being identical to the first VM at the specific time when the configuration state information was collected; and Updating the second VM; detecting a calculated downtime of the vehicle; and During the computing downtime of the vehicle, the first VM is replaced with the updated second VM on the particular device such that once the computing downtime of the vehicle ends, the updated second VM performs the particular set of operations.

Images