Driver domain as security monitor in virtualization environment

Abstract

A virtualization platform includes a number of virtual machines, one of which is configured as a driver domain and includes the network service control for routing network traffic between the other virtual machines. The privileged domain does not include the network service control. The network service control includes network backend interfaces and a virtual switch or bridge. The driver domain includes a PCI driver for direct communication with a network interface card. The driver domain includes hooking software and an inspection agent. Packets passing between the other virtual machines pass through the driver domain, are hooked, and are inspected by inspection agent to determine if they are malicious or not. Malicious packets are blocked. The driver domain may also utilize a PCI driver of the privileged domain for access to the network interface card. Platforms with or without pass-through mode may be used.

Description

FIELD OF THE INVENTION[0001]The present invention relates generally to the detection of malicious software on a computer. More specifically, the present invention relates to detection of malicious software between virtual machines in a virtualization environment.BACKGROUND OF THE INVENTION[0002]While malicious software certainly targets traditional computers running a single operating system, it can also target a computer running multiple operating systems. In a virtualization environment, different operating systems may run on a single computer and these may be subject to unique types of malicious software.[0003]FIG. 1 illustrates a prior art virtualization environment 10 that is subject to attacks by malicious software. Any suitable computer hardware 20 executes a virtualization platform 30 which is a layer of software running directly on the computer hardware and which replaces the traditional operating system. The platform 30 allows the computer hardware to execute multiple oper...

AI Technical Summary

Benefits of technology

This patent describes a way to improve network security in a virtualization environment. By using a driver domain that acts as a security monitor, network service control can be moved from a privileged domain to the driver domain. This eliminates the need for extra relay mechanisms and allows for better security development. Additionally, the architecture is simple and no inter-VM communication is needed to share packets, resulting in improved performance.

Problems solved by technology

While malicious software certainly targets traditional computers running a single operating system, it can also target a computer running multiple operating systems.

In a virtualization environment, different operating systems may run on a single computer and these may be subject to unique types of malicious software.

Unfortunately, the nature of a virtualization environment allows for a new threat called an inter-VM attack, in which malicious software under one operating system attacks programs and data under another operating system executing on the same host computer.

Inter-VM attacks can be especially problematic in a public virtual cloud environment.

Traditional network security software not accustomed to a virtualization environment has difficulty detecting or containing malicious inter-VM traffic between the virtual machines.

More and more, a great deal of data center network traffic occurs between virtual machines on a host computer server, but, administrators find it more and more difficult to monitor such virtual machine traffic or to implement inspection or filtering policies.

While one approach might be to scale back any virtualization efforts, this defeats the promise that virtualization offers in terms of economic benefits.

Physical security products cannot detect attacks that go from one virtual machine to another on the same host computer.

And, attempting to send all inter-VM traffic out to the network in order to detect these inter-VM attacks is undesirable because it increases network latency.

Another approach might be to install security software on each virtual machine; but, this consumes resources on each machine and the need to manage the security software of each virtual machine.

Installing security software in the host system is a possibility but this technique might not be portable between diverse host systems.

Images