Malware Detection System and Methodology Through Hypervisor-Level Virtual Machine Insight

Abstract

The invention relates to a system and method for providing agentless security by externally reading virtual machine memory at the hypervisor level without requiring the installation of a security agent within the virtual machine, converting raw memory data into meaningful operating system structures, evaluating discrepancies between the operating system-reported state and the actual memory state through signature matching and behavioral anomaly analysis to detect concealed malware, and isolating, suspending, or generating an alarm for the virtual machine based on the detected threat.

Description

1 TARIFF Malware Detection System with Hypervisor-Level Virtual Machine Insight and Method TECHNICAL FIELD The invention enables malware detection through virtual machine introspection at the hypervisor level. It is related to a system and method of implementation. The invention has implications for telecommunications data, particularly in the fields of cloud computing and virtualization technologies. In order to ensure the cybersecurity of the servers hosted in the centers, virtual Direct virtualization without installing any additional security software on the machines. a hypervisor-based security system in which threats are detected via infrastructure and hypervisor. and is related to the method. 15 PREVIOUS TECHNIQUE In current systems, the security of virtual servers is usually installed within the operating system. Antivirus or Endpoint Detection and Response (EDR) 20 This is achieved through agents. However, this traditional approach has two fundamental shortcomings. Advanced rootkits, also known as kernel-level kits, are used for rooting. Malicious software can hijack the operating system's kernel, allowing them to hide inside. They can conceal this from security agents and create misleading reports to make it appear as if the system is clean. It can offer this. However, a separate security agent must be installed on each virtual machine and 25 Continuous operation of these agents significantly consumes the server's processor and memory resources. By consuming it, it leads to a performance penalty. This situation, in addition to the risk of security breaches, there is also the inefficient use of data center resources. This reveals a technical problem. As a result of research conducted in the literature, publication number “WO2015065330A1” and “Virtual A patent document titled "virtual machine introspection" was found. The document in question describes offline analysis of the virtual machine hard disk image, kernel Identifying operating system files, a symbol for operating system structure symbols retrieved from the server and using the symbols in question, 35 of the virtual machine memory It is related to analysis through an introspective approach via the hypervisor. However, the mentioned The document states that physical RAM blocks are used while the target virtual machine continues to operate normally. 2 Reading a snapshot or live stream from an external source, the status reported by the operating system. Analyzing the discrepancy between the actual memory state and the detected discrepancy, and using the resulting security findings. an indication of an integrated structure in the form of policy and transfer to the intervention unit None have been encountered. As a result of research conducted in the literature, publication number “US20160224789A1” and “System and "Method for Hypervisor-Based Security" A patent document with the title [patent document title] was found. This document concerns a device located on the hypervisor. probes of system calls and kernel inputs during the virtual machine's boot process Monitoring the objects, comparing the obtained data with a reliable reference image, 10 Identifying anomalies in the root user kit and securing the virtual machine. It is related to being quarantined through policies. However, in the document mentioned, the target While the virtual machine continues its normal operation, physical RAM blocks are accessed from outside. Reading raw memory data in the form of process lists, kernel modules, and hooks converting to operating system structures, the process list reported by the operating system and the actual 15 Identifying discrepancies between the process lists in the memory map and memory using a signature matching algorithm and a behavioral anomaly detection engine in the printouts No evidence has been found regarding an integrated structure in the form of analysis. Ultimately, the problems mentioned above, which cannot be solved with the current technology, are related to the technical 20 This has made it necessary to make an innovation in the field. A BRIEF DESCRIPTION OF THE INVENTION The invention enables the installation of a security agent within a virtual machine without requiring the installation of a hypervisor 25. At this level, the virtual machine reads and analyzes the memory from the outside to operate itself. It detects malware that hides from the system and, based on the detection result, acts as a virtual machine. It relates to a system and method that applies safety intervention to a machine. The main purpose of the invention is to perform security scanning in a hypervisor 30 isolated from a virtual machine. by performing this action at the security layer, it prevents malware from accessing the security system or The goal is to prevent the security system from shutting down. Another purpose of the invention is to eliminate the need to install a separate security agent on each virtual machine. by eliminating it, reducing the load on processor and memory resources, target virtual 35 memory without causing performance loss or interruption in the normal operation of the machine 3 to conduct analysis and to use data center resources more efficiently to provide. Another objective of the invention is to create a root user kit and kernel-level interface hidden from the operating system. Detecting malware by externally analyzing memory traces at the hardware level. 5 to do. Another purpose of the invention is to follow predefined security rules when a threat is detected. According to this, disconnecting the virtual machine from the network, freezing the virtual machine, or Intervention through quarantine or alerting security teams 10 to provide. All the purposes mentioned above and those that will emerge from the detailed explanation below. The invention aims to achieve malicious virtual machine introspection at the hypervisor level. It is a system that performs software detection, and its feature is; 15  physical hardware that runs on and hosts virtual machines, physical hardware It allocates resources to virtual machines, manages the entire system, and a hypervisor layer that manages all memory and processor resources,  Hosted on the hypervisor layer, containing the operating system and a target virtual environment where applications are running and undergoing security scanning machine,  By operating at the hypervisor level, it accesses the target virtual machine's memory from an external source. Reading, analyzing raw memory data, and detecting malware. a VMI and analysis module that performs,  Based on threat information from the VMI and analysis module, 25 predefined a security policy and response unit that enforces security rules, It includes. The structure of the invention and the best understanding of its advantages, including additional components. This should be evaluated together with the figures explained below. 30 BRIEF DESCRIPTION OF THE FIGURES Figure 1 shows the hypervisor layer, target virtual machine, VMI, and analysis module of the system described in the invention. This is a representative block diagram showing the relationship between security policy and the intervention unit. 35 Figure 2 is a representative flowchart illustrating the steps of the method described in the invention. 4 The drawings do not necessarily need to be scaled and are necessary for understanding the invention. Details that are not present may have been overlooked. Furthermore, at least to a significant extent... Identical elements or elements with similar functions are numbered the same way. It is shown. REFERENCE NUMBERS 1. Hypervisor layer 2. Target virtual machine 3. VMI and analysis module 10 4. Security policy and intervention unit 1001. Administering physical hardware resources to the target virtual machine via the hypervisor layer. sharing, and the management of the entire system is provided by the hypervisor layer, and The VMI and analysis module that performs the security scan from the target virtual machine is 15 by completely isolating it, preventing malware from accessing the security system or the security making it physically impossible to shut down the system 1002. A system that houses user applications and the operating system, and is vulnerable to cyberattacks. without any security software installed on the target virtual machine and without the user being aware that they are being monitored. While continuing its normal operations, the hypervisor layer 20 of the target virtual machine's memory will be accessed without interruption. scanning as a passive target from the outer layer 1003. VMI and analysis module for raw data in the RAM memory of the target virtual machine. direct reading by the operating system without requiring reference to the raw data, making the raw data meaningful. conversion to operating system structures and the status reported by the operating system and in memory By analyzing the discrepancies between the actual situation and the 25 malicious actors who conceal themselves... software detection 1004. The security policy and the signal indicating that a threat has been detected from the VMI and analysis module. to be picked up by the intervention unit and targeted according to predetermined security rules. Disconnecting the virtual machine's network connection at the hypervisor layer level, the target virtual at least 30 of the processes of freezing, quarantining or generating an alarm on the machine. the realization of one DETAILED DESCRIPTION OF THE INVENTION This detailed explanation describes the invention, which involves hypervisor-level virtual machine introspection, and 35 The malware detection system and method are solely for the purpose of better understanding the issue. This is explained with examples that will not create any limiting effects. The hypervisor layer (1) included in the invention is a virtual layer that runs on physical hardware. hosting machines, allocating physical hardware resources to virtual machines, all It is the main platform that manages the system and controls all memory and processor resources. The target virtual machine (2) is hosted on the hypervisor layer (1), and runs in it. The system runs user applications, is protected, and is vulnerable to cyberattacks. The virtual server to be scanned is the VMI and analysis module (3), hypervisor layer (1) It runs at the level and completely isolated from the target virtual machine (2), the target virtual reads the machine's (2) memory from outside without contacting the operating system and malicious software It is the main discovery engine that performs the analysis. Security policy and response unit (4), VMI 10 and predetermined security rules according to threat information from the analysis module (3) by activating the isolation, stop or alarm on the target virtual machine (2) It is the decision-making mechanism that performs the appropriate of the operations. Security scan VMI and analysis module (3) performing the action is completely isolated from the target virtual machine (2) This allows malware to access the security system or compromise security 15 It becomes physically impossible to shut down the system. Within the scope of the VMI and analysis module (3), a memory integrated into the hypervisor core The tracing module is a signature module that analyzes memory dumps in real-time. a matching algorithm and a behavioral anomaly that detects encrypted or hidden processes 20 The memory monitoring module uses the physical memory used by the target virtual machine (2). It reads memory blocks from the outside and transfers them to the VMI and analysis module (3). Signature matching The algorithm evaluates data from memory dumps in real-time and detects behavioral anomalies. The detection engine identifies anomalous behavior related to encrypted or hidden processes. This structure Thanks to this, even if the malware hides itself from the operating system, the hardware 25 Memory traces at this level are externally visible and analyzable. During the Monitoring & Acquisition phase, the system uses the hypervisor layer. (1) runs as a continuously active service on the target virtual machine (2), user The working environment, which houses the applications and the operating system, is vulnerable to cyberattacks. 30 as it continues its normal operation, for example, a database application When running, the VMI and analysis module (3) are activated. The VMI and analysis module (3) activates the target virtual without asking permission from the operating system of the machine (2) or informing the operating system, a snapshot of the physical RAM blocks used by the target virtual machine (2) or reads the live stream. 35 Security software is installed in the target virtual machine (2). is not, it continues its normal operations without noticing that it is being monitored and its memory is stored in the hypervisor layer (1) 6 It is scanned as a passive target from the outer layer. During this process, the target is virtual. No performance loss or interruption occurs on the machine (2). VMI and analysis module (3) during the semantic analysis and detection (Introspection & Detection) phase, It reads raw memory data (raw binary data), process list, and kernel modules 5. It converts them into meaningful operating system structures such as kernel modules and hooks, and It compares the structures within memory to known healthy kernel structures. For example, a root when the user kit hides itself from the task manager in the target virtual machine (2) The operating system may not be able to see the process in question. VMI and analysis module (3) access memory from outside. It detects the hidden code block from what it's looking at and includes it in the process list reported by the operating system. discrepancy between the process list in the actual memory map (process discrepancy) determines. VMI and analysis are used when a threat is detected during the response and isolation phase. module (3) sends the signal that a threat has been detected to the security policy and response unit (4) 15 It transmits. Security policy and response unit (4), predefined rules Within this framework, the network connection of the target virtual machine (2) through the hypervisor layer (1) By cutting it off, you can prevent the attack from spreading to other systems, target virtual machine (2) They can quarantine or freeze them, or generate an alarm for security teams. The steps involved in the process are as follows:  Physical hardware resources to the target virtual machine (2) hypervisor layer (1) sharing over the entire system management is done through the hypervisor layer (1) VMI and analysis provided by and performing security scans module (3) completely isolated from the target virtual machine (2) malicious 25 software accessing or disabling the security system making it physically impossible (1001),  Containing user applications and the operating system, and vulnerable to cyberattacks without any security software installed in the target virtual machine (2) and While continuing its normal operations without realizing it is being monitored, the target virtual machine (2) 30 as a passive target from the outer layer via the hypervisor layer (1) of its memory scanning (1002),  VMI and analysis module of raw data in RAM memory of the target virtual machine (2) (3) can be read directly by the operating system without resorting to the raw transforming data into meaningful operating system structures and the operating system's 35 analysis of discrepancies between the reported situation and the actual situation in memory Detection of malicious software that hides itself by doing so (1003), 7  Security signal from VMI and analysis module (3) regarding the detection of a threat policy and intervention unit (4) to be taken and predetermined According to the security rules, the network connection of the target virtual machine (2) to the hypervisor interruption at layer (1) level, freezing of the target virtual machine (2), At least one of the following procedures must be completed: quarantine or alarm generation. implementation (1004). The working principle of the invention is that instead of installing the security agent into the target virtual machine (2) It is based on the principle of analyzing memory from an external observer's perspective. All monitoring and analysis... and the intervention cycle isolates the area beyond the reach of the malware's manipulation capability. Since the attack is executed on the modified hypervisor layer (1), the attacker's security It becomes technically impossible to disable or bypass the measure.

Claims

8 REQUESTS 1. Malware detection through virtual machine introspection at the hypervisor level. It is a system that performs the following functions;  physical hardware that runs on virtual machines, physical 5 Allocating hardware resources to virtual machines, managing the entire system. a hypervisor layer that provides and manages all memory and processor resources (1),  Hosted on the hypervisor layer (1), containing the operating system and applications run normally and without any security agents installed. a target virtual machine (2) that continues its operations,  at the hypervisor layer (1) level and isolated from the target virtual machine (2) By working, the physical RAM blocks used by the target virtual machine (2) are operated Reading raw memory data from outside the system without permission, making it meaningful... converting into operating system structures and the status reported by the operating system 15 by analyzing the discrepancies between the actual situation in memory and the situation itself A VMI and analysis module (3) that detects hidden malware.  Pre-determined according to threat information from VMI and analysis module (3) security on the target virtual machine (2) by enforcing security rules a security policy and intervention unit that carries out the intervention (4) 20 It includes.

2. The system complies with Request 1, and its feature is; VMI and analysis module (3), hypervisor A memory monitoring module integrated into its core provides real-time analysis of memory dumps. a signature matching algorithm and a 25 that detects encrypted or hidden processes It is a module that includes a behavioral abnormality detection engine.

3. The system is compliant with Request 1 and its feature is; VMI and analysis module (3), target virtual an instant copy or live stream of the physical RAM blocks used by the machine (2) It is a module that can read. 30 4. The system complies with Request 1, and its features include; VMI and analysis module (3), raw memory operating system data in the form of process lists, kernel modules and hooks. a healthy kernel that converts into structures and whose structures within memory are known. It is a module that compares structures. 35 9 5. The system is compliant with Claim 1, and its feature is the security policy and intervention unit (4), can disconnect the network connection of the target virtual machine (2) via the hypervisor layer (1), can freeze or quarantine the target virtual machine (2) or security It is a unit that can generate alarms for its teams.

6. Malware detection via virtual machine introspection at the hypervisor level. hypervisor layer (1) that works on the physical hardware to perform It is a method carried out through, and its characteristic feature is;  Physical hardware resources to the target virtual machine (2) hypervisor layer (1) Sharing of the entire system management via the hypervisor layer (1) 10 VMI and analysis provided by and performing security scans module (3) completely isolated from the target virtual machine (2) malicious software accessing the security system or the security system making it physically impossible to close (1001),  User applications and the operating system are housed in 15 vulnerable cyberattacks. no security software is installed in the target virtual machine (2) without realizing they are being watched, they continue their normal operations, the target the virtual machine's (2) memory is transferred from the hypervisor layer (1) to the outer layer scanning as a passive target (1002),  VMI and analysis of raw data in RAM memory of the target virtual machine (2) 20 The module (3) is read directly by the operating system without referring to it. Transforming raw data into meaningful operating system structures and running them. the difference between the status reported by the system and the actual status in memory Detecting hidden malware by analyzing inconsistencies (1003), 25  Security signal from VMI and analysis module (3) regarding the detection of a threat policy and intervention unit (4) to be taken and predetermined According to the security rules, the network connection of the target virtual machine (2) interruption at the hypervisor layer (1) level, target virtual machine (2) at least 30 of the freezing, quarantine or alarm generation processes the realization of one (1004) It includes the steps of the process.

7. The method is compliant with Request 6, and its characteristic is that the physical space used by the target virtual machine (2) A snapshot or live stream of RAM blocks VMI and analysis module (3) 35 It includes the step of being read by the system.

8. The method is compliant with claim 6, and its feature is that the raw memory data is processed by the VMI and analysis module (3) operating system by process list, kernel modules and hooks conversion of structures and known healthy kernel structures within memory Comparing their structures involves the process steps.

9. The method compliant with claim 6 is characterized by the memory dumps VMI and analysis module (3) instant analysis by the signature matching algorithm and encrypted or secret The steps involved in identifying processes using a behavioral abnormality detection engine. It includes.

Images