Malware Detection Method Based on Hardware Performance Counter Time-Series Data

Abstract

This invention belongs to the field of computer technology, and specifically relates to a malware detection method based on hardware performance counter timing data. The method uses the `perf stat` command provided by the Perf toolset of the Linux system to obtain hardware performance counter timing data of malware and benign software from Linux containers. The obtained hardware performance counter timing data is preprocessed using a Python program: the raw timing data is cleaned and saved as a CSV file. The preprocessed hardware performance counter timing data is then input into a Long Short-Term Memory (LSTM) recurrent neural network for training to obtain the final malware detection model. The classification results of the malware detection model are interpreted using SHAP values ​​to obtain time slices and hardware performance counter features with high contribution. Finally, the Perf toolset of the Linux system is used to obtain system call timing data of malware and benign software from Linux containers.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a method for detecting malware based on hardware performance counter timing data. Background Technology

[0002] With the development of the internet, malware attacks have rapidly escalated. Malware is increasingly employing sophisticated techniques to evade detection and analysis. For example, some malware utilizes encryption, obfuscation, polymorphism, and metamorphism to evade static detection, while anti-debugging and anti-virtual machine methods allow malware to identify its current virtual environment, thus refraining from performing genuine malicious behavior to evade dynamic detection. Meanwhile, to enhance detection capabilities, deep learning models trained on massive amounts of data have been widely adopted. However, due to a lack of interpretability, these models are considered black boxes. Malware detection plays a crucial role in system security, and relying on such opaque models for critical decision-making poses significant security risks.

[0003] Static detection methods focus on analyzing the static characteristics of malware, including byte sequences, assembly instructions, strings, and opcodes, to achieve rapid scanning. However, some malware employs obfuscation and encryption techniques to hide its distinguishable static characteristics. Therefore, more advanced techniques are needed to identify malware. Dynamic detection methods require dynamically running samples and collecting data during their execution. Current popular dynamic detection methods mainly use virtual environments to collect data, but malware employing anti-debugging or anti-virtual machine techniques can detect virtual environments, leading to data distortion.

[0004] Hardware performance counters are a set of registers used to store hardware-related activities. These registers reside on the CPU, making them highly reliable and difficult to tamper with or interfere with from the outside world. Hardware performance counters were originally designed for debugging software performance, indicating that their values ​​can effectively reflect the behavioral characteristics of upper-layer software. Malware behavior differs significantly from benign software behavior; therefore, this invention aims to identify corresponding malware behaviors through the timing data characteristics of hardware performance counters. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a malware detection method based on hardware performance counter time-series data. Using the `perf stat` command provided by the Perf toolset of the Linux system, hardware performance counter time-series data of malware and benign software are obtained from Linux containers. Then, a Long Short-Term Memory (LSTM) recurrent neural network is used as a deep learning model. The hardware performance counter time-series data obtained in the previous step is preprocessed using a Python program: the raw time-series data is cleaned and saved as a CSV file. The preprocessed hardware performance counter time-series data is input into the LSM recurrent neural network for training, obtaining the final malware detection model. The classification results of the malware detection model are interpreted using SHAP values ​​to obtain time slices and hardware performance counter features with high contribution. Then, the Perf toolset of the Linux system is used to obtain system call time-series data of malware and benign software from Linux containers. Combining the hardware performance counter time-series data with software behavior analysis further reveals the correlation between hardware performance counters and software behavior, enhancing the interpretability of this detection method.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The malware detection method based on hardware performance counter time-series data is characterized by the following steps:

[0008] 1) Deploy malware samples and benign software samples into Linux containers respectively. Use the Perf toolset provided by the Linux system to collect the hardware performance counter timing data of the corresponding Linux containers. Use a Python program to preprocess the obtained hardware performance counter timing data so that it can be input into the long short-term memory recurrent neural network.

[0009] 2) A long short-term memory recurrent neural network is used as a deep learning model. The preprocessed hardware performance counter time series data is input into the long short-term memory recurrent neural network for training to obtain a malware detection model.

[0010] 3) SHAP graphs are generated using Python programs. The classification results of the malware detection model are interpreted based on SHAP graphs with different features at different time slices, making the detection model interpretable. The Perf toolset provided by the Linux system is used to obtain system call timing data of malware and benign software from Linux containers. Combined with hardware performance counter timing data, software behavior is analyzed to further reveal the correlation between hardware performance counters and software behavior.

[0011] A further optimization of this technical solution is that the time-series data preprocessing method in step 1) is as follows:

[0012] 1.1) Install the Linux system on the experimental machine and build a Linux container on the system;

[0013] 1.2) Use the lxc file command to transfer malware and benign software samples into a Linux container;

[0014] 1.3) Use an automated data acquisition script to collect timing data of hardware performance counters. The specific command for data acquisition in the script is: perf stat -e [hardware performance counter event to be collected] -I [output hardware performance counter value at fixed time intervals] -o [path to output data file].

[0015] The parameter following `-e` specifies the exact hardware performance counter events to be collected. This patent requires the collection of data from 16 hardware performance counter events: branch-instructions, branch-misses, cache-misses, cache-references, CPU-cycles, instructions, L1-dcache-load-misses, L1-dcache-loads, and L1-dcache-stores. The data counters are: L1-icache-load-misses (number of L1 instruction cache load misses), branch-loads (number of branch loads), branch-load-misses (number of branch instruction load misses), dTLB-load-misses (number of data page table cache load misses), dTLB-store-misses (number of data page table cache store misses), iTLB-load-misses (number of instruction page table cache load misses), and bus-cycles (number of bus cycles). To prevent time-division multiplexing of hardware performance counters, the 16 hardware performance counter events are collected in 4 groups, with 4 hardware performance counter events in each group.

[0016] The parameter following the -I command above is 100. This means that the value of the hardware performance counter collected is output every 100ms.

[0017] The parameter following the -o option in the above command, / home / Users / Desktop / data / , means that the data collected by the above command will be output to the directory " / home / Users / Desktop / data / ".

[0018] After the above command is executed, the script uses the `sleep 5s` command to control the runtime of the `perf` tool. Therefore, in this patent, the `perf` tool outputs the values ​​of the hardware performance counter events to be collected 50 times within 5 seconds. After the `sleep 5s` command, the `kill` command is used to close the `perf` tool and stop data collection.

[0019] 1.4) Preprocess the data obtained in step 1.3). Use a Python program to clean the raw timing data of the hardware performance counter, removing invalid words: time, events, started, on, and invalid information: the time of acquisition of the hardware performance counter timing data, the specific information of the container used, and setting the performance counter data that did not have a count to 0.

[0020] 1.5) Save the data obtained in step 1.4) as a CSV file.

[0021] Further optimization of this technical solution, step 2) is as follows:

[0022] 2.1) Design a long short-term memory recurrent neural network model and train the long short-term memory recurrent neural network model with the data preprocessed in step 1); wherein, the label of malicious software is 1 and the label of benign program is 0.

[0023] 2.2) The malware detection model is a modified long short-term memory recurrent neural network. It solves the problems of gradient vanishing and exploding by introducing input gates, forget gates and output gates to control the control memory. The cell state of the long short-term memory recurrent neural network can access and store long-term memory information, enabling the long short-term memory recurrent neural network to remember previous information when processing sequential data.

[0024] This technical solution is further optimized. The Long Short-Term Memory (LSTM) recurrent neural network structure is as follows: Two LSTM layers. The first LSTM layer has 256 neurons, and each input sample is a 50*16 matrix. The activation function of the LSTM layer is hyperbolic tangent, and the activation function of the loop step is linear rectified unit. The second LSTM layer has the same structure as the first layer, but this LSTM layer only returns the last output of the sequence, instead of the output of the complete sequence. A flattening layer is added to transform the multidimensional data into one-dimensional data. After the flattening layer, a dropout layer is added. The dropout layer in this network is set to a ratio of 0.1 to randomly discard some neuron outputs to prevent overfitting. Then, three fully connected layers are added. The first fully connected layer has 64 neurons and uses ReLU as the activation function. The second fully connected layer has 32 neurons that receive the 64 neurons from the previous fully connected layer as input. The third fully connected layer is the output layer of the network, with 2 neurons, used to output whether the software is malicious software.

[0025] Further optimization of this technical solution, step 3) specifically includes the following steps:

[0026] 3.1) Use a Python program to generate the SHAP values ​​of each hardware feature used in the experiment for different time slices, and use Python functions to plot them into a graph to intuitively analyze the contribution of each hardware feature to the model classification effect at different time slices;

[0027] 3.2) Use a Python program to generate SHAP values ​​for different hardware features in the same time slice, and use Python functions to plot them into a graph to intuitively show the contribution of different hardware features in the same time slice to the model classification performance;

[0028] 3.3) Combining the SHAP graphs obtained in steps 3.1) and 3.2), locate the time slice with the greatest distinguishing effect between malicious and benign software, further locate the hardware features that contribute the most to distinguishing between benign and malicious software in this time slice, generate a ranking graph of the top ten contributing hardware features and feed it back to the users to describe the behavioral characteristics of the samples.

[0029] 3.4) Use the perf record command of the Perf tool to collect system call timing data of malware and benign software in a Linux container environment;

[0030] 3.5) Using a Python program, align the hardware feature contribution ranking chart generated in step 3.3) and the system call timing data collected in step 3.4) with the time axis to generate a corresponding comparison chart of hardware features and their system calls. According to the comparison chart, users can see the system calls corresponding to the top ten hardware features with the greatest contribution to the classification in the time slice. This helps users analyze the differences between malicious and benign software in high-level semantics, as well as the inherent correlation between hardware performance counter timing data and system call timing data, thereby enhancing the reliability and transparency of the model.

[0031] Unlike existing technologies, the above technical solution has the following beneficial effects:

[0032] 1) The malware detection method proposed in this invention uses hardware performance counter time series as hardware fingerprints to identify malware. This method can effectively detect malware that uses evasion techniques.

[0033] 2) This invention utilizes an interpretable long short-term memory recurrent neural network model to further verify the correlation between hardware performance counter timing data and software behavior by analyzing system call and hardware performance counter timing data. These features make the malware detection method proposed in this invention more transparent and reliable. Attached Figure Description

[0034] Figure 1 Here is a flowchart of a malware detection method based on hardware performance counter timing data;

[0035] Figure 2 This is a simplified diagram of a long short-term memory recurrent neural network. Detailed Implementation

[0036] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0037] like Figure 1 As shown, the malware detection method based on hardware performance counter timing data provided by this invention includes the following steps performed in sequence:

[0038] 1) Deploy malware samples and benign software samples into Linux containers respectively. Use the Perf toolset provided by the Linux system to collect the timing data of the hardware performance counters of the corresponding containers. Use a Python program to preprocess the obtained timing data of the hardware performance counters so that it can be input into the long short-term memory recurrent neural network.

[0039] 1.1) Install the Linux system on the experimental machine and build a Linux container on the system;

[0040] 1.2) Use the lxc file command to transfer malware and benign software samples into a Linux container;

[0041] 1.3) Use an automated data acquisition script to collect timing data of hardware performance counters. The specific command for data acquisition in the script is: perf stat -e [hardware performance counter event to be collected] -I [output hardware performance counter value at fixed time intervals] -o [path to output data file].

[0042] The parameter following `-e` specifies the exact hardware performance counter events to be collected. This patent requires the collection of data from 16 hardware performance counter events: branch-instructions, branch-misses, cache-misses, cache-references, CPU-cycles, instructions, L1-dcache-load-misses, L1-dcache-loads, and L1-dcache-stores. The data counters are: L1-icache-load-misses (number of L1 instruction cache load misses), branch-loads (number of branch loads), branch-load-misses (number of branch instruction load misses), dTLB-load-misses (number of data page table cache load misses), dTLB-store-misses (number of data page table cache store misses), iTLB-load-misses (number of instruction page table cache load misses), and bus-cycles (number of bus cycles). To prevent time-division multiplexing of hardware performance counters, the 16 hardware performance counter events are collected in 4 groups, with 4 hardware performance counter events in each group.

[0043] The parameter following the -I command above is 100. This means that the value of the hardware performance counter collected is output every 100ms.

[0044] The parameter following the -o option in the above command, / home / Users / Desktop / data / , means that the data collected by the above command will be output to the directory " / home / Users / Desktop / data / ".

[0045] After the above command is executed, the script uses the `sleep 5s` command to control the runtime of the `perf` tool. Therefore, in this patent, the `perf` tool outputs the values ​​of the hardware performance counter events to be collected 50 times within 5 seconds. After the `sleep 5s` command, the `kill` command is used to close the `perf` tool and stop data collection.

[0046] 1.4) Preprocess the data obtained in step 1.3). Use a Python program to clean the raw timing data of the hardware performance counters, removing invalid words: time, events, started, on, and invalid information: the time information of hardware performance counter timing data acquisition and the specific information of the container used. Set the performance counter data that has not been counted to 0.

[0047] 1.5) Save the data obtained in step 1.4) as a CSV file.

[0048] 2) The Long Short-Term Memory Recurrent Neural Network (LSTM Recurrent Neural Network) designed in this invention is used as a deep learning model. Preprocessed hardware performance counter time-series data is input into the LSTM Recurrent Neural Network for training to obtain a malware detection model.

[0049] 2.1) Design a long short-term memory recurrent neural network model and read the CSV file formed in step 1.5). The data in the CSV file is used to train the long short-term memory recurrent neural network model. The label for malicious programs is 1, and the label for benign programs is 0.

[0050] 2.2) The malware detection model is a modified Long Short-Term Memory (LSTM) recurrent neural network that addresses the vanishing and exploding gradient problems by introducing input, forget, and output gates to control the unit memory. The input gate determines how much new information is added to the cell state. It consists of a sigmoid layer and a tanh layer. The sigmoid layer determines which information is updated, and the tanh layer creates new candidate values ​​that may be added to the cell state. The forget gate determines how much information is discarded from the cell state. It uses a sigmoid layer to determine which information in each state cell should be retained and which should be discarded. If the output of the forget gate is close to 0, the information is discarded; if it is close to 1, the information is retained. The output gate determines what the next hidden state should be. The hidden state contains a portion of the cell state but is filtered through a sigmoid layer and a tanh layer, so only specific information is output. The cell states of the LSM recurrent neural network can access and store long-term memory information, enabling the LSM recurrent neural network to remember previous information when processing sequential data. The Long Short-Term Memory (LSTM) recurrent neural network structure designed in this invention consists of two LSTM layers. The first LSTM layer has 256 neurons, with each input sample being a 50*16 matrix. The activation function for the LSTM layer is specified as hyperbolic tangent (tanh), and the activation function for the specified loop step is Rectified Linear Unit (ReLU). The second LSTM layer has the same structure as the first layer, but this LSTM layer only returns the last output of the sequence, instead of the complete sequence output. A flattening layer is added to the second LSTM layer to transform the multidimensional data into one-dimensional data. A dropout layer is added after the flattening layer. The dropout layer in this network has a ratio of 0.1 to randomly discard some neuron outputs to prevent overfitting. After the dropout layer, three fully connected layers are added. The first fully connected layer has 64 neurons, with ReLU activation function. The second fully connected layer has 32 neurons that receive the 64 neurons from the previous fully connected layer as input. The third fully connected layer is the network's output layer, with 2 neurons, used to output whether the software is malicious software.

[0051] 3) SHAP graphs are generated using Python. The classification results of the malware detection model are interpreted based on SHAP graphs with different features at different time slices. These graphs reveal which time slices and hardware performance counter features have a significant impact on the model's classification results during malware detection. This allows us to determine which time slices exhibit the most aggressive malicious behavior, making the detection model's results interpretable. Furthermore, the Perf toolkit provided by the Linux system is used to obtain system call timing data of both malicious and benign software from Linux containers. Combining this system call timing data with SHAP graphs of different features at different time slices allows us to identify the time slices where malicious behavior is most intense and which system call functions were invoked by the malware during these peak times. This further reveals the correlation between hardware performance counters and software behavior, enhancing the transparency of this detection method.

[0052] 3.1) Use a Python program to generate the SHAP values ​​of each hardware performance counter event at different time slices used in the experiment, and use Python functions to plot them into a graph to intuitively analyze the contribution of each hardware feature to the model classification effect at different time slices;

[0053] 3.2) Use a Python program to generate SHAP values ​​for different hardware features in the same time slice, and use Python functions to plot them into a graph to intuitively analyze the contribution of different hardware features in the same time slice to the model classification performance.

[0054] 3.3) Combining the SHAP graphs obtained in steps 3.1) and 3.2), locate the time slice with the highest differentiation between malicious and benign software, and further locate the hardware features that contribute the most to differentiating between benign and malicious software within this time slice. Generate a ranking graph of the top ten contributing hardware features and provide it to the users to describe the behavioral characteristics of the samples.

[0055] 3.4) Use the perf record command of the Perf tool to collect system call timing data of malicious and benign software in a Linux container environment;

[0056] 3.5) Using a Python program, align the hardware feature contribution ranking chart generated in step 3.3) and the system call time series data collected in step 3.4) with the time axis to generate a corresponding comparison chart of hardware features and their system calls. According to the comparison chart, users can see the system calls corresponding to the top ten hardware features with the greatest contribution to the classification in the time slice. This helps users analyze the differences between malicious and benign software in high-level semantics, as well as the inherent correlation between hardware performance counter time series data and system call time series data, thereby enhancing the reliability and transparency of the model.

[0057] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." or "including..." does not exclude the presence of additional elements in the process, method, article, or terminal device that includes said element. Additionally, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the stated number; "above," "below," "within," etc., are understood to include the stated number.

[0058] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A malware detection method based on hardware performance counter timing data, characterized in that, Includes the following steps: Step 1) Deploy malware samples and benign software samples into Linux containers respectively. Use the Perf toolkit provided by the Linux system to collect the hardware performance counter timing data of the corresponding Linux containers. Use a Python program to preprocess the obtained hardware performance counter timing data so that it can be input into the long short-term memory recurrent neural network. Step 2) Using a long short-term memory recurrent neural network as a deep learning model, the preprocessed hardware performance counter time series data is input into the long short-term memory recurrent neural network for training to obtain a malware detection model. Step 3) Generate SHAP graphs using Python. Interpret the classification results of the malware detection model based on SHAP graphs with different features at different time slices, making the detection model interpretable. Then, use the Perf toolset provided by the Linux system to obtain system call timing data of malware and benign software from Linux containers. Combine this with hardware performance counter timing data to analyze software behavior, further revealing the correlation between hardware performance counters and software behavior. Step 3) specifically includes the following steps: 3.1) Use a Python program to generate the SHAP values ​​of each hardware feature used in the experiment for different time slices, and use Python functions to plot them into a graph to intuitively analyze the contribution of each hardware feature to the model classification effect at different time slices; 3.2) Use a Python program to generate SHAP values ​​for different hardware features in the same time slice, and use Python functions to plot them into a graph to intuitively show the contribution of different hardware features in the same time slice to the model classification performance. 3.3) Combining the SHAP graphs obtained in steps 3.1) and 3.2), locate the time slice with the greatest distinguishing effect between malicious and benign software, further locate the hardware features that contribute the most to distinguishing between benign and malicious software in this time slice, generate a ranking graph of the top ten contributing hardware features and feed it back to the users to describe the behavioral characteristics of the samples. 3.4) Use the perf record command of the Perf tool to collect system call timing data of malicious and benign software in a Linux container environment; 3.5) Using a Python program, align the hardware feature contribution ranking chart generated in step 3.3) and the system call time series data collected in step 3.4) with the time axis to generate a corresponding comparison chart of hardware features and their system calls. According to the comparison chart, users can see the system calls corresponding to the top ten hardware features with the greatest contribution to the classification in the time slice. This helps users analyze the differences between malicious and benign software in high-level semantics, as well as the inherent correlation between hardware performance counter time series data and system call time series data, thereby enhancing the reliability and transparency of the model.

2. The malware detection method based on hardware performance counter timing data as described in claim 1, characterized in that, The time series data preprocessing method in step 1) is as follows: Step 1.1) Install the Linux system on the experimental machine and build a Linux container on the system; Step 1.2) Use the lxc file command to transfer malware and benign software samples into the Linux container; Step 1.3) Use an automated data acquisition script to collect timing data from hardware performance counters; Step 1.4) Preprocess the data obtained in Step 1.3) Use a Python program to clean the raw timing data of the hardware performance counter, removing invalid words: time, events, started, on, and invalid information: the time of acquisition of hardware performance counter timing data, the specific information of the container used, and set the performance counter data that did not have a count to 0. Step 1.5) Save the data obtained in step 1.4) as a CSV file.

3. The malware detection method based on hardware performance counter timing data as described in claim 1, characterized in that, Step 2) is as follows: Step 2.1) Design a long short-term memory recurrent neural network model and train the long short-term memory recurrent neural network model with the data preprocessed in Step 1); where the label of malicious software is 1 and the label of benign program is 0. Step 2.2) The malware detection model is a modified long short-term memory recurrent neural network, which controls the unit memory by introducing input gates, forget gates and output gates. The cell state of the long short-term memory recurrent neural network can access and store long-term memory information, enabling the long short-term memory recurrent neural network to remember previous information when processing sequential data.

4. The malware detection method based on hardware performance counter timing data as described in claim 3, characterized in that, The structure of the Long Short-Term Memory (LSTM) recurrent neural network is as follows: Two LSTM layers are used. The first LSTM layer has 256 neurons, with each input sample being a 50*16 matrix. The activation function for the LSTM layer is hyperbolic tangent, and the activation function for the loop step is a linear rectified unit (RCU). The second LSTM layer has the same structure as the first layer, but this LSTM layer only returns the last output of the sequence, instead of the complete sequence output. A flattening layer is added to transform the multidimensional data into one-dimensional data. After the flattening layer, a Dropout layer is added. The Dropout layer in this network has a ratio of 0.1 to randomly discard some neuron outputs to prevent overfitting. Three fully connected layers are then added. The first fully connected layer has 64 neurons, with ReLU activation. The second fully connected layer has 32 neurons that receive the 64 neurons from the previous fully connected layer as input. The third fully connected layer is the network's output layer, with 2 neurons, used to output whether the software is malicious software.

Images