Machine learning cloud platform Trojan horse detection method and system

Abstract

The invention relates to the technical field of network security, in particular to a machine learning cloud platform Trojan horse detection method and system, and the method comprises the following steps: modeling, training and verification, network flow data acquisition, feature extraction, parallelization random forest detection, and Trojan horse identification and alarm. The machine learning-based cloud platform Trojan horse detection method and system provided by the invention have the beneficial effects that the Trojan horse is timely and accurately detected by using the time sequence characteristics of the Trojan horse, the security of the network space situation of the cloud platform is ensured, the leakage events of various privacy information and commercial confidential data are effectively avoided, and the safety of the cloud platform is improved. And the data assets in the cloud platform are protected.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a cloud platform Trojan detection method and system based on machine learning. Background Art

[0002] With the rapid development of cloud computing technology, a large number of information applications and abundant information resources have led to the emergence of more and more traps set by malicious attackers, such as Trojans. This Trojan has a small amount of code and is very easy to disguise or hide in legitimate applications. Some of these Trojans are used to execute some malicious programs, and some are used to illegally obtain permissions to set up some special backdoors for applications. Once the Trojan is executed, it will trigger some specific events, such as stealing user privacy data information and stealing cloud platform commercial confidential information. In recent years, Trojans have been widely spread in cloud platforms, posing serious threats to the security of cloud platforms and causing huge economic losses. Especially in recent years, remote office, remote conferencing, and remote control have become the norm, which has led to the rapid development of various remote control Trojans, which can steal system stored files and data, remotely monitor user desktops, modify processes, etc.

[0003] At present, the detection of Trojans is mainly based on the structural characteristics of Trojan files or dynamic behavior sandboxes. However, as Trojans evolve and evolve, existing network security inspection systems cannot accurately identify highly concealed and variable Trojans. In addition, for some remote Trojans, even if the Trojans are detected, it is very likely that private data and commercial confidential data have already been leaked.

[0004] Conventional cloud platform Trojan detection methods mainly include the following categories:

[0005] Static structural analysis is a static detection method that checks source code or binary files and matches and analyzes them based on the Trojan feature library. Since Trojans are deformed and of various types, there is theoretically no upper limit to the number of static code structure features. Therefore, the feature library of this method needs to be continuously expanded to adapt to the detection of new Trojan types, and it is difficult to detect unknown types of Trojans.

[0006] Dynamic behavior analysis is performed by running Trojan programs. During dynamic detection, testers observe and record the running status of the application through the sandbox, obtain and analyze the behavior of the application, and then determine the maliciousness of the application based on its behavioral characteristics. The analysis process of this method takes a long time, and the recognition accuracy is relatively high, but there is also a certain false negative rate. Summary of the invention

[0007] The purpose of the present invention is to provide a cloud platform Trojan detection method and system based on machine learning to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a cloud platform Trojan detection method based on machine learning, the method comprising the following steps:

[0009] Modeling, training, and verification: Build a parallelized random forest detection model, use training data to train the model, and verify the accuracy and effectiveness of the model through verification data;

[0010] Network traffic data acquisition: Real-time acquisition of network traffic data in the cloud platform, including normal traffic and abnormal traffic, and storage of the data in the form of traffic logs;

[0011] Feature extraction: Extract session information from network traffic data, including session six-tuples and transmission features;

[0012] Parallelized random forest detection: Using parallelization technology, the decision tree construction and classification process in the random forest model are parallelized to improve detection efficiency;

[0013] Trojan identification and alarm: The extracted features are classified through the parallelized random forest detection model, Trojan traffic is identified, and corresponding alarm information is generated.

[0014] Preferably, the six-tuple includes: source IP, destination IP, source port, destination port, transmission protocol, and collection time;

[0015] Transmission characteristics include: application type, URL, protocol type, number of bytes transmitted, data flow direction, and collection time.

[0016] Preferably, the construction of the parallelized random forest detection model includes:

[0017] Create multiple decision trees and distribute the construction of these decision trees to multiple parallel processes;

[0018] Each parallel process independently builds part of the decision tree and returns the completed decision tree to the main process;

[0019] The main process integrates the decision trees returned by each parallel process into a complete random forest model.

[0020] Preferably, the classification step in the parallelized random forest detection process includes:

[0021] Split the test data into multiple parts and assign each part of the data to different parallel classification processes;

[0022] Each parallel classification process uses the decision tree in the random forest model to classify the assigned test data and returns the classification results;

[0023] The main process integrates the classification results returned by each parallel classification process into a complete classification result list.

[0024] Preferably, the method further comprises the following steps:

[0025] Iterative training: For models that do not meet the detection requirements, continue iterative training until the model meets the detection requirements;

[0026] Model verification: Verify the model on the test sample data set to ensure that the model's detection performance, precision, and recall indicators meet the predetermined standards;

[0027] Trojan fingerprint library update: The detected Trojan information is included in the Trojan fingerprint library, and new categories are created in the feature library based on the characteristics of unknown Trojans.

[0028] A machine learning cloud platform Trojan detection system, the system includes the following modules:

[0029] A data acquisition module is used to obtain network traffic data from the cloud platform network environment;

[0030] A feature extraction module is used to extract session six-tuples, session timing features and transmission features from the acquired network traffic data;

[0031] Parallelized random forest detection model, used to parallelize the extracted features and detect Trojans based on the processing results;

[0032] An alarm module is used to generate an alarm message when a Trojan is detected;

[0033] The model tuning module is used to regularly tune the parallelized random forest detection model to improve Trojan detection capabilities.

[0034] Preferably, the parallelized random forest detection model includes multiple decision trees, each of which is independently constructed in a parallel manner, and the final classification result is determined by a voting mechanism to improve detection efficiency and accuracy.

[0035] Preferably, the feature extraction module specifically includes:

[0036] A session information extraction unit, used for extracting session six-tuple data from network traffic;

[0037] A timing feature extraction unit, used to extract timing features from a data packet sequence;

[0038] The transmission feature extraction unit is used to extract transmission features such as application type, URL, protocol type, and number of transmitted bytes from network traffic.

[0039] Preferably, the method for constructing a parallelized random forest detection model includes:

[0040] Divide into multiple sub-processes, each of which independently builds part of the decision tree;

[0041] Merge the decision trees constructed by each subprocess into a complete random forest;

[0042] Allocate test data to each subprocess for parallel classification processing and summarize the classification results.

[0043] Preferably, it also includes:

[0044] An iterative training module is used to iteratively train the model when the parallelized random forest detection model does not meet the detection requirements;

[0045] The verification module is used to verify the detection effect of the model on the test set, including detection performance, precision and recall, and adjust the model parameters according to the verification results;

[0046] The Trojan fingerprint library update module is used to incorporate the detected known Trojan information into the Trojan fingerprint library and create a new classification in the feature library based on the characteristics of unknown Trojans.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The machine learning cloud platform Trojan detection method and system proposed in the present invention utilize the time series characteristics of Trojans to achieve timely and accurate detection of Trojans, thereby ensuring the security of the cloud platform's cyberspace situation, effectively avoiding the occurrence of various types of privacy information and commercial confidential data leakage incidents, and protecting data assets in the cloud platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the method of the present invention;

[0050] Figure 2 This is the parallel random forest detection flow chart of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] For example, see Figure 1 to Figure 2 The present invention provides a technical solution: a cloud platform Trojan detection method based on machine learning, the method comprising the following steps:

[0053] Use the time series characteristics of Trojans to achieve timely and accurate detection of Trojan programs.

[0054] 1. Security technology for Trojan identification

[0055] Common Trojan types are: remote control, information acquisition, and destructive

[0056] (1) Remote control Trojan:

[0057] Remotely control the target computer for interactive access (real-time or non-real-time), can issue corresponding instructions to trigger the functions of malware, and can also obtain various data of the target. Its interactivity is two-way (attacker-controlled end).

[0058] (2) Information acquisition Trojans:

[0059] The function of information acquisition Trojan is to acquire information, and it can obtain valuable information from keyboard input, memory, files, databases, and browser cookies. Its interactivity is one-way interaction, where the controlled end sends data to the attacker, such as to the attacker's third-party space, file server, designated mailbox, etc., or directly opens the FTP service program, and the attacker directly accesses and downloads data.

[0060] (3) Destructive Trojans:

[0061] The function of destructive Trojans is to destroy data and consume resources of local or remote host systems. Its interactivity is also one-way, and the attacker can send instructions to the controlled end to destroy file data.

[0062] Trojans are very deceptive and can be implanted in the following ways:

[0063] (1) Trojan horse implantation

[0064] The typical method is to use web pages to install Trojan horses. Usually, it is necessary to exploit browser or related vulnerabilities, and then combine real-time news and hot topics to create related web pages, combine the vulnerabilities and web pages, and finally generate malicious web pages. When the user opens the web page, the remote Trojan program will be automatically downloaded and installed.

[0065] (2) Email implantation

[0066] The most common way of email implantation is through attachments, where the Trojan is injected when the user opens the attachment; another way is to combine email with malicious web pages. Since email supports HTML body, if the relevant vulnerabilities are implanted into the web page, the relevant effect can also be achieved.

[0067] (3) Document bundling and implantation

[0068] Through exe files, office documents, pdf document vulnerabilities, etc., the Trojan is bundled with the document. When the user opens the document, the vulnerability will be triggered, thereby releasing the Trojan or executing shellcode to perform remote attacks.

[0069] (4) Implantation through Disguise and Deception

[0070] You can modify the file extension of the exe file, change the "exe" file to "docx" to make the exe not displayed, and then modify the document class icon to disguise it.

[0071] In this solution, when identifying Trojans, it is mainly based on network traffic characteristics, using a parallelized random forest machine learning classification algorithm to analyze traffic to identify Trojan behaviors: such as transmitting malicious files, screen monitoring, voice and video interception, keyboard logging, remote shells, and illegal external communications.

[0072] The main characteristics of Trojan behavior based on traffic analysis are as follows:

[0073] (1) In most cases, the number of bytes flowing out is greater than the number of bytes flowing in. This is usually because a Trojan horse is stealing data and sending it out.

[0074] (2) The main connection is responsible for communication link control and command transmission, and is idle most of the time. After receiving the return result, the attacker will have an analysis time, which is usually much longer than the packet time interval.

[0075] (3) There are some characteristic characters of the Trojan in the request body (which vary from Trojan to Trojan).

[0076] (4) Heartbeat traffic characteristics, the Trojan client and the remote Trojan server maintain a heartbeat connection and periodically use fixed handshake protocol data.

[0077] (5) Traffic characteristics in three stages: In the connection establishment stage, there is usually a dynamic domain name (based on DGA); during data transmission, the push flag PSH is set to 1 in the message, resulting in an increase in the number of PSH messages; in the command interaction stage, the upstream and downstream traffic is asymmetric, and the proportion of small data packets is large; in the connection maintenance stage, there will be heartbeat data packets.

[0078] When processing traffic data, the system will extract the above features, create feature vectors, and store them in the traffic data archive feature library for use in machine learning.

[0079] 2. The overall process of Trojan detection

[0080] (1) Modeling, training, and verification: In the initial stage, a parallelized random forest detection model is constructed, trained, and verified.

[0081] (2) Entering the detection phase, first obtain network traffic data.

[0082] (3) Next, feature extraction is performed on the traffic, such as session six-tuple extraction, session timing feature extraction, and transmission feature extraction.

[0083] (4) Finally, by executing the parallelized random forest detection model, Trojans can be detected in a timely and accurate manner and an alert can be generated.

[0084] Repeat steps (2)(3)(4) to continue Trojan detection.

[0085] Regular model tuning needs to be performed (1) to adapt to more Trojan type detection.

[0086] In the present invention, the overall flow chart of the Trojan detection model is as follows: Figure 1 shown.

[0087] 3. Algorithmic idea and data structure of parallel random forest detection model

[0088] Random forest is a machine learning model that uses decision trees as basic classifiers. It contains multiple decision trees trained by bagging ensemble learning technology. When a sample to be classified is input, the final classification result is determined by voting on the output results of a single decision tree.

[0089] Random forest is a random forest that is built in a random way. The forest consists of many decision trees. There is no correlation between each decision tree in the random forest. After the forest is obtained, when a new input sample enters, each decision tree in the forest is allowed to make a judgment to determine which category the sample belongs to.

[0090] Random forest is an effective classification prediction method with high classification accuracy, good robustness to noise and outliers, strong generalization ability, and no overfitting problem. A top-down greedy algorithm is used to build a tree structure, and an optimal attribute is selected at each internal node for splitting. Each branch corresponds to an attribute value, and the tree is recursively built until the termination condition is met. Each leaf node represents the category of the sample along this path.

[0091] The data structures used in this invention mainly include lists and dictionaries, which are used to store the following data:

[0092] (1) Two-dimensional lists store training data and test data

[0093] (2) One-dimensional list to store classification labels

[0094] (3) Dictionary stores decision tree structure

[0095] 4. Parallelization improves processing efficiency

[0096] The construction of the decision tree model is very complex. When the number of training data, features, and attribute partitions increases further, the leaf nodes of the tree grow exponentially. Moreover, the success of random forests is based on the construction of a large number of decision trees. Of course, we can implement decision trees by using non-recursive methods, or reduce the number of selected traffic features, attribute partitions, and training data, but this adjustment is likely to affect the accuracy of random forests. Based on this, this solution solves this processing efficiency problem through parallelization and makes full use of the computer's hardware and software configuration.

[0097] The parallelization steps are as follows:

[0098] (1) First, create N trees and 4 processes. Each process is responsible for creating N / 4 decision trees. The created N / 4 decision trees are returned to the main process in the form of a list.

[0099] (2) After obtaining the decision tree lists of the four sub-processes respectively, the four sub-lists are integrated into a decision tree list L with a length of N.

[0100] (3) Create four classification processes, copy the decision tree list into four copies and pass them to the four classification processes respectively. At the same time, divide the test data into four parts, for example: [0,400] behavior part 1, [401,800] behavior part 2, [801,1200] behavior part 3, [1201,1600] behavior part 4, and pass them to the four classification sub-processes respectively.

[0101] (4) The first subprocess returns the classification results of the [1,400] rows in the form of a list, the second subprocess returns the classification results of the [401,800] rows in the form of a list, the third subprocess returns the classification results of the [801,1200] rows in the form of a list, and the fourth subprocess returns the classification results of the [1201,1600] rows in the form of a list.

[0102] (5) After obtaining the label lists of the four sub-processes respectively, the four sub-lists are integrated into a result label list with a length of 1600.

[0103] 5. Parallelize the Random Forest Trojan Detection Steps

[0104] This solution is applicable to various cloud platform application scenarios, such as cloud platform network traffic monitoring system, network anomaly intrusion detection system, etc. The specific steps of parallelized random forest detection are as follows:

[0105] (1) Network traffic data acquisition: Network traffic may be normal traffic or abnormal traffic. After data collection, it is stored in the form of traffic logs.

[0106] (2) Feature extraction: Extract the session information of network traffic, and then extract the session timing features and transmission features. The session timing features are mainly the six-tuple data in the network traffic data packet sequence (source IP, destination IP, source port, destination port, transmission protocol, collection time); the transmission features are application type, URL, protocol type, number of bytes transmitted, data flow direction, collection time and other data.

[0107] (3) Construction based on parallel random forest detection model: Since the detection efficiency of the traditional random forest detection model is not ideal, parallelization theory is introduced to realize parallel detection, thereby solving the defects of the traditional forest detection model.

[0108] (4) Ensure that the model meets the detection requirements through iterative training: Models that meet the detection requirements are stored; for models that do not meet the detection requirements, iterative training needs to be continued.

[0109] (5) Verify the model based on the test set: Verify the model detection effect on the test sample data set. When the detection performance, precision, recall and other indicators meet the requirements, stop testing and training; otherwise, continue training, adjust parameters, and optimize the detection model until the standards are met.

[0110] (6) Use the parallelized random forest detection model to identify Trojans: Determine whether a Trojan exists based on the detection results. If so, output Trojan information, intercept the application, and generate an alarm. If not, mark it as a normal application. If known, include it in the corresponding category in the Trojan fingerprint library; if unknown, create a new category for the unknown Trojan in the feature library based on the characteristics of the unknown Trojan.

[0111] The specific Trojan detection process based on parallel random forest is as follows: Figure 2 shown.

[0112] The present invention has certain advantages over traditional Trojan detection schemes in terms of real-time performance and detection results (accuracy, query rate, etc.). In a closed network security test environment, Trojan samples were selected as follows: 1,000 remote control Trojans and 1,000 information acquisition Trojans (with a black-white ratio of 7:3). Comparison tests were performed, and the following are the Trojan detection effects of the three schemes:

[0113] (1) Parallelized Random Forest Trojan Detection (this solution)

[0114] (2) Static structural feature Trojan detection (traditional solution)

[0115] (3) Dynamic behavior feature Trojan detection (traditional solution)

[0116] Table 1 Detection results of remote control Trojans

[0117]

[0118]

[0119] Table 2 Detection results of information acquisition Trojans

[0120]

[0121] Embodiment 2, based on embodiment 1, proposes a cloud platform Trojan detection system based on machine learning, the system includes the following modules:

[0122] A data acquisition module is used to obtain network traffic data from the cloud platform network environment;

[0123] The feature extraction module is used to extract the session six-tuple, session timing features and transmission features from the acquired network traffic data; the feature extraction module specifically includes:

[0124] A session information extraction unit, used for extracting session six-tuple data from network traffic;

[0125] A timing feature extraction unit, used to extract timing features from a data packet sequence;

[0126] The transmission feature extraction unit is used to extract transmission features such as application type, URL, protocol type, and number of transmitted bytes from network traffic.

[0127] The parallelized random forest detection model is used to process the extracted features in parallel and detect Trojans based on the processing results. The parallelized random forest detection model includes multiple decision trees, each of which is independently constructed in a parallelized manner, and the final classification result is determined by a voting mechanism to improve the detection efficiency and accuracy. The construction method of the parallelized random forest detection model includes:

[0128] Divide into multiple sub-processes, each of which independently builds part of the decision tree;

[0129] Merge the decision trees constructed by each subprocess into a complete random forest;

[0130] Allocate test data to each subprocess for parallel classification processing and summarize the classification results.

[0131] An alarm module is used to generate an alarm message when a Trojan is detected;

[0132] The model tuning module is used to regularly tune the parallelized random forest detection model to improve Trojan detection capabilities.

[0133] Also includes:

[0134] An iterative training module is used to iteratively train the model when the parallelized random forest detection model does not meet the detection requirements;

[0135] The verification module is used to verify the detection effect of the model on the test set, including detection performance, precision and recall, and adjust the model parameters according to the verification results;

[0136] The Trojan fingerprint library update module is used to incorporate the detected known Trojan information into the Trojan fingerprint library and create a new classification in the feature library based on the characteristics of unknown Trojans.

[0137] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cloud platform Trojan detection method based on machine learning, characterized in that: The method comprises the following steps: Modeling, training, and verification: Build a parallelized random forest detection model, use training data to train the model, and verify the accuracy and effectiveness of the model through verification data; Network traffic data acquisition: Real-time acquisition of network traffic data in the cloud platform, including normal traffic and abnormal traffic, and storage of the data in the form of traffic logs; Feature extraction: Extract session information from network traffic data, including session six-tuples and transmission features; Parallelized random forest detection: Using parallelization technology, the decision tree construction and classification process in the random forest model are parallelized to improve detection efficiency; Trojan identification and alarm: The extracted features are classified through the parallelized random forest detection model, Trojan traffic is identified, and corresponding alarm information is generated.

2. The method for detecting Trojans on a cloud platform using machine learning according to claim 1, characterized in that: The six-tuple includes: source IP, destination IP, source port, destination port, transmission protocol, and collection time; Transmission characteristics include: application type, URL, protocol type, number of bytes transmitted, data flow direction, and collection time.

3. The method for detecting Trojans on a cloud platform using machine learning according to claim 1, characterized in that: The construction of the parallelized random forest detection model includes: Create multiple decision trees and distribute the construction of these decision trees to multiple parallel processes; Each parallel process independently builds part of the decision tree and returns the completed decision tree to the main process; The main process integrates the decision trees returned by each parallel process into a complete random forest model.

4. The method for detecting Trojans on a cloud platform using machine learning according to claim 1, characterized in that: The classification step in the parallelized random forest detection process includes: Split the test data into multiple parts and assign each part of the data to different parallel classification processes; Each parallel classification process uses the decision tree in the random forest model to classify the assigned test data and returns the classification results; The main process integrates the classification results returned by each parallel classification process into a complete classification result list.

5. The method for detecting Trojans on a cloud platform using machine learning according to claim 1, characterized in that: The following steps are also included: Iterative training: For models that do not meet the detection requirements, continue iterative training until the model meets the detection requirements; Model verification: Verify the model on the test sample data set to ensure that the model's detection performance, precision, and recall indicators meet the predetermined standards; Trojan fingerprint library update: The detected Trojan information is included in the Trojan fingerprint library, and new categories are created in the feature library based on the characteristics of unknown Trojans.

6. A machine learning cloud platform Trojan detection system according to the machine learning cloud platform Trojan detection method according to any one of claims 1 to 5, characterized in that: The system includes the following modules: A data acquisition module is used to obtain network traffic data from the cloud platform network environment; A feature extraction module is used to extract session six-tuples, session timing features and transmission features from the acquired network traffic data; Parallelized random forest detection model, used to parallelize the extracted features and detect Trojans based on the processing results; An alarm module is used to generate an alarm message when a Trojan is detected; The model tuning module is used to regularly tune the parallelized random forest detection model to improve Trojan detection capabilities.

7. The machine learning cloud platform Trojan detection system according to claim 6, characterized in that: The parallelized random forest detection model includes multiple decision trees, each of which is independently constructed in a parallel manner, and the final classification result is determined by a voting mechanism to improve detection efficiency and accuracy.

8. The machine learning cloud platform Trojan detection system according to claim 6, characterized in that: The feature extraction module specifically includes: A session information extraction unit, used for extracting session six-tuple data from network traffic; A timing feature extraction unit, used for extracting timing features from a data packet sequence; The transmission feature extraction unit is used to extract transmission features such as application type, URL, protocol type, and number of transmitted bytes from network traffic.

9. The machine learning cloud platform Trojan detection system according to claim 6, characterized in that: The construction method of the parallelized random forest detection model includes: Divide into multiple sub-processes, each of which independently builds part of the decision tree; Merge the decision trees constructed by each subprocess into a complete random forest; Allocate test data to each subprocess for parallel classification processing and summarize the classification results.

10. The machine learning cloud platform Trojan detection system according to claim 6, characterized in that: Also includes: An iterative training module is used to iteratively train the model when the parallelized random forest detection model does not meet the detection requirements; The verification module is used to verify the detection effect of the model on the test set, including detection performance, precision and recall, and adjust the model parameters according to the verification results; The Trojan fingerprint library update module is used to incorporate the detected known Trojan information into the Trojan fingerprint library and create a new classification in the feature library based on the characteristics of unknown Trojans.

Images