Machine learning cloud platform Trojan detection method and system

Abstract

The application relates to the technical field of network security, in particular to a machine learning cloud platform Trojan detection method and system, which comprises the following steps: modeling, training and verification, network traffic data acquisition, feature extraction, parallel random forest detection, and Trojan identification and alarm; beneficial effects are that the machine learning cloud platform Trojan detection method and system can realize timely and accurate detection of Trojans by using time sequence characteristics of the Trojans, guarantee the security of a cloud platform network space situation, effectively avoid the occurrence of various private information and commercial secret data leakage events, and protect data assets in the cloud platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a cloud platform Trojan detection method and system based on machine learning. BACKGROUND

[0002] With the rapid development of cloud computing technology, while a large number of information application and information resources are rich, more and more traps set by malicious attackers, such as Trojans, appear. The Trojan code is small, and it is easy to disguise or hide in legal application programs. Some of these Trojans are designed to execute malicious programs, and some are designed to illegally obtain permissions to set special backdoors for application programs. Once the Trojan is executed, it will trigger some specific events, such as stealing user privacy data information and stealing cloud platform business secret data. In recent years, Trojans have spread widely in cloud platforms, posing a serious threat to the security of cloud platforms and causing huge economic losses. In recent years, remote office, remote conference and remote control have become the norm, which has promoted the rapid development of various remote Trojans, which can steal system stored files, remotely monitor user desktops, and modify processes.

[0003] At present, the detection of Trojans is mainly based on the structured features of Trojan files or dynamic behavior sandboxes to identify them. However, with the deformation and derivation of Trojans, existing network security inspection systems cannot accurately identify highly concealed and variable Trojans. In addition, for some remote Trojans, even if the Trojan is detected, the leakage of private data and business secret data may have already occurred.

[0004] The conventional cloud platform Trojan detection method mainly includes the following types:

[0005] Static structure analysis, which is a static detection method, matches and analyzes the source code or binary file according to the Trojan feature library. Due to the deformation of Trojans and the variety of Trojans, the number of static code structure features theoretically has no upper limit. Therefore, the feature library of this method needs to be continuously expanded to adapt to the detection of new Trojan types, and it is difficult to find unknown types of Trojans.

[0006] Dynamic behavior analysis, which is a dynamic detection method, detects by running the Trojan program. During dynamic detection, the tester observes and records the running state of the application program through the sandbox, obtains and analyzes the behavior of the application program, and then determines the malicious degree of the application program according to the behavior characteristics of the application program. The analysis process of this method takes a long time, the recognition accuracy is relatively high, but there is also a certain rate of false negatives. SUMMARY

[0007] The present application aims to provide a cloud platform Trojan detection method and system based on machine learning to solve the problems raised in the background.

[0008] To achieve the above object, the present application provides the following technical solution: a machine learning cloud platform Trojan detection method, the method comprising the following steps:

[0009] Modeling, training and verification: building a parallelized random forest detection model, using training data for model training, and verifying the accuracy and effectiveness of the model through verification data;

[0010] Network traffic data acquisition: real-time acquisition of network traffic data in the cloud platform, including normal traffic and abnormal traffic, and storing the data in the form of traffic logs;

[0011] Feature extraction: extracting session information from network traffic data, including session six-tuple and transmission features;

[0012] Parallelized random forest detection: using parallelization technology, parallelizing the decision tree construction and classification process in the random forest model to improve detection efficiency;

[0013] Trojan identification and alarm: classifying the extracted features through the parallelized random forest detection model, identifying Trojan traffic, and generating corresponding alarm information.

[0014] Preferably, the six-tuple includes: source IP, destination IP, source port, destination port, transmission protocol, and collection time;

[0015] Transmission features include: application type, URL, protocol type, number of transmitted bytes, data flow direction, and collection time.

[0016] Preferably, the construction of the parallelized random forest detection model includes:

[0017] Create multiple decision trees and distribute the construction tasks of these decision trees to multiple parallel processes;

[0018] Each parallel process independently constructs part of the decision tree, and returns the constructed decision tree to the main process;

[0019] The main process integrates the decision trees returned by each parallel process into a complete random forest model.

[0020] Preferably, the classification step in the parallelized random forest detection process includes:

[0021] Divide the test data into multiple parts and assign each part of data to a different parallel classification process;

[0022] Each parallel classification process uses the decision trees in the random forest model to classify the assigned test data and returns the classification results;

[0023] The main process integrates the classification results returned by the parallel classification processes into a complete classification result list.

[0024] Preferably, the method further comprises the following steps:

[0025] Iterative training: for the model that does not meet the detection requirements, continue iterative training until the model meets the detection requirements;

[0026] Model verification: verify the model on the test sample data set to ensure that the detection performance, precision and recall rate of the model meet the predetermined standard;

[0027] Trojan fingerprint library update: incorporate the detected Trojan information into the Trojan fingerprint library, and create a new classification in the feature library according to the characteristics of unknown Trojan.

[0028] A machine learning cloud platform Trojan detection system, the system comprises the following modules:

[0029] A data acquisition module for acquiring network traffic data from a cloud platform network environment;

[0030] A feature extraction module for extracting session six-tuple, session timing features and transmission features from the acquired network traffic data;

[0031] A parallelized random forest detection model for parallel processing of the extracted features and detecting Trojans based on the processing results;

[0032] An alarm module for generating alarm information when a Trojan is detected;

[0033] A model tuning module for periodically tuning the parallelized random forest detection model to improve Trojan detection capability.

[0034] Preferably, the parallelized random forest detection model includes multiple decision trees, each of which is independently constructed by parallelization and determines the final classification result through a voting mechanism to improve detection efficiency and accuracy.

[0035] Preferably, the feature extraction module specifically includes:

[0036] A session information extraction unit for extracting session six-tuple data from network traffic;

[0037] A timing feature extraction unit for extracting timing features from packet sequences;

[0038] A transmission feature extraction unit for extracting application type, URL, protocol type, transmission byte number and other transmission features from network traffic.

[0039] Preferably, the construction method of the parallelized random forest detection model comprises:

[0040] dividing a plurality of sub-processes, each sub-process independently constructing a partial decision tree;

[0041] combining the decision trees constructed by each sub-process into a complete random forest;

[0042] allocating test data to each sub-process for parallel classification processing, and summarizing the classification results.

[0043] Preferably, it also includes:

[0044] an iterative training module for iteratively training the model when the parallelized random forest detection model does not meet the detection requirements;

[0045] a verification module for verifying the detection effect of the model on the test set, including detection performance, precision and recall, and adjusting the model parameters according to the verification results;

[0046] a Trojan fingerprint library updating module for incorporating the detected known Trojan information into the Trojan fingerprint library, and creating a new classification in the feature library according to the characteristics of unknown Trojans.

[0047] Compared with the prior art, the beneficial effects of the present application are:

[0048] The machine learning cloud platform Trojan detection method and system provided by the present application utilizes the time series characteristics of Trojans to achieve timely and accurate detection of Trojans, ensuring the security of the cloud platform network space situation, effectively avoiding the occurrence of various private information and commercial secret data leakage events, and protecting the data assets in the cloud platform. BRIEF DESCRIPTION OF DRAWINGS

[0049] Fig. 1 is the flowchart of the method of the present application;

[0050] Fig. 2 is the parallelized random forest detection flowchart of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution of the present application clear, complete description, and the advantages are more clear and obvious, the following will be further described in detail with the help of the attached drawings. It should be understood that the specific embodiments described here are part of the embodiments of the present application, not all embodiments, and are used to explain the embodiments of the present application, and do not limit the embodiments of the present application. All other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] Embodiment one, please refer to Figs. 1-2 The present application provides a technical solution: a machine learning cloud platform Trojan detection method, the method comprising the following steps:

[0053] The Trojan horse time sequence characteristics are used to realize the timeliness and accuracy of the Trojan horse program detection.

[0054] 1. Security technology of Trojan horse identification

[0055] Common types of Trojan horses are: remote control type, information acquisition type, and destruction type

[0056] (1) Remote control type Trojan:

[0057] Remote control target computer for interactive access (real-time or non-real-time), can issue corresponding instructions to trigger the function of malicious software, and can also obtain various data of the target, and its interaction is bidirectional (attacker-controlled end).

[0058] (2) Information acquisition type Trojan:

[0059] The function of information acquisition type Trojan is information acquisition, which can obtain valuable information from keyboard input, memory, files, databases, and browser cookies. Its interaction is one-way interaction, which is the controlled end sending data to the attacker, such as sending to the attacker's third-party space, file server, designated mailbox, etc., or directly opening the FTP service program, and the attacker directly accesses to download data.

[0060] (3) Destruction type Trojan:

[0061] The function of destruction type Trojan is to destroy data and consume resources of local or remote host systems. Its interaction is also one-way, and the attacker can send instructions to the controlled end to execute file data destruction.

[0062] Trojan horses have strong deception, and their implantation methods mainly include:

[0063] (1) Web page Trojan implantation

[0064] It is relatively typical to achieve through web page Trojan, which usually needs to use the browser or related vulnerabilities, combined with real-time news and hot topics to make related web pages, combine the vulnerabilities and web pages, and finally generate malicious web pages. When the user opens the web page, the remote Trojan program will be automatically downloaded and installed.

[0065] (2) Email implantation

[0066] The most common way of email implantation is through the form of attachments, which are injected with Trojan horses when the user opens the attachments. Another way is to combine email with malicious web pages. Since email supports HTML text, if related vulnerabilities are implanted into the web page, the relevant effect can also be achieved.

[0067] (3) Document bundling implantation

[0068] Through exe files, office documents, pdf document vulnerabilities, etc., the trojan is bundled with the document, and when the user opens the document, the vulnerability will be triggered, releasing the trojan or executing shellcode to perform remote attacks.

[0069] (4) camouflage deception implant

[0070] The file suffix of exe can be modified, changing the "exe" file to "docx", so that the exe is not displayed, and the document class icon is modified, so as to camouflage.

[0071] In this scheme, when identifying the trojan, the network traffic characteristics are mainly used, and the parallel random forest machine learning classification algorithm is used to analyze the traffic to identify the behavior of the trojan: such as transmitting malicious files, screen monitoring, voice and video interception, keyboard recording, remote shell, and illegal external communication.

[0072] The behavior characteristics of the trojan based on traffic analysis are mainly as follows:

[0073] (1) In most cases, the number of bytes flowing out is greater than the number of bytes flowing in, which is usually a trojan stealing data and sending it out.

[0074] (2) The main connection is responsible for communication link control and command transmission, and is in an idle state most of the time. After receiving the return result, the attacker will have an analysis time, which is usually much longer than the packet time interval.

[0075] (3) There are some characteristic characters of the trojan (different for different trojans) in the request body.

[0076] (4) Heartbeat traffic characteristics: the trojan client and the remote trojan server maintain a heartbeat connection, and periodically use fixed handshake protocol data.

[0077] (5) Three-stage traffic characteristics: connection establishment stage, usually with dynamic domain name (based on DGA); during data transmission, the push flag PSH is set to 1, causing the number of PSH packets to increase; in the command interaction stage, the uplink and downlink traffic is asymmetric, and the proportion of small data packets is large; in the keep-alive stage, there will be heartbeat packets.

[0078] When processing traffic data, the system will extract the above characteristics, establish a feature vector, and store it in the traffic data file feature library for machine learning.

[0079] 2. Overall process of trojan detection

[0080] (1) Modeling, training and verification: in the initial stage, the construction, training and verification of the parallel random forest detection model are carried out.

[0081] (2) Enter the detection phase, first get network traffic data.

[0082] (3) Next, feature extraction is performed on the traffic, such as session six-tuple extraction, session timing feature extraction, and transmission feature extraction.

[0083] (4) Finally, by executing a parallelized random forest detection model, the Trojan can be detected in a timely and accurate manner, and an alarm can be generated.

[0084] Repeat (2), (3), and (4) to continuously perform Trojan detection.

[0085] Periodically optimize the model, and perform (1) to adapt to more Trojan type detection.

[0086] In the present application, the overall flowchart of the Trojan detection model is as shown in Fig. 1 .

[0087] 3. Algorithmic thought and data structure of the parallelized random forest detection model

[0088] Random forest is a machine learning model based on decision trees, which contains multiple decision trees trained by Bagging ensemble learning technology. When a sample to be classified is input, the final classification result is determined by the output results of individual decision trees.

[0089] Random forest is a forest established in a random manner, and the forest is composed of many decision trees. Each decision tree of the random forest is not associated. After obtaining the forest, when a new input sample enters, each decision tree in the forest judges which class the sample belongs to.

[0090] Random forest is an effective classification and prediction method, which has high classification accuracy, good robustness to noise and outliers, and strong generalization ability, and does not have overfitting problem. A tree structure is constructed by using a top-down greedy algorithm, an optimal attribute is selected for splitting at each internal node, each branch corresponds to an attribute value, and the tree is recursively built until the termination condition is met. Each leaf node represents the class to which the samples along the path belong.

[0091] The data structures used in the present application mainly include lists and dictionaries, which are used to store the following data:

[0092] (1) A two-dimensional list stores training data and test data

[0093] (2) A one-dimensional list stores classification labels

[0094] (3) A dictionary stores the decision tree structure

[0095] 4. Parallelization improves processing efficiency

[0096] The model ontology construction of decision tree is very complex, when the training data, the number of features, the number of attribute partitions further increases, the leaf node growth rate of the tree is exponential. And the success of random forest is based on the construction of a large number of decision trees, of course we can achieve decision tree by using non recursive way, or reduce the number of selected traffic features, attribute partition number and the number of training data, but this adjustment is likely to affect the accuracy of random forest. Based on this, the parallelization is used to solve this processing efficiency problem in this scheme, and the hardware and software configuration of the computer is fully utilized.

[0097] The parallelization steps are as follows:

[0098] (1) First, create N trees, 4 processes, each process is responsible for creating N / 4 decision trees, and the created N / 4 decision trees are returned to the main process in the form of a list.

[0099] (2) After getting the decision tree list of the 4 sub processes respectively, integrate the 4 sub lists into a decision tree list L with length N.

[0100] (3) Create 4 classification processes, copy the decision tree list 4 times and pass it to the 4 classification processes, and divide the test data into 4 parts, for example: [0,400] is the first part, [401,800] is the second part, [801,1200] is the third part, [1201,1600] is the fourth part, and pass them to the 4 classification sub processes respectively.

[0101] (4) The first sub process returns the classification result of [1,400] rows in the form of a list, the second sub process returns the classification result of [401,800] rows in the form of a list, the third sub process returns the classification result of [801,1200] rows in the form of a list, and the fourth sub process returns the classification result of [1201,1600] rows in the form of a list.

[0102] (5) After getting the label list of the 4 sub processes respectively, integrate the 4 sub lists into a result label list with length 1600.

[0103] 5. Parallelization of random forest Trojan detection steps

[0104] This scheme is suitable for various cloud platform application scenarios, such as cloud platform network traffic monitoring system, network anomaly intrusion detection system, etc. The specific parallelization based random forest detection steps are as follows:

[0105] (1) Network traffic data acquisition: network traffic may be normal traffic or abnormal traffic, after collecting data, in the form of traffic log storage.

[0106] (2) Feature extraction: extract session information of network traffic, and then extract session timing features and transmission features.

[0107] (3) Parallel random forest detection model construction: due to the low detection efficiency of traditional random forest detection model, parallel theory is introduced to realize parallel detection and solve the defects of traditional forest detection model.

[0108] (4) Ensure that the model meets the detection requirements through iterative training: store the model that meets the detection requirements; for the model that does not meet the detection requirements, continue to perform iterative training.

[0109] (5) Verify the model based on the test set: check the model detection effect on the test sample data set, stop testing and training when the detection performance, precision rate and recall rate meet the requirements; otherwise, continue to train, adjust parameters and optimize the detection model until the requirements are met.

[0110] (6) Trojan identification using parallel random forest detection model: according to the detection result, judge whether there is a Trojan, if there is, output the Trojan information, intercept the application program, and generate an alarm. If not, mark it as a normal application program. If known, it is included in the corresponding classification of the Trojan fingerprint library; if unknown, a new classification needs to be created for the unknown Trojan in the feature library according to the characteristics of the unknown Trojan.

[0111] The specific parallel random forest Trojan detection process is shown in Fig. 2 .

[0112] The present application has certain advantages in real-time performance and detection results (accuracy, query rate, etc.) compared with the traditional Trojan detection scheme. In a closed network security test environment, select Trojan samples: 1000 remote control type Trojans and 1000 information acquisition type Trojans (black and white ratio is 7:3). Perform comparison test, and the Trojan detection effects of the three schemes are as follows:

[0113] (1) Parallel random forest Trojan detection (this scheme)

[0114] (2) Static structure feature Trojan detection (traditional scheme)

[0115] (3) Dynamic behavior feature Trojan detection (traditional scheme)

[0116] Table 1 Detection results of remote control type Trojan

[0117]

[0118]

[0119] Table 2 Detection results of information acquisition type Trojan

[0120]

[0121] In Example Two, on the basis of Example One, a machine learning cloud platform Trojan detection system is proposed, which includes the following modules:

[0122] A data acquisition module is configured to acquire network traffic data from a cloud platform network environment.

[0123] A feature extraction module is configured to extract session six-tuple, session timing feature and transmission feature from the acquired network traffic data.

[0124] A session information extraction unit is configured to extract session six-tuple data from network traffic.

[0125] A timing feature extraction unit is configured to extract timing features from packet sequences.

[0126] A transmission feature extraction unit is configured to extract application type, URL, protocol type, transmission byte number and other transmission features from network traffic.

[0127] A parallelized random forest detection model is configured to perform parallelized processing on the extracted features and detect Trojans based on the processing results. The parallelized random forest detection model includes multiple decision trees, each of which is independently constructed in a parallelized manner and determines the final classification result through a voting mechanism to improve detection efficiency and accuracy. The construction method of the parallelized random forest detection model includes:

[0128] Divide multiple sub-processes, each of which independently constructs part of the decision trees.

[0129] Merge the decision trees constructed by each sub-process into a complete random forest.

[0130] Distribute test data to each sub-process for parallelized classification processing and aggregate the classification results.

[0131] An alarm module is configured to generate alarm information when a Trojan is detected.

[0132] A model tuning module is configured to periodically tune the parallelized random forest detection model to improve the Trojan detection capability.

[0133] Further comprising:

[0134] An iterative training module is configured to iteratively train the model when the parallelized random forest detection model does not meet the detection requirements;

[0135] A verification module is configured to verify the detection effect of the model on a test set, including the detection performance, precision and recall, and adjust the model parameters according to the verification result;

[0136] A Trojan fingerprint library updating module is configured to incorporate the detected known Trojan information into the Trojan fingerprint library, and create a new classification in the feature library according to the characteristics of unknown Trojans.

[0137] Although the embodiments of the present application have been shown and described, it should be understood by those ordinary skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A machine learning-based method for detecting Trojans on cloud platforms, characterized in that: The method includes the following steps: Modeling, training, and validation: Construct a parallelized random forest detection model, train the model using training data, and validate the model's accuracy and effectiveness using validation data; Network traffic data acquisition: Real-time acquisition of network traffic data from the cloud platform, including normal and abnormal traffic, and storage of the data in the form of traffic logs; Feature extraction: Session six-tuple data and transmission features are extracted from network traffic data, and temporal features are extracted from the packet sequence; the six-tuple includes: source IP, destination IP, source port, destination port, transmission protocol, and collection time; Transmission characteristics include: application type, URL, protocol type, number of bytes transmitted, data flow direction, and collection time; Parallelized Random Forest Detection: Parallelization techniques are used to parallelize the decision tree construction and classification processes in the random forest model to improve detection efficiency; Trojan horse identification and alerting: The extracted features are classified using a parallelized random forest detection model to identify Trojan horse traffic and generate corresponding alert information.

2. The cloud platform Trojan detection method based on machine learning according to claim 1, characterized in that: The construction of the parallelized random forest detection model includes: Create multiple decision trees and distribute the construction tasks of these decision trees to multiple parallel processes; Each parallel process independently constructs a portion of the decision tree and returns the completed decision tree to the main process; The main process integrates the decision trees returned by each parallel process into a complete random forest model.

3. The cloud platform Trojan detection method based on machine learning according to claim 1, characterized in that: The classification steps in the parallelized random forest detection process include: The test data is split into multiple parts, and each part is assigned to a different parallel classification process; Each parallel classification process uses a decision tree in a random forest model to classify the assigned test data and returns the classification result; The main process integrates the classification results returned by each parallel classification process into a complete list of classification results.

4. The cloud platform Trojan detection method based on machine learning according to claim 1, characterized in that: It also includes the following steps: Iterative training: For models that do not meet the detection requirements, continue iterative training until the model meets the detection requirements; Model validation: Validate the model on the test sample dataset to ensure that the model's detection performance, precision, and recall meet the predetermined standards; Trojan fingerprint database update: Detected Trojan information is added to the Trojan fingerprint database, and new categories are created in the feature database based on the characteristics of unknown Trojans.

5. A machine learning-based cloud platform Trojan detection system according to any one of claims 1-4, characterized in that: The system includes the following modules: The data acquisition module is used to acquire network traffic data from the cloud platform network environment; The feature extraction module is used to extract session six-tuples, session timing features, and transmission features from the acquired network traffic data; The feature extraction module specifically includes: The session information extraction unit is used to extract session six-tuple data from network traffic; the six-tuple includes: source IP, destination IP, source port, destination port, transport protocol, and collection time; The temporal feature extraction unit is used to extract temporal features from the data packet sequence; The transmission feature extraction unit is used to extract transmission features from network traffic; transmission features include: application type, URL, protocol type, number of bytes transmitted, data flow direction, and collection time; A parallelized random forest detection model is used to process the extracted features in parallel and detect Trojans based on the processing results. The alarm module is used to generate alarm information when a Trojan is detected; The model tuning module is used to periodically tune the parallelized random forest detection model to improve Trojan detection capabilities.

6. The cloud platform Trojan detection system based on machine learning according to claim 5, characterized in that: The parallelized random forest detection model consists of multiple decision trees, each of which is constructed independently in parallel and the final classification result is determined through a voting mechanism to improve detection efficiency and accuracy.

7. The cloud platform Trojan detection system based on machine learning according to claim 5, characterized in that: Methods for constructing parallelized random forest detection models include: Divide the process into multiple sub-processes, and each sub-process independently constructs a portion of the decision tree; The decision trees built by each subprocess are merged into a complete random forest; The test data is distributed to each subprocess for parallel classification processing, and the classification results are then summarized.

8. The cloud platform Trojan detection system for machine learning according to claim 5, characterized in that: Also includes: The iterative training module is used to iteratively train the model when the parallelized random forest detection model does not meet the detection requirements. The validation module is used to validate the model's detection performance on the test set, including detection performance, precision, and recall, and to adjust the model parameters based on the validation results. The Trojan fingerprint database update module is used to add detected known Trojan information into the Trojan fingerprint database and create new categories in the feature database based on the characteristics of unknown Trojans.

Images