System and method for machine learning based malware detection

Abstract

To provide a system and method for machine learning based malware detection.SOLUTION: A method comprises: obtaining a training set of network data that includes benign network data and malware network data; engaging a feature extraction engine to generate a set of dyads for each source-destination pair in the training set of network data; and using the set of dyads to train a machine learning engine so as to differentiate between the benign network data and the malware network data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to machine learning, and more particularly to systems and methods for machine learning-based malware detection. [Background technology]

[0002] Command and control is a post-exploitation tactic that allows attackers to maintain persistence, communicate with infected hosts, exfiltrate data, and issue commands. Once a host is infected, the malware establishes a command and control channel to the attacker. To avoid detection, the agent often remains dormant for long periods of time, periodically communicating with a server for further instructions. These intermittent communications are referred to as malware beacons.

[0003] Detecting the presence of malware beacons in network data is difficult for several reasons. For example, check-in intervals for embedded agents vary, and most command and control systems have built-in techniques to avoid detection, such as by adding random jitter to callback times. As another example, malware beacons are often disguised as network data by mimicking normal communications such as DNS or HTTP requests. Summary of the Invention [Means for solving the problem]

[0004] The present invention provides, for example, the following items. (Item 1) obtaining a training set of network data including benign network data and malware network data; engaging a feature extraction engine to generate a pairwise set for each source-destination pair in the training set of network data; Using the paired sets, a machine learning engine is trained to distinguish between the benign network data and the malware network data. A method comprising: (Item 2) obtaining network data identifying at least one new event relating to at least one source-destination pair; engaging the feature extraction engine to generate a pairwise set of the at least one source-destination pair associated with the at least one new event; sending the pairwise set of the at least one source-destination pair associated with the at least one new event to the machine learning engine for classification; The method according to the above item further comprises: (Item 3) receiving data from the machine learning engine classifying the at least one source-destination pair as one of benign or malware; The method according to any one of the preceding items. (Item 4) receiving data from the machine learning engine classifying the at least one source-destination pair as malware; adding the internet protocol address of at least one of said source or said destination to a blacklist; The method according to any one of the preceding items, further comprising: (Item 5) generating the malware network data using a training malware server such that the malware network data includes Internet Protocol addresses known to be associated with the training malware server; The method according to any one of the preceding items. (Item 6) 10. The method of claim 9, wherein the malware network data is generated to mimic a malware beacon by varying at least one of the communication interval, the amount of jitter, or the data channel. (Item 7) 5. The method of claim 1, wherein the set of pairs for each source-destination pair includes serving interval skewness and serving interval kurtosis. (Item 8) 2. The method of claim 1, wherein the malware network data includes more consistent communication interval skewness than the benign network data. (Item 9) 10. The method of claim 1, wherein the malware network data comprises communication interval kurtosis that is more evenly clustered than the benign network data. (Item 10) The method according to any one of the preceding items, wherein the set of two includes the number of flow events, the byte down mean, the byte down standard deviation, the byte up mean, the byte up standard deviation, the communication interval mean, the communication interval standard deviation, the communication interval skewness, the communication interval kurtosis, the number of local endpoints connected to the destination, and the number of remote endpoints to which the local endpoint is connected. (Item 11) 1. A system comprising: at least one processor; a memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, obtaining a training set of network data including benign network data and malware network data; engaging a feature extraction engine to generate a pairwise set for each source-destination pair in the training set of network data; Using the paired sets, a machine learning engine is trained to distinguish between the benign network data and the malware network data. a memory; and A system comprising: (Item 12) The instructions, when executed by the at least one processor, obtaining network data identifying at least one new event relating to at least one source-destination pair; engaging the feature extraction engine to generate a pairwise set of the at least one source-destination pair associated with the at least one new event; sending the pairwise set of the at least one source-destination pair associated with the at least one new event to the machine learning engine for classification; 2. The system of claim 1, further configured to: (Item 13) The instructions, when executed by the at least one processor, 10. The system of claim 9, further configuring the at least one processor to receive data from the machine learning engine classifying the at least one source-destination pair as one of benign or malware. (Item 14) The instructions, when executed by the at least one processor, receiving data from the machine learning engine classifying the at least one source-destination pair as malware; adding the internet protocol address of at least one of said source or said destination to a blacklist; 2. The system of claim 1, further configured to: (Item 15) The instructions, when executed by the at least one processor, The system of any one of the preceding items, further configuring the at least one processor to generate the malware network data using a training malware server such that the malware network data includes Internet Protocol addresses known to be associated with the training malware server. (Item 16) The system of any one of the preceding items, wherein the malware network data is generated to mimic a malware beacon by varying at least one of the communication interval, the amount of jitter, or the data channel. (Item 17) Item 10. The system of any one of the preceding items, wherein the set of two for each source-destination pair includes serving interval skewness and serving interval kurtosis. (Item 18) The system of any one of the preceding items, wherein the malware network data includes at least one of more consistent communication interval skewness than the harmless network data or more evenly clustered communication interval kurtosis than the harmless network data. (Item 19) The system described in any one of the above items, wherein the set of two includes the number of flow events, the byte down mean, the byte down standard deviation, the byte up mean, the byte up standard deviation, the communication interval mean, the communication interval standard deviation, the communication interval skewness, the communication interval kurtosis, the number of local endpoints connected to the destination, and the number of remote endpoints to which the local endpoints are connected. (Item 20) A non-transitory computer-readable medium having processor-executable instructions stored thereon, the processor-executable instructions, when executed by the processor, causing the processor to: obtaining a training set of network data including benign network data and malware network data; engaging a feature extraction engine to generate a pairwise set for each source-destination pair in the training set of network data; training a machine learning engine using the paired sets to distinguish between the benign network data and the malware network data; A non-transitory computer-readable medium for causing (Summary) The method includes obtaining a training set of network data including benign network data and malware network data, engaging a feature extraction engine to generate a pairwise set for each source-destination pair in the training set of network data, and using the pairwise sets to train a machine learning engine to distinguish between the benign network data and the malware network data. [Brief explanation of the drawings]

[0005] Reference will now be made to the accompanying drawings which illustrate, by way of example, exemplary embodiments of the present application.

[0006] [Figure 1] FIG. 1 illustrates a high-level block diagram of a system for machine learning-based malware detection, according to one embodiment.

[0007] [Figure 2] FIG. 2 provides a flowchart illustrating a method for training a machine learning engine for malware detection, according to one embodiment.

[0008] [Figure 3A] FIG. 3A is a graph showing data transmission within malware network data used to train a machine learning engine according to the method of FIG. 2.

[0009] [Figure 3B]FIG. 3B is a graph illustrating data transmission within harmless network data used to train a machine learning engine according to the method of FIG.

[0010] [Figure 4A] FIG. 4A is a graph showing communication intervals in malware network data used to train a machine learning engine according to the method of FIG. 2.

[0011] [Figure 4B] FIG. 4B is a graph showing communication intervals in benign network data used to train a machine learning engine according to the method of FIG. 2.

[0012] [Figure 5] FIG. 5 provides a flowchart illustrating a method for machine learning-based malware detection.

[0013] [Figure 6] FIG. 6 provides a flowchart illustrating a method for adding an Internet Protocol address to a blacklist.

[0014] [Figure 7] FIG. 7 illustrates a high-level block diagram of an exemplary computing device, according to one embodiment.

[0015] Like reference numerals are used in the drawings to refer to like elements and features. DETAILED DESCRIPTION OF THE INVENTION

[0016] DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS Thus, in one aspect, a method is provided that includes obtaining a training set of network data including benign network data and malware network data; engaging a feature extraction engine to generate a set of pairs for each source-destination pair in the training set of network data; and using the sets of pairs to train a machine learning engine to distinguish between the benign network data and the malware network data.

[0017] In one or more embodiments, the method further includes obtaining network data identifying at least one new event for the at least one source-destination pair; engaging a feature extraction engine to generate a pairwise set for the at least one source-destination pair associated with the at least one new event; and transmitting the pairwise set for the at least one source-destination pair associated with the at least one new event to a machine learning engine for classification.

[0018] In one or more embodiments, the method further includes receiving data from the machine learning engine classifying the at least one source-destination pair as one of benign or malware.

[0019] In one or more embodiments, the method further includes receiving data from the machine learning engine that classifies at least one source-destination pair as malware, and adding an Internet Protocol address of at least one of the source or destination to a blacklist.

[0020] In one or more embodiments, the method further includes using a training malware server to generate malware network data such that the malware network data includes Internet Protocol addresses known to be associated with the training malware server.

[0021] In one or more embodiments, the malware network data is generated to mimic a malware beacon by varying at least one of the communication interval, the amount of jitter, or the data channel.

[0022] In one or more embodiments, the set of pairs for each source-destination pair includes a serving interval skewness and a serving interval kurtosis.

[0023] In one or more embodiments, the malware network data includes more consistent communication interval skewness than the benign network data.

[0024] In one or more embodiments, the malware network data includes communication interval kurtosis that is more evenly clustered than the benign network data.

[0025] In one or more embodiments, the paired sets include a number of flow events, a byte down mean, a byte down standard deviation, a byte up mean, a byte up standard deviation, a communication interval mean, a communication interval standard deviation, a communication interval skewness, a communication interval kurtosis, a number of local endpoints connected to the destination, and a number of remote endpoints to which the local endpoint is connected.

[0026] According to another aspect, a system is provided comprising: at least one processor; and a memory coupled to the at least one processor, storing instructions that, when executed by the at least one processor, configure the at least one processor to: obtain a training set of network data including benign network data and malware network data; engage a feature extraction engine to generate a set of pairs for each source-destination pair in the training set of network data; and train a machine learning engine using the sets of pairs to distinguish between the benign network data and the malware network data.

[0027] In one or more embodiments, the instructions, when executed by the at least one processor, further configure the at least one processor to: obtain network data identifying at least one new event for the at least one source-destination pair; engage a feature extraction engine to generate a pairwise set for the at least one source-destination pair associated with the at least one new event; and transmit the pairwise set for the at least one source-destination pair associated with the at least one new event to a machine learning engine for classification.

[0028] In one or more embodiments, the instructions, when executed by the at least one processor, further configure the at least one processor to receive data from the machine learning engine that classifies the at least one source-destination pair as one of benign or malware.

[0029] In one or more embodiments, the instructions, when executed by the at least one processor, further configure the at least one processor to receive data from the machine learning engine that classifies at least one source-destination pair as malware, and add an Internet Protocol address of at least one of the source or destination to a blacklist.

[0030] In one or more embodiments, the instructions, when executed by the at least one processor, further configure the at least one processor to generate malware network data using a training malware server, such that the malware network data includes Internet Protocol addresses known to be associated with the training malware server.

[0031] In one or more embodiments, the malware network data is generated to mimic a malware beacon by varying at least one of the communication interval, the amount of jitter, or the data channel.

[0032] In one or more embodiments, the set of pairs for each source-destination pair includes a serving interval skewness and a serving interval kurtosis.

[0033] In one or more embodiments, the malware network data includes at least one of a more consistent communication interval skewness than the benign network data or a more evenly clustered communication interval kurtosis than the benign network data.

[0034] In one or more embodiments, the paired sets include a number of flow events, a byte down mean, a byte down standard deviation, a byte up mean, a byte up standard deviation, a communication interval mean, a communication interval standard deviation, a communication interval skewness, a communication interval kurtosis, a number of local endpoints connected to the destination, and a number of remote endpoints to which the local endpoint is connected.

[0035] According to another aspect, a non-transitory computer-readable medium is provided having processor-executable instructions stored thereon that, when executed by a processor, cause the processor to obtain a training set of network data including benign network data and malware network data; engage a feature extraction engine to generate a set of pairs for each source-destination pair in the training set of network data; and train a machine learning engine using the sets of pairs to distinguish between the benign network data and the malware network data.

[0036] Other exemplary embodiments of the present disclosure will be apparent to those skilled in the art from review of the following detailed description taken in conjunction with the drawings.

[0037] As used herein, the term "and / or" is intended to cover all possible combinations and subcombinations of the listed elements, including any one of the listed elements alone, any subcombination, or all of the elements, and does not necessarily exclude additional elements.

[0038] As used herein, the phrase "at least one of or" is intended to encompass any one or more of the listed elements, including any one of the listed elements alone, any subcombination, or all of the elements, without necessarily excluding any additional elements, and without necessarily requiring all of the elements.

[0039] 1 is a high-level block diagram of a system 100 for machine learning-based malware detection, according to one embodiment. System 100 includes a server computer system 110 and a data store 120.

[0040] The data store 120 may include various data records. At least some of the data records may include network data. The network data may include a training set of network data including benign network data and malware network data. As will be explained, benign network data may include network data that is known to be benign, i.e., known not to contain malware. Malware network data may include network data that is known to be malware. Each network log may be referred to as a flow event.

[0041] In one or more embodiments, the network data stored in data store 120 may include network logs, where each network log is a flow event between a particular destination and a particular remote endpoint. Each network log may include a timestamp, a source Internet Protocol (IP) address, a destination IP address, a source port, a destination port, etc. The network logs may additionally include data transmission information, such as packet size, byte up / down, etc.

[0042] Data store 120 may additionally maintain one or more whitelists containing IP addresses known to be trusted and one or more blacklists containing IP addresses known to be malware. As will be explained, one or more whitelists may be consulted, so that any flow events associated with a whitelisted IP address may not undergo classification. Similarly, one or more blacklists may also be consulted, so that any flow events associated with a blacklisted IP address may be blocked or an alert may be raised.

[0043] Data store 120 may only store network data that occurred within a threshold period of time. For example, data store 120 may only store network data for the past seven days, and therefore may discard, purge, or otherwise delete network data that is older than seven days.

[0044] In one or more embodiments, system 100 includes a malware engine 130, a feature extraction engine 140, and a machine learning engine 150. Malware engine 130 and feature extraction engine 140 are in communication with server computer system 110. Malware engine 130 may log network data locally and may export the logged network data to data store 120 via server computer system 110. Machine learning engine 150 is in communication with feature extraction engine 140 and server computer system 110. Malware engine 130, feature extraction engine 140, and machine learning engine 150 may be separate computing devices in different environments.

[0045] Malware engine 130 may be configured to generate malware network data that may be used to train machine learning engine 150. The malware network data may include malware beacons communicated between source-destination pairs. In one or more embodiments, the malware engine may include a virtual server, such as a training malware server, and one or more virtual computing devices, and communications between the virtual server and the one or more virtual computing devices may be logged as malware network data.

[0046] Malware engine 130 may be configured to generate malware network data, for example, by varying beacon intervals, jitter amounts, data channels, etc. In this manner, extensive beacon obfuscation is obtained. The malware network data may include IP addresses of training malware servers, which may be used to train machine learning engine 150. For example, any network log that includes IP addresses of training malware servers may be identified as malware network data.

[0047] As will be explained, malware network data generated by malware engine 130 may be stored in data store 120 and used to train machine learning engine 150.

[0048] Feature extraction engine 140 is configured to analyze the network data received from data store 120 and generate a pairwise set for each source-destination pair in the network data. To generate the pairwise set for each source-destination pair in the network data, feature extraction engine 140 may analyze the network data and categorize the network data by source-destination pair. For each source-destination pair, the pairwise set may include a number of flow events, a byte down mean, a byte down standard deviation, a byte up mean, a byte up standard deviation, a communication interval mean, a communication interval standard deviation, a communication interval skewness, a communication interval kurtosis, a number of local endpoints connected to the destination, and a number of remote endpoints to which the local endpoint is connected.

[0049] The number of flow events may include a count of flow events that occur in the network data for a source-destination pair. Because each network log is a flow event, feature extraction engine 140 may count the number of network logs for a source-destination pair in the network data, which may determine the number of flow events per source-destination pair.

[0050] The bytedown average for each source-destination pair may be generated by calculating the average bytedown size for the source-destination pair for all flow events in the network data for the source-destination pair.

[0051] The byte down standard deviation for each source-destination pair may be generated by calculating the byte down standard deviation for the source-destination pair for all flow events in the network data for the source-destination pair.

[0052] The byteup average for each source-destination pair may be generated by calculating the average size of the byteup for the source-destination pair for all flow events in the network data for the source-destination pair.

[0053] The byte up standard deviation for each source-destination pair may be generated by calculating the byte up standard deviation for the source-destination pair for all flow events in the network data for the source-destination pair.

[0054] The communication interval average may include an average of the number of seconds between flow events, or may be generated by calculating the average number of seconds between flow events, where it will be appreciated that the number of seconds between flow events may be the amount of time between adjacent flow events.

[0055] The serving interval standard deviation may include the standard deviation of the number of seconds between flow events and may be generated by calculating the standard deviation of the number of seconds between flow events.

[0056] Serving interval skewness may include a metric that indicates the degree to which the distribution is skewed towards one end.

[0057] Serving interval kurtosis may include a metric that indicates the tapering of a probability distribution. Serving interval kurtosis may be generated by determining a measure of the combined weight of the tails of the distribution relative to the center of the distribution.

[0058] The number of local endpoints connected to a destination may include a count of the local endpoints that have flow events involving the destination in the network data.

[0059] The number of remote endpoints to which the local endpoint is connected may include a count of the remote endpoints that had one or more flow events involving the source in the network data.

[0060] Machine learning engine 150 may include or utilize one or more machine learning models. For example, machine learning engine 150 may be a classifier, such as a random forest classifier, that may be trained to classify network data as one of malware network data or benign network data. Other machine learning methods that may be used include support vector machines and decision tree-based boosting methods, such as AdaBoost™ and XGBoost™.

[0061] In one or more embodiments, the pairwise sets generated by the feature extraction engine using the training network data may be used to train the machine learning engine 150 for malware detection.

[0062] 2 is a flowchart illustrating operations performed by server computer system 110 to train a machine learning engine for malware detection, according to an embodiment. The operations may be included within method 200, which may be performed by server computer system 110. For example, computer-executable instructions stored in a memory of server computer system 110, when executed by a processor of the server computer system, may configure server computer system 110 to perform method 200, or portions thereof. It will be understood that server computer system 110 may offload at least some of the operations to malware engine 130, feature extraction engine 140, and / or machine learning engine 150.

[0063] The method 200 includes obtaining a training set of network data (step 210), the training set including benign network data and malware network data.

[0064] In one or more embodiments, server computer system 110 may obtain a training set of network data from data store 120. As mentioned, malware network data may be generated by malware engine 130. Beneficial network data includes network data that is known to be benign, and malware network data includes network data that is known to be malware.

[0065] FIG. 3A is a graph showing data transmission within malware network data used to train a machine learning engine.

[0066] FIG. 3B is a graph showing data transmission within benign network data used to train a machine learning engine.

[0067] Comparing Figures 3A and 3B, it can be seen that the malware network data contains very consistent packet sizes, while the benign network data has inconsistent data patterns and packet sizes.

[0068] FIG. 4A is a graph showing communication intervals within malware network data used to train a machine learning engine.

[0069] FIG. 4B is a graph showing communication intervals in the benign network data used to train the machine learning engine.

[0070] Comparing Figures 4A and B, it can be seen that the malware network data contains consistent and regular communication intervals, while the harmless network data contains communication intervals with long periods of inactivity and has a large number of communication intervals near zero.

[0071] In one or more embodiments, the malware network data may include more consistent communication interval skewness than the benign network data and / or more evenly clustered communication interval kurtosis than the benign network data.

[0072] The method 200 includes engaging a feature extraction engine to generate a set of pairs for each source-destination pair in a training set of network data (step 220).

[0073] As mentioned, feature extraction engine 140 may analyze the network data and categorize the network data by source-destination pair to generate a paired set for each source-destination pair in the network data. For each source-destination pair, the paired set may include the number of flow events, the byte down mean, the byte down standard deviation, the byte up mean, the byte up standard deviation, the communication interval mean, the communication interval standard deviation, the communication interval skewness, the communication interval kurtosis, the number of local endpoints connected to the destination, and the number of remote endpoints to which the local endpoint is connected.

[0074] The method 200 includes using the pairwise sets to train a machine learning engine to distinguish between benign network data and malware network data (step 230).

[0075] The paired sets are fed to a machine learning engine and used to train the machine learning engine to classify the network data as benign network data or malware network data. Once trained, the machine learning engine may classify the network data as one of benign network data or malware network data.

[0076] In embodiments where the machine learning engine 150 includes a random forest classifier, the paired sets may be labeled with a zero (0), indicating benign network data, or with a one (1), indicating malware network data. Additionally, package functions may be used to fit the model to the data. For example, a fitting method may be used for each decision tree associated with the random forest classifier, which may include selecting features and values ​​such that the network data is split based on the features. In this manner, the purity of each split data chunk is maximized. This may be repeated multiple times for each decision tree, and thus the classifier is trained for a predictive task.

[0077] 5 is a flowchart illustrating operations performed for machine learning-based malware detection, according to an embodiment. The operations may be included in a method 500 that may be performed by server computer system 110. For example, computer-executable instructions stored in a memory of server computer system 110, when executed by a processor of the server computer system, may configure server computer system 110 to perform method 500, or portions thereof.

[0078] The method 500 includes obtaining network data (step 510) that identifies at least one new event for at least one source-destination pair.

[0079] In one or more embodiments, data store 120 may receive new flow events in the form of network data, which may occur periodically, for example, every minute, every five minutes, every 30 minutes, every hour, every 24 hours, etc. Specifically, server computer system 110 may send requests for new flow events to one or more source or destination computer systems connected to it, and the new flow events may be received in the form of network data. Server computer system 110 may send the received network data to data store 120 for storage.

[0080] Server computer system 110 may analyze the at least one new event and determine whether the at least one new event is associated with a source-destination pair that is known to be trusted. For example, server computer system 110 may consult a whitelist stored in data store 120 that includes a list of IP addresses that are known to be trusted and determine that the at least one new event is associated with a source-destination pair that is known to be trusted. In response to determining that the at least one new event is associated with a source-destination pair that is known to be trusted, server computer system 110 may discard the at least one new event and take no further action.

[0081] In response to determining that the at least one new event is not associated with a source-destination pair that is known to be trusted, server computer system 110 may send a request for all available network data for the at least one source-destination pair to data store 120. In other words, server computer system 110 does not just request network data associated with the at least one new event, but rather, server computer system 110 requests all available network data for the at least one source-destination pair associated with the at least one new event.

[0082] The method 500 includes engaging a feature extraction engine to generate a pairwise set for at least one source-destination pair associated with at least one new event (step 520).

[0083] The network data acquired by the server computer system 110 is sent to a feature extraction engine to generate a pairwise set for at least one source-destination pair associated with at least one new event. As mentioned, the pairwise set may include the number of flow events, the byte down mean, the byte down standard deviation, the byte up mean, the byte up standard deviation, the communication interval mean, the communication interval standard deviation, the communication interval skewness, the communication interval kurtosis, the number of local endpoints connected to the destination, and the number of remote endpoints to which the local endpoint is connected.

[0084] The method 500 includes sending the pairwise sets to a machine learning engine for classification (step 530).

[0085] The paired sets are sent to a machine learning engine for classification.

[0086] The method 500 includes receiving data from a machine learning engine that classifies a source-destination pair as one of benign or malware (step 540).

[0087] As stated, the machine learning engine is trained to classify the network data as one of benign network data or malware network data. Specifically, the machine learning engine analyzes the pairwise sets and classifies the network data as benign network data or malware network data.

[0088] The machine learning engine may classify the source-destination pair as one of benign or malware, which may be based on classifying the network data as benign network data or malware network data. For example, in an embodiment in which the network data is classified as malware network data, at least one source-destination pair may be classified as malware.

[0089] In embodiments where at least one source-destination pair is classified as benign, server computer system 110 may determine that no further action is required.

[0090] In embodiments in which at least one source-destination pair is classified as malware, server computer system 110 may implement one or more corrective actions. For example, server computer system 110 may raise a flag or alarm indicating that the source-destination pair is malware.

[0091] In another example, server computer system 110 may add at least one of the source or destination of a source-destination pair to a blacklist. Figure 6 is a flowchart illustrating operations performed to add an Internet Protocol address to a blacklist, according to an embodiment. This operation may be included within a method 600 that may be performed by server computer system 110. For example, computer-executable instructions stored in a memory of server computer system 110, when executed by a processor of the server computer system, may configure server computer system 110 to perform method 600, or portions thereof.

[0092] The method 600 includes receiving data from a machine learning engine that classifies a source-destination pair as malware (step 610).

[0093] The machine learning engine may perform operations similar to those described herein with reference to method 500 to classify the source-destination pair as malware. In response, server computer system 110 may receive data from the machine learning engine classifying the source-destination pair as malware.

[0094] The method 600 includes adding at least one Internet Protocol address of the source or destination to a blacklist (step 620).

[0095] The server computer system 110 may determine the IP address of at least one of the source or destination by analyzing network data associated with it, and may send a signal to the data store 120 to add the IP address to a blacklist maintained thereby.

[0096] It will be appreciated that in addition to, or as an alternative to, identifying a destination IP address as malware, in one or more embodiments, a fully qualified domain name (FQDM) may be identified as malware.

[0097] As mentioned, the server computer system 110 is a computing device. Figure 7 shows a high-level block diagram of an exemplary computing device 700. As shown, the exemplary computing device 700 includes a processor 710, a memory 720, and an I / O interface 730. The aforementioned modules of the exemplary computing device 700 communicate with each other via and are communicatively coupled by a bus 740.

[0098] Processor 710 includes a hardware processor and may include, for example, one or more processors using the ARM, x86, MIPS, or PowerPC™ instruction set. For example, processor 710 may include an Intel™ Core™ processor, a Qualcomm™ Snapdragon™ processor, or the like.

[0099] Memory 720 comprises physical memory. Memory 720 may include random access memory, read-only memory, persistent storage such as flash memory, solid-state drive, or the like. Read-only memory and persistent storage are computer-readable media, and more specifically, may each be considered non-transitory computer-readable storage media. Computer-readable media may be organized using a file system that may be managed by software that governs the overall operation of exemplary computing device 700.

[0100] I / O interface 730 is an input / output interface. I / O interface 730 allows exemplary computing device 700 to receive input and provide output. For example, I / O interface 730 may allow exemplary computing device 700 to receive input from a user or provide output to a user. In another example, I / O interface 730 may allow exemplary computing device 700 to communicate with a computer network. I / O interface 730 may serve to interconnect exemplary computing device 700 with one or more I / O devices, such as, for example, a keyboard, a display screen, a pointing device such as a mouse or trackball, a fingerprint reader, a communications module, a hardware security module (HSM) (e.g., a trusted platform module (TPM)), or the like. Virtual counterparts of I / O interface 730 and / or devices accessed via I / O interface 730 may be provided by, for example, a host operating system, etc.

[0101] Software comprising instructions is executed by processor 710 from a computer-readable medium. For example, software corresponding to a host operating system may be loaded into random access memory from persistent storage or flash memory of memory 720. Additionally, or alternatively, software may be executed by processor 710 directly from read-only memory of memory 720. In another embodiment, software may be accessed via I / O interface 730.

[0102] It will be appreciated that the malware engine 130, feature extraction engine 140, and machine learning engine 150 may also be computing devices similar to those described herein.

[0103] It will be understood that some or all of the above-described operations of the various above-described exemplary methods may be performed in orders other than those illustrated and / or may be performed in parallel without changing the overall operation of the methods.

[0104] The various embodiments presented above are merely examples and are not intended to limit the scope of the present application in any way. Variations of the innovations described herein will be apparent to those skilled in the art, and such variations would fall within the intended scope of the present application. In particular, features from one or more of the exemplary embodiments described above may be selected to create alternative exemplary embodiments including subcombinations of features that may not be explicitly described above. In addition, features from one or more of the exemplary embodiments described above may be selected and combined to create alternative exemplary embodiments including combinations of features that may not be explicitly described above. Features suitable for such combinations and subcombinations will be readily apparent to those skilled in the art upon review of the present application as a whole. The subject matter described in this specification and the enumerated claims is intended to cover and encompass all suitable modifications in technology.

Claims

1. A method, The method involves obtaining a training set of network data, which includes harmless network data and malware network data, wherein the malware network data is generated to mimic malware beacons by varying the amount of jitter. The feature extraction engine is engaged to generate a set of two for each source-destination pair in the training set of the aforementioned network data. Using the aforementioned set of two, a machine learning engine is trained to distinguish between the harmless network data and the malware network data. Obtain network data that identifies at least one new event relating to at least one source-destination pair, Obtaining all available network data relating to the aforementioned at least one source-destination pair, Using the acquired network data, the feature extraction engine is engaged to generate a pair of at least one source-destination pairs associated with the at least one new event. Send the set of two, relating to the at least one source-destination pair associated with the at least one new event, to the machine learning engine for classification. Methods that include...

2. The machine learning engine further includes receiving data classifying the at least one source-destination pair as either harmless or malware. The method according to claim 1.

3. The machine learning engine receives data classifying the at least one source-destination pair as malware, Adding at least one Internet Protocol address of the source or destination to a blacklist The method according to claim 1, further comprising:

4. The process further includes using a training malware server to generate the malware network data such that the malware network data includes an Internet Protocol address known to be associated with the training malware server, The method according to claim 1.

5. The method according to claim 1, wherein the malware network data is further generated to mimic the malware beacon by varying the communication interval or data channel.

6. The method according to claim 1, wherein the set of two for each source-destination pair includes communication interval distortion and communication interval kurtosis.

7. The method according to claim 1, wherein the malware network data includes a more consistent communication interval skew than the harmless network data.

8. The method according to claim 1, wherein the malware network data includes communication interval kurtosis that is more uniformly clustered than the harmless network data.

9. The method according to claim 1, wherein the set of two includes the number of flow events, the byte-down average, the byte-down standard deviation, the byte-up average, the byte-up standard deviation, the communication interval average, the communication interval standard deviation, the communication interval skewness, the communication interval kurtosis, the number of local endpoints connected to the destination, and the number of remote endpoints to which the local endpoints are connected.

10. It is a system, At least one processor, A memory connected to the at least one processor and storing instructions, wherein the instructions are executed by the at least one processor. The method involves obtaining a training set of network data, which includes harmless network data and malware network data, wherein the malware network data is generated to mimic malware beacons by varying the amount of jitter. The feature extraction engine is engaged to generate a set of two for each source-destination pair in the training set of the aforementioned network data. Using the aforementioned set of two, a machine learning engine is trained to distinguish between the harmless network data and the malware network data. Obtain network data that identifies at least one new event relating to at least one source-destination pair, Obtaining all available network data relating to the aforementioned at least one source-destination pair, Using the acquired network data, the feature extraction engine is engaged to generate a pair of at least one source-destination pairs associated with the at least one new event. Send the set of two, relating to the at least one source-destination pair associated with the at least one new event, to the machine learning engine for classification. To perform the above, the at least one processor is configured with memory and A system equipped with these features.

11. When the instruction is executed by the at least one processor, The system according to claim 10, further comprising the configuration of the at least one processor to receive data from the machine learning engine classifying the at least one source-destination pair as either harmless or malware.

12. When the instruction is executed by the at least one processor, The machine learning engine receives data classifying the at least one source-destination pair as malware, Adding at least one Internet Protocol address of the source or destination to a blacklist The system according to claim 11, further comprising configuring the at least one processor to perform the following:

13. When the instruction is executed by the at least one processor, The system according to claim 10, further comprising configuring the at least one processor to generate the malware network data using a training malware server such that the malware network data includes an Internet protocol address known to be associated with the training malware server.

14. The system according to claim 10, wherein the malware network data is further generated to mimic the malware beacon by varying the communication interval or data channel.

15. The system according to claim 10, wherein each set of two for each source-destination pair includes communication interval distortion and communication interval kurtosis.

16. The system according to claim 10, wherein the malware network data includes at least one of a more consistent communication interval skewness than the harmless network data, or a more uniformly clustered communication interval kurtosis than the harmless network data.

17. The system according to claim 10, wherein the set of two includes the number of flow events, the byte-down average, the byte-down standard deviation, the byte-up average, the byte-up standard deviation, the communication interval average, the communication interval standard deviation, the communication interval skewness, the communication interval kurtosis, the number of local endpoints connected to the destination, and the number of remote endpoints to which the local endpoints are connected.

18. A non-transient computer-readable medium having processor-executable instructions stored thereon, wherein when the processor executes the processor, the processor executes the instructions. The method involves obtaining a training set of network data, which includes harmless network data and malware network data, wherein the malware network data is generated to mimic malware beacons by varying the amount of jitter. The feature extraction engine is engaged to generate a set of two for each source-destination pair in the training set of the aforementioned network data. Using the aforementioned set of two, a machine learning engine is trained to distinguish between the harmless network data and the malware network data. Obtain network data that identifies at least one new event relating to at least one source-destination pair, Obtaining all available network data relating to the aforementioned at least one source-destination pair, Using the acquired network data, the feature extraction engine is engaged to generate a pair of at least one source-destination pairs associated with the at least one new event. Send the set of two, relating to the at least one source-destination pair associated with the at least one new event, to the machine learning engine for classification. A non-transient, computer-readable medium that enables the following action.

Images