Network Threat Analysis System

Abstract

Machine learning techniques and models are described for alerting users to attacks on their accounts in real-time or near real-time. In some embodiments, the attack detection model uses natural language processing (NLP) and multi-level classification techniques to monitor login attempts and detect attacks. The model may use NLP to convert text associated with account activity into a number vector, where the vector includes a score and / or other numerical value calculated based on the meaning of the converted text. The model may further include a set of classifiers trained to learn patterns in the number vector that are predictive of network attacks. The model may assign a label to an event based on the predicted likelihood that the event is an attack. The system may deploy real-time prevention or corrective measures based on the ML model output to combat or mitigate the effects of the attack.

Description

[Technical Field]

[0001] Technical Field This disclosure relates to network attack detection, prevention, and mitigation. In particular, this disclosure relates to using machine learning to adaptively predict and prevent attacks on accounts accessible over a network. [Background technology]

[0002] background A network attack is an attempt to gain unauthorized access to a set of computing resources accessible over a network. A successful network attack may allow an unauthorized party to view and copy sensitive data, thereby compromising data security. In more serious cases, an attacker may modify, encrypt, or otherwise tamper with data. A data breach can lead to serious repercussions for individuals and organizations, including legal liability arising from the loss or unauthorized use of private data.

[0003] Network administrators may deploy preventative measures to combat network attacks. For example, network administrators may set a threshold number of password attempts before locking a user account, install antivirus software to monitor the network for viruses, and encrypt sensitive data to reduce the likelihood of unauthorized access. However, network attacks are constantly evolving, and it may be difficult to proactively address all attack techniques.

[0004] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Thus, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by their inclusion in this section.

[0005] Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. It should be noted that references to "one" or "an" embodiment in this disclosure are not necessarily to the same embodiment, but that such references mean at least one. In the drawings: [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an example system for network threat analysis, according to some embodiments. [Figure 2] FIG. 10 illustrates an example set of operations for converting text tokens to numeric values, according to some embodiments. [Figure 3] FIG. 10 illustrates an example conversion of log data into a number score according to some embodiments. [Figure 3] FIG. 1 illustrates an example set of operations for training a machine learning model for adaptively predicting network attacks, according to some embodiments. [Figure 4] FIG. 1 illustrates an example set of operations for training a machine learning model to perform real-time monitoring of network attacks, according to some embodiments. [Figure 5] FIG. 1 illustrates an example set of operations for tuning a machine learning model to perform real-time monitoring of network attacks, according to some embodiments. [Figure 6] FIG. 1 illustrates an example set of operations for applying machine learning models to perform real-time monitoring of network attacks, according to some embodiments. [Figure 7] FIG. 1 illustrates an example application of a model for analyzing a network threat associated with an event, according to some embodiments. [Figure 8] FIG. 1 illustrates an example computer system according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] Detailed Description In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in a different embodiment. In some instances, well-known structures and devices are described with reference to block diagram form in order to avoid unnecessarily obscuring the present invention.

[0008] 1. General Overview 2. System Architecture for Network Threat Analysis 3. A Model for Network Threat Monitoring 3.1 NLP-based Feature Engineering 3.2 Classifier training 3.3 Model Evaluation and Tuning 4. Real-time threat analysis and detection 5. Real-time attack response and mitigation 6. Computer Networks and Cloud Networks 7. Microservices Applications 8. Hardware Overview 9.Other extensions 1. General Overview Machine learning techniques and models are described for alerting users to attacks on their accounts in real time or near real time. In some embodiments, the attack detection model uses natural language processing (NLP) and multi-level classification techniques to monitor login attempts and detect attacks. The model may use NLP to convert text associated with account activity into a number vector, where the vector includes a score and / or other numerical value calculated based on the meaning of the converted text. The model may further include a set of classifiers trained to learn patterns in the number vector that are predictive of network attacks. The model may assign a label to an event based on the predicted likelihood that the event is an attack. The system may deploy real-time preventative or corrective measures based on the model output to counter or mitigate the effects of the attack.

[0009] During the training phase, a machine learning (ML) engine may receive a training dataset including multiple examples of user log events associated with one or more user accounts. The ML engine may use the training dataset to train an attack detection model to learn atypical behaviors that are predictive of attacks, including the type and severity of network attacks. The training process may use NLP during feature extraction and engineering to transform the text included in the examples into a set of number vectors. The number vectors may include scores for words based on how meaningful the words are to the log entry relative to how meaningful the words are to a list of historical events, such as all events in the past three to five days. The ML engine may then build one or more ML classification models as a function of varying feature values, including varying NLP-based scores, to learn what behaviors associated with the log events are most predictive of network attacks.

[0010] In some embodiments, the ML engine may build ML models on a per-account basis. By building separate ML models for different accounts, the system may learn different archetype behaviors for various users. Behavior that is atypical for one user may not be atypical for another user. Additionally, or alternatively, the number and / or severity of likely attacks may vary for different users, even when they exhibit similar behaviors. Machine learning allows archetype behaviors to be learned at application runtime, thereby avoiding hard-coded rules that may not be universally applicable to all user accounts. ML models may further evolve as archetype behaviors change over time, adapting to new attack techniques. ML models may be retrained periodically or continuously as new behaviors are observed.

[0011] During the inference phase, the ML engine may generate predictions by applying the trained ML model to newly generated log data associated with the user account. When applying the model, the ML engine may perform feature extraction and transformation to generate feature vectors in the same manner as the training phase. For example, the ML engine may use NLP to convert the log text into a number vector and apply a trained classifier to the number vector to generate predictions. The newly observed data may be unique, without exactly matching any previous examples in the training dataset, due in part to the extremely large number of possible permutations of the extracted feature values. The ML model may receive the feature vectors as input and output a set of one or more predictions regarding whether the observed behavior is a network attack.

[0012] The system may use the ML model predictions to provide analytical insights and / or trigger responsive actions to address attacks in real time. For example, the system may generate real-time alerts identifying accounts in which the ML model detected a network attack. Additionally or alternatively, the system may implement preventative actions at runtime, including selectively enabling or disabling security measures on a per-account basis based on the predicted network attack risk.

[0013] One or more embodiments described herein and / or recited in the claims may not be included in this General Overview section.

[0014] 2. System Architecture for Network Threat Analysis In some embodiments, a network threat analysis system provides real-time monitoring of a set of user accounts for accessing one or more network services and / or one or more networked computing resources. The user accounts may provide a mechanism by which the system identifies, tracks, and / or authenticates distinct users. A user may log in to a user account through an authentication process that may require the user to submit a password and / or other authentication credentials. Once logged in, the user may access files, applications, and / or other resources that the user is authorized to access.

[0015] In some embodiments, each user account is associated with a different home directory that serves as a root directory for the corresponding user account and may store files generated based on the activity of a user logged into the user account. Access to the root directory may be restricted to the corresponding user account and one or more administrator accounts, thereby preventing unauthorized access to the user's files by other users of the network service. Furthermore, when a user logs into a user account, the system may restrict the user's access to the root directory associated with the user account to prevent unauthorized access to private system resources.

[0016] In some embodiments, the set of user accounts may include accounts for accessing one or more cloud services. Cloud services may include computing infrastructure, platforms, and / or software hosted by a third-party service provider and made available over the Internet. Example cloud service models include Software as a Service (SaaS), Database as a Service (DBaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). A user may create an account as part of a subscription for one or more cloud services.

[0017] In some embodiments, cloud services may enable subscribing entities to build and deploy network services accessible to other users. For example, cloud services may host software and / or hardware resources provisioned to subscribers to customize and launch e-commerce websites. Online shoppers may visit websites and create separate accounts to access websites and / or subscribe to online services. Thus, a primary subscriber account may manage or otherwise be associated with multiple shoppers, secondary subscribers, and / or other user accounts that have access to online services created by the primary subscriber using provisioned cloud resources, resulting in a multi-level hierarchy of user accounts. Network attack monitoring techniques, as further described herein, may be applied to one or more levels of user accounts.

[0018] FIG. 1 illustrates an example system for network threat analysis, according to some embodiments. As illustrated in FIG. 1, system 100 includes network service 102, network 122, data repository 124, and clients 130a-b. System 100 may include more or fewer components than those illustrated in FIG. 1. The components illustrated in FIG. 1 may be proximate to each other or remote from each other. The components illustrated in FIG. 1 may be implemented in software and / or hardware. Each component may be distributed across multiple applications and / or machines. Multiple components may be combined into one application and / or machine. Operations described with respect to one component may instead be performed by another component.

[0019] In some embodiments, network service 102 includes a set of hardware and / or software resources accessible via network 122. Network service 102 may represent one or more cloud services, such as IaaS, PaaS, DBaaS, and / or SaaS applications. Additionally or alternatively, network service 102 may include a set of components for managing a set of user accounts to identify, track, and / or authenticate distinct users. The set of components may include account manager 104, authentication service 106, applications 108, tracking service 110, ML service 112, and interface engine 120. As previously mentioned, the components in system 100, including network service 102, may vary. In some cases, functions performed by one component may be combined or otherwise implemented by another component in system 100. Additionally or alternatively, the components of network service 102 may execute locally or remotely from each other.

[0020] In some embodiments, the account manager 104 manages user accounts that have access to the network service 102. For example, the account manager 104 may manage the creation of new user accounts and the deletion of accounts when users subscribe to a service. Additionally or alternatively, the account manager 104 may assign identifiers that uniquely identify distinct user accounts. Additionally or alternatively, the account manager 104 may manage other aspects of user accounts, such as privacy settings, identity and access management (IAM) policies, and user account access permissions.

[0021] Once a user account is created, the user may log in to the user account upon successful authentication by the authentication service 106. In some embodiments, the authentication service 106 implements one or more authentication protocols to verify user identity. Example authentication protocols include Password Authentication Protocol (PAP), Challenge Handshake Authentication Protocol (CHAP), and Authentication, Authorization, and Accounting (AAA) protocols. During a login attempt, the user may submit a username, password, digital certificate, and / or other credentials. The authentication service 106 checks the credentials and may block the login attempt if the credentials are not successfully verified.

[0022] If the credentials are successfully authenticated, the user may be granted permission to access a restricted set of network resources, such as applications 108, which may include software and / or services for performing tasks directed by the end user. For example, a SaaS application may include software and services for managing customer relations, operations, social media, inventory, website design, and / or e-commerce functions. However, application-specific functionality may vary depending on the network service and / or user subscription.

[0023] The tracking service 110 may generate logs that track the activity of users who have logged in to and / or attempt to log in to user accounts. In some embodiments, the tracking service 110 includes one or more monitoring agents, such as daemons and / or log-generating processes, that trace or otherwise capture user requests. For example, the tracking service 110 may track the number of directory traversals, the number of standard query language (SQL) injection attempts, the number of successful login attempts, the number of failed login attempts, the location of the login attempts, and / or the number of vulnerability scans triggered for one or more user accounts. Additionally or alternatively, other metrics may be logged by the tracking service 110 to track the behavior of online users.

[0024] In some embodiments, the ML service 112 includes components for profiling user behavior and learning which behavioral patterns are predictive of future network attacks. Rather than relying on a static instruction set to accomplish its tasks, the ML service 112 may make inferences and adjustments during application runtime. In this way, the system 100 may adapt in real time to changing and evolving behaviors that are indicative of attacks, without requiring additional hard-coding of new attack patterns.

[0025] In some embodiments, the ML service 112 includes a training engine 114 for training ML models, a tuning engine 116 for adjusting ML model parameters and / or hyperparameters, and a prediction engine 118 for applying the trained ML models. Techniques for training and tuning ML models are further described in Section 3, entitled "Models for Network Threat Monitoring."

[0026] The interface engine 120 may provide a user interface for interacting with the network services 102. Example user interfaces may include a graphical user interface (GUI), an application programming interface (API), a command line interface (CLI), or some other interface for accessing network resources. The interface engine 120 may serve interface components to client applications, including clients 130a-b, which may render the elements on a display. For example, the clients may be browsers, mobile apps, or application front ends that display user interface elements for invoking one or more of the network services 102 through GUI windows. Examples of user interface elements include check boxes, radio buttons, drop-down lists, list boxes, buttons, toggles, text fields, date and time selectors, command lines, sliders, pages, and forms.

[0027] Users may use clients 130a-b, which may include client applications and / or devices, to connect with network service 102 over network 122. Network 122 represents one or more interconnected data communication networks, such as the Internet. Clients may connect with network service 102 via one or more communication protocols. Example communication protocols may include Hypertext Transfer Protocol (HTTP), Simple Network Management Protocol (SNMP), and other communication protocols of the Internet Protocol (IP) suite.

[0028] In some embodiments, the network resources include a data repository 124. The data repository 124 may include volatile and / or non-volatile storage for storing behavioral profiles 126 and ML model data 128. The behavioral profiles 126 may include metrics and learned patterns that represent typical user behavior for one or more user accounts. The ML model data 128 may store model artifacts and outputs. For example, the ML model data 128 may store weights, biases, hyperparameter values, and / or other artifacts obtained through model training. Additionally or alternatively, the ML model data 128 may include predictions and / or other values ​​obtained from evaluating and applying the trained ML models.

[0029] In some embodiments, the ML model predictions and associated functionality are exposed through a cloud service or microservice. A cloud service may support multiple tenants, further referred to as subscribing entities. A tenant may correspond to a corporation, organization, enterprise, or other entity that accesses shared computing resources. Different tenants may be managed independently, even if they share computing resources. For example, different tenants may have different account identifiers, access credentials, identity and access management (IAM) policies, and configuration settings. Additional embodiments and / or examples related to computer networks and microservice applications are described below in Section 6 entitled Computer Networks and Cloud Networks and Section 7 entitled Microservice Applications.

[0030] 3. A Model for Network Threat Monitoring 3.1 NLP-based Feature Engineering In some embodiments, the ML service 112 generates a set of feature vectors for training the ML model. The feature vectors may include a set of values ​​for various features that capture behavioral attributes associated with a user account. For example, the feature vector

[0031]

number

[0032] is [x1,x2,....,x n ], where x1 is the value for the first feature, x2 is the value for the second feature, and x n is the value for the nth feature.

[0033] The features selected for training and the number of features in a vector may vary depending on the particular implementation. One or more features may be curated by a domain expert. Additionally or alternatively, the ML service 112 may select one or more features during the training and / or tuning phase based on which features produce the ML model with the highest performance. The ML service 112 may extract, generate, and / or select features based on activity tracked by the tracking service 110.

[0034] In some embodiments, the set of features includes values ​​extracted from log data associated with a user account. User activity, such as a login attempt, may trigger the tracking service 110 to generate login data that captures attributes associated with the activity. For example, the log data may include one or more of the attributes shown in Table 1 below.

[0035] [Table 1] TIFF2024540116000004.tif126169

[0036] Example event log attributes capture various aspects of login attempt behavior. Additionally or alternatively, other attributes associated with account activity may be extracted and used as features to build ML models.

[0037] In some embodiments, the ML service 112 includes an NLP engine that converts text-based features into numerical values. Illustratively, the numerical value for a token may be a score that represents the degree to which the word is meaningful for a log entry relative to the degree to which the word is meaningful for a list of historical events. An example technique for assigning a score is to calculate a term frequency inverse-document frequency (TF-IDF) score. With TF-IDF, the score for a token increases proportionally to the frequency with which the token appears in log records, offset by the number of logs that contain the token. TF-IDF scores may be calculated on a per-account basis to account for varying user behavior.

[0038] 2 illustrates an example set of operations for converting text tokens to numeric values, according to some embodiments. One or more of the operations illustrated in FIG. 2 may be modified, rearranged, or omitted altogether. Thus, the particular sequence of operations illustrated in FIG. 2 should not be construed as limiting the scope of one or more embodiments.

[0039] Referring to Figure 2, the process includes identifying text tokens in the log entries (operation 202). For example, the process may extract one or more of the attribute values ​​listed above in Table 1 for recent or historical login attempts. As used herein, text tokens may include words and / or phrases. Additionally, or alternatively, text tokens may include numeric values ​​in string format. By way of illustration, IP addresses and timestamps may include numeric values. The process may generate a score based on the frequency and / or uniqueness of the tokens, as further described herein.

[0040] The process then determines the frequency of the text token in the log entry (operation 204). For example, the process may calculate the term frequency for a token as the number of repetitions of the token in the log entry divided by the total number of tokens in the log entry. In some embodiments, a weighting scheme such as logarithmic scaling or expanded frequency may be applied to prevent bias toward longer log entries. However, unweighted frequency values ​​may also be used, depending on the particular implementation.

[0041] The process further determines the frequency of the text token in the list of historical events (operation 206). For example, the process may determine the frequency of the token in the list of log records in the past five days, or over some other time frame. A logarithmically scaled inverse document frequency may be calculated by taking the logarithm of the value obtained by dividing the total number of log entries in the specified time frame by the number of log entries that contain the token.

[0042] In some embodiments, the process calculates a score for the text token based on the frequency of the text token in the log entry relative to the list of historical events (operation 210). For example, the TF-IDF score for the token may be calculated as the product of the token frequency and the inverse log record frequency.

[0043] The process further determines whether there are any remaining text tokens in the log entry to analyze (operation 210). If so, the process may iterate through the remaining text tokens to calculate a score for the tokens.

[0044] In some embodiments, the process calculates a score for the log record based on the scores of the text tokens included in the log record (operation 212). For example, the process may sum, average, and / or otherwise aggregate the scores to calculate the score for the log record. The individual and / or aggregate token scores may be used to train an ML model classifier, as described further below.

[0045] 3 illustrates an example conversion of log data to a numeric score, according to some embodiments. Table 300 shows example path attributes extracted from different log records. Each path attribute contains a set of text tokens, including services, rest, v1, and atg_settlement. Table 302 shows the term frequency and inverse document (log record) frequency values ​​for each of the tokens, and table 304 shows the resulting score values.

[0046] 3.2 Classifier training The training engine 114 may use the set of feature vectors associated with the user account to train one or more ML models. In some embodiments, the training engine 114 trains one or more classification models that classify activities based on the detected threat level. For example, the trained classifier may assign a green label to activities that are not detected to be network attacks, an amber label when a low risk of network attack is predicted, and a red label to activities that involve a high risk of network attack. Additionally or alternatively, other labels may be assigned depending on the particular implementation.

[0047] The classification model trained may vary depending on the particular implementation. In some embodiments, the training engine 114 builds one or more decision trees, which may include random forests and / or gradient-boosted trees. However, the training engine 114 may train other ML classifiers, such as cluster-based classifiers, support vector machines (SVMs), and / or artificial neural networks.

[0048] 4 illustrates an example set of operations for training a machine learning model to perform real-time monitoring of network attacks, according to some embodiments. One or more of the operations illustrated in FIG. 4 may be modified, rearranged, or omitted altogether. Thus, the particular sequence of operations illustrated in FIG. 4 should not be construed as limiting the scope of one or more embodiments.

[0049] Referring to FIG. 4 , the process includes generating a set of NLP-based scores for text features in a set of log data used to train an ML model (operation 402). For example, the process may receive a set of training examples, each example including one or more historical log records and an indication of whether a network attack occurred. The process may then generate TF-IDF scores for individual text tokens and / or log records, as described above. The process may generate a feature vector for the examples that includes the scores. Additionally or alternatively, the feature vector may include values ​​for other attributes, such as the number of vulnerability scanners detected, the number of directory traversals, and the number of SQL injection attempts.

[0050] In some embodiments, the process selects features for splitting the decision tree (operation 404). The process selects features for splitting the decision tree (operation 404). a *(1-p a ), where p a represents the proportion of training examples in a particular class for the prediction node. a may represent the percentage of data in which an attack or a distinct category of attack (e.g., severe attack, moderate attack, no attack) was detected. As another example, the process may determine that a TF-IDF score of 5.1 for a distinct feature minimizes an error function. However, the selected features and feature values ​​used to split the tree may vary depending on the distinct activity detected in the account. The process may implement a greedy algorithm to identify the features and feature values ​​used to split the tree, but the manner in which the selection is made may vary depending on the particular implementation.

[0051] The process then splits the training data set based on the selected features (operation 406). For example, if a TF-IDF score of 5.1 for an individual feature is selected, training examples with values ​​less than 5.1 may be assigned to one branch of the tree, and training examples with values ​​greater than 5.1 may be assigned to another branch of the tree. If different values ​​and / or features are selected, the process splits along the learned boundaries.

[0052] In some embodiments, the process determines whether to continue splitting the decision tree (operation 408). The process may continue splitting the tree until a set of one or more stopping criteria is satisfied. For example, the process may split the tree until the number of examples assigned to one or more leaf nodes falls below a minimum threshold. If the stopping criteria are not satisfied, the process may return to operation 408 and recursively split the tree.

[0053] Once the stopping criteria are satisfied, the process may prune the decision tree based on which features, including TF-IDF scores, are least predictive of attacks (operation 410). For example, a node may be pruned if it separates two groups of training examples that have little or no difference in observed attacks. Additionally or alternatively, the process may determine the difference in the error function when the node is pruned. If the difference is greater than a threshold, the pruning may be reversed and the node may be reinserted into the tree. If the difference in the error function is less than a threshold, the pruning may be maintained. As a result, the split examples or branches may be merged. The process may continue pruning nodes until removing one of the remaining nodes changes the result of the error function by more than a threshold amount or until a minimum threshold number of nodes remain.

[0054] Once a decision tree is built, the process may determine whether to build additional decision trees (operation 412). Multiple decision trees may be built in the case of random forests and gradient-boosted decision trees. For example, to generate a random forest, training data may be divided into several groups of examples. Each separate set of training examples may be used to independently build a separate decision tree. For gradient-boosted decision trees, several trees are built sequentially, with each new decision tree minimizing an error function, such as the mean squared error or logarithmic loss, of one or more previous trees in the sequence. Random forests and gradient-boosted decision trees may reduce overfitting and improve the predictive accuracy of a trained ML model.

[0055] 3.3 Model Evaluation and Tuning In some embodiments, the tuning engine 116 evaluates the trained ML model and tunes the ML model to optimize performance. The tuning engine 116 may measure performance using an F-measure, such as an F1-score. The F-measure evaluates a model based on precision and recall, and the F1-score represents the harmonic mean between the two factors. The tuning engine 116 may adjust the trained ML model parameters and hyperparameters until the F-score satisfies a threshold. While the F-score is used in the examples herein, in other embodiments, the tuning engine 116 may use other measures of accuracy to tune the ML model, such as mean average precision (MAP) and R-Precision metrics.

[0056] 5 illustrates an example set of operations for tuning a machine learning model to perform real-time monitoring of network attacks, according to some embodiments. One or more of the operations illustrated in FIG. 5 may be modified, rearranged, or omitted altogether. Thus, the particular sequence of operations illustrated in FIG. 5 should not be construed as limiting the scope of one or more embodiments.

[0057] 5, the process includes applying the trained ML model to test data and / or new incoming data to generate attack predictions (operation 502). The process may then compare the predictions to observed attacks to evaluate the model.

[0058] In some embodiments, the process determines a precision rate of the ML model prediction (operation 504). The process may calculate the precision rate by dividing the number of correctly predicted attacks by the total number of predicted attacks, including attacks that were predicted but not observed. The precision rate may then be used as a measure to indicate how effective the ML model is in avoiding false flag alerts.

[0059] In some embodiments, the process determines the recall of the ML model predictions (operation 506). The process may calculate the recall by dividing the number of correctly predicted attacks by the total number of observed attacks. The recall may then be used as a measure to indicate how sensitive the ML model is at detecting attacks.

[0060] In some embodiments, the process determines whether the balance between precision and recall satisfies a threshold value (operation 508). For example, the process may determine whether the harmonic mean is above a threshold value. An F1 score above 85% may indicate a good balance in some applications. However, the threshold value may vary depending on the particular implementation.

[0061] If the balance does not satisfy the threshold, the process may tune the ML model by adjusting one or more model hyperparameters and / or parameters (operation 510). Example hyperparameters and parameters may include the depth of the decision tree, the number of decision trees in the random forest, the length of the time window of historical log records used to calculate the TF-IDF score, the minimum number of training examples per leaf, and the set of features selected to use to build the decision tree. Additionally, or instead, the tuning engine 116 may adjust other parameter values ​​associated with the model. The process may continue adjusting values ​​until a balance between precision and recall is satisfied.

[0062] Once the threshold balance has been achieved, the process may store the ML model parameter and hyperparameter values ​​(operation 512). The prediction engine 118 may access the stored values ​​to apply the ML model to new incoming data as user activity is monitored in real time.

[0063] 4. Real-time threat analysis and detection 6 illustrates an example set of operations for applying a machine learning model to perform real-time monitoring of network attacks, according to some embodiments. One or more of the operations illustrated in FIG. 6 may be modified, rearranged, or omitted altogether. Thus, the particular sequence of operations illustrated in FIG. 6 should not be construed as limiting the scope of one or more embodiments.

[0064] 6, the process detects a new event log for the account (operation 602). For example, the process may detect a log associated with a new login attempt or other activity associated with the user account.

[0065] In response to detecting the event log, the process generates a score for the text token in the event log (operation 604). In some embodiments, the process generates the score using TF-IDF, as described above. Thus, the score for a token may be calculated as a function of how frequently the token occurs in the currently detected event log relative to how frequently the token occurs in historical account log records within a threshold time frame, such as the past 3 to 5 days.

[0066] The process further generates a score for the log record based on the scores of the text tokens included in the log record (operation 606). For example, the process may sum the TF-IDF scores of the text tokens, average those scores, or otherwise aggregate those scores.

[0067] Once the scores have been calculated, the process applies one or more trained classifiers to predict whether new event log activity represents a current attack (operation 608). For example, the process may traverse one or more decision vines based on the calculated scores. Additionally or alternatively, the process may identify the nearest cluster in a cluster-based model or a hyperplane boundary in a trained SVM model to classify the log record.

[0068] Based on the applied classifier, the process generates an output based on the predicted likelihood that the current account activity merits an attack (operation 610). In some embodiments, the output includes a label, such as red, yellow, or green, based on the probability of the attack and / or the predicted severity of the attack. For example, a probability or severity above a high threshold may be assigned the label red, a probability or severity above a lower threshold but below the high threshold may be assigned the label yellow, and a probability or severity below the lowest threshold may be assigned the label green. Additionally or alternatively, the output of the ML model may estimate the type of network attack that will occur, such as an SQL injection attempt, a directory traversal attack, or a credential stuffing attack.

[0069] In embodiments where multiple decision trees are used, such as by a random forest, the process may calculate a final prediction by aggregating the predictions of the multiple trees. For example, the process may calculate the mean, median, or mode prediction of the decision trees. The process may then output the aggregated result.

[0070] Table 2 illustrates an example set of sample outputs from a trained ML model, according to some embodiments.

[0071] [Table 2]

[0072] As illustrated in Table 2, the first column contains a set of raw log data, the second column identifies feature-engineered TF-IDF scores for various text tokens in the log data, the third column identifies an overall score for the event, and the fourth column shows the inferred label. The model output provides real-time insight into whether current account activity warrants a network attack. If warranted, the model output may be consumed by a user, application, and / or system to take appropriate prevention and mitigation actions.

[0073] FIG. 7 illustrates an example application of a model for analyzing network threats associated with events, according to some embodiments. Table 700 identifies a set of text tokens and scores associated with an event log. Table 702 identifies an overall score for the event log. Classification model 704 includes a set of decision trees that are traversed based on the overall score for the log event. For example, the process may determine whether to traverse to the left or right of the node based on the score until a leaf node is reached. The leaf node may be associated with a classification label, such as red, yellow, or green. Classification model 704 uses a voting system, whereby the majority classification is used as the final classification. In this example, a majority of the decision trees classified the log event as an attack, as reflected in table 706.

[0074] In some embodiments, system 100 may generate and render charts, lists, and / or other objects to be presented to a user based on the ML model output. For example, interface engine 120 may present a list of subscribers who have experienced attacks within the last five minutes. Additionally or alternatively, interface engine 120 may highlight the top n shoppers associated with subscriber accounts with the highest attack severity.

[0075] Additionally or alternatively, system 100 may generate alerts to notify administrators, primary subscribers, and / or other users of network attacks. For example, system 100 may send email and / or short message service (SMS) messages to primary subscribers if a severe attack is detected based on shopper log events. As another example, system 100 may send alert messages to primary subscribers if a threshold number of subscribers have experienced a severe attack within the last five minutes or some other time frame as detected by an ML model.

[0076] In some embodiments, an administrator may search and filter a list of accounts based on model predictions. By way of illustration, a user may request to view only a list of accounts that have experienced severe attacks within a threshold time frame, accounts with a threshold number of log records classified as red, accounts with a predicted type of attack, and / or accounts with behavior predicted to be atypical within a given time frame. In response to the filter request, the interface engine 120 may identify a list of accounts and / or shoppers that satisfy the filter criteria and present the end user with information about the accounts, such as account names, current status, and / or other account attributes.

[0077] 5. Real-time attack response and mitigation In some embodiments, system 100 may perform one or more attack prevention or mitigation actions based on the output of one or more trained ML models. When an attack is detected at login, system 100 may implement responsive actions at runtime, including selectively enabling or disabling security measures on a per-account basis. For example, system 100 may lock an account, selectively enable two-part authentication, block an IP address, send a one-time password to a user, run a vulnerability scan, and / or perform other actions to prevent or minimize the damage of an ongoing attack.

[0078] In some embodiments, system 100 compares the ML model output for newly detected account activity with one or more thresholds. If one or more thresholds are satisfied, system 100 may trigger one or more adaptive attack prevention and mitigation actions. For example, system 100 may compare the estimated number and / or severity of attacks on an account in the past 15 minutes with a corresponding threshold. If a threshold is satisfied, system 100 may enable one or more of the additional security measures described above. Additionally or alternatively, the type of security measure activated may vary depending on the severity, number, and / or type of network attacks detected by the ML model. Illustratively, different security measures may be deployed for SQL injection attempts than for directory traversal attacks.

[0079] In some embodiments, an administrator may configure thresholds and / or actions taken by the system 100 to address attacks in real time. For example, an administrator may define rules to be followed.

[0080] If (NrOfAttackInPrevious5minutes=5) and (AlertLabelforCurrentAttack==Red) SendOneTimePassword(useraccount) The system 100 may evaluate a rule based on the output of an ML model associated with a user account. If the number and labels of detected attacks satisfy the criteria specified by the custom rule, the system 100 may send a one-time password to the user via email or SMS message to verify that the user is active. Activity on the user account may be locked until the one-time password is received. In other embodiments, the conditions and actions specified by the rule may vary depending on administrator input. Additionally or alternatively, the system 100 may perform a default action or a learned action to stop the attack pattern based on the ML model output.

[0081] 6. Computer Networks and Cloud Networks In some embodiments, a computer network provides connectivity between a set of nodes. The nodes may be close to each other and / or remote from each other. The nodes are connected by a set of links. Examples of links include coaxial cable, unshielded twisted cable, copper cable, optical fiber, and virtual links.

[0082] A subset of nodes implements computer networks. Examples of such nodes include switches, routers, firewalls, and network address translators (NATs). Another subset of nodes uses computer networks. Such nodes (further referred to as "hosts") may execute client processes and / or server processes. A client process makes requests for computing services (such as running a particular application and / or storing a particular amount of data). A server process responds by performing the requested service and / or returning corresponding data.

[0083] A computer network may be a physical network, including physical nodes connected by physical links. A physical node is any digital device. A physical node may be a function-specific hardware device, such as a hardware switch, a hardware router, a hardware firewall, and a hardware NAT. Additionally or alternatively, a physical node may be a general-purpose machine configured to run various virtual machines and / or applications that perform respective functions. A physical link is a physical medium connecting two or more physical nodes. Examples of links include coaxial cable, unshielded twisted cable, copper cable, and optical fiber.

[0084] A computer network may be an overlay network. An overlay network is a logical network implemented on top of another network (such as a physical network). Each node in the overlay network corresponds to a respective node in the underlying network. Thus, each node in the overlay network is associated with both an overlay address (for addressing the overlay node) and an underlay address (for addressing the underlay node that implements the overlay node). Overlay nodes may be digital devices and / or software processes (such as virtual machines, application instances, or threads). Links connecting overlay nodes are implemented as tunnels through the underlying network. Overlay nodes at either end of the tunnel treat the underlying multi-hop path between them as a single logical link. Tunneling is accomplished through encapsulation and decapsulation.

[0085] In some embodiments, a client may be local to and / or remote from a computer network. A client may access a computer network over a private network or another computer network, such as the Internet. A client may communicate a request to a computer network using a communication protocol, such as the Hypertext Transfer Protocol (HTTP). The request is communicated through an interface, such as a client interface (e.g., a web browser), a program interface, or an application programming interface (API).

[0086] In some embodiments, a computer network provides connectivity between clients and network resources. The network resources include hardware and / or software configured to run server processes. Examples of network resources include processors, data storage, virtual machines, containers, and / or software applications. The network resources are shared among multiple clients. The clients request computing services from the computer network independently of each other. The network resources are dynamically allocated to requests and / or clients on an on-demand basis. The network resources allocated to each request and / or client may be scaled up or down based on, for example, (a) the computing services requested by individual clients, (b) the aggregated computing services requested by individual tenants, and / or (c) the aggregated computing services requested of the computer network. Such a computer network may be referred to as a "cloud network."

[0087] In some embodiments, a service provider offers a cloud network to one or more end users. Various service models may be enabled by the cloud network, including, but not limited to, Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). In SaaS, a service provider offers end users the ability to use the service provider's applications running on network resources. In PaaS, a service provider offers end users the ability to deploy custom applications onto the network resources. The custom applications may be created using programming languages, libraries, services, and tools supported by the service provider. In IaaS, a service provider offers end users the ability to provision the processing, storage, network, and other basic computing resources provided by the network resources. Any optional applications, including operating systems, may be deployed on the network resources.

[0088] In some embodiments, a computer network may implement various deployment models, including, but not limited to, private clouds, public clouds, and hybrid clouds. In a private cloud, network resources are provisioned for exclusive use by a discrete group of one or more entities (the term "entity" as used herein refers to a corporation, organization, individual, or other entity). The network resources may be local to and / or remote from the premises of the discrete group of entities. In a public cloud, cloud resources are provisioned for multiple entities (further referred to as "tenants" or "customers") that are independent of each other. The computer network and its network resources are accessed by clients corresponding to different tenants. Such a computer network may be referred to as a "multi-tenant computer network." Several tenants may use the same discrete network resources at different times and / or at the same time. The network resources may be local to and / or remote from the tenants' premises. In a hybrid cloud, the computer network includes a private cloud and a public cloud. The interface between the private cloud and the public cloud allows for data and application portability. Data stored in the private cloud and data stored in the public cloud may be exchanged through the interface. Applications implemented in the private cloud and applications implemented in the public cloud may have dependencies on each other. Calls from applications in the private cloud to applications in the public cloud (and vice versa) may be made through the interface.

[0089] In some embodiments, tenants of a multi-tenant computer network are independent of one another. For example, the business or operations of one tenant may be separate from the business or operations of another tenant. Different tenants may desire different network requirements for the computer network. Examples of network requirements include processing speed, amount of data storage, security requirements, performance requirements, throughput requirements, latency requirements, resiliency requirements, quality of service (QoS) requirements, tenant isolation, and / or consistency. The same computer network may be required to achieve different network requirements desired by different tenants.

[0090] In one or more embodiments, in a multi-tenant computer network, tenant isolation is implemented to ensure that applications and / or data of different tenants are not shared with each other. Various tenant isolation techniques may be used.

[0091] In some embodiments, each tenant is associated with a tenant ID. Each network resource in a multi-tenant computer network is tagged with a tenant ID. A tenant is granted access to an individual network resource only if the tenant and the individual network resource are associated with the same tenant ID.

[0092] In some embodiments, each tenant is associated with a tenant ID. Each application implemented by the computer network is tagged with a tenant ID. Additionally or alternatively, each data structure and / or dataset stored by the computer network is tagged with a tenant ID. A tenant is granted access to individual applications, data structures, and / or datasets only if the tenant and the individual applications, data structures, and / or datasets are associated with the same tenant ID.

[0093] As an example, each database implemented by a multi-tenant computer network may be tagged with a tenant ID. Only the tenant associated with the corresponding tenant ID may access the data in the individual database. As another example, each entry in a database implemented by a multi-tenant computer network may be tagged with a tenant ID. Only the tenant associated with the corresponding tenant ID may access the data in the individual entry. However, a database may be shared by multiple tenants.

[0094] In some embodiments, the subscription list indicates which tenants have permission to access which applications. For each application, a list of tenant IDs of tenants authorized to access the application is stored. A tenant is granted access to a particular application only if the tenant's tenant ID is included in the subscription list corresponding to the particular application.

[0095] In some embodiments, network resources (such as digital devices, virtual machines, application instances, and threads) corresponding to different tenants are isolated to tenant-specific overlay networks maintained by a multi-tenant computer network. As an example, packets from any source device in a tenant overlay network may only be sent to other devices in the same tenant overlay network. An encapsulation tunnel is used to prohibit any transmission from a source device on a tenant overlay network to a device in another tenant overlay network. Specifically, a packet received from a source device is encapsulated in an outer packet. The outer packet is sent from a first encapsulation tunnel endpoint (communicating with a source device in the tenant overlay network) to a second encapsulation tunnel endpoint (communicating with a destination device in the tenant overlay network). The second encapsulation tunnel endpoint decapsulates the outer packet to obtain the original packet sent by the source device. The original packet is sent from the second encapsulation tunnel endpoint to a destination device in the same individual overlay network.

[0096] 7. Microservices Applications According to some embodiments, the techniques described herein are implemented in a microservices architecture. A microservice in this context refers to software logic designed to be independently deployable, with endpoints that may be logically coupled to other microservices to build various applications. Applications built using microservices are distinct from monolithic applications, which are designed as a single, fixed unit and typically include a single logical executable. A microservices application allows different microservices to be independently deployable as separate executables. Microservices may communicate via API endpoints, using HTTP messages, and / or other communication protocols. Microservices may be managed and updated separately, written in different languages, and run independently of other microservices.

[0097] Microservices provide flexibility in managing and building applications. Different applications can be built by connecting different sets of microservices without changing the microservices' source code. Microservices thus act as logical building blocks that can be arranged and configured in various ways to build different applications. A microservice may provide a monitoring service that notifies a microservice manager (such as If-This-Then-That (IFTTT), Zapier, or Oracle Self-Service Automation (OSSA)) when a trigger event occurs from a set of trigger events exposed to the microservice manager. A microservice exposed for an application may alternatively or additionally provide an action service that performs an action in the application based on data received from the microservice manager (controllable and configurable by the microservice manager by passing values ​​in and connecting the action to other triggers and / or data passed in from other actions in the microservice manager). Microservice triggers and / or actions may optionally be chained together to form recipes of actions that occur in different applications that are not otherwise aware of or have no control or dependency on each other. These managed applications may be authenticated or plugged into the microservice manager, for example, by application credentials supplied by a user to the manager, without having to re-authenticate each time the managed application is used alone or in combination with other applications.

[0098] In some embodiments, microservices may be connected through a GUI. For example, microservices may appear as logical blocks within windows, frames, or other elements of the GUI. A user may drag and drop microservices into areas of the GUI used to build applications. A user may use directional arrows or any other GUI elements to connect the output of one microservice into the input of another microservice. An application builder may run validation tests to ensure that the outputs and inputs are consistent (e.g., by checking data types, size constraints, etc.).

[0099] Trigger The techniques described above may be encapsulated into a microservice, according to some embodiments. In other words, a microservice may trigger a notification (into a microservice manager for optional use by other plugged-in applications, referred to herein as a "target" microservice) based on the techniques described above and / or may be represented as a GUI block and connected to one or more other microservices. The trigger condition may include an absolute or relative threshold for a value and / or an absolute or relative threshold for the amount or duration of data to be analyzed, such that a trigger to the microservice manager occurs whenever a plugged-in microservice application detects that a threshold is exceeded. For example, a user may request a trigger into the microservice manager when a microservice application detects that a value has exceeded a triggering threshold.

[0100] In some embodiments, a trigger, when satisfied, may output data for ingestion by a target microservice. In other embodiments, a trigger, when satisfied, may output a binary value indicating that the trigger was satisfied, or may output the name of the field or other contextual information for which the trigger condition was satisfied. Additionally or alternatively, the target microservice may be connected to one or more other microservices, whereby the alert is input to the other microservices. The other microservices may perform responsive actions based on the above techniques, including, but not limited to, deploying additional resources, adjusting system configuration, and / or generating a GUI.

[0101] action In some embodiments, a plugged-in microservice application may expose actions to the microservice manager that may receive as input data, or an identification of a data object, or a location of data that causes the data to be moved into the data cloud.

[0102] In some embodiments, the exposed action may receive as input a request to increase or decrease an existing alert threshold. The input may identify an existing in-application alert threshold and whether the threshold should be increased, decreased, or deleted. Additionally or alternatively, the input may request the microservice application to create a new in-application alert threshold. An in-application alert may cause a user to trigger an alert while logged into the application, or may cause a user to trigger an alert using default or user-selected alert mechanisms available within the microservice application itself, rather than by another application plugged into the microservice manager.

[0103] In some embodiments, a microservice application may generate and provide output based on inputs that identify, locate, or provide historical data and define limits or ranges of desired output. When triggered, an action causes the microservice application to provide, store, or display the output, for example, as a data model or as aggregate data that describes the data model.

[0104] 8. Hardware Overview According to some embodiments, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hardwired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or network processing units (NPUs) that are permanently programmed to perform the techniques, or may include one or more general-purpose hardware processors that are programmed to perform the techniques according to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may even combine custom hardwired logic, ASICs, FPGAs, or NPUs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices, or any other devices that incorporate hardwired and / or program logic to implement the techniques.

[0105] For example, Figure 8 illustrates a computer system according to some embodiments. Computer system 800 includes a bus 802 or other communication mechanism for communicating information, and a hardware processor 804 coupled with bus 802 for processing information. Hardware processor 804 may be, for example, a general-purpose microprocessor.

[0106] Computer system 800 also includes a main memory 806, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 802 for storing information and instructions to be executed by processor 804. Main memory 806 may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 804. Such instructions, when stored in a non-transitory storage medium accessible to processor 804, transform computer system 800 into a special-purpose machine that is customized to perform the operations specified in the instructions.

[0107] Computer system 800 further includes a read-only memory (ROM) 808 or other static storage device coupled to bus 802 for storing static information and instructions for processor 804. A storage device 810, such as a magnetic disk or optical disk, is provided and coupled to bus 802 for storing information and instructions.

[0108] Computer system 800 may be coupled via bus 802 to a display 812, such as a cathode ray tube (CRT) or light-emitting diode (LED) monitor, for displaying information to a computer user. An input device 814, which may include alphanumeric and other keys, is coupled to bus 802 for communicating information and command selections to processor 804. Another type of user input device is a cursor control 816, such as a mouse, trackball, touch screen, or cursor direction keys, for communicating directional information and command selections to processor 804 and for controlling cursor movement on display 812. Input device 814 typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), which allows the device to specify a position in a plane.

[0109] Computer system 800 may implement the techniques described herein using customized hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, causes or programs computer system 800 to be a special-purpose machine. According to some embodiments, the techniques herein are performed by computer system 800 in response to processor 804 executing one or more sequences of one or more instructions contained in main memory 806. Such instructions may be read into main memory 806 from another storage medium, such as storage device 810. Execution of the sequences of instructions contained in main memory 806 causes processor 804 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0110] The term "storage medium" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a particular manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 810. Volatile media include dynamic memory, such as main memory 806. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM, and EPROM, flash EPROM, NVRAM, any other memory chip or cartridge, content addressable memory (CAM), and ternary content addressable memory (TCAM).

[0111] Storage media is distinct from but may be used in conjunction with transmission media. Transmission media involves transferring information between storage media. For example, transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise bus 802. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0112] Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 804 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a network line, such as a telephone line, fiber optic cable, or coaxial cable using a modem. A modem local to computer system 800 may receive the data on the network line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector may receive the data carried in the infrared signal and appropriate circuitry may place the data on bus 802. Bus 802 carries the data to main memory 806, from which processor 804 retrieves and executes the instructions. The instructions received by main memory 806 may optionally be stored on storage device 810 either before or after execution by processor 804.

[0113] Computer system 800 also includes a communication interface 818 coupled to bus 802. The communication interface 818 provides a two-way data communication coupling to a network link 820 that is connected to a local network 822. For example, the communication interface 818 may be an Integrated Services Digital Network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, the communication interface 818 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. A wireless link may also be implemented. In any such implementation, the communication interface 818 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

[0114] Network link 820 typically provides data communication through one or more networks to other data devices. For example, network link 820 may provide a connection through local network 822 to a host computer 824 or to data equipment operated by an Internet Service Provider (ISP) 826. ISP 826 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the "Internet" 828. Local network 822 and Internet 828 both use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 820 and through communication interface 818, which carry the digital data to and from computer system 800, are example forms of transmission media.

[0115] Computer system 800 can send messages and receive data, including program code, through the network(s), network link 820 and communication interface 818. In the Internet example, a server 830 might transmit a requested code for an application program through Internet 828, ISP 826, local network 822 and communication interface 818.

[0116] The received code may be executed by processor 804 as it is received, and / or stored in storage device 810, or other non-volatile storage for later execution.

[0117] 9.Other extensions Embodiments are directed to systems involving one or more devices that include a hardware processor and are configured to perform any of the operations described herein and / or recited in any of the claims below.

[0118] In some embodiments, a non-transitory computer-readable storage medium includes instructions that, when executed by one or more hardware processors, cause the performance of any of the operations described herein and / or recited in any of the claims.

[0119] Any combination of the features and functionality described herein may be used by one or more embodiments. In the foregoing specification, the embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings should therefore be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indication of the scope of the invention, and what is intended by the applicant to be the scope of the invention, is the literal and equivalent range of the set of claims that are obtained from this application in the specific form in which such claims are obtained, including any subsequent amendments.

Claims

1. A program executed by one or more hardware processors, the program comprising: identifying a first set of text tokens in a set of log records associated with an account for accessing the network service; training a machine learning model to identify network attacks based on the set of text tokens; detecting a new log record associated with the account for accessing the network service; and generating an output indicating whether the new log record is associated with a network attack based on a second set of text tokens in the new log record.

2. The method of claim 1 , wherein training the machine learning model to identify network attacks comprises converting the first set of text tokens into numeric values.

3. The program of claim 2 , wherein the number is based at least in part on a first frequency of the text token in an individual log record and an inverse frequency of the text token across multiple log records.

4. 2. The program of claim 1, wherein training the machine learning model to identify network attacks includes generating a score for each respective log record in the set of log records based at least in part on what text tokens are included in the respective log record.

5. The method of claim 4 , wherein generating the score for each respective log record comprises aggregating a set of individual scores assigned to the text tokens included in the respective log record.

6. 2. The program of claim 1 , wherein the machine learning model comprises one or more decision trees, and training the machine learning model comprises splitting training examples from the set of log records based at least in part on scores associated with the set of text tokens.

7. The program of claim 6, further causing the one or more hardware processors to prune the one or more decision trees based at least in part on the scores associated with the set of text tokens.

8. The program described in claim 1, which causes the one or more hardware processors to adjust at least one model hyperparameter to achieve a balance between precision and recall of the machine learning model.

9. The program of claim 1, further causing the set of text tokens to include values ​​identifying a network address, language, browser, and location associated with a login attempt to the account to access the network service.

10. The method of claim 1 , wherein the new log records are generated based on login attempts to the account.

11. The program of claim 1, wherein generating a prediction includes scanning one or more decision trees based on a set of one or more scores associated with the second set of text tokens.

12. 12. The program of claim 11, wherein the score is based at least in part on a first frequency of the second set of tokens in the new log record and a second inverse frequency of the second set of tokens in the set of log records.

13. The program of claim 1, causing the one or more hardware processors to perform one or more actions to counter a network attack detected based on the output.

14. The method of claim 13 , wherein the one or more actions are performed in response to determining that a severity of the detected network attack meets a threshold.

15. 14. The method of claim 13, wherein the one or more actions include at least one of locking a previous user account, sending a one-time password to a user, or enabling two-factor authentication.

16. The program of claim 1 , wherein the output includes a label that classifies the new log record.

17. 2. The method of claim 1, wherein the trained machine learning model includes at least three classification labels based on at least one of a predicted likelihood that a new low record will be associated with the network attack or a predicted severity of the network attack.

18. 2. The program of claim 1, wherein the at least three classification labels include a first label for events having an estimate above a first threshold, a second label for events having estimates above a second threshold and below the first threshold, and a third label for events having estimates below a third threshold.

19. one or more hardware processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more hardware processors, cause the performance of the operations recited in any of claims 1 to 18.

20. A method carried out by one or more hardware processors comprising the operations of any of claims 1 to 18.

Images