Techniques for input classification and response using generative neural networks

Abstract

Disclosed herein is a technique for detecting invalid web traffic. The method includes obtaining an input sequence. The method further includes inputting the input sequence to an encoder model configured to generate an embedding from the input sequence. The method includes generating the embedding as an output of the encoder model. The method includes inputting the embedding to a classifier model configured to predict a vector of probability values ​​corresponding to a plurality of classes using the embedding. A first class of the plurality of classes can be attributed to variable traffic and a second class of the plurality of classes can be attributed to repetitive traffic. The method includes predicting the vector of probability values ​​as an output of the classifier. The method includes determining a machine actor score from the vector of probability values. The method also includes outputting the machine actor score.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit and priority of U.S. Application No. 17 / 492,341, filed on October 1, 2021, and U.S. Application No. 17 / 690,953, filed on March 9, 2022. Each of these applications is hereby incorporated by reference in its entirety for all purposes.

[0002] Technical Field The present disclosure relates to methods and systems for detecting invalid web interactions, and more particularly, to techniques that use neural networks to generate embeddings from input sequences of user interface interaction data, classify the embeddings to determine a machine actor score, and / or authenticate or block web interactions based on the machine actor score.

Background Art

[0003] Background Invalid interactions with online resources, such as those originating from bots or other automated systems, represent a persistent and significant challenge to maintaining control over data and limiting the cost of computing resources associated with responding to requests. Invalid interactions are, or may include, user interface interaction data generated by automated systems configured to emulate valid human users by simulating input from peripherals such as mouse cursor trace data, keystroke data, or voice input data. For example, a bot or other automated scraping system may be configured to repeatedly access publicly accessible databases, such as those on social media platforms, to generate and / or access and store non-authentic interaction statistics. In another example, an automated system “bot” may be configured to point to a social networking platform user base of all users violating the platform’s terms of service. In some cases, social media platforms may even fool users when presenting pop-up tests to verify that users are not scraping data from the platform. For example, automated systems are often configured to detect and bypass such tests by simulating pop-ups and human interactions to scam the tests. [Overview of the Initiative]

[0004] overview In various embodiments, techniques are provided for analyzing interaction data and classifying the data according to the characteristics of the interaction. In various further embodiments, the interaction data is collected from web interactions, including input sessions, and takes the form of, for example, mouse cursor movement and / or click sequence data. The systems and methods described herein include the classification of interaction data based on a machine actor score related to predicting interaction effectiveness (e.g., classification as relating to a human user or a bot). The machine actor score may indicate the type and / or classification of the interaction determined by a machine learning system and can be compared against one or more thresholds of tolerance to determine downstream actions to be taken with respect to the interaction. For example, a predicted malicious interaction may be isolated and rejected from further interaction with the web-based item. Other entities that are not predicted to be significantly malicious but are still suspicious may receive lower response actions, such as interaction isolation. Finally, a predicted tolerable interaction may not be prevented by a response action.

[0005] An invalid interaction is input originating from an automated system (e.g., a robot or “bot”) or other malicious input. Invalid interactions cover a range of interaction types, including malicious automated interactions driven by bot activity, malicious interactions driven by real users (non-automated) such as incentivized browsing, and non-malicious automated activities such as web crawling. In contrast, a valid interaction describes an interaction generated by a human user performing authorized or otherwise authentic activity and input. Some techniques disclosed herein involve obtaining a cursor trace dataset, including the identification of the sequential position of a cursor, in order to generate embeddings of the cursor trace dataset using a generative machine learning model (e.g., an encoder). Generative machine learning models may be used as part of techniques to distinguish human interactions from automated “bot” interactions. However, it should be understood that the technologies, methods, and systems may be applied to distinguish different classes of interactions of various origins, including interactions with invalid humans (e.g., “click farming”). Where used herein, the term “interaction” is used interchangeably and may refer to “traffic.”

[0006] Attempts to address these issues involve using machine learning models and datasets trained with supervised learning to detect and analyze invalid input from bots. Machine learning models trained with supervised learning and training datasets do not require direct manual rule-based refinement and respond more efficiently to adaptations by bots. Therefore, machine learning models are well-suited to addressing the "arms race" in detecting bot behavior. However, machine learning models trained with supervised learning and training datasets may require expensive and cumbersome data labeling for training and operation. Machine learning models trained with supervised learning and training datasets cannot efficiently refine and implement the detection model if label generation is difficult or labels are unavailable. For example, machine learning models trained with supervised learning and training datasets typically cannot learn to detect input from relatively new bot configurations with the required accuracy unless training labels exist to train the supervised model, and the bot detector is at a disadvantage when bots are refined with newly developed techniques and input patterns.

[0007] To address invalid traffic issues while eliminating resource-intensive requirements for labeled training data, cursor trace data for analysis is grouped and embedded according to one or more generative machine learning models, such as generative neural networks (e.g., variational autoencoders consisting of recurrent neural networks or transformer models), which can generate unique input embeddings (compact vector representations of the input). The resulting embedded inputs can be analyzed using approaches such as embedding clustering subsystems (e.g., K-means) for clustering the embeddings and cluster membership classifiers (which may include standard classifiers such as logistic regression). The generative machine learning models can process input sequence data (e.g., identifying the presence, time, duration, and / or location of cursor movements, clicks, etc.), generate unique embeddings of the input, classify the embeddings via a cluster membership classifier (trained on clustering the embeddings learned by the embedding clustering subsystem), and predict a machine actor score that measures the likelihood that the input sequence is from a bot (e.g., as opposed to a human user).

[0008] In a rule-based approach, the system can use static rules to predict that a user is a bot. For example, if the input cursor speed exceeds a threshold, the bot might evaluate the system's response to reverse-engineer the rules and then modify its interaction (e.g., by slowing down) to avoid detection. Adapting supervised detection models typically requires effortful reverse engineering to capture the bot's characteristic behavior.

[0009] For machine learning models trained using supervised learning and training datasets, the requirement to train the machine learning model using labeled training data is a resource-intensive process. For example, training a machine learning model using supervised learning and training datasets to detect input from a new bot may require generating and inputting a highly representative training dataset into the model. Regularly generating a sufficiently representative training dataset of this form with human annotations for each training sample is either impractical or too noisy, especially if little is known about the bot. A machine learning model trained with supervised learning may continue to fool the model unless it has already been trained to detect input from the new bot. By implementing an unsupervised learning model instead of a supervised learning model, the detection system can learn to identify new types of invalid traffic in real time with virtually fewer resources.

[0010] In contrast, the techniques disclosed herein utilize machine learning models trained with unsupervised learning. The trained models may be configured to be trained online or at frequent intervals with more recent data, thereby becoming robust to data drift from the evolving bot ecosystem and corresponding evasion attempts. Unsupervised learning and machine learning models trained on training datasets may include an autoencoder (e.g., a variational autoencoder including a recurrent neural network or transformer) configured to generate embeddings (compact vector representations of the inputs), an embedding clustering subsystem (e.g., K-means) for clustering the embeddings, and a cluster membership classifier (e.g., logistic regression) trained with the clustering learned by the embedding clustering subsystem and configured to predict cluster membership likelihoods that can be used with the embeddings to derive a machine actor score that measures the likelihood that the original input corresponds to a bot. As described herein, an unsupervised machine learning model can be a machine learning mode such as a generative model, trained using unsupervised machine learning techniques.

[0011] In relation to obtaining raw cursor trace data for downstream analysis, using unsupervised machine learning models offers advantages over traditional approaches (e.g., rule-based and supervised learning approaches), including, but not limited to, improved accuracy, robustness, and responsiveness of cursor trace data grouping by using embeddings of cursor trace data generated by generative machine learning models. Rather than utilizing raw cursor trace data collected from applications or attempting to extract metrics directly from raw cursor trace data, using unsupervised generative machine learning models allows for more efficient grouping and embedding of cursor trace data relevant to the input being analyzed. Because generative machine learning models are unsupervised, they can learn from modern bot interactions without requiring the generation of labeled training datasets as is required for supervised machine learning models.

[0012] The generative machine learning model may be an encoder trained in conjunction with a decoder as part of a unified autoencoder model configured to receive embeddings in cursor data and generate a predicted reconstruction of the cursor data. The encoder and decoder may be trained using a loss function that penalizes the error between the reconstruction of the input generated by the decoder and the input initially received and processed by the encoder. This training approach facilitates the generation of highly useful and efficient representations of the cursor data.

[0013] The above, along with other features and embodiments, will become more apparent by referring to the following specification, claims, and accompanying drawings. [Brief explanation of the drawing]

[0014] [Figure 1] This is a diagram of a bot interaction detection system incorporating a data processor / formatter, an autoencoder subsystem, and a machine actor detection subsystem, according to one embodiment described herein. [Figure 2] This is a diagram of a training system configured for unsupervised training of an autoencoder in a bot traffic detection system, according to one embodiment described herein. [Figure 3] This is a diagram of a training system for an embedded clustering subsystem and a cluster membership classifier (both components of the machine actor detection subsystem) according to one embodiment described herein. [Figure 4] This is a schematic diagram of an inference-stage method for detecting bot traffic from mouse cursor trace data, according to one embodiment described herein. [Figure 5] This is a block flow diagram of a method 500 for the inference stage of detecting bot traffic from mouse cursor trace data, according to one embodiment described herein. [Figure 6] This figure shows an exemplary interface for displaying mouse trace data analysis and data analysis reports. [Figure 7] This is a diagram of a distributed system for implementing a certain embodiment described herein. [Figure 8] This is a block diagram of a cloud-based system environment according to one embodiment described herein. [Modes for carrying out the invention]

[0015] Detailed explanation In the following description, certain details are included for illustrative purposes to fully understand a particular embodiment. However, it will be apparent that various embodiments can be carried out without these specific details. The figures and descriptions are not intended to be restrictive. The term “exemplary” is used herein to mean “serving as an example, illustration, or illustration.” Embodiments or designs described herein as “exemplary” or “example” are not necessarily construed as being preferable or advantageous to other embodiments or designs.

[0016] A trainable browser interaction processing system is provided that can store, process, and / or embed cursor trace datasets into an embedding representation using a generative machine learning model. The resulting embedding representation can be used downstream to validate interaction data. Interaction data may be generated as part of an active application, such as a web browser, that detects browser input. The trainable interaction embedding, analysis, and processing system may be provided as a standalone device, a cloud-accessible service, etc. In each of these forms, the system can access and / or receive data generated from interactions with a user interface, use the data to determine sequences that characterize sequential interactions, generate embeddings of sequences using a trained encoder model, classify the embeddings using a suitable cluster membership classifier (e.g., logistic regression) trained on clustering of embeddings learned by an embedding clustering subsystem (e.g., K-means), determine a machine actor score indicating the likelihood that the input is a bot using the resulting cluster membership probabilities, determine whether an access-blocking condition is met based on the machine actor score (e.g., comparing the machine actor score to a threshold score), and output a result indicating whether it was determined that the access-blocking condition was met. In some cases, outputting results may include outputting results to a web server or specific code on a web server so that the web server or specific code can determine how to respond to content requests received from a device associated with a user (for example, returning the requested content if it is determined that the access blocking conditions are not met, and not returning the requested content if it is determined that the access blocking conditions are met), and / or so that the web server or specific code can push updated web content lacking specific content if the access blocking conditions are met.

[0017] To this end, the techniques, methods, and systems presented herein implement unsupervised learning to overcome the limitations of obtaining comprehensive cursor trace data for analysis. One model system may be an autoencoder, which may be an unsupervised neural network model. An autoencoder may learn latent representations of input data called embeddings, which may be reduced-dimensional versions of the input data that retain only the most prominent features. An autoencoder model may be trained using unsupervised learning, as described by the methods and embodiments presented herein. Unsupervised learning means that the model does not use labeled data to guide the model training. For example, an unsupervised training framework does not provide class labels indicating a specific interaction type (e.g., bot or human) during training, as is done in supervised learning.

[0018] Training an encoder to embed a cursor trace dataset An autoencoder may include two components: an encoder and a decoder. The encoder may include a learnable neural network that maps the input to embeddings (compressed potential reduced-dimensional vector representations) in the embedding space (a vector space containing all possible embeddings). The distance in the embedding space between user input embeddings is particularly important because proximity in the embedding space can indicate similarity between embeddings, and thus similarity between the original user inputs. In various embodiments, one or more calculated sets of distances between embeddings may be used by an embedding clustering subsystem to group similar embeddings together, which serves as the basis for the final cluster membership classification and the generation of machine actor scores, as described later.

[0019] The decoder part of an autoencoder can include a neural network that can be trained to reconstruct the uncompressed / unembedded input from the embedding output by the encoder. Different types of neural network models are available for constructing the encoder and decoder. The selection of the type of neural network model may depend on the intended use of the model. For example, when processing image data, a feedforward neural network (FNN) or a convolutional neural network (CNN) can be selected and used. When processing sequence data, among other types of neural network models, a recurrent neural network (RNN) or a transformer can be selected and used. To encode and decode multimodal data, FNN, CNN, and RNN can be combined and used.

[0020] An autoencoder may be trained "end-to-end" where the encoder and decoder are jointly trained. Joint training of the encoder and decoder may involve defining a reconstruction error, or "loss", that serves as the loss function for the autoencoder model. The reconstruction error compares the input with the decoder's reconstruction of the input from the embedding and penalizes the difference (e.g., via squared loss). During training, the model parameters of both the encoder and decoder can be optimized using stochastic gradient descent (or other gradient-based optimization algorithms such as Adam). The decoder plays an important role during training in formulating the autoencoder's loss function from the perspective of reconstruction error but does not play a role during inference, at which time the embedding output from the encoder functions as an input to the cluster membership classifier described below.

[0021] A variational autoencoder (VAE) is an implementation of a specific architecture of an autoencoder model. In addition to the standard autoencoder formulation described above, a VAE is an autoencoder that learns to output an embedding by sampling the embedding from a Gaussian probability distribution with parameters learned and output by the encoder. Thus, the parameters of the sampled Gaussian distribution are a learned function of the input data. Since the embedding is sampled from a probability distribution, the VAE produces a non-deterministic output for sequence inputs. The loss formulation of the VAE is different from that of the standard autoencoder in that, in addition to the reconstruction error term described above, the VAE includes an additional (additive) term in the loss function that encourages the learned Gaussian distribution over the embedding space to resemble a "standard" Gaussian distribution with zero mean and unit variance.

[0022] A recurrent neural network may include, but is not limited to, gradient recurrent units (GRUs) or long short-term memory (LSTM) units. A long short-term memory model includes at least one long short-term memory cell, which can include multiple interaction layers that can hold some information for a long or short period of time during operation and protect the gradients inside the cell from harmful changes during training. For example, an LSTM cell can include three gates (input, forget, output) and a cell unit. The gates are configured to use sigmoid activation, and the input and the cell state can be transformed by a hyperbolic tangent, i.e., a tanh function.

[0023] Gradient recurrent unit cells also regulate the flow of information within the unit, without having separate memory cells. Activation of a GRU cell at time t can involve linear interpolation between previous and candidate activations, and the update gate determines how much the unit updates the activation. This procedure, which determines a linear sum based on the existing state and the newly calculated state, is similar to the procedure in an LSTM cell. However, GRU cells lack a mechanism to control the degree to which the state is exposed, exposing the entire state each time. While the above description focused on LSTM and GRU cells, the encoder and decoder components of an autoencoder are not limited to such systems. For example, an RNN architecture may be fully recurrent, bidirectional, recurrent, etc., but is not limited to these.

[0024] Another example of a model that can be used to build the encoder and decoder of an autoencoder may be a transformer model. A transformer is a deep learning model that can be used for processing and modeling sequential data for various tasks, such as machine translation or modeling mouse cursor trace data for invalid interaction detection. A transformer typically consists of an encoder and a decoder. The encoder of a transformer converts data into an intermediate output sequence called an encoding, and its decoder decodes it into an output sequence. The encoder and decoder each incorporate multiple encoder and decoder layers. The output of the encoder contains latent information from the input sequence for the decoder to generate the output sequence.

[0025] Each encoding layer of the transform encoder takes a sequence of tokens as input, where each token contains a small, meaningful unit of the input. In the context of bot interaction detection, the input sequence could be a vector representation of input data, such as the position of the mouse cursor at a given time, or it could correspond to a specific interaction state (describing one or more state conditions surrounding an interaction, such as "clicked" or "hovering"). Each encoding layer takes a list of vector representations of the input sequence and passes it to a self-attention layer, which then passes it to a feedforward neural network. The self-attention layer attempts to isolate the most prominent parts of the input sequence. The output of the feedforward neural network is passed to a subsequent encoder layer, and the output of the encoder's final encoding layer can represent the encoder's final output encoding. These encodings are token-level encodings, as they are output by the encoder for each token, and represent the user's input sequence as a whole, in contrast to sequence-level embeddings, which, by definition, involve the output of an RNN-based encoder. To generate such sequence-level embeddings from a transform encoder, it is necessary to aggregate token-level encodings, which can be achieved, for example, by summing or averaging all token-level encodings to obtain the sequence-level embeddings. For these sequence-level embeddings to be a learning function of the input, they must be characterized by a loss function, which can be achieved, for example, by passing them as input to a decoder and concatenating them with the input of a standard decoder. Similar to the formulation of an RNN, the transform decoder is also trained to reconstruct the original input sequence from its inputs.

[0026] For an autoencoder to process user session data, which includes various user input sequences (such as mouse cursor movements and clicks), the input data must first be formatted, normalized, and otherwise transformed. A specific embodiment of this data preprocessing is described when the autoencoder is included in an RNN encoder and decoder; however, other (similar) schemes are possible, and modifications may be necessary when using other models (such as transformer-based autoencoders).

[0027] Data preprocessing first involves extracting only a specific subset of sequential user input signals (e.g., mouse cursor movements and click sequences) from past web interactions, including user session data, among many other signals (sometimes of a non-sequential type). The elements of these movement and click sequences are tuples (x, y, c) of the horizontal (x) and vertical (y) mouse position coordinates on the web page when the web browser triggered the event, and a timestamp offset c when the event was triggered for the start of the user session. These sequences are first sorted by timestamp c, and duplicate (or other invalid) events are removed. Next, the sequences are transformed into sequences of positional and time deltas (dx, dy, dc) by taking the difference between adjacent elements in the original sequence.

[0028] It is assumed that all mouse event types (movement, click, etc.) are mutually exclusive at a given position (x, y, c). A mouse state can be defined as the state of the mouse when a given event occurs, and the number of possible states is the number of event types being tracked, and there is also a "lift" state corresponding to a large difference in timestamps between consecutive events, which can be controlled, for example, by a numerical threshold. In this method, sequences of different modalities (movement and click, etc.) are then matched into a unified sequence of vectors, where the first three elements of each vector contain the position and time delta (dx, dy, dc) of the current event, and the later elements contain a one-hot vector specifying the mouse state. More specifically, all positions in the one-hot portion of the vector have a value of 0 (inactive), except for the position representing the current mouse state, which has a value of 1 (active). This vector representation facilitates processing by downstream machine learning models, particularly by autoencoders trained to generate embeddings for each user session input.

[0029] Once the data is cast to the format described above, further processing may include normalization, such as column-level scaling of position and time values, and truncation of sequences by a uniform length limit, to improve the efficiency of the autoencoder. In some embodiments, for some or all of the downstream training stages, stratified samples of user session input are used based on whether a given session has been flagged by another fraud detection methodology, rather than using the methods described herein. Using stratified samples can provide the training process with a better distribution of fraudulent and non-fraudulent interactions, whereas with unstratified samples, the training data is heavily biased towards non-fraudulent interactions.

[0030] Generating an embedded representation of a cursor trace dataset Once the encoder, or the generative machine learning model used, is trained using unsupervised learning as described above, the encoder is used to generate an embedding representation of cursor trace data that will be analyzed in downstream processes to determine whether the interaction is human or non-human. That is, the trained encoder takes formatted / processed cursor trace data to be embedded as input. The encoder uses the input formatted / processed cursor trace data to generate an embedding representation that preserves the non-fluid, non-human cursor patterns present in the original raw cursor trace data and the corresponding formatted / processed data. It is important to preserve the present non-fluid, non-human cursor patterns so that downstream analysis systems can properly classify non-human embeddings when determining the classification of the cursor trace data.

[0031] The nature of cursor trace data can be represented by an embedded representation of the cursor trace data generated by the encoder, for example, by identifying the sequential position of the cursor. Binary input instructions, such as button presses, may also be stored in the embedded representation. The binary input may be included in the embedded representation along with the corresponding sequential position of the cursor at the point when the binary input changed (i.e., when the button was pressed or released). By processing clusters of data stored in the embedded representation, the instructions / nature in the embedding can be identified.

[0032] Analysis of embeddings in cursor trace datasets A classifier may be trained to predict machine actor scores from clustered embeddings. For example, a classifier may be trained to predict machine actor scores based on embeddings generated by an autoencoder after the embeddings have been clustered according to an embedding clustering subsystem. The embedding clustering subsystem may be a set of steps for clustering the embeddings output by the encoder part of an autoencoder, and may be separate from the classifier (for example, the clustering algorithm may be a known clustering algorithm such as K-means or DBSCAN). During training, the embeddings are clustered by the embedding clustering subsystem and a cluster label is assigned to each embedding. A classifier can then be trained to predict cluster membership using a training dataset consisting of embeddings along with the cluster labels learned by the embedding clustering subsystem. The classifier may use a machine learning model to generate predicted cluster labels, and the machine learning model may include, for example, a probabilistic classification model (such as logistic regression). In other embodiments, the clustering and classifiers may include a single model, for example, in the case of a mixture model, which learns a multimodal distribution of input embeddings whose modes can be considered clusters, and each embedding can be assigned a cluster label by associating it with the mode most likely to have generated them.

[0033] The classifier can output a probability vector for each input, where each element of the vector is the probability that the input belongs to the corresponding cluster. During inference, the embedded clustering subsystem may no longer be used. New inputs may be passed to an encoder to create an embedding, which may be passed to a classifier that can make cluster membership predictions in the form of probability vectors. The machine actor score for the input may then be computed from the probability vector, the details of which are described later. The fact that clustering may not occur during inference is true in embodiments using separate clustering and classifiers, as well as embodiments using a model that combines clustering and classification functions, such as a mixed model. In the latter case, the mixed model can remain in use during inference, but only its predictive function that generates probability vectors for the input is used.

[0034] As described above, the prediction may be based on a vector of probability values ​​output by the cluster membership classifier subsystem, which includes representing the confidence (or probability) that a particular encoded input corresponds to each of the set of clusters (e.g., a confidence of 25% for the user type associated with cluster A, a confidence of 40% for the user type associated with cluster B, and a confidence of 35% for the user type associated with cluster C). In this embodiment, the encoded input may be assigned to any cluster associated with the highest assignment confidence. The determined confidences can be further generated by first filtering out confidences associated with the "largest" cluster (one that has been assigned more encoded input datasets than other clusters, or one whose cluster variance is greater than other clusters). In other embodiments, a measure of density can be calculated for each cluster, and confidences for clusters whose density measure is below a predetermined threshold (which may itself be cluster-dependent, e.g., the average density of the clusters) can be excluded. The machine actor score can then be (or at least partially) generated based on the remaining confidences. For example, the machine actor score can be defined to be the highest confidence level among the remaining confidence levels.

[0035] In various embodiments, clusters within an overall clustering learned by a clustering instrument describe similarly patterned groupings of inputs. For example, a cluster may describe a grouping of mouse cursor trace embeddings indicating that the original mouse cursor trace inputs exhibit similar patterns. Such groupings of mouse cursor traces may be analyzed to detect suspicious behavior in order to help determine whether an input is invalid. For example, mouse cursor traces with similar patterns may be analyzed to determine that the mouse cursor trace exhibits rigid, repetitive behavior, indicating suspicious input behavior, and should be determined to be an invalid interaction.

[0036] In some cases (for example, instead of using clustering embeddings), a signature can be generated for each of one or more clusters. The signature may include one or more properties of the cursor trace represented by the embeddings within the cluster. One or more properties may include the mean or median property for the entire cluster. In some cases, generating a signature involves identifying a subset of properties whose standard deviation, variance, or range across all members of the cluster is below a predefined threshold, and then defining a signature to represent the values ​​of that subset of properties. Signatures can be used by humans to understand what types of cursor trace activity are represented by the clusters. Furthermore, or alternatively, signatures can be used to facilitate assigning a given cursor trace data element to a cluster (for example, by identifying the cluster signature that most closely matches the embedding of the cursor trace data element).

[0037] Embedding clustering and classification may be achieved by separate embedding clustering subsystems and classifier algorithms. For example, an invalid interaction detection system can use separate clustering (e.g., a system including an embedding clustering subsystem) and classifiers to perform the embodiments described herein. The clustering may be stored, implemented, and utilized in a different system from the classifier, and the only interaction between entities may be the sending and receiving of cluster labels assigned to the input embeddings. In various alternative embodiments, the clustering and classifiers may be part of a single model, such as a mixed model.

[0038] A mixture model is a probabilistic model that employs mixture distributions to represent the existence of subgroups within an overall population. A mixture distribution is a probability distribution composed of combinations of subdistributions called components. For example, a mixture distribution may be a linear combination of subdistributions, where each term in the sum contains the product of the prior probability of the component (coefficient) and the posterior probability of the component given the input data. Mixture models come in various types depending on the distribution types chosen for the subdistributions, which can be (but not limited to) Gaussian, Dirichlet, or categorical distributions. Rather than explicitly assigning the input data to subdistributions, a mixture model allows for the construction of a probability vector where the i-th element is the probability that the input "originates" from the i-th component. More specifically, the posterior probabilities of each component given the input data become elements of this probability vector, and the maximum value of the probability vector is the predicted value of the most likely component given the input data. From this, it is easy to see how a mixture model can simultaneously function as both a clustering machine and a cluster membership classifier. When the components of the overall distribution are interpreted as clusters, the calculation of the probability vector described above can be interpreted as a cluster membership classification. Thus, a mixture model condenses the roles of an embedding clusterer and an embedding cluster membership classifier into a single model. In an exemplary embodiment, during training, a Gaussian mixture model (GMM) fits input embeddings with a Gaussian mixture distribution contained in a combination of individual Gaussian distributions. During inference, the GMM constructs a probability vector for new input embeddings, which are contained in the posterior probabilities of each given Gaussian subdistribution.

[0039] Figure 1 is a diagram of an exemplary bot traffic detection system 100 (also called the “Detection System”) according to an embodiment described herein. The bot traffic detection system 100 utilizes a data processor / formatter 104, an autoencoder subsystem 107, and a machine actor detection subsystem 110. The machine actor detection subsystem 110 may internally include several subsystems, including a model training subsystem and an inference subsystem. The training subsystem 111 may be a subsystem for training model components. The training subsystem may be used separately from other subsystems that perform inference-related operations, such as an inference subsystem. The inference subsystem may be a subsystem for performing inference-related operations, such as generating machine actor scores / decisions from input sequence embeddings. The bot interaction detection system 100 depicted in Figure 1 will be recognized as a general “separate models” paradigm, where the embedding clustering subsystem and the cluster membership classifier subsystem 114 are separate entities. In various embodiments, the cluster membership classifier subsystem 114 may also be part of the inference subsystem as a means for generating cluster membership probability vectors for input sequence embeddings from which individual machine actor scores can be derived. In various embodiments, the embedded clustering subsystem and the cluster membership classifier subsystem 114 may be implemented as a single model / component within the training subsystem ("single model" paradigm), for example, in embodiments where a mixed model is employed for clustering and cluster membership classification. The detection system 100 is configured to receive, format, and otherwise process user session input using a data processor / formatter 104.

[0040] Input data may be generated by an entity 101, such as a human or bot, which simulates data generated by the data input component of the interaction system 102. For example, entity 101 can input interactions into a user web interaction system such as the interaction system 102. Interaction data generated by the interaction system 102 is transferred to the data processor / formatter 104 as interaction data 103 by recording mouse cursor data from the interactions performed by entity 101, and can be combined with mouse state data for interacting with the browser or console environment. The state data may include the specification of different types of mutually exclusive interaction states that occur in real time, such as "click," "release click," "move," and "hover." Data preparation may further include cleaning and dedensification, for example, by ordering the interaction data 103 according to timestamps and removing duplicate events. In various embodiments, the interaction data 103 is then transferred to a data processor / formatter 104 in the invalid interaction detection system 100. The data processor / formatter 104 can parse the input data 103 and process / format the interaction data 103 into another state usable by other subsystems, such as the autoencoder subsystem 107. The resulting processed / formatted sequence data 105 may then be sent to the autoencoder system 107, which may recognize the format of the processed / formatted data sequence data 105 and utilize it according to the embodiments described herein. In various embodiments, the data processor / formatter 104 is configured to receive data prepared by related systems, such as a data aggregation system, where the detection system 100 is used as part of a security system.

[0041] In some embodiments, the input data includes historical interaction data collected from interactions over a past and / or present period. For example, interaction data may be collected over a period and stored in a data repository. Interaction data may be collected over a period and transmitted from the repository to another system as historical interaction data. For example, instead of ingesting and transferring interaction data in real time, interaction data may be ingested in real time and collected in the repository. At a later point in time, all collected interaction data may be exposed in a newly collected dataset that includes historical interaction data.

[0042] An invalid interaction detection system, or “detection system” 100, can perform or cause to perform one or more actions using interaction data 103, for example, by using a data processor / formatter 104, an autoencoder subsystem 107, or a machine actor detection subsystem 110, a cluster membership classifier subsystem 114, a machine actor score generator 116, and a thresholding subsystem 118. Each of these subsystems is a component of the machine actor detection subsystem 110. The cluster membership classifier subsystem 114 may also be part of a training system, in a manner that it is trained on cluster labels to make cluster membership predictions during the inference phase. In this way, the detection system 100 can process the interaction data 103 using the machine actor detection subsystem 110 and output interaction decisions and interaction authorizations, for example, by an output component 120. For example, the output component 120 may be a virtual network interface card or a physical router device that can output data to the network 160, for example, to an associated bot interaction detection system or interaction control system, or may include such a device.

[0043] In various embodiments, the detection system 100 may be further configured to forward requests indicated by interaction data 103 for human interactions and block requests for bot interactions. Furthermore, the output component 120 may be communicatively coupled to one or more display devices to enable the presentation of visualizations of the outputs of one or more subsystems of the detection system 100 on the display screen of a display device. For example, a reduced dimensional projection of the classifier output (e.g., a cluster plot) can be visually presented to the user, and a comparison of the outputs of bot interaction detection using alternative modalities as a validation technique can be similarly displayed.

[0044] In various embodiments, the detection system 100 is a specialized computing system that can be used to process large amounts of data by using a large number of computer processing cycles for normal operation. The number of devices depicted in Figure 1 is provided for illustrative purposes only. Different numbers of devices may be used. For example, although each device, server, and system in Figure 1 is shown as a single device, multiple devices may be used instead to facilitate the operation of any entity shown therein.

[0045] In various embodiments, the processing performed by the detection system 100 is implemented by a pipeline of components or subsystems, including a data processor / formatter 104, an autoencoder subsystem 107, and a machine actor detection subsystem 110, and may include an output component 120. The subsystems described above may be implemented in software (e.g., using code, programs, or instructions executable by one or more processors or cores), hardware, or a combination of hardware and software. In some implementations, multiple subsystems described herein can be combined into a single subsystem. Furthermore, or alternatively, in some implementations, the functions described herein performed by a particular subsystem may be implemented by multiple separate subsystems.

[0046] The data processor / formatter 104 includes hardware and software configured to receive interaction data 103. In some cases, the data processor / formatter 104 may be part of the detection system 100. In some other cases, the data processor / formatter 104 may be separate from the detection system 100 and communicatively coupled to the detection system 100. The processor / formatter 104 may include, for example, data input hardware and software communicatively coupled to a data reception / processing / formatting / conversion system. In an exemplary example, the processor / formatter 104 may include an application programming interface (API) that communicates with a sessionizer system or other subsystem configured to interface with a computer system over a network. In another exemplary example, the processor / formatter 104 may include a personal computer configured to receive a set of interaction data 103 for processing by a subsystem of the detection system 100.

[0047] Interaction data 103 will be parsed by a data processor / formatter 104 to generate an input sequence of processed / formatted sequence data 105, which can be sent to an autoencoder subsystem 107 and associated with a separate user session. The input sequence may, but is not limited to, a finite-length vector of mouse cursor position data and interaction state data for a single session. To define a session, start and end events may be recorded or identified for the session so that the input sequence is bounded before it is stored and / or sent to a subsequent subsystem of the detection system 100. The input sequence may be matched with input sequences from content interactions based on separate user sessions, which may be associated with the presentation of content on a user device (e.g., a tablet or display). A session may be further divided into periods during which interactions occur, and the input sequence may be grouped into these periods to form a bounded time sequence of input data. In an exemplary example, a distinct session may correspond to the period during which a web page is loaded and displayed on a display, during which mouse cursor event data may be collected.

[0048] Once the processed / formatted sequence data 105 input sequence is generated, the autoencoder subsystem 107 is configured to access or receive the input sequence. For example, the input sequence may be received from a data storage utility communicating with the detection system 100 to generate output embeddings. The output embeddings may be an embedded version of the cursor trace data of the input sequence. For example, the embeddings may be a low-dimensional representation of the input cursor trace. In some examples, the embedding representation is a fixed-length multidimensional vector representing one or more latent dimensions of the sequential position of the cursor. The fixed-length multidimensional vector represents one or more latent dimensions, as will be explained in more detail with reference to Figure 4. As part of its processing, the autoencoder subsystem 107 uses the encoder model 108 to generate output embeddings.

[0049] In various embodiments, the autoencoder system employs a variational autoencoder (VAE) architecture and learns output embeddings by sampling the output embeddings from a Gaussian probability distribution output by the encoder. The autoencoder subsystem 107 may be trained using historical interaction data. For example, the dataset may be provided as part of the interaction data 103, as processed / formatted sequence data 105, or accessed by the detection system 100 via a database communicating with the detection system 100, and the dataset includes historical interaction data of human and bot interactions. The interaction data may include data already classified by a detection system employing alternative modalities. In this way, the training data may be stratified to address biases introduced during training by relatively infrequent bot interaction requests that may occur in typical interaction patterns. The training of the model of the autoencoder subsystem 107 is described in detail with reference to Figure 2.

[0050] In various embodiments, interaction classification is performed by a machine actor detection subsystem 110. The machine actor detection subsystem 110 accesses or receives output embeddings generated by the autoencoder subsystem 107. The embeddings received by the machine actor detection subsystem 110 may include fixed-length vectors of multidimensional encoder outputs. To this end, in various embodiments, the machine actor detection subsystem 110 may include entities 114-118 as depicted in Figure 1. The machine actor detection subsystem 110 may include a cluster membership classifier subsystem 114. The cluster membership classifier subsystem 114 accesses data received by the machine actor detection subsystem 110, such as output embeddings generated by the encoder model 108, and may implement a classifier that predicts a vector of probability values ​​corresponding to the number of clusters determined using a system such as an embedding clustering subsystem 340 as shown in Figure 3. For example, the cluster membership classifier subsystem 114 may be trained to predict the cluster membership to be assigned to the embeddings. Cluster membership can correspond to groupings of similar mouse cursor trace patterns known or learned by the model. For example, cluster membership may be assigned to groups of embeddings generated from input patterns, and embeddings grouped into cluster membership have some degree of similarity in properties, such as shape, distribution, probability, vector, or any other applicable features. Thus, the cluster membership classifier subsystem 114 can be trained to predict the cluster membership to be assigned to an incoming embedding.

[0051] The machine actor detection subsystem 110 may further include a machine actor score generator 116. The machine actor score generator 116 may be an entity that uses predictions formed by the cluster membership classifier subsystem 114 to quantify a score associated with the suspicious level of the embedding received by the machine actor detection subsystem 110. For example, the machine actor score generator 116 may calculate the highest probability value obtained by subtracting the value corresponding to the "largest" cluster found by the embedding clustering subsystem from the probability values ​​output by the cluster membership classifier subsystem 114, and output this as the machine actor score. The machine actor detection subsystem 110 may further include a thresholding subsystem 118. The thresholding subsystem 118 may be a system or entity that stores and applies thresholds to the generated machine actor score, such as one generated by the machine actor score generator 116. In various embodiments, the thresholding subsystem 118 may generate a decision on whether the embedding received by the machine actor detection subsystem 110 can be classified as a valid or invalid input based on one or more specified thresholds.

[0052] As shown in Figure 1, in various embodiments, the unsupervised training system 150 trains an encoder model 108. For example, the encoder model 108 may be or include a recurrent neural network or a transformer model. The unsupervised training system 150 may access and / or receive one or more training datasets and may train the encoder model 108 concurrently with a decoder model that learns to reconstruct the input sequence from the output embeddings. When trained as part of an autoencoder framework, the autoencoder loss function takes the form of a reconstruction error that penalizes the difference between the input sequence and the decoder reconstruction of the input sequence.

[0053] In various embodiments, additional processes can be used to train the cluster membership classifier subsystem 114, reduce bias, and balance classification techniques. In addition to training the cluster membership classifier subsystem 114 using training, the training system 152 may use these additional processes, such as stratification, to further refine the cluster membership classifier subsystem 114 for operation in the detection system 100. Training data for both the autoencoder subsystem 107 and the cluster membership classifier subsystem 114 may be drawn from stratified samples of impressions based on whether a given input has been flagged as a bot interaction by other detection techniques, possibly based on other modalities. Using stratified samples improves the training process by providing a distribution of malicious and non-malicious interactions that reduces bias towards valid inputs. Bias in this sense could result in the classifier predicting false negatives at the classification stage. In this way, the training data can be balanced between valid and invalid interactions. This stratification process may provide the above advantages when used to supplement the main training methods described herein.

[0054] In various embodiments, models trained according to the methods described herein may be retrained to improve the detection of valid and invalid inputs. For example, retraining an autoencoder may enable improved embedding generation by capturing more prominent features of the input sequence. In another example, retraining an embedding clustering subsystem may enable improved clustering of input embeddings by picking up finer-grained embedding quality. In yet another example, as illustrated with reference to Figures 3-4, retraining a cluster membership classifier can improve cluster membership classification by more accurately predicting the classification of clustered embeddings using additional criteria or subtle features of the clustered embeddings.

[0055] This retraining can improve the performance of various components, including the thresholding subsystem 118, by facilitating the retraining and dynamic adjustment of the learned model parameters of the autoencoder subsystem 107 and the cluster membership classifier subsystem 114. Such retraining can improve the robustness of the classifier in the "arms race" between the bot interaction system and the detection system 100. For example, the autoencoder subsystem 107 and the cluster membership classifier subsystem 114 can be retrained periodically by updating the training data with a new input sequence. In this way, adaptations by the bot or other sources of invalid interaction access attempts are learned by the cluster membership classifier subsystem 114.

[0056] As described above, the various subsystems of the detection system 100 work together to provide functionality that enables the detection system 100 to receive interaction data 103 and determine whether the interaction data 103 represents a human interaction or a bot interaction. The various subsystems described above may be implemented using a single computer system or using multiple computer systems working together. For example, in the case of a device implementing the detection system 100, the subsystems of the detection system 100 described above may be fully implemented on the device with which the user interacts. In some other implementations, some components or subsystems of the detection system 100 may be implemented on the device with which the user interacts, while other components may be implemented remotely from the device, possibly on some other computing device, platform, or server.

[0057] Figure 2 shows an unsupervised training system 150 configured for unsupervised training of the autoencoder subsystem 107 of the bot interaction detection system 100 through the use of an autoencoder training subsystem 153, according to one embodiment described herein. As described above, the autoencoder subsystem may also implement a transformer model. The training data 210 accessed and / or received by the autoencoder training subsystem 153 may be, or include, historical data from human-interaction or bot-interaction sources that has been processed to prepare formatted / processed sequence data 105. The formatted / processed sequence data 105 may be received from a data processor / formatter 104, as described herein. In various embodiments, the training data 210 may be, or include, synthetic data in addition to, or instead of historical data.

[0058] For example, generating synthetic data using a generative neural network model can improve the accuracy and speed of the autoencoder training subsystem 153 when a large amount of processed / formatted sequence data 105 is used to train the encoder model 108. Furthermore, the training data 210 can be stratified to provide a balanced dataset between bot-human interactions as an approach to training the encoder 108, where bot interactions may not be adequately reflected in the input data. In various embodiments, the autoencoder training subsystem 153 includes, or may include, one or more devices and / or virtual devices configured to access and / or receive processed / formatted sequence data 105 from the training data 210 source and provide data to one or more configuration subsystems of the unsupervised training system 150.

[0059] In various embodiments, the unsupervised training system 150 is implemented in or as part of a computing device such as a server. The unsupervised training system 150 may be implemented as a dedicated hardware device, as program code, or a combination of both. For example, the operations described herein as being performed by the unsupervised training system 150 may also be embodied in program code that implements the unsupervised training system 150, such program code being executable by one or more processing units. For example, the data input subsystem 220, encoder model 108, and decoder model 230, the objective function 240, and other embodiments of the autoencoder training subsystem 153 may each be implemented as one or more software functions or dedicated hardware devices that work together to implement the unsupervised training system 150 described herein. The encoder model 108 and decoder model 230 may each be implemented as separate machine learning models, such as recurrent neural networks. The unsupervised training system 150 may implement separate unsupervised training algorithms for training each subsystem of the invalid interaction detection system 100, such that each subsystem is trained using unlabeled data. Thus, the cluster membership classifier subsystem 114 may be trained to generate machine actor scores for input embeddings using labels predicted by the embedding clustering subsystem, and embedding clustering subsystems such as the embedding clustering subsystem 340 may be trained using unlabeled training data.

[0060] The encoder model 108 may be unaware of the origin of the formatted / processed sequence data 105. Therefore, the encoder model 108 can be trained to generate embeddings from input sequences that can correspond to human or bot interactions. The decoder model 230 may be configured to take the output embeddings of the encoder model 108 as input and generate an output sequence provided to the objective function 240. The objective function 240 can then access and / or receive the output sequence from the decoder model 230 and the input sequence from the data input subsystem 220. Through comparison of the output and input sequences, the objective function 240, which may be realized, for example, as a squared loss, adjusts one or more learned parameters of the encoder model 108 and / or the decoder model 230. Thus, the encoder model 108 and the decoder model 230 can be trained together as part of a unified autoencoder training process.

[0061] In various embodiments, the autoencoder training subsystem 153 processes the formatted / processed sequence data 105 until it minimizes an objective function 240, for example, defining the changes in the comparison between the output sequence and the input sequence, and / or estimating the limiting changes in the learning parameters with each training cycle. In this way, the autoencoder training subsystem 153 can provide the classifier with fixed-length vectors that can be effectively clustered and classified, while verifying that the output embeddings of the encoder model 108 retain significant information about the input sequence. In this context, “significant” information may refer to a reduced-dimensional representation of the input sequence that can serve as input to a clustering model. After training, the encoder model 108 can be used as part of a bot interaction detection system, as shown in Figure 1.

[0062] Figure 3 shows a training system 152 for training the embedding clustering subsystem 340 and the cluster membership classifier subsystem 114 of the machine actor detection subsystem 110 according to one embodiment described herein. As described above, the embedding clustering subsystem 340 may be a standard clustering algorithm such as K-means that learns the clustering of the sequence embeddings 320, i.e., assigns cluster labels to each input embedding. Since proximity in the embedding space may mean similarity, each cluster in the learned clustering may correspond to a grouping of inputs that feature similar characteristics, and consequently, such a grouping may correspond to an invalid interaction pattern. As described above, the cluster membership classifier subsystem 114 may be a standard classifier such as a logistic regression model, or may include such a classifier. Given the cluster labels assigned to the input sequence embeddings 320 by the embedding clustering subsystem 340, the classifier 114 may be trained to predict the probability of belonging to each cluster discovered by the embedding clustering subsystem 340 for a given embedding. Thus, the training system 152 may include applying learning techniques using one or more prepared sets of training data 310, which are converted into sequence embeddings 320 by the autoencoder subsystem 107. The training data 310 may be the same training data used for unsupervised training of the autoencoder subsystem 107, or may include the same training data.

[0063] The sequence embedding data 320 may be data corresponding to embeddings of valid and invalid interaction data, used to train the autoencoder subsystem 107. When the classifier is trained by the training approach described above, the cluster membership classifier subsystem 114 is not provided with labeled embeddings. In various embodiments, the training data 310 includes a set of input embeddings generated using the input sequence data for each of several subgroups representing different types of invalid interactions (e.g., bot interactions) and genuine human interactions, which the cluster membership classifier subsystem 114 will learn. In various further embodiments, the classification is a binary classification representing whether the interaction is a bot interaction or not. Similar to training the encoder model 108, the cluster membership classifier subsystem 114 may be trained using a stratified dataset containing a balanced proportion of sequence embeddings 320 as an approach to limit the classifier's bias in order to classify the input sequences in an unbiased manner. The training data 310 can be processed via the data input subsystem 330. In various embodiments, the data input subsystem 330 can perform processing similar to that of the data input subsystem 220 described above.

[0064] The learning subsystem 155 can implement one or more objective functions. The learning subsystem 155 may include an objective function 350(a) suitable for the learning setup, more specifically, for clustering (of the embeddings output by the encoder model 108). For example, if the embedding clustering subsystem 340 is implemented as a K-means clustering subsystem, the objective function 350(a) may employ an objective function that penalizes the intra-cluster distance to the cluster centroid, summed across all clusters. Training using the objective function 350(a) may involve tuning one or more weights and / or coefficients of the embedding clustering subsystem over multiple iterations until the value of the objective function 350(a) converges to a local or global minimum. For example, parameters defining each cluster of the embedding clustering subsystem, such as the central position and distribution, may be tuned in each training iteration, and the objective function 350(a) may be evaluated using the current values ​​of each cluster.

[0065] Values ​​of the objective function 350(b) suitable for a supervised learning setup may be used to tune one or more parameters of the cluster membership classifier subsystem 114 in each training iteration to better recognize the classifiable elements of the clustered embeddings (output by the encoder model 108) and predict the cluster membership of the embeddings. Objective function 350(b) may differ from objective function 350(a) in that it may be suitable for a supervised learning setup to train the cluster membership classifier subsystem 114 to predict embedding cluster memberships using, for example, the predicted values ​​of the embedding clustering subsystem 340 as labels. For example, if the cluster membership classifier subsystem 114 is implemented as a logistic regression model, objective function 350(b) may include a cross-entropy loss function used to train the cluster membership classifier subsystem 114.

[0066] For example, in various embodiments using a mixture model (such as a Gaussian mixture model), the objective functions 350(a) and 304(b) may be folded into a single objective function 340 used to train the connected system, i.e., the embedding clustering subsystem 340 and the cluster membership classifier subsystem 114 are folded into a single component containing the mixture model. More specifically, for example, a Gaussian mixture model (GMM) can fit multimodal Gaussian distributions to input embeddings (individual Gaussian distributions playing the role of clusters), from which "cluster labels" can be inferred as labels for the individual distributions having the highest posterior probability given the input (which covers the role of classification). The objective function in this latter example can be derived from an expectation maximization algorithm that may be used to train the GMM.

[0067] A typical parametric learning model may be trained and tested with various variable configurations and / or hyperparameters to determine the optimal configuration for a particular application of the model. The autoencoder training subsystem 153 and the learning subsystem 155 may implement separate hyperparameter tuning algorithms to optimize the autoencoder subsystem 107 in the former case, and the embedded clustering subsystem 340 and the cluster membership classifier subsystem 114 in the latter case. For example, one or more terms of the objective function 340 and / or the cluster membership classifier subsystem 114 and / or the embedded clustering subsystem 340 may be tuned by changing unlearned parameters, such as scalar weighting coefficients. Since the embedded clustering subsystem 340 and the cluster membership classifier subsystem 114 are separate entities, each of their various components may be optimized individually at a finer granularity than in situations where clustering and classification are combined in a single model, such as a mixed model.

[0068] Figure 4 is a schematic diagram of the inference stage for detecting bot interaction from mouse cursor event data according to one embodiment described herein. As an exemplary example of the technique described with reference to Figures 1-3, Figure 4 illustrates an encoder 400 that receives mouse cursor event data (e.g., including both motion and click sequences) as an input sequence 402, as part of an inference process for determining some output according to the embodiments described herein. The encoder 400 may be part of an autoencoder including an RNN encoder and decoder, as described herein.

[0069] Other user interface interaction data can be handled similarly, but as an example involving positional changes according to a series of positional deltas, mouse cursor event data 404 is shown in Figure 4. Positional deltas can be constructed from a specific subset of sequential types of user input signals extracted from past web interactions, including user session data. In various embodiments, deltas between positions in a coordinate system (e.g., "x", "y", "r", "θ") may be accompanied by corresponding deltas between timestamps (e.g., "t", or "c"). State values ​​("M") may be used to further indicate the current state of the received input. For example, state value M may include a vector representing the state of the input, such as when a mouse button is pressed, not pressed, clicked, or released. When constructing a sequence of deltas, a rule may be enforced that only a single mouse cursor event can occur at a given position and time (e.g., arbitrarily, or prioritizing privileged event types such as motion), and thus state M may be represented as a one-hot vector, taking a value of 1 at the coordinates representing the active event type and a value of 0 at all other coordinates. These methods allow us to match sequences of inputs with different modalities (such as mouse movements and clicks) into a unified sequence of vectors that include deltas and states.

[0070] It is crucial that embeddings generated by generative machine learning models retain the non-fluid, non-human cursor patterns present in the original raw cursor trace data. Non-fluid, non-human cursor patterns generally indicate invalid bot-based interactions with applications. For example, humans often interact with input devices using fluid, inaccurate, and non-periodic inputs. In contrast, bots often generate inputs using non-fluid, non-human, accurate, and / or periodic inputs (i.e., inputs that change in response to a fixed time interval, such as steps in a script manipulated by the bot). In some cases, bots may even exploit their ability to attempt to simulate the fluid and random movements of humans. Nevertheless, these bot interactions include some patterns that do not fully represent human interactions. Therefore, proper, thorough, and continuous training of the generative models described herein is a critical feature in detecting inputs from these simulated bots.

[0071] As described herein, input devices such as a mouse, tablet, or other electronic input devices may be used to generate input interaction data. Simultaneously with the use of an electronic input device, a cursor representing the relative position of the input on the electronic input device may be displayed on the graphical user interface. When the electronic input device is operated to generate interaction data, the cursor displayed on the graphical user interface also changes its position or state accordingly. This can be used to facilitate interaction with applications that utilize the graphical user interface. For example, the electronic input device can interact with one or more additional elements on the graphical user interface implemented as part of an application by aligning the cursor with these additional elements and triggering some interaction.

[0072] For each entry in the input sequence 402, the mouse cursor position may be defined as the difference between the current position and the preceding position (e.g., in Cartesian or polar coordinates). The input sequence 402 may be generated based on formatted / processed sequence data 105 derived from cursor trace data 404. In this way, the input sequence may contain tuples of position data and velocity data in a single coordinate set, by including a difference time value "c" that describes the elapsed time between the current data point and the preceding data point. The state value "M" may be or contain a one-hot vector of a defined state.

[0073] For an input sequence 402 to encoder 400, each time point may be provided as an input state 408 to a cell 410 of encoder 400, which may contain any number of cells 410 corresponding to the variable length of the input sequence 402. As previously mentioned, encoder 400 may be an RNN implementation, in which case the cells 410 may include LSTM cells or GRU cells, among other approaches to context-sensitive sequence embedding.

[0074] Each cell 410 receives the input state 408, cell state 412, and hidden state 414 from the preceding cell 410 in the neural network. For the first cell of cell 410 corresponding to the first time step in memory, the first cell state and the first hidden state are provided to the first cell. LSTM cells and GRU cells differ in that GRU cells do not pass the cell state 412. The hidden state 414 passed to subsequent cells 410 facilitates information retention and provides the encoder 400 with the ability to identify potential information that enables bot interaction detection, such as cursor position movement or changes in interaction state. In various embodiments, the final hidden state 414 of the encoder 400 is "h nThis serves as an embedding 416 that can be taken up by the cluster membership classifier subsystem 114. The final hidden state may be selected to include latent information from the entire input sequence 402 and reduce the possibility of omitting entries that could affect the classification. However, intermediate hidden states 414 may serve as embeddings 416 that would improve the classification of the input sequence 402, for example. For example, if a bidirectional encoder is used, the intermediate hidden state 414 may represent latent dimensional information better than either of the final hidden states 414.

[0075] The obtained embedding data can then be sent to the cluster membership classifier subsystem 114 to predict the probability distribution for each of the numerous clusters 418 using an unlabeled input sequence. As described above, in various embodiments, the encoder 400 is included as part of an autoencoder system (e.g., autoencoder subsystem 107). In some cases, the clustering and / or classifier can output a visualization of the clustered data, for example, as a three-dimensional or two-dimensional cluster projection. A two-dimensional (2D) cluster projection 420 can represent the clustering of data for each of the clusters 418 on a pair of axes selected to explain the separation between the clustered embeddings.

[0076] As illustrated, different clusters 418 can have different locations and different distribution densities in the 2D coordinate system. In some cases, bot interactions and human interactions may be distinguishable by distribution density, in which case the aspects of the input sequence generated by the bot algorithm may contain a high degree of replication or exhibit other detectable patterns, in which case clustering may produce high-density groups of input that display similar repeating patterns, and otherwise low-density groups. For example, depending on the input dataset, the largest low-density cluster (in terms of points) is likely to belong to a mixture of genuine non-repetitive human interactions and bot interactions indistinguishable from human interactions, while other smaller low-density clusters may contain inputs with similar patterns that are likely to be bot interactions. For example, cluster 418-2 may be characterized by a lower cluster density and a larger population size than cluster 418-1. The output of the classifier may be a probability distribution describing, among other things, the likelihood that the input sequence 402 belongs to each of these clusters. For example, in the 2D cluster projection example 420, the classifier output includes a four-element vector of values ​​between 0 and 1 that describes the probability that the input sequence 402 belongs to cluster 418-2, which exhibits relatively variable behavior, or to a cluster that exhibits repetitive behavior (e.g., cluster 418-1).

[0077] During inference, the classifier output may be processed by a machine actor score generator 116. The machine actor score generator can generate machine actor scores based on the classified clusters 418. In various embodiments, the machine actor score generator 116 may modify the output vector by removing the probability values ​​that belong to the largest cluster. The remaining probability values, such as the highest probability value, may then be used to determine the machine actor score. In an exemplary example, the classifier output may be a four-element vector P=[p1, p2, p3, p4]=[0.01, 0.002, 0.89, 0.098], indicating that the input sequence 402 is most likely to belong to class 3.

[0078] As an example, but not always, if class i=1 corresponds to the largest cluster, the machine actor score generator 116 can remove the first element from the vector P, leaving P=[0.002, 0.89, 0.098]. From the maximum probability value p3=0.89, the machine actor score can be predicted. In the simplest example, the machine actor score may simply be p3 such that the probability that the input sequence 402 corresponds to a bot interaction is greater than 89%. Next, the machine actor score generator 116 can send the machine actor score to the thresholding subsystem 118. The thresholding subsystem 118 can use a threshold or metric to determine whether the generated machine actor score meets, exceeds, or does not meet the threshold. This decision may be sent to output 422.

[0079] Output 422, which may include the operation of output component 120 as described with reference to Figures 1 and 5, may include providing verification that the input sequence corresponds to a genuine interaction with a human user. In contrast, if the input sequence is deemed suspicious (i.e., indicating invalid traffic), output 422 may include instructions and / or messages to the relevant system to flag the corresponding request and / or the source of the request. Output may also include, but is not limited to, generating and / or presenting a visualization of the 2D cluster projection 420, such as on the user device's display, or communicating probability values, input sequences, and / or embeddings to the relevant security system for use in further bot interaction detection processing.

[0080] Figure 5 is a block flow diagram of an inference-stage method 500 for detecting bot interactions from mouse cursor trace data, according to one embodiment described herein. Specifically, the detection system 100 can perform this method 500 or a similar method, in which the cluster membership classifier subsystem 114 is trained to classify embedded cluster memberships generated by the encoder 108 from the input data 105, and the clustering is learned by the embedded clustering subsystem during training. Thus, this method 500 or a similar method can be performed to distinguish bot interactions from human interactions and to perform a gatekeeping function by blocking or forwarding requests based on the classification of the input data 105.

[0081] The method 500 depicted in Figure 5, like other methods described herein, may be implemented in software (e.g., as code, instructions, or a program) executed by one or more processing units (e.g., a processor or processor core), hardware, or a combination thereof. The software may be stored in a non-temporary storage medium such as a memory device. The method 500 is intended to be illustrative and non-limiting. Figure 5 depicts various activities occurring in a particular order or sequence, but this is not intended to limit it. In some embodiments, for example, the activities may be executed in a different order, or one or more activities of the method 500 may be executed in parallel. In some embodiments, the method 500 may be executed by a detection system 100.

[0082] In block 505, the detection system 100 acquires input data 105, which may be, or include, mouse cursor trace data generated as part of an interaction with a user interface element such as a button. In various embodiments, the data is collected from past web interactions. For example, the data may be collected from a user web session that includes a large number of inputs from entities that interact with the browser during the duration of the session. This collection process is advantageous because it collects raw data from the web session, and there is no possibility of input bias by requiring the user or entity to interact with the input system to generate the data. The data collection period may be continuous or may occur periodically during the web session. The input data 105 may include, but is not limited to, mouse cursor position data, motion data, timestamp data, and state data describing the state of the mouse as a function of time. As will be described in more detail with reference to Figures 1-4, the input data 104 may be sessioned to isolate separate input sessions. In some cases, the input data 104 may be an unbound sequence, for example, collected and stored while the detection system 100 is active. Thus, the subsystem components can bind the input data 104 based on identifying a single session.

[0083] In block 510, the detection system 100 generates an output embedding by providing an input sequence to an encoder 108 trained to generate output embeddings from an input sequence. The encoder 108 is or may include an RNN containing LSTM and / or GRU cells, trained to generate fixed-length embeddings from a variable-length input sequence. In a specific embodiment, the embedding may be or may include the hidden state output of a cell in an RNN, such that the embedding includes latent dimensional information along with clustering learned during training by the embedding clustering subsystem (i.e., the embedding clustering subsystem 340), and the cluster membership classifier subsystem 114 is trained to distinguish between human interactions and bot interactions.

[0084] In block 515, the detection system 100 classifies the embeddings according to cluster membership by predicting a probability distribution. As will be explained in more detail with reference to Figures 1 and 4, the classifier can output a vector of probability values ​​that describes the likelihood that the input data 104 belongs to a particular group of similar patterns.

[0085] In block 520, the detection system 100 generates a machine actor score from the probability distribution output by the cluster membership classifier subsystem 114. In various embodiments, the machine actor score corresponds to the probability that the input data 104 belongs to a group of similar cluster patterns when embedded and clustered. For example, the machine actor score may correspond to the probability that the input data is similar to a group of clustered inputs that exhibit a particular repetitive pattern. This indicates the possibility that the input data is invalid input, since repetitive user input behavior is typical of a bot. More specifically, the machine actor score may be an element of the classifier output corresponding to the highest probability value for a clustering class excluding the "largest" cluster found during training, as described above, or may include such an element. The machine actor score acts as a surrogate for the probability that the input is generated by a bot, insofar as it captures the possibility that the input belongs to a group of inputs that have similar repetitive patterns.

[0086] In decision block 525, the detection system 100 determines whether the machine actor score meets the threshold for attributing the input data to bot traffic. In various embodiments, the decision includes comparing the machine actor score to the threshold and / or threshold range so that if the score exceeds the threshold and / or falls outside the threshold range, the detection system 100 attributes the input data to bot interaction. The threshold can be defined based on input received from a data scientist or client. Alternatively, the threshold may be determined by using a machine learning model to identify (for example) the optimal threshold with respect to performance metrics (e.g., precision, recall, sensitivity, specificity, accuracy, etc.) or their weighted sum. In some cases, different thresholds may be defined for different use cases. In some cases, different types of models, different algorithms, and / or different prioritized performance metrics (or combinations thereof) may be defined for different use cases.

[0087] In various embodiments, the output of the detection system 100 may include a determination of whether the input data represents a human interaction or a bot interaction. In addition, or alternatively, the detection system 100 may function as an intermediate check on one or more interactions, such as a gatekeeping function, to verify the interaction before granting access to the data targeted by the request. For example, the detection system may process the input data, make a suspicious determination, and then block or grant the request based on the suspicious determination. In this way, the detection system 100 can significantly improve data security and enhance automated efforts to detect and thwart automated data access systems.

[0088] It will be understood that variations of process 500 are intended. For example, block 525 shows that the machine actor score is compared to a threshold to predict whether the input data corresponds to bot traffic. In some embodiments, the machine actor score can be used to prioritize requests rather than to generate binary predictions (which may lead to a binary decision on whether to allow an access request, for example). For example, each of several requests (e.g., data access, processing resources, etc.) may be ranked in a queue according to its machine actor score. In some cases, requests associated with a low machine actor score (e.g., corresponding to a prediction that the input is from a human) may be preferred over requests associated with a high machine actor score and placed closer to the front of the queue. This approach may facilitate improvements in user experience and performance. In some cases, requests associated with a high machine actor score may be preferred over requests associated with a low machine actor score and placed closer to the front of the queue. This approach may facilitate ensuring that critical automated processes (e.g., backup generation, security scans, etc.) are executed reliably.

[0089] Different thresholds that alter how responses to machine actor scores are generated may also exist, or alternatively. For example, the highest machine actor scores may be immediately discarded or sent for analysis. Machine actor scores that are not so high but are important may be isolated, and normal machine actor scores may be periodically reviewed by data scientists. Low machine actor scores may be approved for processing of requests.

[0090] In various embodiments, the generated machine actor score may be further utilized and / or transmitted to another system to determine one or more attributes of the machine actor score. For example, in some embodiments, the generated machine actor score can be incorporated as a feature into a downstream detection / suspicious pipeline that utilizes multiple signals of different kinds to characterize and validate input sessions. In some embodiments, the score can be used to assist researchers in knowing which input sessions should be investigated more thoroughly. Investigation may include manually inspecting mouse cursor signals and / or other signals from the input session to examine individual elements of valid and / or invalid interactions. In some embodiments, the machine actor score can be used to generate synthetic input data to further refine a training dataset for training a generative neural network.

[0091] In some embodiments, the decision may operate in real time (for example, when a mouse device generates input for the application). The decision may be updated in real time based on incoming input, and the decision may be modified based on that real-time update. For example, process 500 may iteratively complete the steps of blocks 505-525 over a period of time. The decision in 525 may be made at the end of the period, at which point the decision proceeds to either block 530 or 535.

[0092] In block 530, if the detection system 100 determines that the input data was generated by a valid entity, the detection system 100 may forward the request and the output of the detection system 100 to another system for processing, for example, via the network 160, in order to allow additional input. In contrast, if the detection system 100 identifies the input data as an invalid interaction, the system may block the request or take other actions to identify the originating system and prevent further access attempts 535, such as forwarding the request and the output of the detection system 100 to another system for processing. In some embodiments, the input may be isolated in a suspension environment until it is manually reviewed by an entity such as a data scientist.

[0093] Advantageously, the detection system 100 provides an improved technique for detecting and / or rejecting bot interactions in requests generated by an automated data access system. In various embodiments, the detection system 100 can be trained to process input data from a variety of input modalities, including but not limited to touch data, scroll data, mouse cursor trace data, speech / voice input, keystrokes, or other computer input data that can be expressed as sequences. In particular, the detection system 100 is a robust system that can be trained periodically and learn to detect bot interactions without explicit reverse engineering of bot interaction algorithms.

[0094] Weighting of classification It will be understood that there can be multiple clusters corresponding to bot activity. For example, one cluster might correspond to trace activity, such as quickly and directly navigating the cursor to the center pixel, while another cluster might correspond to clicking pixels in the corners of an image. Different types of activity may arise from different entities with defined rules for bot control (e.g.), different contexts (e.g., different browser environments with different event firing mechanisms), different time zones, etc. Another example is that device type or other types of data characterizing user devices may have different characteristics across clusters. It should be understood that these examples are simplifications that convey the idea that clusters ideally correspond to specific, distinct bot patterns. In reality, clusters may represent more complex patterns and behaviors that cannot be easily described or identified, but may still represent bot traffic.

[0095] Some disclosures in this specification relate to predicting whether a given user is human or a bot. In a purely binary classification instance, each cluster associated with either type of bot activity may be characterized as a bot cluster. And if a given cursor trace dataset is assigned to one of the bot clusters, it can be predicted that it was generated by a bot.

[0096] However, there may be differences between clusters. For example, clusters may differ in their likelihood of corresponding to bot activity. Therefore, multiple clusters may have a much higher likelihood of corresponding to bot activity compared to another cluster or compared to unclustered data elements, while the confidence metric representing the confidence that a represented activity corresponds to bot activity may be higher than that of another. As another example, clusters may differ in their confidence that the elements within each cluster correspond to a clear and uniform type of activity. As yet another example, a content source or other entity may decide that one bot, a certain type of bot, or a certain type of bot activity is less desirable than another bot, a different type of bot, or a different type of bot activity.

[0097] Therefore, each cluster may be assigned a weight. In some cases, the weight may influence the cluster assignment. The weight may influence the cluster assignment, as it is done by evaluating both the degree to which the characteristics of a given cursor trace match the characteristics of the cluster and the weight of the cluster. Alternatively or additionally, the weight may influence the machine actor score or determined threshold (for example, to determine whether to characterize input activity as coming from a bot or to determine whether to allow a given action). For example, if a given cursor trace dataset is assigned to a cluster with a higher weight (compared to if the cursor trace dataset is assigned to a cluster with a lower weight), the machine actor score may be higher or the threshold may be lower.

[0098] Weights can be assigned to clusters reliably or to specific instances. In addition or alternatively, weights may be used reliably (e.g., to potentially influence cluster assignment, to influence machine actor scores, or to influence thresholds) or only in specific instances. Exemplary specific instances include when an input (representing a cursor trace) has no nearest neighbors within a certain radius, when an input is tightly clustered with other known inputs of the same type, when the classification of other nearby inputs has a certain importance, when an input indicates that the cursor trace was generated by a particular type of input device (e.g., mouse, trackpad, keyboard, VR), when an input indicates that the cursor trace was received at a particular time of day (e.g., nighttime), or when an input indicates that the cursor trace originated from a device within a particular geographical region (e.g., a particular country).

[0099] Periodic sampling In some cases, instead of evaluating each input dataset to predict whether it is from a bot or a human (or generating another type of prediction), periodic sampling can be used to reduce the resources used for monitoring. As a result of periodic sampling, inputs may be monitored over a certain period, inputs may be evaluated every nth instance, or a subset of random or pseudo-random inputs may be evaluated.

[0100] User interface and analytics dashboard In some cases, instead of using machine actor scores to decide whether to process a request, determine its placement in a queue, or allocate resources, the scores may be used for analysis in addition to these other purposes. Specifically, analysis could track how many times a particular piece of content is viewed or clicked, etc. Machine actor scores may be used to distinguish whether some of the views, clicks, etc., or some of the views, clicks, etc., are associated with a machine actor score that might indicate that such instances are due to bot interaction.

[0101] For example, a dashboard might show the distribution of machine actor scores across a set of one or more types of interactions (e.g., clicks, views, etc.). Another example is a dashboard that identifies the percentage of interactions predicted to be bot-based. This data can be presented for each of one or more content objects, or aggregated. In some cases, one or more specific cursor trace datasets might be flagged as particularly suspicious or primed for human review (to determine whether they are bot-based or human-based).

[0102] In some cases, the content source may have pledged to pay another entity for the placement of the content on a webpage and / or for various types of interactions. Therefore, dashboard data can inform the source about the value of the placement and / or potential fraud. Furthermore, or alternatively, the dashboard can be made available to other entities, who can use that information to identify strategies to reduce bot interactions (for example, by influencing decisions regarding the placement of content within a webpage, various frames, or JavaScript® preferences).

[0103] Exemplary Implementation Environment Figure 7 shows a distributed system 700 for implementing a certain embodiment. In the illustrated embodiment, the distributed system 700 includes one or more client computing devices 702, 704, 706, and 708 coupled to a server 712 via one or more communication networks 710. The client computing devices 702, 704, 706, and 708 may be configured to run one or more applications.

[0104] In various embodiments, the server 712 may be adapted to run one or more services or software applications that enable the use of backpropagation to train the cluster membership classifier subsystem 114 of the detection system 100, as described herein. For example, the server 712 may run some or all aspects of the unsupervised training system 150, or some or all aspects of the detection system 100.

[0105] In a particular embodiment, the server 712 may also provide other services or software applications that may include non-virtual and virtual environments. In various embodiments, these services may be provided to users of client computing devices 702, 704, 706, and / or 708 as web-based or cloud services, such as under a Software as a Service (SaaS) model. Users operating client computing devices 702, 704, 706, and / or 708 can then utilize one or more client applications that interact with the server 712 to use the services provided by these components. More specifically, for example, each of the client computing devices 702, 704, 706, and / or 708 may be configured to run the discovery system 100 and may also be an embedded device configured to communicate with the server 712 to enable the server 712 to train the cluster membership classifier subsystem 114 of the discovery system 100 through backpropagation, as described herein.

[0106] In the configuration depicted in Figure 7, server 712 may include one or more components 718, 720, and 722 that implement the functions performed by server 712. These components may include one or more processors, hardware components, or software components that can be executed by a combination thereof. It should be understood that various different system configurations are possible that may differ from the distributed system 700. Therefore, the embodiment shown in Figure 7 is an example of a distributed system for implementing the embodiment system and is not intended to be limiting.

[0107] Users can interact with aspects of the detection system 100 provided by the server 712 in accordance with the teachings of this disclosure using client computing devices 702, 704, 706, and / or 708. Client devices can provide interfaces (e.g., voice interfaces) that allow users of the client devices to interact with the client devices. Client devices can also output information to the user through this interface. Although only four client computing devices are depicted in Figure 7, any number of client computing devices may be supported.

[0108] Client devices can include various types of computing systems, such as PA devices, portable handheld devices, general-purpose computers like personal computers and laptops, workstation computers, wearable devices, game systems, thin clients, various messaging devices, sensors or other sensing devices. These computing devices can run various types and versions of software applications and operating systems (e.g., Microsoft Windows®, Apple Macintosh®, UNIX® or UNIX-like operating systems, Linux® or Linux-like operating systems like Google Chrome® OS), including various mobile operating systems (e.g., Microsoft Windows Mobile®, iOS®, Windows Phone®, Android®, BlackBerry®, Palm OS®). Portable handheld devices may include mobile phones, smartphones (e.g., iPhone®), tablets (e.g., iPad®), and personal digital assistants (PDAs). Wearable devices may include Google Glass® head-mounted displays. Game systems include various portable game consoles and internet-enabled game consoles (for example, Microsoft Xbox® game consoles with or without Kinect® gesture input devices, Sony PlayStation® systems, and various game systems provided by Nintendo®). Client devices can run a variety of different applications, such as various internet-related applications and communication applications (for example, email applications and short message service (SMS) applications), and can use various communication protocols.

[0109] Network 710 may be any type of network well known to those skilled in the art that can support data communication using any of the various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk®, etc. Just as an example, Network 710 may be a local area network (LAN), an Ethernet®-based network, Token Ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating on any of the IEEE 802.11 suite protocols, Bluetooth®, and / or other wireless protocols), and / or any combination of these networks and / or other networks.

[0110] Server 712 can consist of one or more general-purpose computers, dedicated server computers (including, for example, PC (personal computer) servers, UNIX® servers, midrange servers, mainframe computers, rack-mount servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. Server 712 may include other computing architectures with virtualization, such as one or more virtual machines running a virtual operating system, or one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for the server. In various embodiments, Server 712 can be adapted to run one or more services or software applications that provide the functionality described in the foregoing disclosure.

[0111] The computing system in Server 712 may run one or more operating systems, including any of the above, and any server operating system available on the market. Server 712 may also run any of a variety of other server applications and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Java® servers, and database servers. Exemplary database servers include, but are not limited to, those available on the market from Oracle Corporation®, Microsoft Corporation®, Sybase Corporation®, IBM Corporation® (International Business Machines), and others.

[0112] In some implementations, server 712 may include one or more applications for analyzing and organizing data feeds and / or event update information received from users of client computing devices 702, 704, 706, and 708. For example, data feeds and / or event update information may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party sources and continuous data streams, and may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring. Server 712 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices on client computing devices 702, 704, 706, and 708.

[0113] The distributed system 700 may also include one or more data repositories 714, 716. In a certain embodiment, these data repositories may be used to store data and other information. For example, one or more of the data repositories 714, 716 may be used to store training data 210 or other data necessary to train the cluster membership classifier subsystem 114 of the detection system 100, as described herein. The data repositories 714, 716 may be in various locations. For example, a data repository used by server 712 may be local to server 712, or it may be remote from server 712 and communicate with server 712 via a network-based connection or a dedicated connection. The data repositories 714, 716 may be of different types. In a certain embodiment, a data repository used by server 712 may be a database, such as a relational database, provided by Oracle Corporation® and other vendors. One or more of these databases may be adapted to enable the storage, updating, and retrieval of data to and from a database that responds to SQL-formatted commands.

[0114] In one embodiment, one or more of the data repositories 714, 716 may also be used by an application to store application data. The data repositories used by the application may be of different types, such as a key-value store repository, an object store repository, or a general storage repository supported by a file system.

[0115] In a particular embodiment, as described herein, all or part of the training of the cluster membership classifier subsystem 114 of the detection system 100 may be provided as a service via a cloud environment. Figure 8 is a block diagram of a cloud-based system environment in which, according to a particular embodiment, the training of the cluster membership classifier subsystem 114 described herein may be provided at least partially as a cloud service. In the embodiment shown in Figure 8, the cloud infrastructure system 802 may provide one or more cloud services that may be requested by a user using one or more client computing devices 804, 806 and 808. The cloud infrastructure system 802 may include one or more computers and / or servers, which may include the ones described above for server 712. The computers in the cloud infrastructure system 802 may be organized as general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0116] Network 810 can facilitate communication and data exchange between client computing devices 804, 806, and 808 and the cloud infrastructure system 802. Network 810 may include one or more networks. These networks may be of the same type or of different types. Network 810 may support one or more communication protocols, including wired and / or wireless protocols, to facilitate communication.

[0117] The embodiment shown in Figure 8 is merely one example of a cloud infrastructure system and is not intended to be limiting. It should be understood that in some other embodiments, the cloud infrastructure system 802 may have more or fewer components than those shown in Figure 8, may combine two or more components, or may have different configurations or arrangements of components. For example, while Figure 8 shows three client computing devices, any number of client computing devices may be supported in alternative embodiments.

[0118] The term "cloud service" is generally used to refer to services made available to users on demand via communication networks such as the Internet by a service provider's system (e.g., cloud infrastructure system 802). Generally, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own on-premises servers and systems. The cloud service provider's system is managed by the cloud service provider. Therefore, customers can use these services without having to purchase separate licenses, support, or hardware and software resources for the cloud services provided by the cloud service provider. For example, the cloud service provider's system can host an application, and users can order and use this application on demand via the Internet without having to purchase the infrastructure resources to run this application. Cloud services are designed to provide easy and scalable access to applications, resources, and services. Several providers offer cloud services. For example, some cloud services such as middleware services, database services, and Java cloud services are offered by Oracle Corporation (registered trademark) in Redwood Shores, California.

[0119] In one embodiment, the cloud infrastructure system 802 may provide one or more cloud services using various models, including a hybrid service model, a software-as-a-service (SaaS) model, a platform-as-a-service (PaaS) model, and an infrastructure-as-a-service (IaaS) model. The cloud infrastructure system 802 may include a set of applications, middleware, databases, and other resources that enable the provision of various cloud services.

[0120] The SaaS model allows applications or software to be delivered to customers as a service over a communication network such as the internet, without requiring customers to purchase the hardware or software for the basic application. For example, the SaaS model can be used to provide customers with access to on-demand applications hosted by a cloud infrastructure system. Examples of SaaS services offered by Oracle Corporation (registered trademark) include, but are not limited to, a variety of services for human resources / capital management, customer relationship management (CRM), enterprise resource planning (ERP), supply chain management (SCM), enterprise performance management (EPM), analytics services, and social applications.

[0121] The IaaS model is generally used to provide customers with resilient computing and storage capabilities by offering infrastructure resources (e.g., servers, storage, hardware, and networking resources) as cloud services. Various IaaS services are offered by Oracle Corporation®.

[0122] The PaaS model is generally used to provide resources as a service that enable customers to develop, run, and manage applications and services without having to procure, build, or maintain platform and environmental resources themselves. Examples of PaaS services offered by Oracle Corporation (registered trademark) include, but are not limited to, Oracle Java Cloud Services (JCS), Oracle Database Cloud Services (DBCS), Data Management Cloud Services, and various application development solution services.

[0123] Cloud services are generally provided in an on-demand self-service, subscription-based, scalable, reliable, highly available, and secure manner. For example, a customer may order one or more services provided by the cloud infrastructure system 802 through a subscription order. The cloud infrastructure system 802 then processes the services requested in the customer's subscription order. For example, a customer may subscribe to information services or other services provided by the discovery system 100 in a conversational manner. The cloud infrastructure system 802 may be configured to provide one or more cloud services.

[0124] The cloud infrastructure system 802 can provide cloud services through various deployment models. In a public cloud model, the cloud infrastructure system 802 may be owned by a third-party cloud service provider, and the cloud services may be provided to any general public customer, who may be an individual or a company. In another embodiment, under a private cloud model, the cloud infrastructure system 802 may be operated within an organization (e.g., within a corporate organization), and services may be provided to customers within that organization. For example, customers may be various departments of the company, such as the human resources department or the payroll department, or individuals within the company. In yet another embodiment, under a community cloud model, the cloud infrastructure system 802 and the services provided may be shared by several organizations within a relevant community. Various other models, such as hybrids of the above models, may also be used.

[0125] Client computing devices 804, 806, and 808 may be of different types (such as client computing devices 702, 704, 706, and 708 shown in Figure 7) and may be capable of running one or more client applications. Users may use the client computing devices to interact with the cloud infrastructure system 802, such as requesting services provided by the cloud infrastructure system 802. For example, an attacker may use the client devices to send malicious requests.

[0126] In various embodiments, the processing performed by the cloud infrastructure system 802 may include big data analytics. This analytics may involve using, analyzing, and manipulating large datasets to detect and visualize various trends, behaviors, relationships, etc., within the data. This analytics may be performed by one or more processors and may include parallel processing of the data and running simulations using the data. For example, big data analytics may be performed by the cloud infrastructure system 802 to provide training for the cluster membership classifier subsystem 114, as described herein. The data used for this analytics may include structured data (e.g., data stored in a database or data structured according to a structured model) and / or unstructured data (e.g., data blobs (binary large objects)).

[0127] As shown in the embodiment of Figure 8, the cloud infrastructure system 802 may include infrastructure resources 830 that are used to facilitate the provision of various cloud services offered by the cloud infrastructure system 802. The infrastructure resources 830 may include, for example, processing resources, storage or memory resources, networking resources, etc.

[0128] In one embodiment, to facilitate the efficient provisioning of these resources to support various cloud services provided to various customers by the cloud infrastructure system 802, the infrastructure resources 830 may be bundled into sets of resources or resource modules (also called “pods”). Each resource module or pod may contain a pre-integrated and optimized combination of one or more types of resources. In one embodiment, different pods may be pre-provisioned for different types of cloud services. For example, a first set of pods may be provisioned for a database service, and a second set of pods may be provisioned for a Java service, which may contain a different combination of resources than the pods in the first set of pods. For some services, the resources allocated to provisioning the service may be shared among the services.

[0129] The cloud infrastructure system 802 may internally use services 832 that are shared by different components of the cloud infrastructure system 802 and facilitate the provisioning of services by the cloud infrastructure system 802. These internally shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, and file transfer services.

[0130] The cloud infrastructure system 802 may include multiple subsystems. These subsystems may be implemented in software, hardware, or a combination thereof. As depicted in Figure 8, the subsystem may include a user interface subsystem 812 that enables users or customers of the cloud infrastructure system 802 to interact with the cloud infrastructure system 802. The user interface subsystem 812 may include various different interfaces, such as a web interface 814, an online store interface 816 where cloud services provided by the cloud infrastructure system 802 are advertised and available for purchase by consumers, and other interfaces 818. For example, a customer may use a client device to request one or more services provided by the cloud infrastructure system 802 (service request 834) using one or more of interfaces 814, 816, and 818. For example, a customer may access the online store, browse the cloud services provided by the cloud infrastructure system 802, and place a subscription order for one or more services provided by the cloud infrastructure system 802 that the customer wishes to subscribe to. A service request may include information that identifies the customer and one or more services that the customer wishes to subscribe to.

[0131] In one embodiment, such as the embodiment depicted in Figure 8, the cloud infrastructure system 802 may include an Order Management Subsystem (OMS) 820 configured to process new orders. As part of this process, the OMS 820 may be configured to create a customer account if one has not already been made, receive invoice issuance and / or billing information from the customer which will be used to issue an invoice to the customer in order to provide the requested services to the customer, verify the customer information, confirm the customer order upon verification, and organize various workflows to prepare the order for provisioning.

[0132] Once properly authenticated, OMS820 may then invoke the Order Provisioning Subsystem (OPS)824. OPS824 is configured to provision resources for an order, including processing, memory, and networking resources. Provisioning may involve allocating resources for an order and configuring these resources to facilitate the services requested by the customer order. The manner in which resources are provisioned for an order and the types of resources provisioned may depend on the type of cloud service ordered by the customer. For example, according to one workflow, OPS824 may be configured to determine that a particular cloud service is requested and identify the number of pods that would have been pre-configured for that particular cloud service. The number of pods allocated to an order may depend on the size / volume / level / scope of the requested service. For example, the number of pods allocated may be determined based on the number of users supported by the service, the duration for which the service is requested, etc. The allocated pods may then be customized to suit the specific requesting customer to provide the requested service.

[0133] The cloud infrastructure system 802 may send a response or notification 844 to the requesting customer indicating when the requested service became available. In some cases, information (e.g., a link) that enables the customer to begin using and taking advantage of the requested service may also be sent to the customer.

[0134] The cloud infrastructure system 802 can provide services to multiple customers. For each customer, the cloud infrastructure system 802 is responsible for managing information related to one or more subscription orders received from the customer, maintaining customer data related to the orders, and providing the requested services to the customer. The cloud infrastructure system 802 can also collect usage statistics regarding the use of services subscribed to by the customer. For example, it can collect statistics such as the amount of storage used, the amount of data transferred, the number of users, system uptime, and system downtime. This usage information can be used for billing the customer. Billing can be done, for example, on a monthly cycle.

[0135] The cloud infrastructure system 802 can provide services to multiple customers in parallel. The cloud infrastructure system 802 can store information about these customers, which may include proprietary information. In one embodiment, the cloud infrastructure system 802 includes an identity management subsystem (IMS) 828 configured to manage customer information and provide managed information isolation so that information related to one customer is not accessed by another customer. The IMS 828 may be configured to provide a variety of security-related services, such as identity services, including information access management, authentication and authorization services, and services for managing customer identities and roles and related functions.

[0136] Figure 9 is a block diagram of an exemplary computer system 900 that may be used to implement a particular embodiment. For example, in various embodiments, computer system 900 may be used to implement any of the systems, subsystems, and components described herein. For example, multiple host machines may provide and perform training of the cluster membership classifier subsystem 114 of the detection system 100, as described herein. A computer system such as computer system 900 can be used as a host machine. As shown in Figure 9, computer system 900 includes various subsystems, including a processing subsystem 904 that communicates with a number of other subsystems via a bus subsystem 902. These other subsystems may include a processing acceleration unit 906, an I / O subsystem 908, a storage subsystem 918, and a communication subsystem 924. The storage subsystem 918 may include a non-temporary computer-readable storage medium, including a storage medium 922 and system memory 910.

[0137] The bus subsystem 902 provides a mechanism for various components and subsystems of the computer system 900 to communicate with each other as intended. Although the bus subsystem 902 is schematically shown as a single bus, alternative embodiments of the bus subsystem can utilize multiple buses. The bus subsystem 902 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of various bus architectures. For example, such architectures may include the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MCA) bus, the Extended ISA (EISA) bus, the Video Electronics Standards Institute (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0138] The processing subsystem 904 controls the operation of the computer system 900 and may include one or more processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The processors may include single-core or multi-core processors. The processing resources of the computer system 900 can be organized into one or more processing units 932, 934, etc. A processing unit may include one or more processors, one or more cores from the same or different processors, a combination of cores and processors, or other combinations of cores and processors. In various embodiments, the processing subsystem 904 may include one or more special-purpose coprocessors, such as graphics processors or digital signal processors (DSPs). In various embodiments, some or all of the processing units of the processing subsystem 904 may be implemented using customized circuits such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

[0139] In various embodiments, the processing units of the processing subsystem 904 can execute instructions stored in system memory 910 or computer-readable storage medium 922. In various embodiments, the processing units can execute various program or code instructions and can maintain multiple concurrently running programs or processes. At any given time, some or all of the program code being executed can reside in system memory 910 and / or computer-readable storage medium 922, which may include one or more storage devices. With appropriate programming, the processing subsystem 904 can provide the various functions described above. If the computer system 900 is running one or more virtual machines, one or more processing units may be assigned to each virtual machine.

[0140] In one embodiment, a processing acceleration unit 906 may be provided as an option to perform customized processing to accelerate the overall processing performed by the computer system 900, or to offload a portion of the processing performed by the processing subsystem 904.

[0141] The I / O subsystem 908 may include devices and mechanisms for inputting information into and / or outputting information from or through the computer system 900. Generally, the use of the term input device is intended to include all possible types of devices and mechanisms for inputting information into the computer system 900. User interface input devices may include, for example, keyboards, pointing devices such as mice and trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion sensing and / or gesture recognition devices such as Microsoft Kinect® motion sensors, Microsoft Xbox® 360 game controllers, and devices that provide interfaces for receiving input using gestures and voice commands, allowing users to control and interact with input devices. Furthermore, user interface input devices may also include eye gesture recognition devices, such as the Google Glass® Blink Detector, which detects eye activity from the user (e.g., blinking while taking a photo and / or selecting a menu) and translates eye gestures into input to an input device (e.g., Google Glass®). In addition, user interface input devices may include voice recognition sensing devices that allow the user to interact with a voice recognition system (e.g., Siri® Navigator) through voice commands.

[0142] Other examples of user interface input devices include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphics tablets, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include medical imaging input devices such as computed tomography devices, magnetic resonance imaging devices, positional radiography devices, and medical ultrasound devices. User interface input devices may also include audio input devices such as MIDI keyboards and digital musical instruments.

[0143] Generally, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 900 to a user or another computer. User interface output devices may include non-visual displays such as display subsystems, indicator lights, or audio output devices. Display subsystems may include flat panel devices such as cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. For example, user interface output devices may include, but are not limited to, a variety of display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0144] The storage subsystem 918 provides a repository or datastore for storing information and data used by the computer system 900. The storage subsystem 918 provides a tangible, non-temporary, computer-readable storage medium for storing basic programming and data configurations that provide functionality in various embodiments. The storage subsystem 918 can store software (e.g., programs, code modules, instructions) that provides the above-described functionality when executed by the processing subsystem 904. The software may be executed by one or more processing units of the processing subsystem 904. The storage subsystem 918 may also provide a repository for storing data used in accordance with the teachings of this disclosure.

[0145] The storage subsystem 918 may include one or more non-temporary memory devices, including volatile memory devices and non-volatile memory devices. As shown in Figure 9, the storage subsystem 918 includes system memory 910 and computer-readable storage medium 922. The system memory 910 may include a number of memories, including volatile main random access memory (RAM) for storing instructions and data during program execution, and non-volatile read-only memory (ROM) or flash memory for storing fixed instructions. In some implementations, a basic input / output system (BIOS), which includes basic routines that help transfer information between elements within the computer system 900, such as at startup, may be stored, typically in ROM. The RAM typically contains data and / or program modules currently being operated and executed by the processing subsystem 904. In some implementations, the system memory 910 may include several different types of memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM).

[0146] As an example, and not an limitation, as shown in Figure 9, system memory 910 can load running application programs 912, program data 914, and operating systems 916, which may include various applications such as web browsers, mid-tier applications, and relational database management systems (RDBMS). For example, operating systems 916 may include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS.

[0147] In one embodiment, as described herein, software instructions or code for training the cluster membership classifier subsystem 114 of the detection system 100 may be executed in system memory 910.

[0148] The computer-readable storage medium 922 can store programming and data structures that provide functionality in various embodiments. The computer-readable storage medium 922 can provide storage for computer-readable instructions, data structures, program modules, and other data for the computer system 900. Software (programs, code modules, instructions) that, when executed by the processing subsystem 904, provides the aforementioned functionality may be stored in the storage subsystem 918. As an example, the computer-readable storage medium 922 may include non-volatile memory such as hard disk drives, magnetic disk drives, optical disk drives such as CD-ROMs, DVDs, Blu-ray® discs, or other optical media. The computer-readable storage medium 922 includes, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, and digital videotapes. Computer-readable storage media 922 may also include solid-state drives (SSDs) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, and solid-state ROMs, as well as SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs.

[0149] In one embodiment, the storage subsystem 918 may also include a computer-readable storage medium reader 920 which may be further connected to the computer-readable storage medium 922. The reader 920 may be configured to receive and read data from memory devices such as disks and flash drives.

[0150] In a particular embodiment, the computer system 900 may support virtualization technologies, including but not limited to virtualization of processing and memory resources. For example, the computer system 900 may provide support for running one or more virtual machines. In a particular embodiment, the computer system 900 may run a program, such as a hypervisor, that facilitates the configuration and management of virtual machines. Each virtual machine may be allocated memory, compute (e.g., processors, cores), I / O, and networking resources. Each virtual machine typically runs independently of other virtual machines. A virtual machine typically runs its own operating system, which may or may not be the same as the operating system run by other virtual machines run by the computer system 900. Thus, multiple operating systems may potentially run simultaneously by the computer system 900.

[0151] The communication subsystem 924 provides interfaces to other computer systems and networks. The communication subsystem 924 functions as an interface for receiving data from computer system 900 and transmitting data to other systems. For example, the communication subsystem 924 may enable computer system 900 to establish communication channels with one or more client devices via the internet for receiving information from and transmitting information to client devices.

[0152] The communication subsystem 924 can support both wired communication protocols and / or wireless communication protocols. For example, in one embodiment, the communication subsystem 924 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular phone technology, advanced data network technologies such as 3G, 4G or EDGE (Enhanced Data Communication Speed ​​for Global Evolution), WiFi (IEEE 802.XX family standards, or other mobile communication technologies, or any combination thereof)), a Global Positioning System (GPS) receiver component, and / or other components. In various embodiments, the communication subsystem 924 can provide wired network connectivity (e.g., Ethernet) in addition to, or instead of, the wireless interface.

[0153] The communication subsystem 924 can receive and transmit data in various forms. For example, in various embodiments, in addition to other forms, the communication subsystem 924 can receive input communications in the form of structured and / or unstructured data feeds 926, event streams 928, event updates 930, etc. For example, the communication subsystem 924 may be configured to receive (or transmit) data feeds 926 in real time from users of social media networks and / or other communication services, such as web feeds like Twitter® feeds, Facebook® updates, rich site summary (RSS) feeds, and / or real-time updates from one or more third-party sources.

[0154] In one embodiment, the communication subsystem 924 may be configured to receive data in the form of a continuous data stream, which may include an event stream 928 of real-time events and / or event updates 930 that may be continuous or unbound in nature and have no explicit termination. Examples of applications that generate continuous data include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0155] The communication subsystem 924 may also be configured to communicate data from the computer system 900 to other computer systems or networks. The data can communicate to one or more databases that can communicate with one or more streaming data source computers coupled to the computer system 900 in various different forms, such as structured data feeds 926 and / or unstructured data feeds 926, event streams 928, event updates 930, etc.

[0156] The computer system 900 can be one of many types, including handheld portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), personal computers, workstations, mainframes, kiosks, server racks, or any other data processing systems. Because the nature of computers and networks is constantly changing, the description of the computer system 900 depicted in Figure 9 is intended only as a specific example. Many other configurations are possible with more or fewer components than the system depicted in Figure 9. Based on the disclosures and teachings provided herein, those skilled in the art will understand other methods and / or techniques for implementing various embodiments.

[0157] While specific embodiments have been described, various modifications, changes, alternative structures, and equivalents are possible. The embodiments are not limited to operation within a particular data processing environment, but are free to operate within multiple data processing environments. Furthermore, while some embodiments are described using a specific set of transactions and steps, it will be apparent to those skilled in the art that this is not intended as limitation. While some flowcharts describe operations as sequential processes, many operations can be performed in parallel or concurrently. Furthermore, the order of operations can be rearranged. Processes may include additional steps not shown in the diagrams. The various features and aspects of the embodiments described above can be used individually or in combination.

[0158] Furthermore, while certain embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also possible. Certain embodiments can be implemented using hardware alone, software alone, or a combination thereof. The various processes described herein can be implemented on the same processor or in any combination of different processors.

[0159] Where a device, system, component, or module is described as being configured to perform a certain operation or function, such configuration can be achieved, for example, by designing electronic circuits to perform an operation; by programming programmable electronic circuits (such as a microprocessor) to perform an operation by executing computer instructions or code, or a processor or core programmed to execute code or instructions stored in a non-temporary memory medium, or any combination thereof. Processes can communicate using a variety of techniques, including but not limited to prior art for inter-process communication, different sets of processes may use different techniques, or the same set of processes may use different techniques at different times.

[0160] This disclosure provides specific details to give a complete understanding of the embodiments. However, embodiments may be carried out without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary details to avoid obscuring the embodiments. This description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of other embodiments. Rather, the prior description of embodiments provides to those skilled in the art a possible explanation for carrying out various embodiments. Various modifications may be made in the function and arrangement of elements.

[0161] This specification and drawings are therefore intended to be illustrative rather than restrictive. However, it will be apparent that additions, reductions, deletions, and other modifications and alterations may be made without departing from the broad spirit and scope set forth in the claims. Thus, while specific embodiments are described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims. Modifications and / or variations include any relevant combination of the disclosed features.

[0162] The features disclosed according to the embodiments are summarized below. Example 1 is a computer implementation method, the method comprising: obtaining interaction data generated as part of a specific interaction between a user and a computer application; formatting the interaction data into a formatted sequence containing a bounded sequence of interaction events over time that can be processed by a neural network; inputting the formatted sequence into an encoder model to generate embeddings; inputting the embeddings into a classifier model and outputting a vector of probability values ​​corresponding to multiple classes characterizing the embeddings; and determining a machine actor score for a particular interaction, at least partially based on the vector of probability values, wherein the machine actor score indicates that the particular interaction is invalid, and the method outputs the machine actor score.

[0163] Example 2 is an arbitrary preceding or succeeding embodiment of the method, in which the encoder model is trained as part of an autoencoder including an encoder model and a decoder model, and the encoder model includes a recurrent neural network including a plurality of gradient recurrent unit cells or long-term short-term memory cells.

[0164] Example 3 is an arbitrary preceding or succeeding embodiment of the method, where the autoencoder is a variational autoencoder configured to learn to output an embedding by sampling from a Gaussian probability distribution whose parameters are learned and output by an encoder model.

[0165] Example 4 is an arbitrary preceding or succeeding embodiment of the method, where the encoder model is a deep learning transformation model comprising a transformation encoder and a transformation decoder, the transformation encoder being configured to transform a formatted sequence into a sequence of intermediate outputs, and the transformation decoder being configured to decode the intermediate outputs into an embedded output sequence.

[0166] Example 5 is an arbitrary preceding or succeeding method further comprising: clustering the embeddings with a clustering model separate from the classifier model to form a clustered set of embeddings; training a classifier model to learn embedding cluster membership probabilities using the embeddings as input features and the corresponding cluster labels from the clustering as labels; thereby forming a trained classifier model, wherein inputting the embeddings into the classifier model includes inputting the embeddings into the trained classifier model.

[0167] Example 6 is an arbitrary preceding or succeeding embodiment of the method, where the cluster model and classifier model are part of a single mixed model.

[0168] Example 7 is an arbitrary preceding or succeeding embodiment of a method in which determining a machine actor score includes identifying a first value in a vector of probability values ​​belonging to a first class and identifying a second value in a vector of probability values ​​belonging to a second class, the second value being the largest value in the vector of probability values ​​other than the first value, and determining a machine actor score includes predicting a machine actor score based at least in part on the second value.

[0169] Example 8 is an arbitrary preceding or succeeding method in which determining a first value includes evaluating multiple variances of multiple clusters generated by a classifier model, and assigning the cluster with the largest variance among the multiple variances to the first class.

[0170] Example 9 is an arbitrary preceding or succeeding method further comprising generating a visualization of interaction inputs in the embedding space, optionally as an annotated two-dimensional or three-dimensional projection, and presenting the visualization on a computer display so that the embedding can be qualitatively monitored and addressed as necessary.

[0171] Example 10 is an arbitrary preceding or succeeding method in which a bounded sequence of interaction events includes a sequence of positional and temporal deltas generated by the differences in the positional and temporal elements of the input over a period of time.

[0172] Example 11 is an arbitrary preceding or succeeding embodiment of the method, in which the bounded sequence further includes mouse click data corresponding to a sequence of mouse clicks over a period of time.

[0173] Example 12 is an arbitrary preceding or succeeding method in which a first class of multiple classes belongs to human traffic and a second class of multiple classes belongs to bot traffic.

[0174] In some examples, a cursor trace embedding system includes one or more processors and non-temporary memory that, when executed by one or more processors, stores computer-executable instructions that cause one or more processors to perform operations including those described above.

[0175] In some examples, a computer program product includes a computer-readable storage medium having program instructions, the program instructions being executable by one or more processors, causing one or more processors to perform operations including those described above.

Claims

1. A method for detecting automated web traffic performed by one or more processors, The aforementioned method, This involves obtaining interaction data generated as part of a specific interaction between a user and a computer application, The interaction data is formatted into a formatted sequence containing a bounded sequence of interaction events over a time period that can be processed by a neural network. To generate the embedding, the formatted sequence is input to the encoder model, The embedding is input into a classifier model, and a vector of probability values ​​corresponding to multiple classes characterizing the embedding is output. This includes determining the machine actor score for the particular interaction based at least partially on the vector of probability values, A high machine actor score corresponds to the prediction that the particular interaction is invalid. The method further comprises outputting the machine actor score.

2. The method according to claim 1, wherein the encoder model is trained as part of an autoencoder including the encoder model and a decoder model, and the encoder model includes a recurrent neural network including a plurality of gradient recurrent unit cells or long-term short-term memory cells.

3. The method according to claim 2, wherein the autoencoder is a variational autoencoder configured to learn parameters to output an embedding by sampling from a Gaussian probability distribution learned and output by the encoder model.

4. The encoder model is a deep learning transformation model comprising a transformation encoder and a transformation decoder, wherein the transformation encoder is configured to transform the formatted sequence into an intermediate output sequence, and the transformation decoder is configured to decode the intermediate output into an embedded output sequence, according to any one of claims 1 to 3. method.

5. The embeddings are clustered using a clustering model separate from the aforementioned classifier model to form a clustered set of embeddings. The method further includes using the aforementioned embeddings as input features and using the corresponding cluster labels from the clustering as labels to train the classifier model to learn the embedding cluster membership probabilities, thereby forming a trained classifier model. The method according to any one of claims 1 to 4, wherein inputting the embedding into a classifier model includes inputting the embedding into a trained classifier model.

6. The method according to claim 5, wherein the clustering model and the classifier model are part of a single mixed model.

7. Determining the aforementioned machine actor score means Identifying the first value of the vector of probability values ​​belonging to the first class, This includes identifying a second value in the vector of probability values ​​belonging to the second class, The second value is the maximum value in the vector of probability values ​​other than the first value. The method according to any one of claims 1 to 6, wherein determining the machine actor score further comprises predicting the machine actor score based at least in part on the second value.

8. Identifying the first value mentioned above is Evaluating multiple variance values ​​of multiple clusters generated by the aforementioned classifier model, The method according to claim 7, further comprising assigning the cluster having the largest variance among the plurality of variances to the first class.

9. To generate a visualization of interaction inputs in the embedded space as an annotated two-dimensional or three-dimensional projection, The method according to any one of claims 1 to 8, further comprising presenting a visualization on a computer display so that the embedding can be qualitatively monitored and addressed as necessary.

10. The method according to any one of claims 1 to 9, wherein the bounded sequence of interaction events includes a sequence of positional and temporal deltas generated by the differences in the positional and temporal elements of the input over a certain period of time.

11. The method according to claim 10, wherein the bounded sequence further includes mouse click data corresponding to the sequence of mouse clicks over a certain period of time.

12. The method according to claim 7, wherein the first class of the plurality of classes belongs to human traffic, and the second class of the plurality of classes belongs to bot traffic.

13. An automated web traffic detection system, One or more processors, It includes non-temporary memory for storing computer executable instructions, When the computer executable instruction is executed by the one or more processors, the one or more processors will: This involves obtaining interaction data generated as part of the user's interaction with computer applications, The interaction data is formatted as a bounded sequence of interaction events over time, which can be modeled and formatted by a neural network to form a sequence. The formatted sequence is input to an encoder model configured to generate an embedding from the bounded sequence, The output of the encoder model is to generate an embedding, The embedding is input to a classifier model configured to predict a vector of probability values ​​corresponding to multiple classes using the embedding, The output of the classifier model is to predict the vector of the probability values, The machine actor score of the interaction is determined from the vector of probability values, An automated web traffic detection system that performs the following actions: outputting the aforementioned machine actor score.

14. The automated web traffic detection system according to claim 13, wherein the encoder model is trained as part of an autoencoder including the encoder model and a decoder model, and the encoder model includes a recurrent neural network including a plurality of gradient recurrent unit cells or long-term short-term memory cells.

15. The embeddings are clustered using a clustering model separate from the aforementioned classifier model to form a clustered set of embeddings. The method further includes training the classifier model using the clustered embedding set to form a trained classifier model, The automated web traffic detection system according to claim 13 or 14, wherein inputting the embedding into a classifier model includes inputting the embedding into a trained classifier model.

16. Determining the aforementioned machine actor score means Identifying the first value of the vector of probability values ​​belonging to the first class, This includes identifying a second value in the vector of probability values ​​belonging to the second class, The second value is the maximum value in the vector of probability values ​​other than the first value. The automated web traffic detection system according to any one of claims 13 to 15, wherein determining the machine actor score further comprises predicting the machine actor score based at least in part on the second value.

17. Identifying the first value mentioned above is Evaluating multiple variance values ​​of multiple clusters generated by the aforementioned classifier model, The automated web traffic detection system according to claim 16, further comprising assigning the cluster having the largest variance among the plurality of variances to a first class among the plurality of classes.

18. When the computer executable instruction is executed by the one or more processors, the one or more processors will: Visualization of interaction inputs in the embedded space, potentially as annotated two-dimensional or three-dimensional projections, An automated web traffic detection system according to any one of claims 13 to 17, further comprising performing an operation that includes qualitatively monitoring the embedding and presenting a visualization on a computer display so that it can be taken action as necessary.

19. The automated web traffic detection system according to any one of claims 13 to 18, wherein the bounded sequence of interaction events includes a sequence of positional and temporal deltas generated by differences in the positional elements of the input over a period of time.

20. A program that causes one or more processors to execute the method according to any one of claims 1 to 12.

Images