Anomaly detection system and method

Abstract

Some embodiments utilize a novel procedure for training an artificial intelligence system (e.g., a set of deep neural networks) for anomaly detection in applications such as natural language processing and computer security. Selected token sequences from a training corpus are distorted according to at least one of a plurality of predefined sequence transformations before being fed to a sequence analyzer. The sequence analyzer is further trained to correctly guess which transformation was used to generate each input token sequence.

Description

[Technical Field]

[0001]

[0001] The present invention relates to artificial intelligence, and more particularly to a system and method for automatically detecting anomalies in data for application fields of natural language processing and computer security. [Background technology]

[0002]

[0002] Artificial intelligence (AI) and machine learning technologies are increasingly being used to process large amounts of data, particularly in application areas such as pattern recognition, automatic classification, and anomaly detection. Anomaly detection involves identifying specimens that deviate significantly from a standard or "normal" collectively defined by a group of criteria. Anomaly detection can present considerable technical challenges in the case of complex data, in which case the meaning and boundaries of normality may not be clear or defined in advance. Modern artificial intelligence systems (e.g., deep neural networks) have been shown to perform well to such challenges by using their ability to automatically infer sophisticated models from data.

[0003]

[0003] However, implementing machine learning to train anomaly detectors presents its own set of technical challenges. Some conventional methods can result in extremely computationally expensive training, require very large training corpora, and be unstable and / or inefficient. Thus, there has been considerable interest in developing novel detector architectures and methods for training anomaly detectors for the application areas of natural language processing and computer security. [Overview of the Initiative]

[0004]

[0004] In one embodiment, a computer-implemented anomaly detection method includes the step of using at least one hardware processor of a computer system to apply a selected transformation to a training token sequence in response to selecting a training token sequence from a training corpus of token sequences and selecting a transformation from a predetermined set of sequence transformations to generate a modified token sequence. The method further includes the step of running a sequence analyzer having a set of tunable parameters and configured to determine a transformation prediction indicator according to the modified token sequence, wherein the transformation prediction indicator indicates the possibility that a selected transformation has been applied to generate the modified token sequence. The method further includes the step of adjusting at least one parameter of the set of tunable parameters according to the transformation prediction indicator in response to determining the prediction indicator, and the step of using the sequence analyzer to determine whether a target token sequence is anomaly in response to adjusting at least one parameter.

[0005]

[0005] In another embodiment, the computer system comprises at least one hardware processor configured to apply the selected transformation to the training token sequence to generate a modified token sequence in response to selecting a training token sequence from a training corpus of token sequences and selecting a transformation from a predetermined set of sequence transformations. The at least one hardware processor is further configured to run a sequence analyzer having a set of tunable parameters and configured to determine a transformation prediction indicator according to the modified token sequence, the transformation prediction indicator indicating the possibility that the selected transformation has been applied to generate the modified token sequence. The at least one hardware processor is further configured to adjust at least one parameter of the set of tunable parameters according to the transformation prediction indicator in response to determining the prediction indicator, and to use the sequence analyzer to determine whether the target token sequence is abnormal in response to adjusting at least one parameter.

[0006]

[0006] In another embodiment, when a non-temporary computer-readable medium is executed by at least one hardware processor of a computer system, it stores instructions that cause the computer system to apply a selected transformation to the training token sequence to generate a modified token sequence, in response to selecting a training token sequence from a training corpus of token sequences and selecting a transformation from a predetermined set of sequence transformations. The instructions further cause the computer system to run a sequence analyzer having a set of tunable parameters and configured to determine a transformation prediction indicator according to the modified token sequence, the transformation prediction indicator indicating the possibility that the selected transformation has been applied to generate the modified token sequence. The instructions further cause the computer system to adjust at least one parameter of the set of tunable parameters according to the transformation prediction indicator in response to determining the prediction indicator, and to use the sequence analyzer to determine whether the target token sequence is abnormal in response to adjusting at least one parameter.

[0007]

[0007] The above-mentioned aspects and advantages of the present invention will be better understood by reading the following detailed description and referring to the drawings. [Brief explanation of the drawing]

[0008] [Figure 1]

[0008] This figure shows a set of client systems that cooperate with a utility server when detecting anomalies, according to some embodiments of the present invention. [Figure 2]

[0009] This figure shows exemplary operation of an anomaly detector according to several embodiments of the present invention. [Figure 3]

[0010] This figure shows exemplary training of an anomaly detector according to several embodiments of the present invention. [Figure 4]

[0011] A diagram showing an exemplary operation of an input corrector according to some embodiments of the present invention. [Figure 5]

[0012] A diagram showing an exemplary token embedding space according to some embodiments of the present invention. [Figure 6]

[0013] A diagram showing an exemplary sequence conversion according to some embodiments, wherein the illustrated conversion includes nudging the representative vector of a selected token. [Figure 7]

[0014] A diagram showing an exemplary structure of a sequence classifier according to some embodiments of the present invention. [Figure 8]

[0015] A diagram showing an exemplary sequence of steps performed during training of an anomaly detector according to some embodiments of the present invention. [Figure 9]

[0016] A diagram showing an exemplary sequence of steps performed by a trained anomaly detector according to some embodiments of the present invention. [Figure 10]

[0017] A diagram showing an exemplary computing appliance configured to perform some of the methods described herein.

Mode for Carrying Out the Invention

[0009]

[0018] In the following explanation, it should be understood that all connections between structures can be directly operable or indirectly operable through intermediate structures. A set of elements contains one or more elements. Any description of an element should be understood to refer to at least one element. Multiple elements contain at least two elements. Unless otherwise specified, any use of "OR" refers to a non-exclusive OR. Unless otherwise required, any method steps described do not necessarily have to be performed in a specific illustrated order. A first element derived from a second element (e.g., data) includes a first element equal to the second element, as well as a first element produced by processing the second element and, optionally, other data. Making a decision or judgment according to a parameter includes making a decision or judgment according to the parameter and, optionally, according to other data. Unless otherwise specified, some quantity / data labels can be the quantity / data itself or a label different from the quantity / data itself. A computer program is a sequence of processor instructions that perform a task. The computer programs described in some embodiments of the present invention may be standalone software entities or sub-entities of other computer programs (e.g., subroutines, libraries). Computer-readable media include non-temporary media such as magnetic, optical, and semiconductor storage media (e.g., hard drives, optical discs, flash memory, DRAM), as well as communication links such as conductive cables and optical fiber links. According to some embodiments, the present invention provides, among other things, a computer system comprising hardware (e.g., one or more processors) programmed to carry out the methods described herein, and computer-readable media encoding instructions for carrying out the methods described herein.

[0010]

[0019] The following description illustrates embodiments of the present invention as examples, not necessarily as limitations.

[0020] Figure 1 shows an exemplary set of client systems 10a-c that can work with a utility server 12 to detect anomalies in data according to several embodiments of the present invention. For the purposes of this specification, an anomaly should be understood as an item that deviates significantly from the standard or "normal" as collectively represented by a reference set / corpus of items. This description focuses on detecting anomaly token sequences, such as anomaly text fragments or computing event sequences. In such embodiments, exemplary anomaly detection includes determining that the author of the target text differs from the reference text. Another exemplary anomaly detection includes determining that a sequence of computing events deviates from the normal behavior of each computer and possibly indicates a security breach or the presence of malicious software. Several exemplary anomaly detection use case scenarios are described below.

[0011]

[0021] Client systems 10a-c generally represent any electronic appliance having a processor, memory, and communication interfaces. Exemplary client systems 10a-c include, among other things, personal computers, enterprise mainframe computers, servers, laptops, tablet computers, mobile telecommunications devices (e.g., smartphones), media players, TVs, game consoles, home appliances, and wearable devices (e.g., smartwatches). The illustrated client systems are interconnected by a communication network 15, which may include a local area network (LAN) and / or a wide area network (WAN) such as the Internet. Servers 12 generally represent a set of communicatively coupled computer systems that may or may not be physically close to each other.

[0012]

[0022] Figure 2 shows the operation of an exemplary anomaly detector 20 according to several embodiments of the present invention. The anomaly detector 20 may be implemented as software, i.e., as a set of computer programs containing instructions that cause each appliance to perform its respective task when loaded into memory and executed by the hardware processor of a computing appliance such as a personal computer or smartphone. However, those skilled in the art will understand that such embodiments do not mean limiting. Rather, the detector 20 may be implemented as any combination of software and hardware. For example, some or all of the functions of the detector 20 may be implemented in firmware and / or dedicated hardware such as a field-programmable gate array (FPGA) or other application-specific integrated circuit (ASIC). Each hardware module may be highly optimized for its respective function and may, for example, directly implement a particular version of a deep neural network architecture, thus enabling processing speeds considerably faster than those achievable with respect to a general-purpose processor. Furthermore, those skilled in the art will understand that the anomaly detector 20 and / or separate components of a computer system configured to train the detector 20, as described below, may run on separate but communicatively coupled machines and / or on separate hardware processors of the same computer system.

[0013]

[0023] Anomaly detector 20 may be configured to receive a computer-readable encoding of a target token sequence 22 and, in response, output an anomaly indicator 26 indicating whether each token sequence 22 is anomaly. Exemplary token sequences include, among other things, ordered arrays of tokens such as fragments of text constructed in natural languages ​​such as English or Chinese. Without losing generality, the following description will focus primarily on examples of natural language processing, and exemplary tokens may include, among other things, individual words, phrases, sentences, numbers, punctuation marks (e.g., ?!;: / (),...), special characters (e.g., $#%), abbreviations (e.g., USA, LOL, IMHO, etc.), social media handles (e.g., @POTUS), hashtags, and emoticons. Those skilled in the art will understand that the systems and methods described herein may be adapted, among other things, to process other types of token sequences such as sequences of computing events and sound sequences (e.g., music, speech).

[0014]

[0024] An exemplary anomaly indicator 26 includes a numerical score indicating the likelihood that each target token sequence is anomaly. The score may be Boolean (e.g., YES / NO) or may fluctuate gradually between a given boundary (e.g., between 0 and 1). In such an example, a larger value indicates a higher likelihood that each sequence is anomaly. An alternative anomaly indicator 26 may include a classification label (e.g., anomaly, normal, unknown, suspected) indicating the category of token sequences to which sequence 22 is likely to belong.

[0015]

[0025] In one exemplary scenario, separate instances of the anomaly detector 20 may run on each client system 10a-c, so that each client can perform its own anomaly detection activity locally and independently. In an alternative embodiment, the anomaly detector 20 may run on a utility server 12, so that the utility server 12 can perform centralized anomaly detection activity on behalf of the multiple client systems 10a-c. In such an embodiment, the server 12 may receive the encoding of the target token sequence 22 from each client system 10a-c and return the respective anomaly indicator 26 to the respective client. In such an example, clients 10a-c may access the anomaly detection service via a web interface exposed by the utility server 12.

[0016]

[0026] Figure 3 shows exemplary components of an anomaly detector, such as a sequence analyzer 42 connected to a decision module 44. In some embodiments, the sequence analyzer 42 comprises an artificial intelligence (AI) system, such as a deep neural network trained on a corpus 18 of reference token sequences. In a natural language processing scenario, the corpus 18 may include a collection of text fragments written in natural language (e.g., English). More specific examples of the corpus 18 may consist of a collection of texts by a particular author, a collection of electronic messages (e.g., short message service - SMS messages, emails, social media posts, etc.), a collection of texts on a particular topic or area of ​​interest (e.g., business news, sports, the Middle East, etc.), and a collection of texts written in a particular style (e.g., fiction, poetry, science articles, news, etc.). Individual corpus items may be tagged, labeled, and / or annotated, for example, using metadata. Exemplary metadata may include markers of membership for an item to a selected class / category (e.g., email messages sent by a particular user, financial news, etc.). Corpus 18 can be organized and stored in any format well known in the art, for example, as a relational database, a simple list, or as structured data specified in XML or JSON format.

[0017]

[0027] The contents of Corpus 18 collectively define a reference pattern or "normal" pattern for communication, and in some embodiments, the anomaly detector 20 may be trained to build an internal model of each reference pattern and, in response, determine whether a target text fragment fits the learned pattern. When it is found that the target token sequence 22 does not fit the reference pattern corresponding to "normal" text (according to Corpus 18), the sequence 22 is considered an anomaly and may be reported as such via an anomaly marker 26.

[0018]

[0028] In some embodiments, the training of the anomaly detector 20 is carried out by a separate dedicated computer system, shown as the AI ​​training appliance 14 in Figure 1. The appliance 14 may be communicatively coupled to a utility server 12 and / or client systems 10a-c and may have dedicated hardware such as a graphics processing unit (GPU) farm to facilitate the computationally intensive training procedure. The term “training” is typically used in the art to describe a machine learning procedure in which a variety of training inputs are presented to an artificial intelligence system (e.g., a neural network), and the AI ​​system is gradually adjusted according to the output produced by each input. For each training input / batch, training may include processing each input to produce a training output, determining the value of a problem-specific utility function according to each training output and / or input, and adjusting a set of parameters for each AI system according to each utility value. Adjusting the parameters may aim to maximize (and in some cases minimize) the utility function. In one example of training a neural network, the tunable parameters may include a set of synaptic weights, while the utility function may quantify the deviation of the training output from an expected or desired output. In one such example, training may involve tuning synaptic weights, and possibly other network parameters, to bring the training output closer to a desired output corresponding to each training input. Known training flavors include, among others, supervised, unsupervised, self-supervised, and reinforcement learning. In some embodiments, the number of tunable parameters for a typical detector 20 can vary from thousands to millions. Successful training may generate a set of optimized detector parameter values ​​24 (Figure 2), which can be used to instantiate local instances of the anomaly detector 20 running on client systems 10a-c and / or utility server 12.

[0019]

[0029] The training of the detector 20 is schematically shown in Figure 3. Those skilled in the art will understand that not all of the illustrated components are necessary to run the AI ​​training appliance 14 on the same hardware processor or physical machine.

[0020]

[0030] In some embodiments of the present invention, at least some of the samples contained within the training corpus 18 are strained before being fed into the anomaly detector 20, and the detector 20 is then trained to identify the type of strain applied. In one example shown in Figure 3, the input modifier 40 is configured to receive a selected training token sequence 32 from the training corpus 18 and output a modified token sequence 34 which includes the result of applying at least one of a predetermined set of sequence transformations 30 to the training sequence 32.

[0021]

[0031] An exemplary sequence transformation 30 includes, among other things, replacing a selected subset of tokens in sequence 32 with alternative tokens, deleting a selected subset of tokens from sequence 32, inserting a set of tokens into sequence 32, and rearranging a selected subset of tokens in sequence 32. The subset of tokens targeted for modification by each transformation may be selected according to the position of each token in the training sequence. The targeted positions may be indicated by a binary mask, where 0 marks the positions of tokens to be left immutable and 1 marks the positions affected by each transformation. For example, a rearrangement transformation defined by the mask [0 0 1 0 1] may transform the token sequence "They were prepared to leave" into the modified token sequence "They were leave to prepared", in which the third token is replaced with the fifth token.

[0022]

[0032] In alternative embodiments, the tokens targeted by the transformation 30 may be selected according to the type of each token. For example, some transformations may target tokens having a specific part of speech (e.g., noun, verb, adjective) or a specific grammatical role (e.g., subject of a sentence). One such exemplary transformation may replace a verb with an alternative verb or verb phrase. Each alternative token or token sequence may be selected to be a synonym or antonym of the target token / sequence. More advanced examples of sequence transformations 30 may include paraphrasing, i.e., replacing an entire token sequence with an alternative sequence while preserving its meaning. An example of a paraphrasing transformation includes replacing the training sequence "Kids by the lake were being eaten alive by mosquitoes" with the modified sequence "Mosquitoes ferociously attacked the kids by the lake".

[0023]

[0033] Similar transformations may be applied in computer security embodiments where each token in sequence 32 comprises an individual computing event. For example, exemplary transformation 30 may remove tokens of type "create process" from training sequence 32. An equivalent of paraphrasing in such embodiments may include replacing the target sequence of events with an alternative sequence of events that would bring each computer system to the same final state.

[0024]

[0034] Figure 4 shows exemplary operation of an input modifier 40 implementing a plurality of predefined sequence transforms 30 according to several embodiments of the present invention. The training sequence 32 contains tokens 35a-e, and in the present example, tokens 35a-e are individual words. In the illustrated embodiments, each transform 30 comprises replacing a set of tokens from the training sequence 32 with substitute tokens. Each transform 30 may be defined by a distinct binary mask, such as the one shown, which indicates a distinct set of positions in the training sequence 32 that are targeted for token substitution. In the illustrated example, applying the T2 mask to the sequence 32 masks the third and fifth tokens of the sequence 32, effectively marking the tokens for substitution.

[0025]

[0035] In some embodiments, the input modifier 40 further comprises a token generator 41 configured to output a set of alternative tokens to replace the masked tokens in sequence 32. In the illustrated example, the token generator outputs alternative tokens 35f and 35g to replace tokens 35c and 35e, respectively. A simple embodiment of the generator 41 may be configured to randomly draw alternative tokens from a reference pool. In more advanced embodiments, the token generator 41 may comprise a dictionary / thesaurus and be configured to output synonyms or antonyms for each masked token. In another exemplary embodiment, the generator 41 may determine alternative tokens according to the masked token, and further according to the context of the masked token, the context consisting of fragments of sequence 32 preceding the masked token and / or fragments of sequence 32 following the masked token. For example, in the example of Figure 4, the context of token 35c ("ready") may include tokens 35b and 35d ("were to"). Such embodiments may utilize a statistical language model to generate alternative tokens according to the probability of occurrence of each alternative token within the context of the masked token. In other words, the token generator 41 may be configured to generate a valid alternative token given the context of each token sequence.

[0026]

[0036] An exemplary embodiment of a token generator 41 that generates valid alternative tokens comprises an AI system (e.g., a set of deep neural networks) trained on a corpus of token sequences representing various anomaly detection applications. Such a version of the generator 41 may output alternative tokens according to a subsequence of tokens preceding a masked token in sequence 32. In the illustrated example, the trained generator 41 can generate alternative token 35g ("leave") as having a high probability following the sequence of tokens 35a-d ("they were ready to"). An example of such an AI-based token generator 41 implements the Bidirectional Encoder Representation From Transformers (BERT) model for language, as described, for example, in J. Devlin et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," arXiv:1810.04805. In some embodiments, during training, the token generator 41 is connected to the sequence analyzer 42 as shown in Figures 3-4, and the token generator 41 is trained together with the sequence analyzer 42 in the sense that the generator 41 has a set of tunable functional parameters (e.g., synaptic weights) separate from the tunable parameters of the sequence analyzer 42, and at least one tunable parameter of the trained generator 41 is adjusted according to the output of the sequence analyzer 42.

[0027]

[0037] Another exemplary sequence transformation 30 implemented by the input modifier 40 may include manipulation of token embedding vectors. In such embodiments, the modified sequence 34 may include an array of embedding vectors rather than the sequence of tokens itself. The input modifier 40 may include a token encoder configured to determine a set of coordinates indicating the position of each token in the training sequence 32 in an abstract multidimensional vector space commonly referred to in the art as the embedding space. Each set of coordinates collectively defines the token embedding vectors associated with each token. Figure 5 shows an exemplary token embedding space 50 and a set of token embedding vectors 55a-b representing tokens 35a-b, respectively.

[0028]

[0038] An exemplary embedding space is spanned by a set of axes, each axis representing a distinct token feature, or a linear combination of token features (for example, in a principal component / singular value decomposition embodiment). In embodiments configured to detect anomalies in a sequence of computing events, the token features may include various event features of each event (e.g., event type, routing indicator, network address, etc.). In preferred embodiments, tokens are embedded according to their position in the training sequence, in other words, according to their context. In such cases, the embedding space 50 may include an abstract context space, where two tokens that primarily occur in similar contexts are placed together relatively close to each other. Several such embeddings are well known in the art, including word2vec, GloVe, and BERT, among others. To generate the embedding vector representations 55a~b, the token encoder must be trained with respect to a corpus of token sequences, which may coincide with a training corpus 18, i.e., the corpus used to train the sequence analyzer 42. Training can proceed according to any method well known in the art, for example, according to the bag-of-words and / or skip-gram algorithms. In some embodiments, the token encoder is trained together with the analyzer 42, in the sense that the tunable parameters of the token encoder are adjusted according to the output of the sequence analyzer 42.

[0029]

[0039] As shown in Figure 6, several sequence transformations 30 (Figure 3) for distorting the training sequence 32 can act directly on the embedding vector. Exemplary transformation T jThis can be used to change the original token embedding vector 55c, which represents the selected tokens of the training sequence 32, into the modified vector 55d. An exemplary embedding transformation involves nudging the vector by a small amount ε along one of the axes or along a predetermined direction specific to the transformation. Another exemplary transformation may involve rotation and reflection around a given plane. Each transformation may be applied to all tokens in the training sequence 32, or only to selected tokens identified, for example, by a binary mask (as shown above) or other selection criteria.

[0030]

[0040] In some embodiments, the sequence analyzer 42 is configured to process an input token sequence and generate a set of predictive indicators, including transform predictive indicators 36 and token predictive indicators 38 determined according to the input token sequence. The transform predictive indicators 36 indicate which sequence transform 30 is most likely to have been used to generate the input token sequence. In exemplary embodiments, the transform predictive indicators 36 include a plurality of numerical scores P(T1), P(T2), ... P(T k ) includes each score P(T j ) is used to generate each input token sequence, each transformation T j This indicates the possibility that it has been applied. For example, indicator 36 may include a separate score for each separate sequence transformation 30 implemented by the input modifier 40. Score P(T j ) can be scaled between predetermined boundaries (for example, between 0 and 1), with larger values ​​indicating a higher probability.

[0031]

[0041] Next, several embodiments of the token prediction indicator 38 indicate which tokens in the input sequence are likely to have been modified by the input modifier 40. In exemplary embodiments, the token prediction indicator 38 includes a plurality of numerical scores S1, S2, ..., and score S n This indicates that the nth token of the input sequence may have been modified by the input modifier 40. In the training process shown in Figure 3, each score S nThis may indicate that the nth token of the modified token sequence 34 may be different from the nth token of each training sequence 32.

[0032]

[0042] From an intuitive standpoint, the transform prediction marker 36 represents an attempt to infer which transform was used to distort the training sequence 32, and the token prediction marker 38 represents an attempt to infer which individual tokens were destroyed. Although the markers 36 and 38 appear to convey redundant information (after all, each transform has a specific token mask), they are unrelated in the sense that they are generated by separate subsystems of the sequence analyzer 42 (e.g., separate groups of neurons in a deep neural network). Furthermore, there is no prior knowledge built within the sequence analyzer 42 of the association or correlation between the sequence transform 30 and its associated token mask. Instead, the analyzer 42 may automatically learn such correlations during training. Some embodiments rely on the observation that using both markers 36 and 38 can significantly accelerate the training of the sequence analyzer 42, for example, by speeding up learning and allowing the use of a considerably smaller training corpus to achieve similar anomaly detection performance as using only one of the markers 36 and 38.

[0033]

[0043] An exemplary architecture of the sequence analyzer 32 is shown in Figure 7, comprising a stack of layers / neural network modules, where each layer receives the output of the previous layer / module and provides the input to the next layer in the stack. The sequence analyzer 32 may receive input in the form of an array of token representations 48, each token representation containing a vector of numbers characterizing each token in the input sequence. In a training scenario, each token representation 48 in Figure 7 may represent a distinct token in the modified sequence 34 (Figure 3). In one exemplary embodiment using what is called one-hot coding in the art, the token representation 48 contains an N×1 vector, where each row represents a distinct token type, N represents the size of the token vocabulary, and non-zero elements indicate that each token is of its respective token type. The exemplary size N of the token vocabulary ranges from hundreds to millions in a particular application domain. In another example, each token representation 48 may contain token embedding vectors generated according to embedding algorithms well known in the art. See, for example, the embedding vectors 55a-b in Figure 5 discussed above.

[0034]

[0044] Each of the consecutive layers L i Each layer transforms the input received from the previous layer according to a set of parameters specific to that layer (e.g., activation, weights, biases), generating an internal vector 49 whose size and range of values ​​may vary among the separate layers / modules of the analyzer 32. For example, some layers achieve dimensionality reduction of their respective input vectors, as in the case of pooling or loss layers. The type and architecture of each layer may differ across each embodiment.

[0035]

[0045] An exemplary architecture of the sequence analyzer 42 includes a convolutional neural network (CNN) layer, followed by a dense layer (i.e., a fully connected layer) further coupled to a rectifier (e.g., ReLU or other activation function) and / or a loss layer. Alternative embodiments may include a CNN layer fed within a recurrent neural network (RNN), followed by a fully connected layer and a ReLU / loss layer. The convolutional layer efficiently multiplies the internal vector 49 with a matrix of weights, called filters in the art, to generate an embedding tensor, such that each element of each tensor has a contribution from the selected token, but also has a contribution from another token adjacent to the selected token. Thus, the embedding tensor represents the input token sequence collectively with a coarser granularity than the individual tokens. The filter weights are adjustable parameters that can be adjusted during the training process.

[0036]

[0046] Recurrent neural networks (RNNs) form a special class of artificial neural networks where the connections between network nodes form a directed graph. Some flavors of RNNs, including in particular long short-term memory (LSTM) networks and graph neural networks (GNNs), are well known in the art. A typical RNN includes a set of hidden units (e.g., individual neurons), and the network topology is specifically configured such that each hidden unit receives not only an input (e.g., an embedding vector) characterizing each token m j but also an input provided by an adjacent hidden unit which receives an input characterizing a token m j preceding token m j-1 in the input token sequence. As a result, the output of each hidden unit is affected not only by each token m j but also by preceding token m j-1It is also affected by this. In other words, the RNN layer can process information about each token in the context of the previous token. A bidirectional RNN architecture can process information about each token in the context of both the previous and subsequent tokens in the input token sequence.

[0037]

[0047] A further exemplary embodiment of the sequence analyzer 42 may comprise a stack of transformer neural network layers. The transformer architecture is described, among other things, in A. Vaswani et al., "Attention is all you need," arXiv:1706.03762. For each input token sequence, the transformer layer may generate a sequence of contextualized token embedding vectors, where each token embedding vector h j This is multiple (for example, all) tokens in the input sequence m j The information from is encoded. The output of the transformer layer, referred to in the art as a prediction head, can be fed into a number of separate classifier modules (e.g., a dense layer) shown as blocks H1 and H2 in Figure 7. Heads H1 and H2 can then output a transform prediction marker 36 and a token prediction marker 38, respectively.

[0038]

[0048] Figure 8 shows an exemplary sequence of steps performed by the AI ​​training appliance 14 (Figure 1) to train an anomaly detector 20 according to several embodiments of the present invention. In response to selecting a training sequence 32 from the corpus 18, the sequence of steps 204-206 may select a transformation (for example, randomly) from an available set of sequence transformations 30, apply each transformation to the training sequence 32, and thus generate a modified token sequence 34.

[0039]

[0049] The modified sequence 34 is fed to the sequence analyzer 42, which processes the modified sequence 34 to generate predictive markers 36 and / or 38. Step 208 may then determine a utility function according to the training sequence 32 and / or predictive markers 36 and / or 38. An exemplary utility function, also called loss in the field of machine learning, may be expressed as follows:

[0040] Σ χ U(θ,χ) [1] In the above equation, χ represents the training sequence, and θ represents the set of tunable parameters. Training may involve adjusting the parameter θ in a direction that minimizes the utility U.

[0041]

[0050] A simple utility function can quantify the deviation of the analyzer 42's output from a desired output. For example, an exemplary utility function could indicate whether the analyzer 42 correctly guessed which transformation was applied in step 206 and / or which tokens of the original training sequence 32 were destroyed in step 206, and could penalize the analyzer 42 for incorrect guesses.

[0042]

[0051] Some embodiments rely on the observation that combining utilities determined according to labels 36 and 38 can facilitate training and / or yield a more powerful sequence analyzer 42. A preferred embodiment may use an aggregate utility function that combines a sequence-level component (indicating the selection of sequence transformations) with a token-level component (indicating whether each individual token has been destroyed or not).

[0043] U=λ¹U S +λ2U T [2] In the above equation, U S and U Tλ1 and λ2 represent the sequence-level and token-level components, respectively, and are weights that allow modification of the relative contributions of each utility function. In some embodiments that implement the maximum likelihood training method,

[0044]

number

[0045] In the above formula

[0046]

number

[0047] This represents the expected value,

[0048]

number

[0049] is a distorted sequence

[0050]

number

[0051] Sequence transformation T κ This represents the probability generated by the application of (see, for example, the transformation prediction marker 36 in Figure 3), θ A This generally represents the adjustable parameters of the sequence analyzer 42. On the other hand,

[0052]

number

[0053] In the above formula

[0054]

number

[0055] This is where token i of the training sequence χ is transformed into sequence T. κ This represents the probability affected by the application of the formula (see, for example, transformation prediction marker 38 in Figure 3).

[0052] In some embodiments, components of the input modifier 40 (such as a token generator 41 and / or token encoder configured to generate token embedding vectors) are trained together with the sequence analyzer 42. Such embodiments are described above U S and U T In addition, generator utility functions can be used.

[0056] U=λ¹U S +λ2U T +λ3U G [5] In the above equation, λ3 is the generator utility function U G This represents another weight used to adjust the contribution of global utilities,

[0057]

number

[0058] In the above equation, θ G This generally represents the adjustable parameters of the token generator 41.

[0059]

number

[0060] , token t i The probability that token t appears in the modified sequence 34, or in other words, the probability that token t i This represents the probability that the modified sequence 34 is valid within its context.

[0053] Another step 210 is to determine the parameter θ according to the determined utility function. A and / or θ GThe set of parameters can be adjusted. Such adjustments can implement a backpropagation procedure by gradient descent or any other maximum likelihood search that aims to minimize a chosen utility function. Training can continue until a termination condition is met, for example, until a predetermined number of training sequences have been analyzed for a predetermined number of epochs, or until a predetermined level of anomaly detection performance is demonstrated (step 212). In response to successful training, the optimal values ​​of the tunable parameters of the sequence analyzer 42 (e.g., synaptic weights) are exported in the form of detector parameter values ​​24 (Figure 2) and sent to client systems 10a-c.

[0061]

[0054] Figure 9 shows an exemplary sequence of steps performed by client systems 10a-c and / or utility server 12 to detect anomalies according to some embodiments of the present invention. The sequence of steps 222-224 prepares the detector 20 for operation by receiving detector parameter values ​​24 from the AI ​​training appliance 14 (Figure 2) and instantiating local instances of the detector 20 with the respective values. Then, for each target token sequence 22, step 228 may run the sequence analyzer 42 to determine the token prediction marker 38 for each target token sequence 22.

[0062]

[0055] Another step 230 may involve applying the decision module 44 to generate an anomaly indicator 26. In some embodiments, the decision module 44 may use, for example, the individual token prediction scores S taken across the entire target token sequence 22. i The system is configured to determine the anomaly marker 26 according to the token prediction marker 38 as an average of the individual scores S. i As described above in relation to Figure 3, this can quantify the probability that each token in sequence 22 has been destroyed by the application of sequence transformation 30. In some embodiments, a sequence-specific anomaly score A may be determined according to the following:

[0063]

number

[0064]

[0056] In the above formula, L S This represents the length (token count) of the target sequence 22. i However, in one embodiment where it is highly likely that each token has been destroyed, a large value of A may indicate that the target token sequence 22 is likely to be abnormal. Conversely, a large S i However, a large value of A may indicate that the target sequence 22 is not abnormal, when it is highly likely that each token has not been destroyed. Some embodiments of the determination module 44 compare the calculated value of the abnormality score A with a predetermined threshold and determine whether the target token sequence 22 is abnormal or not according to the result of the comparison.

[0065]

[0057] Figure 10 shows an exemplary hardware configuration of a computing appliance 70 programmed to perform some of the methods described herein. The computing appliance 70 may represent any of the client systems 10a-c, the utility server 12, and the AI ​​training appliance 14 in Figure 1. The illustrated computing appliance is a personal computer, and other devices such as servers, mobile phones, tablet computers, and wearables may have slightly different configurations. The processor 72 may comprise a physical device (e.g., a microprocessor, a multicore integrated circuit formed on a semiconductor substrate) configured to perform computations and / or logical operations with a set of signals and / or data. Such signals or data may be encoded in the form of processor instructions, e.g., machine code, and delivered to the processor 72.

[0066]

[0058] The processor 72 is generally characterized by an instruction set architecture (ISA), which specifies, among other things, the respective sets of processor instructions (e.g., x86 family and ARM® family) and the size of the registers (e.g., 32-bit processor and 64-bit processor). The architecture of the processor 72 can vary according to the primary intended use. A central processing unit (CPU) is a general-purpose processor, while a graphics processing unit (GPU) is optimized for image / video processing and some forms of parallel computing. The processor 72 may further include application-specific integrated circuits (ASICs), such as tensor processing units (TPUs) by Google®, Inc. and neural processing units (NPUs) by various manufacturers. TPUs and NPUs may be particularly well-suited for machine learning applications such as those described herein.

[0067]

[0059] The memory unit 74 may include a volatile computer-readable medium (e.g., dynamic random access memory - DRAM) for storing data / signal / instruction coding that is accessed or generated by the processor 72 in the course of performing its operations. Input devices 76 may include, among other things, a computer keyboard, mouse, and microphone, including their respective hardware interfaces and / or adapters that enable a user to introduce data and / or instructions into the appliance 70. Output devices 78 may include, among other things, display devices such as monitors and speakers, as well as hardware interfaces / adapters such as graphics cards that enable their respective computing appliances to communicate data to a user. In some embodiments, the input and output devices 76-78 share common hardware (e.g., a touchscreen). The storage device 82 includes a computer-readable medium that enables non-volatile storage, reading, and writing of software instructions and / or data. Exemplary storage devices include magnetic disks, optical disks, and flash memory devices, as well as removable media such as CDs and / or DVD disks and drives. The network adapter 84 enables the computing appliance 70 to connect to an electronic communications network (for example, network 15 in Figure 1) and / or other devices / computer systems.

[0068]

[0060] The controller hub 80 generally represents all other circuits that enable communication between multiple systems, peripherals, and / or chipset buses, and / or between the processor 72 and the rest of the hardware components of the appliance 70. For example, the controller hub 80 may include a memory controller, an input / output (I / O) controller, and an interrupt controller. Depending on the hardware manufacturer, some such controllers may be incorporated into a single integrated circuit and / or integrated with the processor 72. In another example, the controller hub 80 may include a northbridge connecting the processor 72 to the memory 74, and / or a southbridge connecting the processor 72 to devices 76, 78, 82, and 84.

[0069]

[0061] The exemplary systems and methods described above enable efficient automated detection of anomalies in various application areas. In some embodiments, token sequences extracted from a training corpus are distorted according to at least one of a plurality of predetermined sequence transformations before being fed to a sequence analyzer that is being trained. The sequence analyzer is then trained to correctly infer which transformation was used to generate each input token sequence.

[0070]

[0062] Several conventional procedures for training anomaly detectors involve disrupting a training token sequence by randomly replacing some of the tokens, and then training the detector to guess which tokens were replaced. However, such training methods are relatively costly in terms of computational resources and can be unstable for some choice of training corpora and / or applications. In contrast to this conventional approach, some embodiments of the present invention distort an input token sequence using a predefined set of transformations and train according to an aggregate utility function that combines a token-level component (indicating whether each individual token was disrupted or not) with a sequence-level component (indicating how the entire input sequence is distorted). Using sequence-level utilities in addition to more conventional token-level utilities can seem counterintuitive, because in some embodiments, each sequence transformation has a specific token mask, and therefore guessing which transformation was applied can effectively mean guessing which tokens were disrupted. However, in some embodiments, setting up a token-level task (inferring whether a particular token has been destroyed) simultaneously with setting up a sequence-level learning task (inferring the applied transformation) can enhance correct learning by preventing the sequence analyzer from predicting non-existent mask patterns. In some embodiments, token-level and sequence-level predictions are generated by separate prediction heads of the deep neural network. Thus, there is no prior knowledge of the correlation between the transformation and the token mask built within the anomaly detector. Instead, the detector learns such latent correlations during training, which can lead to a more robust model.

[0071]

[0063] Computer experiments have shown that combining token-level tasks with sequence-level tasks facilitates learning by improving the performance of the detector. Conversely, by using the training method described herein, the same level of anomaly detection performance can be achieved with a smaller training corpus and / or fewer network parameters. This may be particularly advantageous for anomaly detection tasks such as author attributes in situations where the training corpus 18 has a relatively small size (for example, when the training corpus consists of social media posts). Training the detector as described herein intuitively corresponds to artificially increasing the size of the training corpus, because the same training token sequence 32 can produce multiple distinct modified sequences 34 in response to the application of distinct sequence transformations 30 (see Figure 3).

[0072]

[0064] Some embodiments utilize a second AI system to generate reasonable distortions in the training sequence. For example, given the context of the rest of each token sequence, a token generator implementing a BERT language model may be used to replace selected tokens with reasonable substitutions. Some embodiments rely on the observation that a pre-trained generator may hinder learning by generating modified training sequences that are, in a sense, "too reasonable" to be considered anomalous, and instead of using a pre-trained, high-performance version of the token generator, they explicitly train the token generator together with an anomaly detector. Training them together can ensure that the token generator becomes increasingly skilled at generating reasonable modified training sequences as the sequence analyzer becomes more skilled at detecting the modifications. Furthermore, pre-training the token generator on a corpus separate from the corpus used to train the anomaly detector may introduce anomaly information and prevent the anomaly detector from recognizing the anomaly information as such.

[0073]

[0065] Anomaly detectors trained according to some embodiments of the present invention can be used in a variety of scenarios, including, among others: Automatic text classification

[0066] In an exemplary natural language processing (NLP) application, the anomaly detector 20 may be trained on a corpus of texts belonging to a specific category (e.g., business news) and then used to determine whether a target text fragment belongs to each category. In such an embodiment, a high anomaly score may indicate that each text does not belong to each category.

[0074]

[0067] In the computer experiment, the anomaly detector described herein was trained on a subset of a standard reference corpus of news articles (20 Newsgroups), the subset consisting of articles from a selected category (computing, recreation, science, miscellaneous, politics, or religion). The experiment used a sequence analyzer with four stacked transformer layers and two predictive heads at the top. Each transformer layer included four self-attention heads, a hidden layer of size 256, and feedforward layers of sizes 1024 and 256. Each predictive head had two linear layers separated by nonlinearity and terminated in a classification layer. The maximum size of the training token sequence was 128. Sequence transformation 30 consisted of replacing tokens according to distinct mask patterns using a random token generator. Mask patterns of various counts and coverages were tried with between 5 and 100 distinct mask patterns covering between 25% and 50% of the input training sequence.

[0075]

[0068] The trained detector was then asked to identify whether an article randomly selected from the set belonged to a category for which the detector was trained. Anomaly detectors trained according to several embodiments of the present invention consistently outperformed conventional anomaly detectors of the current art in each task, with typical area under the receiver operating curve (AUROC) values ​​ranging from approximately 70% (when trained for science categories) to over 92% (when trained for computing news). Experiments revealed that generally increasing the number of distinct transformations improved the performance of the trained anomaly detector and facilitated expressibility within the token embedding. Transformations with a proportion of 25% to 50% destroyed tokens appeared to produce the best results. Auto-creator attributes

[0069] Some embodiments of the anomaly detector 20 may be trained on a corpus of text (e.g., letters, articles, blog posts, emails, social media posts) written by selected authors and then used to determine whether a target text fragment was created by each of those authors. Exemplary applications include determining the author of an anonymous letter and verifying the authenticity of various documents and the posthumous attributes of literary works. Some embodiments may also have criminological applications. For example, law enforcement may be interested in identifying authors of messages posted on forums that serve as a collection point for users involved in criminal activities such as the dealing of stolen goods, credit card data, child pornography, guns, drugs, etc., or identifying authors or users of Dark Web resources. The term “Dark Web” is used herein to refer to content that is not indexed by search engines and / or is accessible only through private peer-to-peer networks or anonymization software such as Tor.

[0076]

[0070] Next, an exemplary embodiment of an anomaly detector trained on a corpus of online content created by a set of suspects and made publicly available (e.g., comments posted by each suspect on popular social media sites and / or user forums) may be used to analyze a target text fragment taken from the Dark Web. An anomaly score indicating that the target text is not anomaly may indicate that the author of the target text matches one of the authors of the corpus of texts on which the detector was trained. Social media monitoring

[0071] One embodiment of the anomaly detector 20 may be trained on a specific set of web content associated with a selected set of social media accounts, for example, a specific collection of Twitter® feeds. The training corpus may be further limited to content published within a specific time window (e.g., one day, one week, etc.). The detector can then be used to analyze newly posted content. Anomalies may indicate a change in topic and / or a change in the tone of an ongoing exchange, thus enabling timely automated detection of new topics and trends. Detection of fake and automatically generated content

[0072] One embodiment of the anomaly detector 20 may be trained on a corpus of text written by selected human authors (e.g., actual news articles, social media posts by actual users). The corpus may be further narrowed to articles written about selected periodicals, newspapers, or news websites, or articles written by selected journalists. The trained anomaly detector may then be used to analyze target text fragments. An anomaly score indicating that the target text is anomaly may indicate that each text may contain fake news and / or may be machine-generated. Data protection and privacy

[0073] Some cloud computing services allow users to upload files to remote servers for sharing with other users or for various operations (e.g., malware scanning). As an example, a software agent running on a user's computer may automatically upload the contents of selected folders, which may be indicated by each user. Uploading data to the cloud can involve privacy risks, especially when the user does not explicitly select the content to upload. For example, if a user accidentally drops some sensitive data (e.g., personal files or photos, medical records, etc.) into an upload folder, each piece of data will be automatically uploaded against the user's wishes.

[0077]

[0074] One embodiment of the anomaly detector 20 may be installed on a user's computer and trained on files typically uploaded by each user, for example, 100 files recently uploaded for remote scanning. Additional filters may select only specific types of files, such as Portable Document Format (PDF) documents or Microsoft® Office® files. Such embodiments may use the natural language processing techniques described above, where tokens include individual words, etc. The trained anomaly detector may then be used to analyze each file that is currently marked for remote scanning in preparation for upload. When the anomaly score determined for each file indicates a potential anomaly, some embodiments may prevent the file from being uploaded and notify the user. Computer security

[0075] Some embodiments of the anomaly detector 20 may be trained with respect to a sequence of computing events occurring during a reference time interval that is considered to represent a normal pattern of activity, and then used to monitor the behavior of a set of client computer systems. Anomaly behavior detected on a client may indicate a computer security threat, for example, that each client is running malicious software or that an intruder / hacker has gained access to each client.

[0078]

[0076] In some embodiments, anomaly detection involves analyzing a sequence of events occurring during the execution of a software entity being monitored (e.g., a process, a virtual machine, etc.). Examples of such events include, among other things, the initiation of a process / thread (e.g., a user launching an application, a parent process creating a child process, etc.), attempts to access input devices of each client system (e.g., a camera, a microphone), attempts to access local or remote network resources (e.g., a hypertext transfer protocol - HTTP request to access a specific URL, an attempt to access a document repository over a local network), requests constructed with a specific uniform resource identifier scheme (e.g., a mailto: or FTP: request), the execution of a specific processor instruction (e.g., a system call), attempts to load a library (e.g., a dynamic link library - DLL), attempts to create a new disk file, attempts to read from or write to a specific location on disk (e.g., an attempt to overwrite an existing file, an attempt to open a specific folder or document), and attempts to send an electronic message (e.g., an email, a short message service - SMS, etc.). In some embodiments, periods of inactivity, time gaps between events, and / or time intervals when each client system is idle and not registering user activity or performing only internal system tasks may also be given eligibility as events. Those skilled in the art will understand that the systems and methods described herein can be adapted to analyze other types of events, such as, in particular, events relating to user activity on social media, user browsing history, and user gaming activity.

[0079]

[0077] Event detection may include any method known in the Art. For example, a security agent running on a protected client may register a set of software entities to be monitored with an OS40 event logging service such as event tracking for Windows®. In response, the agent may receive notifications of various events occurring during the execution of each process, either in real time or in log format. An event logging tool typically generates a list of event descriptors for each event, which include a timestamp, a numeric code identifying the event type, an indicator of the type of process or application that generated each event, and other event parameters. An event sequence may be constructed by parsing the log.

[0080]

[0078] In some embodiments, each event may be treated as a separate token. The tokens may be encoded according to an event vocabulary, which may contain thousands to millions of distinct event types. Training the anomaly detector may then involve applying various transformations to the training event sequence as described above. Exemplary sequence transformations may include deleting, inserting, and rearranging selected events in the training sequence, as well as replacing selected events with alternative events of a different type.

[0081]

[0079] In an alternative embodiment, an event log entry may be considered as a sequence of text tokens. For example, a log entry: 20:10|INFO|manager.strage|Found block rdd_2_3 locally This can also be parsed into the following token sequence.

[0082] 20:10;INFO;manager;storage;Found;block;rdd_2_3;locally However, each token is separated by a semicolon. Next, the input modifier 40 may distort each token sequence by replacing the selected token with a substitute. The positions of the tokens selected for substitution may be indicated by a mask as described above. As such an example, the modified token sequence determined above may be understood as follows:

[0083] 20:10;DEBUG;manager;thread;Found;block;rdd_2_3;globally However, alternative tokens are shown in bold. In some embodiments, a mask is chosen so that the selected field of the blog entry is not modified. The token generator 41 may be configured to select an alternative token from a pool of candidate field-specific or location-specific tokens. In the example above, the pool of candidate alternatives for the second token may consist of {WARNING,DEBUG,INFO,CRITICAL}.

[0084]

[0080] An exemplary application of anomaly detection to logs includes detecting zero-day attacks by analyzing access and / or event logs recorded on a honeypot system. Anomaly detectors, such as those described herein, may be trained with respect to a first portion of the logs and thus learn the "normal" behavior of each honeypot during the time frame corresponding to the first portion of the logs. The anomaly detector may then be used to analyze a second portion of the logs. Anomalies may indicate a change in honeypot behavior between the first and second portions of the logs, suggesting the possible emergence of new malware, the activation of a botnet, and so on. In some embodiments, the anomaly detector may be retrained periodically (for example, hourly with respect to log data from the previous time) and used to monitor new threats in real time.

[0085]

[0081] It will be apparent to those skilled in the art that the above embodiments can be modified in many ways without departing from the scope of the present invention. Accordingly, the scope of the present invention should be determined by the following claims and their legal equivalents.

Claims

1. A computer-implemented anomaly detection method, wherein the method comprises at least one hardware processor of a computer system, The steps of generating a modified token sequence by applying the selected transformation to the training token sequence in response to selecting a training token sequence from a training corpus of token sequences and selecting a transformation from a predetermined set of sequence transformations, A step of running a sequence analyzer having a set of adjustable parameters and configured to determine a transformation prediction indicator according to the modified token sequence, wherein the transformation prediction indicator indicates the possibility that the selected transformation was applied to generate the modified token sequence. A step of adjusting at least one parameter from the set of adjustable parameters according to the transformation prediction marker in response to determining the transformation prediction marker, The steps include: determining whether the target token sequence is abnormal using the sequence analyzer in response to adjusting at least one of the aforementioned parameters; A computer-implemented anomaly detection method, including the implementation of [the specified action].

2. A method according to claim 1, wherein the step of applying the selected transformation includes the step of replacing the selected tokens in the training token sequence with alternative tokens.

3. The method according to claim 2, wherein the at least one hardware processor is The steps include: running a token generator having another set of adjustable parameters and configured to generate the alternative tokens according to the training token sequence; A step of adjusting another parameter of the other set of tunable parameters according to the conversion prediction marker in response to determining the conversion prediction marker. A method that further includes this.

4. The method according to claim 1, A method comprising an item selected from a group of steps, the steps of applying the selected transformation, the steps of deleting selected tokens from the training token sequence, inserting additional tokens into the training token sequence, and sorting a selected subset of tokens in the training token sequence.

5. The method according to claim 1, The sequence analyzer is further configured to determine a token prediction indicator according to the modified token sequence, the token prediction indicator indicating that a selected token in the modified token sequence may have been altered by the application of the selected transformation. A method comprising the step of adjusting the at least one adjustable parameter, further comprising the step of adjusting the at least one adjustable parameter according to the token prediction indicator.

6. The method according to claim 1, The aforementioned training token sequence and target token sequence include text constructed in natural language. The method further includes, in response to the at least one hardware processor determining whether the target token sequence is abnormal, determining that if the target token sequence is abnormal, the creator of the target token sequence is different from the creator of the training token sequence. method.

7. The method according to claim 1, The aforementioned training token sequence and target token sequence include text constructed in natural language. The method further includes, in response to the at least one hardware processor determining whether the target token sequence is abnormal, determining that if the target token sequence is abnormal, the subject of the target token sequence is different from the subject of the training token sequence. method.

8. The method according to claim 1, The aforementioned training token sequence and target token sequence include text constructed in natural language. The method further includes, in response to the determination by at least one hardware processor whether the target token sequence is abnormal, determining that the target token sequence was machine-generated if the target token sequence is abnormal. method.

9. The method according to claim 1, The aforementioned training corpus includes text fragments selected according to selection criteria, The method further includes, in response to the determination by at least one hardware processor whether the target token sequence is abnormal, determining that if the target token sequence is abnormal, the target token sequence does not satisfy the selection criteria. method.

10. The method according to claim 1, The aforementioned training token sequence and target token sequence include a sequence of computing events, The method further includes, in response to the determination by at least one hardware processor whether the target token sequence is abnormal, determining that if the target token sequence is abnormal, the target token sequence indicates a computer security threat. method.

11. A computer system comprising at least one hardware processor, wherein the at least one hardware processor is In response to selecting a training token sequence from a training corpus of token sequences, and in response to selecting a transformation from a predetermined set of sequence transformations, the selected transformation is applied to the training token sequence to generate a modified token sequence. Running a sequence analyzer having a set of adjustable parameters and configured to determine a transformation prediction indicator according to the modified token sequence, wherein the transformation prediction indicator indicates the possibility that the selected transformation was applied to generate the modified token sequence; In response to determining the transformation prediction marker, adjusting at least one parameter from the set of adjustable parameters according to the transformation prediction marker, In response to adjusting at least one of the aforementioned parameters, the sequence analyzer is run to determine whether the target token sequence is abnormal. A computer system configured to perform the following actions.

12. A computer system according to claim 11, wherein applying the selected transformation includes replacing the selected tokens in the training token sequence with alternative tokens.

13. The computer system according to claim 12, wherein the at least one hardware processor is Executing a token generator having another set of tunable parameters and configured to generate the alternative tokens according to the training token sequence, In response to determining the transformation prediction marker, adjust another parameter of the other set of tunable parameters according to the transformation prediction marker. A computer system further configured to perform the following tasks.

14. A computer system according to claim 11, A computer system that applies the selected transformation, which includes items selected from a group of actions including deleting selected tokens from the training token sequence, inserting additional tokens into the training token sequence, and reordering a selected subset of tokens in the training token sequence.

15. A computer system according to claim 11, The sequence analyzer is further configured to determine a token prediction indicator according to the modified token sequence, the token prediction indicator indicating the possibility that a selected token in the modified token sequence has been altered by the application of the selected transformation. Adjusting the at least one adjustable parameter includes further adjusting the at least one adjustable parameter in accordance with the token prediction indicator. Computer system.

16. A computer system according to claim 11, The aforementioned training token sequence and target token sequence include text constructed in natural language. The at least one hardware processor is further configured, in response to determining whether the target token sequence is abnormal, to determine, if the target token sequence is abnormal, that the creator of the target token sequence is different from the creator of the training token sequence. Computer system.

17. A computer system according to claim 11, The aforementioned training token sequence and target token sequence include text constructed in natural language. The at least one hardware processor is further configured, in response to determining whether the target token sequence is abnormal, to determine, if the target token sequence is abnormal, that the subject of the target token sequence is different from the subject of the training token sequence. Computer system.

18. A computer system according to claim 11, The aforementioned training token sequence and target token sequence include text constructed in natural language. The at least one hardware processor is further configured to determine, in response to determining whether the target token sequence is abnormal, that if the target token sequence is abnormal, the target token sequence was machine-generated. Computer system.

19. A computer system according to claim 11, The aforementioned training corpus includes text fragments selected according to selection criteria, The at least one hardware processor is further configured to determine, in response to determining whether the target token sequence is abnormal, that if the target token sequence is abnormal, the target token sequence does not satisfy the selection criteria. Computer system.

20. A computer system according to claim 11, The aforementioned training token sequence and target token sequence include a sequence of computing events, The at least one hardware processor is further configured, in response to determining whether the target token sequence is abnormal, to determine that the target token sequence indicates a computer security threat if the target token sequence is abnormal. Computer system.

21. A non-temporary computer-readable medium for storing instructions, wherein the instructions, when executed by at least one hardware processor of a computer system, are stored in the computer system. In response to selecting a training token sequence from a training corpus of token sequences, and in response to selecting a transformation from a predetermined set of sequence transformations, the selected transformation is applied to the training token sequence to generate a modified token sequence. Running a sequence analyzer having a set of adjustable parameters and configured to determine a transformation prediction indicator according to the modified token sequence, wherein the transformation prediction indicator indicates the possibility that the selected transformation was applied to generate the modified token sequence; In response to determining the transformation prediction marker, adjusting at least one parameter from the set of adjustable parameters according to the transformation prediction marker, In response to adjusting at least one of the aforementioned parameters, the sequence analyzer is run to determine whether the target token sequence is abnormal. A non-temporary computer-readable medium that enables the operation of [the process].

Images