Systems, methods, and products for detecting network activity using transit tensor analysis

Abstract

This disclosure relates to systems, methods, and products for detecting network activity using transitive tensor analysis, describing a system for detecting network activity using transitive tensor analysis. The system partitions a tensor into multiple subtensors, where each tensor represents communication on a communication network carrying streaming network data. Each subtensor is decomposed individually and independently into subtensor modal factors. Transmittive modal factor matching is used to determine the ordering of the subtensor modal factors. A set of subtensor factor coefficients is determined for the subtensor modal factors, and these coefficients are used to determine the relative weighting of the subtensor modal factors, detecting the activity patterns represented by the subtensor modal factors. Based on the detection, an alert of an anomaly is generated, indicating the location and time of occurrence within the communication network.

Description

[0001] Cross-reference to related applications

[0002] This application is a continuation-in-part of U.S. Application No. 16 / 298,367, filed March 11, 2019, entitled “Streaming Data Tensor Analysis Using Blind Source Separation,” which is a continuation-in-part of U.S. Application No. 16 / 034,780, filed July 13, 2018, entitled “Independent Component Analysis of Tensors for Sensor Data Fusion and Reconstruction,” which is a non-provisional patent application filed September 13, 2017, entitled “Independent Component Analysis of Tensors for Sensor Data Fusion and Reconstruction,” the entire contents of which are incorporated herein by reference. U.S. Application No. 16 / 298,367 is also a continuation-in-part of U.S. Application No. 16 / 127,927, filed September 11, 2018, entitled "Mixed Domain Blind Source Separation for Sensor Array Processing," which is a non-provisional patent application filed January 30, 2018, entitled "Mixed Domain Blind Source Separation for Sensor Array Processing," the entire contents of which are incorporated herein by reference. U.S. Application No. 16,298,367 is also a non-provisional application filed June 13, 2018, entitled "Streaming Data Tensor Analysis Using Blind Source Separation," the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to systems for detecting network activity, and more particularly to systems for detecting network activity using transitive tensor analysis (TTA). Background Technology

[0004] Tensors are generalizations of matrices to higher dimensions (e.g., multidimensional tables of data values). Tensors can represent multiple relationships between entities in a communication network or multidimensional sensor data, such as hyperspectral image data. It is well known that tensors can be decomposed into a sum of tensor factors, where these factors represent the structure or relationships in the data, such as communication patterns between entities in a computer or social network.

[0005] Current techniques for tensor decomposition are based on least-squares fitting from data to a model. Examples of existing methods include PARAFAC (Parallel Factor Analysis), NPLS (Nonlinear Partial Least Squares), and ALS (Alternating Least Squares). Because these methods operate on the entire tensor and use iterative optimization, they are slow, require significant memory, and do not scale well to higher-dimensional tensors (scaling is an exponential of the tensor's dimensionality). They also do not handle sparse data well, as gradients cannot be accurately computed when data is sparse, a common problem in web applications. Furthermore, they lack mechanisms for using cloud computing to partition the tensor and process these parts independently and in parallel.

[0006] Therefore, there is still a need for a method to analyze large tensors by dividing them into smaller subtensors that can be independently decomposed into factors that represent tensors much larger than those that can be processed by any single processor. Summary of the Invention

[0007] This invention relates to a system for detecting network activity, and more particularly to a system for detecting network activity using Transitive Tensor Analysis (TTA). The system includes one or more processors and a non-transitory computer-readable medium thereon encoding executable instructions, such that when the executable instructions are executed, the one or more processors perform multiple operations. The system partitions a tensor having multiple tensor modes into multiple sub-tensors, wherein the tensors represent communication on a communication network including streaming network data. Each sub-tensor is separately and independently decomposed into multiple sub-tensor mode factors on separate processors. Transitive mode factor matching is used to determine the ordering of the multiple sub-tensor mode factors. A set of sub-tensor factor coefficients is determined for the multiple sub-tensor mode factors. The set of sub-tensor factor coefficients is used to determine the relative weighting of the multiple sub-tensor mode factors. Multiple activity patterns represented by the multiple sub-tensor mode factors are detected. Based on this detection, an anomaly warning is generated, indicating the location of the anomaly in the communication network and the time of its occurrence.

[0008] On the other hand, the plurality of tensor modes include: two modes representing the connections between nodes in the communication network, and a third mode representing the time dependency of the connections between nodes in the communication network.

[0009] On the other hand, decomposing each subtensor includes: converting the subtensor into a one-dimensional vector signal set; using the one-dimensional vector signal set as input for independent component analysis; and extracting the modal factors of the multiple subtensors.

[0010] On the other hand, determining the order of the plurality of sub-tensor mode factors includes matching the sub-tensor mode factors that are common among the plurality of sub-tensors.

[0011] On the other hand, determining the order of the plurality of sub-tensor mode factors further includes: matching non-common sub-tensor mode factors via transitivity using matching of sub-tensor mode factors common among the plurality of sub-tensors.

[0012] On the other hand, determining the sub-tensor factor coefficient set for the plurality of sub-tensor modal factors includes using a pseudo inverse of the sub-tensor factor dictionary matrix, wherein the columns of the sub-tensor factor dictionary matrix are vectorizations of the plurality of sub-tensor factors.

[0013] On the other hand, tensor analysis is applied to the sliding time window of streaming network data.

[0014] Finally, the invention also includes a computer program product and a computer-implemented method. The computer program product includes computer-readable instructions stored on a non-transitory computer-readable medium, which are executable by a computer having one or more processors to cause the one or more processors to perform the operations listed herein when the instructions are executed. Alternatively, the computer-implemented method includes the action of causing a computer to execute such instructions and perform the resulting operations. Attached Figure Description

[0015] The objects, features, and advantages of the present invention will become apparent from the following detailed description of various aspects of the invention, taken in conjunction with the accompanying drawings, wherein:

[0016] Figure 1 This is a block diagram depicting the components of a system for detecting network activity according to some embodiments of the present disclosure;

[0017] Figure 2 These are illustrations of computer program products according to some embodiments of the present disclosure;

[0018] Figure 3 This is an example diagram of the canonical polyadic decomposition (CPD) of a third-order tensor based on existing technology, showing how tensor mode factors are combined to form tensor factors to reconstruct the tensor;

[0019] Figure 4A This is an example diagram of tensor decomposition using independent component analysis of tensors (ICAT) for detecting network activity according to some embodiments of this disclosure, illustrating the detection of weak activity;

[0020] Figure 4B This is an example diagram of tensor decomposition using tensor independent component analysis (ICAT) for detecting network activity based on existing technology, showing the difference accuracy decomposition;

[0021] Figure 5 This is an example diagram of reshaping or matrixing a tensor into a matrix according to an embodiment of the present disclosure, wherein the rows of the matrix represent a set of vector signals to be processed using independent component analysis (ICA);

[0022] Figure 6A This is an illustrative diagram of step 1 of ICAT tensor decomposition according to some embodiments of the present disclosure, which uses matrix transformation and ICA demixing to determine the modality factor of the tensor.

[0023] Figure 6BThis is an illustrative diagram illustrating the correct ordering of modal factors of two modes using ICA layering according to some embodiments of this disclosure;

[0024] Figure 7A This is a flowchart illustrating ICAT tensor modal factor extraction using ICA, factor matching using transitive modal matching (TMM), and calculation of modal factor weights for tensor reconstruction according to some embodiments of this disclosure.

[0025] Figure 7B This is a symbolic illustration of ICAT tensor modal factor extraction and factor matching using TMM according to some embodiments of this disclosure;

[0026] Figure 8 This is an example diagram of tensor analysis according to some embodiments of the present disclosure, which involves dividing a tensor into subtensors that are individually decomposed into subtensor mode factors.

[0027] Figure 9 This is an illustrative diagram of subtensor mode factor matching across subtensors using TMM according to some embodiments of this disclosure;

[0028] Figure 10 This is an example diagram illustrating the calculation of the transitive tensor analysis (TTA) of the subtensor factor coefficients according to some embodiments of this disclosure;

[0029] Figure 11 This is an example diagram illustrating the application of TTA to sliding time window analysis of streaming tensor data according to some embodiments of this disclosure;

[0030] Figure 12 This is a flowchart illustrating the TTA processing steps of a tensor according to an embodiment of the present disclosure, which involves dividing a tensor into subtensors, decomposing the subtensors, matching the subtensor mode factors, and solving the tensor factor coefficients. Detailed Implementation

[0031] This invention relates to systems for detecting network activity, and more particularly, to systems for detecting network activity using transitive tensor analysis. The following description is presented to enable those skilled in the art to make and use the invention in the context of a particular application. Various modifications and multiple uses in different applications will be apparent to those skilled in the art, and the general principles defined herein can be applied to a wide range of aspects. Therefore, the invention is not intended to be limited to the aspects presented, but is consistent with the broadest scope according to the principles and novel features disclosed herein.

[0032] In the following detailed description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without being limited to these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the invention.

[0033] Readers are advised to pay attention to all documents and files submitted concurrently with this specification and made available to the public for consultation, the contents of which are incorporated herein by reference. Unless otherwise expressly stated, all features disclosed in this specification (including any appended claims, abstracts, and drawings) may be replaced by alternative features for the same, equivalent, or similar purposes; therefore, unless otherwise expressly stated, the disclosed features are merely examples of a series of equivalent or similar features.

[0034] (1) Main aspects

[0035] Various embodiments of the present invention comprise three “main” aspects. The first aspect is a system for detecting network activity. This system typically takes the form of computer system operating software or a “hard-coded” instruction set. The system can be incorporated into a wide variety of devices providing different functionalities. The second main aspect is a method, typically in software form, utilizing a data processing system (computer). The third main aspect is a computer program product. A computer program product typically represents computer-readable instructions stored on a non-transitory computer-readable medium such as an optical storage device (e.g., an optical disc (CD) or a digital versatile disc (DVD)) or a magnetic storage device (e.g., a floppy disk or magnetic tape). Other non-limiting examples of computer-readable media include hard disks, read-only memory (ROM), and flash memory. These aspects will be described in more detail below.

[0036] Figure 1 Block diagrams illustrating an example of a system (i.e., computer system 100) of the present invention are provided herein. Computer system 100 is configured to perform calculations, processing, operations, and / or functions associated with a program or algorithm. In one aspect, certain processes and steps discussed herein are implemented as a series of instructions (e.g., a software program) residing in one or more computer-readable storage units and executed by one or more processors of computer system 100. Upon execution, these instructions cause computer system 100 to perform specific actions and exhibit specific behaviors as described herein.

[0037] Computer system 100 may include an address / data bus 102 configured to transmit information. Additionally, one or more data processing units (such as processor 104 (or multiple processors)) are coupled to the address / data bus 102. Processor 104 is configured to process information and instructions. In one aspect, processor 104 is a microprocessor. Alternatively, processor 104 may be a different type of processor, such as a parallel processor, application-specific integrated circuit (ASIC), programmable logic array (PLA), complex programmable logic device (CPLD), or field-programmable gate array (FPGA).

[0038] Computer system 100 is configured to utilize one or more data storage units. Computer system 100 may include volatile memory units 106 (e.g., random access memory (“RAM”), static RAM, dynamic RAM, etc.) coupled to address / data bus 102, wherein volatile memory units 106 are configured to store information and instructions for processor 104. Computer system 100 may also include non-volatile memory units 108 (e.g., read-only memory (“ROM”), programmable ROM (“PROM”), erasable programmable ROM (“EPROM”), electrically erasable programmable ROM (“EEPROM”), flash memory, etc.) coupled to address / data bus 102, wherein non-volatile memory units 108 are configured to store static information and instructions for processor 104. Alternatively, computer system 100 may execute instructions retrieved from online data storage units, such as in “cloud” computing. In one aspect, computer system 100 may also include one or more interfaces, such as interface 110, coupled to address / data bus 102. The one or more interfaces are configured to enable the computer system 100 to connect to other electronic devices and computer systems. The communication interfaces implemented by the one or more interfaces may include wired (e.g., serial cable, modem, network adapter, etc.) and / or wireless (e.g., wireless modem, wireless network adapter, etc.) communication technologies. Furthermore, one or more processors 104 (or devices, such as autonomous platforms) may be associated with one or more memories, wherein each associated memory is a non-transitory computer-readable medium. Each associated memory may be associated with a single processor 104 (or device) or a network of interactive processors 104 (or devices), such as a network of autonomous platforms (e.g., autonomous vehicles, robots).

[0039] In one aspect, computer system 100 may include an input device 112 coupled to address / data bus 102, wherein input device 112 is configured to transmit information and command selections to processor 104. According to one aspect, input device 112 is an alphanumeric input device (such as a keyboard), which may include alphanumeric keys and / or function keys. Alternatively, input device 112 may be another input device besides an alphanumeric input device. In one aspect, computer system 100 may include a cursor control device 114 coupled to address / data bus 102, wherein cursor control device 114 is configured to transmit user input information and / or command selections to processor 104. In one aspect, cursor control device 114 is implemented using a device such as a mouse, trackball, trackpad, optical tracking device, or touchscreen. Nevertheless, in one aspect, cursor control device 114 may be guided and / or enabled via input from input device 112, such as in response to commands using specific keys and key sequences associated with input device 112. Alternatively, the cursor control device 114 is configured to be managed or guided via voice commands.

[0040] In one aspect, the computer system 100 may also include one or more optional computer-usable data storage devices, such as storage device 116 coupled to address / data bus 102. Storage device 116 is configured to store information and / or computer-executable instructions. In one aspect, storage device 116 is a storage device such as a magnetic or optical disk drive (e.g., hard disk drive (“HDD”), floppy disk, optical disk read-only memory (“CD-ROM”), digital versatile disk (“DVD”)). According to one aspect, display device 118 is coupled to address / data bus 102, wherein display device 118 is configured to display video and / or graphics. In one aspect, display device 118 may include: cathode ray tube (“CRT”), liquid crystal display (“LCD”), field emission display (“FED”), plasma display, or any other display device suitable for displaying video and / or graphic images and user-recognizable alphanumeric characters.

[0041] The computer system 100 presented herein is an example computing environment according to one aspect. However, the non-limiting example of computer system 100 is not strictly limited to being a computer system. For example, one aspect provides that computer system 100 represents a class of data processing and analysis that can be used according to the various aspects described herein. Furthermore, other computing systems can also be implemented. In fact, the spirit and scope of this technology are not limited to any single data processing environment. Thus, in one aspect, computer-executable instructions (such as program modules) executed by a computer are used to control or implement one or more operations of various aspects of this technology. In one implementation, such program modules include routines, programs, objects, components, and / or data structures configured to perform a specific task or implement a specific abstract data type. Additionally, one aspect provides that one or more aspects of this technology are implemented by utilizing one or more distributed computing environments, such as in which tasks are performed by remote processing devices linked via a communication network, or in which various program modules reside in local and remote computer storage media including memory-storage devices.

[0042] Figure 2 The diagram illustrates a computer program product (i.e., a storage device) embodying the present invention. The computer program product is depicted as a floppy disk 200 or an optical disk 202 such as a CD or DVD. However, as previously mentioned, the computer program product generally refers to computer-readable instructions stored on any compatible non-transitory computer-readable medium. As used with respect to this invention, the term "instructions" generally indicates a set of operations to be performed on a computer and may represent a fragment of an entire program or a single, separate software module. Non-limiting examples of "instructions" include computer program code (source or object code) and "hard-coded" electronic devices (i.e., computer operations encoded into a computer chip). The "instructions" are stored on any non-transitory computer-readable medium, such as stored in the computer's memory or on a floppy disk, CD-ROM, and flash drive. In any case, these instructions are encoded on a non-transitory computer-readable medium.

[0043] (2) Specific details of each implementation method

[0044] This paper describes a method for applying Transitive Tensor Analysis (TTA) to the problem of detecting activity in network data. TTA is a unique approach for decomposing tensors into a sum of simpler factors. Tensors are generalizations of matrices to higher dimensions (e.g., multidimensional tables of data values). Tensors can represent multiple relationships between entities in a communication network or multidimensional sensor data, such as hyperspectral image data. It is well known that tensors can be decomposed into a sum of tensor factors, where these factors represent the structure or relationships in the data, such as the communication patterns between entities in a computer or social network over time.

[0045] Tensor factorization (TTA) is a unique approach for analyzing large tensors by dividing a large tensor into smaller subtensors that can be independently factored, and then combining the results into a decomposition of the full tensor. TTA enables the distribution of large tensors across multiple processors (including in the cloud) for analyzing tensors representing networks much larger than any single processor can handle. In addition to significantly reducing memory requirements, TTA also greatly reduces the computational demands on individual processors. Existing tensor factorization methods are based on the Alternating Least Squares (ALS) algorithm, where iterative gradient-based optimization is used to find the tensor factors that best fit the data. A fundamentally different tensor factorization method, Tensor Independent Component Analysis (ICAT), is described in U.S. Application No. 16 / 034,780, which offers superior computational and memory usage advantages over ALS. This application is incorporated herein by reference as if fully described herein. ICAT uses signal processing methods instead of iterative data fitting to analyze tensors. Both ALS and ICAT estimate tensor factors directly from data sampled from the full tensor.

[0046] As described in this paper, TTA is an extension of ICAT for cloud (Internet) implementations. It can process very large tensors by partitioning them into subtensors and then independently decomposing each subtensor using a separate processor, thus creating factor sets for each subtensor. TTA then uses transitive factor matching to align and order the components of the subtensor factors so that they can be combined into a decomposition of the full tensor. Since the computational complexity of tensor decomposition is at least quadratic in terms of the number of tensor elements, and the structural complexity of subtensors is less than that of the full tensor, performing ICAT tensor decomposition on subtensors is much faster than on the full tensor. Furthermore, subtensor decomposition can be performed in parallel, thus TTA can leverage cloud computing. Cloud computing is the on-demand availability of computer system resources (such as data storage (cloud storage) and computing power) without the need for direct, active management by the user.

[0047] TTA is highly efficient in terms of processing speed and memory usage. It is faster than existing methods because it does not require solving an alternating least-squares fitting problem to decompose the tensor, as current methods do. TTA is also more memory efficient because the entire tensor does not require memory suitable for any processor. Furthermore, only the non-zero elements of the tensor are used in computation and storage. Memory requirements are linearly proportional to the number of tensor dimensions, rather than exponentially, making processing tensors with more than three dimensions feasible. Finally, TTA can be naturally used for real-time streaming network data analysis using sliding time windows.

[0048] Figure 3 This illustrates the concept of tensor decomposition, which can be used in network analysis. A tensor is a multidimensional table of data values. For ease of illustration, a three-dimensional (3D) tensor is used as an example. A 3D tensor can represent communication on a network by using two-dimensional or modal representations of connections between nodes. The third modality can be used to represent different topics or temporal dependencies in communication activities. Such a tensor can be decomposed into tensor factors, where each tensor factor is a simpler tensor formed by the outer product of the three modal factor vectors. Tensor factors represent the structure in the data. For example, if the rows and columns of a tensor represent people in a social network, and the third dimension represents different types of links between them, then tensor factors can represent different social groups or communities with different interests. Figure 3 The regularized multivariate decomposition (CPD) of a third-order tensor is illustrated. This CPD represents a tensor as a weighted sum of R tensor factors, where each tensor factor is formed by the outer product of three one-dimensional (1D) modal factor vectors. Each mode or dimension of the tensor has a 1D modal factor vector. The rank of the tensor is defined as the number of factors required to reconstruct the tensor.

[0049] Tensor decomposition using conventional alternating least squares iterative optimization has been successfully applied in many other applications involving multidimensional data, such as for movie recommendation systems (e.g., Netflix). TM Challenges include sensor fusion, chemometrics, and network activity analysis. Tensor decomposition using tensor independent component analysis (ICAT) is described in U.S. Application No. 16 / 034,780. Compared to conventional methods, ICAT offers significant advantages in computational and memory usage. Figure 4A This demonstrates an example of ICAT used for tensor analysis of network activity. The dataset consists of packet data transmitted between nodes in a local area network. Figure 4B As shown, conventional ALS-based methods cannot detect the illustrated traffic patterns and weakly embedded activities.

[0050] like Figure 5As shown, step 1 of ICAT tensor decomposition (finding the modality factor vector) is based on transforming the tensor (element 500) into a 1D vector signal set (i.e., vectorization) by reorganizing the tensor (element 500) into a matrix (matrixification) and using the rows of the data elements (element 502) as 1D signal vectors. Then, as... Figure 6A As shown, the tensor (element 500) can be decomposed by using the signal (element 600) as input (element 602) to Independent Component Analysis (ICA), which separates the signal into statistically independent components in step 1 to generate a demixed signal output (element 604). The demixed 1D signal output (element 604) is then rearranged into 2D "slices" or matrices (element 606), which are the outer products of two modal factors among the modal factors of the individual tensor factors. The 1D margins of the slices are then calculated by summing along the rows and columns of the slices. Individual modal factor vectors (element 608) can be extracted by marginalizing or integrating the slices in orthogonal directions. However, because the ordering of the ICA outputs is not predictable or consistent, the modal factor vectors (element 608) across different tensor factors may not necessarily be aligned or matched. This mismatch can be corrected using transitive modal factor matching.

[0051] Figure 6B Depicts Figure 6A The notational representation of the process of generating factors of modes j and k using a mixture indexed by mode i is shown. The mode j factor (br(j)) (element 610) is automatically matched with the mode k factor (cr(k)) (element 612) via an ICA operation (element 602), because the br(j) (element 610) factor and the cr(k) (element 612) factor are entangled due to the fact that they are parts of the same 1D ICA output signal r. The next step is to convert the mode i factor a r (i) and b r (j)(element 610) and c r The (k)(element 612) factor is matched transitively, which will be described in detail below.

[0052] In step 2 of ICAT, using Figure 7A and Figure 7BThe Transient Modal Matching (TMM) (element 700) shown is used to align modal factor vectors. TMM (element 700) is based on the idea that by dividing a tensor into 3D "slabs," integrating these slabs to form 2D "slices" perpendicular to the "mixed modes," and vectorizing these slices into individual 1D signals, the modal factor vectors belonging to these slices become "entangled" because they are parts of the same 1D signal processed by ICA. Therefore, when the signal is separated or layered into different components by ICA, the layered entangled modal factors will always match correctly. If another mode is used as the mixed mode, another set of entangled modal factors will be extracted and matched correctly. Then, the matching of modal factors common to these two layering operations can be used to match modal factors not common to these two layering operations. In other words, as... Figure 7A As illustrated, ICA (element 702) is used to automatically match modality 2 and modality 3 factors, and modality 1 is used to define slices. In parallel, modality 1 and modality 3 modality factors are matched by forming slices defined by modality 2. After TMM (element 700) pass-through matching of modality factors for all three modalities, the following steps are taken: Figure 10 The process shown and described below is used to calculate the weight coefficients of the tensor factors (element 706) and generate the following... Figure 3 The CPD tensor is defined in (element 708). According to this transitivity principle, if the modality 3 factors shared by the two layers match (element 712), then all factors will automatically match (element 710).

[0053] In the third step of ICAT, the tensor factor weighting coefficients λ are determined as follows: r The tensor is sampled to form a measurement vector T(q), and the known values ​​of the tensor mode factors at those measurement locations are used to establish the following system of linear equations:

[0054]

[0055] Alternatively, a single exponent form can be used:

[0056]

[0057] Alternatively, we can use matrix form: Y = DΛ, which has the solution Λ = D. -1 Y, where D -1 It is the generalized inverse (Moore-Penrose pseudo-inverse) of matrix D.

[0058] TTA's innovation is an extension of ICAT, enabling the handling of very large tensors by dividing the tensor into subtensors, decomposing each subtensor independently, and then combining the outputs, while avoiding combinatorial explosion of possible combinations of subtensor modal factors. Figure 8 The diagram illustrates a concept for analyzing tensors by partitioning a tensor (element 500) into subtensors (element 800), then decomposing the subtensors (element 800) into subtensor mode factors (element 802), and finally into multiple subtensor mode factors. These multiple subtensor mode factors can be combined using an outer product operation to form a one-dimensional vector of subtensors. A subtensor is defined by the intersection of regions of tensors addressed by the non-zero parts of the tensor mode factors. Any subtensor can be reconstructed by changing the positions of the subtensor mode factor segments on the tensor mode factor vector. If it is desired to address only the indicated subtensors, the remainder of the tensor mode factor vector must be zero, as shown by the dashed lines in the vector.

[0059] Clearly, any subtensor can be reconstructed by changing the positions of the subtensor modal factor segments on the full tensor modal factor vector. Therefore, the question naturally arises: can the tensor decomposition problem be simplified to a set of subtensor decompositions, and then the result used to decompose the full tensor? Doing so would be advantageous because storage and computational loads could be shared across multiple processors. The reason this hasn't been done is that the ordering of subtensor factors resulting from multiple subtensor decompositions would be inconsistent (i.e., cross-factor contamination would exist). While the correct ordering could be selected by measuring the difference between a sample of the original tensor data and the reconstructed values, the number of possible combinations of tensor orderings used to test consistency would grow exponentially, making such decomposition impractical. Solutions to the ordering problem, such as those implemented with TTA, are based on transitive subtensor modal factor matching, such as... Figure 9The following illustrates the case of two subtensors (elements 900 and 902). A tensor (element 500) is partitioned into subtensors (elements 900 and 902), and each subtensor is decomposed individually and independently using ICAT (or a conventional tensor decomposition method). For illustrative purposes, it is assumed that the combination of subtensors (elements 900 and 902) has rank 4 (i.e., it can be reconstructed using four factors). Subtensors (elements 900 and 902) also have rank 4, and thus, they are decomposed into four subtensor factors (elements 904, 906, 908, and 910) using, for example, ICAT. Since the subtensor modal factors (e.g., elements 904, 906, 908, and 910) contain the same information as the full tensor modal factors, it should be possible to combine the subtensor factors to form a decomposition of the full tensor. However, because the ordering of the subtensor factors is inconsistent across the subtensors, naturally combining them will lead to cross-factor contamination. The invention described herein uses TMM to address this problem. The subtensor modal factors (elements 904, 906, 908, and 910) are shown with distinctly different shadings, where factors with the same shading have the same value. It should be noted that the outputs of the factors for subtensors 900 and 902 are ordered in... Figure 9 The top is inconsistent (element 912). In other words, combining modal factor element 906 with modal factor element 904 would be incorrect because the combination would have a mismatched element. The correct ordering can be determined by matching the subtensor modal factors common to the subtensors (elements 904, 906, 908, and 910). Based on this transitive property, the other modal factors will then also be correctly matched. The four combinations of subtensor modal factors that form consistent tensor factors are in Figure 9 The bottom shows (element 914).

[0060] Figure 10 The next step in the TTA shown is to calculate the subtensor factor coefficients λ. n (Element 1000). When factor coefficients are summed together to reconstruct the tensor, these factor coefficients determine the relative weighting of the tensor factors. A sub-tensor factor dictionary matrix can be used ( Figure 10The factor coefficients are determined by the pseudo-inverse matrix of the dictionary matrix (element 1002), where the columns of the sub-tensor factor dictionary matrix are vectorizations of the sub-tensor factors. This can be achieved by establishing a system of linear equations using samples of the tensor and the dictionary matrix (element 1002), where the columns are formed by the vectorized sub-tensor factors. Multiplying the dictionary matrix (element 1002) by the column vector of factor coefficients (element 1000) yields a value (element 1004) that matches the measured tensor values. The system of linear equations can be represented in matrix-vector form, and the factor coefficients (element 1000) are solved by multiplying the measured data vector (element 1004) by the generalized inverse of the dictionary matrix (element 1002). If the coefficient vector is sparse, a useful alternative is to use a reconstruction method that imposes sparsity constraints by minimizing the L1 norm of the coefficient vector.

[0061]

[0062] Here, ||·||1 is the L1 norm (the absolute sum of the entries). It is the squared Euclidean norm. Many methods can be used to solve the above L1 minimization and related objective functions to recover the provably optimal sparse representation.

[0063] By using time as a modality of a tensor, TTA can naturally and efficiently process streaming tensor data using sliding time windows, such as... Figure 11 As shown. At each time increment, it is only necessary to compute the decomposition of the new "head" sub-tensor (element 1100) and discard the "tail" sub-tensor (element 1102) in a sliding window. This method can be used, for example, to analyze time-varying behaviors and events in communication networks or social media data.

[0064] like Figure 12 The TTA processing steps illustrated can be summarized as follows. First, the tensor is divided into subtensors (1200 elements each). Although the subtensors are in... Figure 11The subtensors are shown as continuous, but they can also be interleaved. Since subtensor decomposition can be performed independently, if multiple processors are available, they can be efficiently utilized by assigning one or more subtensors to each processor (element 1202). Each processor then decomposes the individual subtensors into modal factors using ICAT or regular tensor decomposition (element 1204). Next, the subtensor modal factors are collected, and the correct ordering and combination of the subtensor modal factors are determined using TMM (element 1206). Then, a system of linear equations is established using samples of the tensor values, and the tensor factor coefficients are solved (element 1208). Finally, a modality is used as the time dimension to enable the use of a sliding time window to process the streaming tensor data (element 1210). An example application of the decomposed tensor data is detecting different network activities or patterns of communication between nodes in a network over time. Different activities are represented by individual tensor factors describing which nodes are communicating, the amount of communication, and the duration of communication.

[0065] Previously, subtensor analysis was not used for tensor decomposition due to the arbitrary and inconsistent ordering of subtensor factor outputs. The ordering of subtensor factors can differ for each subtensor, and thus, typically, the modal factors of one subtensor will not "match" with those of other subtensors. Analyzing the ordering using conventional methods is impractical because the number of possible combinations of factors to search is enormous. TTA is unique in that it uses Transient Tensor Modality Matching (TMM) based on Independent Component Analysis (ICA) to address this "factor mismatch" problem. TMM works by "entangling" or concatenating the dimensions or modes of individual subtensors in a vectorized representation. When the vectors are separated into factors using ICA, the factors of entangled modes remain together in the correct combination because they are parts of the same signal. TMM is used to determine the correct modal combinations of other modes. Simple linear equations can then be solved to determine the weighting coefficients of the factors in the modeling data.

[0066] The invention described herein can be used for computer network traffic analysis. A computer network can be described as a directed graph consisting of vertices and edges or links connecting the vertices. A vertex or node is a host computer identified by an Internet Protocol (IP) address, while a link between nodes represents a path along which information, in the form of data packets, travels. A link is directional in the sense that each packet contains the packet's destination address and source address. Routers direct packet packets to other gateway computers and routers that know the network's local domain (e.g., the addresses of computers within that domain). If the domain does not contain the destination address in the packet, the gateway routes the packet to the next gateway until it reaches a gateway capable of delivering the packet to its final destination. Routers maintain a data record table containing the origin, destination address, and routing time of each packet. The router data table can be represented as a multidimensional tensor, where the tensor dimension or modality can be, for example, the packet source address, packet destination address, and routing time. Tensor values ​​can represent the number of packets flowing between addresses within a fixed time interval. In short, a tensor represents the data traffic flow over time within a specific network domain or set of domains, where a domain is a subnet of a larger network.

[0067] One important application of this invention is analyzing the structure or patterns in network traffic flows to detect network attacks, bottlenecks, or specific communication activities indicating high utilization of network resources by a group or community of network nodes. Tensor decomposition is a method of finding such structures in data by representing tensors as the sum of factor tensors formed by the outer product of 1D modal factors. Each tensor factor describes a communication pattern between subgroups of source and destination nodes in the network. For example, a denial-of-service network attack can be represented by tensor factors where many source nodes suddenly connect to the destination node in a "star" pattern. Tensor decomposition can be performed in real time at the router level after data is received from one or more routers, or on a host computer or group of computers in offline mode. The output is a list of warnings for such patterns in the traffic flow, along with information about the nodes involved. The invention described herein covers Transitive Tensor Analysis (TTA), a method for performing tensor decomposition more efficiently. This is valuable because tensors representing network traffic can be so large that a single processor cannot perform the decomposition due to processing time and / or storage space limitations. TTA enables decomposition into smaller subproblems, which can be distributed across multiple processors for parallel computation of one subproblem per processor. The results can then be combined to form a full tensor factor.

[0068] While TTA can be applied to any data analysis application involving tensor data, a non-limiting example is using TTA to more efficiently detect communicating entities and groups of activity in computers and social networks. For instance, it can be used to detect intrusions or attacks on computer networks, or to detect groups with shared interests on social networks. Intrusion detection can be used to alert network operators to intrusions, attacks, or other anomalies, and, when they occur, indicate the location of the intrusion, attack, or anomaly within the network (e.g., node identification). Additionally, the system can provide guidance for mitigating attacks by identifying the targeted network node and removing it from the network, or cutting off / blocking communication from a node exhibiting anomalies.

[0069] Finally, although the invention has been described according to several embodiments, those skilled in the art will readily recognize that the invention can have other applications in other settings. It should be noted that many embodiments and implementations are possible. Furthermore, the appended claims are by no means intended to limit the scope of the invention to the specific embodiments described above. Additionally, any use of the phrase "means for..." is intended to induce a means-plus-function interpretation of the elements and claims, and any element not specifically using the phrase "means for..." should not be interpreted as a means-plus-function element, even if the claims otherwise include the word "means". Furthermore, although specific method steps have been described in a particular order, these method steps can be performed in any desired order and fall within the scope of the invention.

Claims

1. A system for detecting network activity using transitive tensor analysis, the system comprising: Non-transitory computer-readable medium and one or more processors; The non-transitory computer-readable medium is encoded with executable instructions such that, when the executable instructions are executed, the one or more processors perform the following operations: A tensor with multiple tensor modes is divided into multiple sub-tensors, wherein the tensor represents communication on a communication network including streaming network data. Each sub-tensor is decomposed into multiple sub-tensor mode factors individually and independently on a separate processor; Transient mode factor matching is used to determine the ordering of the plurality of sub-tensor mode factors; Determine the sub-tensor factor coefficient set for the plurality of sub-tensor mode factors; The relative weighting of the plurality of subtensor mode factors is determined using the subtensor factor coefficient set; Detect multiple activity modes represented by the plurality of sub-tensor mode factors; and Based on the detection, an anomaly warning is generated, indicating the location of the anomaly in the communication network and the time of its occurrence.

2. The system according to claim 1, wherein, The plurality of tensor modes include: two tensor modes representing connections between nodes of the communication network, and a third tensor mode representing the time dependency of the connections between nodes of the communication network.

3. The system according to claim 1, wherein, Decomposing each subtensor includes: The subtensor is converted into a one-dimensional vector signal set; The one-dimensional vector signal set is used as input for independent component analysis; and Extract the multiple sub-tensor mode factors.

4. The system according to claim 1, wherein, Determining the order of the plurality of sub-tensor mode factors includes matching the sub-tensor mode factors that are common among the plurality of sub-tensors.

5. The system according to claim 4, wherein, Determining the order of the plurality of sub-tensor modal factors further includes: using the matching of the sub-tensor modal factors common among the plurality of sub-tensors to match non-common sub-tensor modal factors via transitivity.

6. The system according to claim 1, wherein, Determining the sub-tensor factor coefficient set for the plurality of sub-tensor modal factors includes using a pseudo-inverse matrix of the sub-tensor factor dictionary matrix, wherein the columns of the sub-tensor factor dictionary matrix are vectorizations of the plurality of sub-tensor factors.

7. The system according to claim 1, wherein, The one or more processors also perform the following operations: apply tensor analysis to a sliding time window of streaming network data.

8. A computer-implemented method for detecting network activity using transitive tensor analysis, the method comprising the following actions: To cause one or more processors to execute instructions encoded in one or more associated memories, each of which is a non-transitory computer-readable medium, such that, when the instructions are executed, the one or more processors perform the following operations: A tensor with multiple tensor modes is divided into multiple sub-tensors, where, The tensor represents communication on a communication network, including streaming network data; Each sub-tensor is decomposed into multiple sub-tensor mode factors individually and independently on a separate processor; Transient mode factor matching is used to determine the ordering of the plurality of sub-tensor mode factors; Determine the sub-tensor factor coefficient set for the plurality of sub-tensor mode factors; The relative weighting of the plurality of subtensor mode factors is determined using the subtensor factor coefficient set; Detect multiple activity modes represented by the plurality of sub-tensor mode factors; as well as Based on the detection, an anomaly warning is generated, indicating the location of the anomaly in the communication network and the time of its occurrence.

9. The method according to claim 8, wherein, The plurality of tensor modes include: two tensor modes representing connections between nodes of the communication network, and a third tensor mode representing the time dependency of the connections between nodes of the communication network.

10. The method according to claim 8, wherein, Decomposing each subtensor includes: The subtensor is converted into a one-dimensional vector signal set; The one-dimensional vector signal set is used as input for independent component analysis; and Extract the multiple sub-tensor mode factors.

11. The method according to claim 8, wherein, Determining the order of the plurality of sub-tensor mode factors includes matching the sub-tensor mode factors that are common among the plurality of sub-tensors.

12. The method according to claim 11, wherein, Determining the order of the plurality of sub-tensor modal factors further includes: using the matching of the sub-tensor modal factors common among the plurality of sub-tensors to match non-common sub-tensor modal factors via transitivity.

13. The method according to claim 8, wherein, Determining the sub-tensor factor coefficient set for the plurality of sub-tensor modal factors includes using a pseudo-inverse matrix of the sub-tensor factor dictionary matrix, wherein the columns of the sub-tensor factor dictionary matrix are vectorizations of the plurality of sub-tensor factors.

14. The method of claim 8, further comprising causing the one or more processors to perform the following action: applying transit tensor analysis to a sliding time window of streaming network data.

15. A computer program product for detecting network activity using transitive tensor analysis, the computer program product comprising: Computer-readable instructions stored on a non-transitory computer-readable medium, the computer-readable instructions being executable by a computer having one or more processors to cause the one or more processors to perform the following operations: A tensor with multiple tensor modes is divided into multiple sub-tensors, wherein the tensor represents communication on a communication network including streaming network data. Each sub-tensor is decomposed into multiple sub-tensor mode factors individually and independently on a separate processor; Transient mode factor matching is used to determine the ordering of the plurality of sub-tensor mode factors; Determine the sub-tensor factor coefficient set for the plurality of sub-tensor mode factors; The relative weighting of the plurality of subtensor mode factors is determined using the subtensor factor coefficient set; Detect multiple activity modes represented by the plurality of sub-tensor mode factors; and Based on the detection, an anomaly warning is generated, indicating the location of the anomaly in the communication network and the time of its occurrence.

16. The computer program product according to claim 15, wherein, The plurality of tensor modes include: two tensor modes representing connections between nodes of the communication network, and a third tensor mode representing the time dependency of the connections between nodes of the communication network.

17. The computer program product according to claim 15, wherein, Decomposing each subtensor includes: The subtensor is converted into a one-dimensional vector signal set; The one-dimensional vector signal set is used as input for independent component analysis; and Extract the multiple sub-tensor mode factors.

18. The computer program product according to claim 15, wherein, Determining the order of the plurality of sub-tensor mode factors includes matching the sub-tensor mode factors that are common among the plurality of sub-tensors.

19. The computer program product according to claim 18, wherein, Determining the order of the plurality of sub-tensor modal factors further includes: using the matching of the sub-tensor modal factors common among the plurality of sub-tensors to match non-common sub-tensor modal factors via transitivity.

20. The computer program product according to claim 15, wherein, Determining the sub-tensor factor coefficient set for the plurality of sub-tensor modal factors includes using a pseudo-inverse matrix of the sub-tensor factor dictionary matrix, wherein the columns of the sub-tensor factor dictionary matrix are vectorizations of the plurality of sub-tensor factors.

Images