A computer-based network monitoring system and method

Abstract

The application provides a computer-based network monitoring system and method, which projects a traffic data vector in a computer network to a low-rank subspace representing a normal traffic pattern of the network, and then obtains a square prediction error of the computer network at a current time; if the square prediction error exceeds a behavior security baseline of the computer network, the network traffic matrix within a current time granularity window is decomposed into a low-rank matrix and a sparse matrix, and then a plurality of abnormal source node and destination node pairs are screened based on the low-rank matrix and the sparse matrix; alarm information containing specific abnormal addresses and attack types within the current time granularity window is generated according to the confidence features of each abnormal source node and destination node pair, and then a network controller is triggered to perform shaping processing on abnormal traffic in the abnormal source node and destination node pair. Based on the above scheme, sparse decomposition positioning based on low-rank subspace projection can be realized, thereby improving the real-time response capability of network attack identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer networks, and more particularly, to a computer-based network monitoring system and method. BACKGROUND

[0002] A computer network is a system formed by connecting multiple computers through communication equipment and lines, used for resource sharing and information exchange. Computer networks connect computer devices distributed in different geographical locations based on network protocols to form an organic whole. Computer networks can be divided into local area networks, metropolitan area networks and wide area networks, and are widely used in office, education, scientific research and commercial fields, bringing great convenience to people's life and work.

[0003] Traditional network traffic anomaly detection methods are based on fixed thresholds or simple statistical models, which are essentially a shallow fitting of historical traffic data and cannot deeply depict the inherent and complex dynamic association structure of the network in the normal operating state. Network traffic is regarded as a set of independent data points, and a global static threshold is set to determine anomalies. However, advanced persistent threats and other covert attacks embed their traffic patterns in normal business fluctuations and background noise to evade detection. This low signal-to-noise ratio characteristic makes it easy for traditional methods to misjudge these slow and small malicious activities as harmless normal fluctuations due to the lack of fine expression of the multi-dimensional traffic cooperative change rule in the model, resulting in serious false negatives and failing to meet the current requirements for early detection of unknown threats. Therefore, how to realize sparse decomposition positioning based on low-rank subspace projection to improve the real-time response capability of network attack identification has become a difficult problem in the industry. SUMMARY

[0004] The present application provides a computer-based network monitoring system and method, which can realize sparse decomposition positioning based on low-rank subspace projection, thereby improving the real-time response capability of network attack identification.

[0005] In a first aspect, the present application provides a computer-based network monitoring method, comprising:

[0006] Collecting historical traffic data between all source node and destination node pairs in the computer network in a preset time granularity window, and constructing a network traffic matrix with rows representing source node and destination node pairs and columns representing time;

[0007] Generating a low-rank subspace representing normal traffic patterns of the network based on the network traffic matrix, monitoring traffic data vectors in the computer network, projecting the traffic data vectors into the low-rank subspace, and then obtaining the squared prediction error of the computer network at the current time;

[0008] if the squared prediction error exceeds a behavior safety baseline of the computer network, decompose a network traffic matrix in a current time granularity window into a low-rank matrix and a sparse matrix, and further screen a plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix, and further determine a confidence feature of an abnormal state in each abnormal source node and destination node pair;

[0009] generate alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence feature of each abnormal source node and destination node pair, and further trigger a network controller to perform shaping processing on abnormal traffic in the abnormal source node and destination node pair.

[0010] In some embodiments, generating a low-rank subspace representing a normal traffic pattern of the network based on the network traffic matrix specifically includes:

[0011] perform principal component analysis on the network traffic matrix to obtain a covariance matrix of the computer network;

[0012] perform eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvectors;

[0013] screen a plurality of low-rank eigenvectors from all the eigenvectors to form a low-rank subspace.

[0014] In some embodiments, projecting the traffic data vector to the low-rank subspace to obtain the squared prediction error of the computer network at the current time specifically includes:

[0015] projecting the traffic data vector to the low-rank subspace to obtain a real-time reconstruction vector of the network traffic of the computer network at the current time;

[0016] calculating a residual vector between the real-time reconstruction vector and a reconstruction vector of the low-rank subspace;

[0017] determining the squared prediction error of the computer network at the current time according to the residual vector.

[0018] In some embodiments, decomposing the network traffic matrix in the current time granularity window into a low-rank matrix and a sparse matrix specifically includes:

[0019] obtaining a network traffic matrix in a current time granularity window;

[0020] performing model convergence solution on the network traffic matrix to obtain a low-rank matrix and a sparse matrix.

[0021] In some embodiments, screening a plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix specifically includes:

[0022] calculating relative abnormal contribution of each source node and destination node pair based on the low-rank matrix and the sparse matrix;

[0023] determining a node pair abnormal threshold in the computer network through all the relative abnormal contribution;

[0024] screening abnormal source node and destination node pairs from the computer network based on the node pair abnormal threshold, and obtaining a plurality of abnormal source node and destination node pairs.

[0025] In some embodiments, determining the confidence feature of the abnormal state in each abnormal source node and destination node pair specifically comprises:

[0026] for each abnormal source node and destination node pair, obtaining an abnormal intensity sequence of the source node and destination node pair from the sparse matrix;

[0027] calculating the average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity in the abnormal intensity sequence;

[0028] determining the confidence feature of the abnormal state in the source node and destination node pair through the average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity, thereby obtaining the confidence feature of the abnormal state in each abnormal source node and destination node pair.

[0029] In some embodiments, generating the alarm information containing specific abnormal address and attack type within the current time granularity window according to the confidence feature of each abnormal source node and destination node pair specifically comprises:

[0030] determining the attack type within the current time granularity window according to the confidence feature of each abnormal source node and destination node pair;

[0031] determining the network address of each abnormal source node and destination node pair;

[0032] generating the alarm information containing specific abnormal address and attack type within the current time granularity window through the attack type and the network address.

[0033] In some embodiments, the historical traffic data between all source node and destination node pairs in the computer network is collected using network probes.

[0034] In some embodiments, the behavior security baseline is a threshold value between normal and abnormal computer network traffic.

[0035] In a second aspect, the present application provides a computer-based network monitoring system, comprising:

[0036] The collecting module is configured to collect historical traffic data between all source node and destination node pairs in a computer network in a preset time granularity window, and construct a network traffic matrix with rows representing source node and destination node pairs and columns representing time;

[0037] The processing module is configured to generate a low-rank subspace representing normal traffic patterns of the network based on the network traffic matrix, monitor a traffic data vector in the computer network, project the traffic data vector to the low-rank subspace, and obtain a squared prediction error of the computer network at a current time instant.

[0038] The processing module is further configured to, if the squared prediction error exceeds a behavior security baseline of the computer network, decompose the network traffic matrix in a current time granularity window into a low-rank matrix and a sparse matrix, filter a plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix, and determine a confidence feature of an abnormal state in each abnormal source node and destination node pair.

[0039] The executing module is configured to generate alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence feature of each abnormal source node and destination node pair, and trigger a network controller to perform shaping processing on abnormal traffic in the abnormal source node and destination node pair.

[0040] The technical scheme provided by the embodiments disclosed in the present application has the following beneficial effects:

[0041] In the computer-based network monitoring system and method provided in the present application, historical traffic data between all source node and destination node pairs in a computer network is collected in a preset time granularity window, and a network traffic matrix with rows representing source node and destination node pairs and columns representing time is constructed. A low-rank subspace representing normal traffic patterns of the network is generated based on the network traffic matrix, a traffic data vector in the computer network is monitored, the traffic data vector is projected to the low-rank subspace, and a squared prediction error of the computer network at a current time instant is obtained. If the squared prediction error exceeds a behavior security baseline of the computer network, the network traffic matrix in the current time granularity window is decomposed into a low-rank matrix and a sparse matrix, a plurality of abnormal source node and destination node pairs are filtered based on the low-rank matrix and the sparse matrix, and a confidence feature of an abnormal state in each abnormal source node and destination node pair is determined. Alarm information containing specific abnormal addresses and attack types in the current time granularity window is generated according to the confidence feature of each abnormal source node and destination node pair, and a network controller is triggered to perform shaping processing on abnormal traffic in the abnormal source node and destination node pair.

[0042] It can be seen that, in the present application, the alarm information containing specific abnormal addresses and attack types in the current time granularity window is generated according to the confidence features of each abnormal source node and destination node pair, and then the network controller triggers the shaping processing of the abnormal traffic in the abnormal source node and destination node pair. First, the overall abnormality degree of the entire computer network at the current time can be obtained by determining the square prediction error. The low-rank subspace represents the normal traffic pattern of the network. When the traffic data vector deviates from the low-rank subspace, the square prediction error will increase, and the system can quickly detect traffic anomalies. Because any traffic deviating from the normal pattern will be immediately detected, once the square prediction error exceeds the safety baseline, the system will trigger further analysis, thereby achieving rapid response to network attacks, and improving the real-time response capability of network attack identification, and detecting anomalies at the early stage of attack. Then, the quantifiable abnormal behavior features of each abnormal source node and destination node pair can be obtained by determining the confidence features. The confidence features can clearly indicate the specific features of abnormal traffic, such as abnormal addresses and attack types, and the system can more accurately identify the nature and source of network attacks, thereby improving the accuracy of network attack identification. At the same time, accurate confidence features also provide a reliable basis for subsequent alarm information generation and traffic shaping processing, further enhancing the real-time response capability of network attack identification, and ensuring that the network can respond to various attacks in a timely and effective manner. In summary, based on the above scheme, sparse decomposition positioning based on low-rank subspace projection can be realized, thereby improving the real-time response capability of network attack identification. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0044] Figure 1 is an exemplary flowchart of a computer-based network monitoring method according to some embodiments of the present application;

[0045] Figure 2 is a flowchart of determining an abnormal source node and destination node pair according to some embodiments of the present application;

[0046] Figure 3 is a structural schematic diagram of a computer-based network monitoring system according to some embodiments of the present application;

[0047] Figure 4is a structural schematic diagram of a computer device for implementing a computer-based network monitoring method according to some embodiments of the present application. DETAILED DESCRIPTION

[0048] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in combination with the drawings of the specification and specific embodiments.

[0049] Reference Figure 1 The figure is an exemplary flowchart of a computer-based network monitoring method according to some embodiments of the present application, which mainly includes the following steps:

[0050] In step 101, historical traffic data between all source node and destination node pairs in the computer network is collected in a preset time granularity window, and a network traffic matrix is constructed, in which rows represent source node and destination node pairs and columns represent time.

[0051] It should be noted that in the present application, the preset time granularity window is the basic time unit for data collection and aggregation; the source node and destination node pair is the combination of the starting network device and the ending network device of the communication flow in the computer network; the historical traffic data is a data set reflecting the network load condition in the computer network; and the network traffic matrix is a two-dimensional data structure in which the traffic data of all source node and destination node pairs in the computer network at consecutive multiple time granularity windows are systematically structured.

[0052] In specific implementation, first, the operation and maintenance console of the computer network presets a fixed time interval (for example, five minutes) as the preset time granularity window according to management requirements; then, the network probe automatically queries and aggregates the total number of bytes flowing between each group of source node and destination node pairs in the network within the preset time granularity window at the end of each such window as the traffic value of the preset time granularity window, so as to take the set of traffic values of the preset time granularity window within a specified time period (by default, the last week) as the historical traffic data between all source node and destination node pairs in the computer network; finally, the data processing unit of the computer network arranges and arranges the traffic snapshots of multiple consecutive time windows, takes each source node and destination node pair as a row of the data matrix, takes each preset time granularity window as a column of the data matrix, and fills the corresponding traffic values to the corresponding positions of the matrix, so as to take the arranged structured data as the network traffic matrix.

[0053] In step 102, a low-rank subspace representing the normal traffic pattern of the network is generated based on the network traffic matrix, the traffic data vector in the computer network is monitored, the traffic data vector is projected into the low-rank subspace, and then the squared prediction error of the computer network at the current time is obtained.

[0054] In some embodiments, generating a low-rank subspace representing normal traffic patterns of a computer network based on the network traffic matrix can be achieved by the following steps:

[0055] performing principal component analysis on the network traffic matrix to obtain a covariance matrix of the computer network;

[0056] performing eigen decomposition on the covariance matrix to obtain a plurality of eigenvectors;

[0057] selecting a plurality of low-rank eigenvectors from all the eigenvectors to form the low-rank subspace.

[0058] It should be noted that in the present application, the low-rank subspace is a matrix space that captures and represents the dominant and stable normal traffic patterns in the computer network; the covariance matrix is a mathematical representation that can completely depict the linear correlation of the traffic changes between each source node and destination node pair of the computer network; and the eigenvector is a basis vector that can reveal the mutually independent change patterns hidden inside the network traffic data.

[0059] In a specific implementation, first, principal component analysis is performed on the network traffic matrix to obtain the covariance matrix of the computer network, which can be achieved in the following manner: the computing unit of the computer network performs centralized preprocessing on the network traffic matrix, i.e., subtracts the mean value of each row of data to eliminate the influence of different traffic baselines of each source node and destination node pair, and based on the centralized network traffic matrix, the covariance value between each pair of source node and destination node pair is calculated according to the mathematical definition of covariance, and all the covariance values together form a square matrix as the covariance matrix of the computer network; then, eigenvalue decomposition is performed on the covariance matrix to obtain a plurality of eigenvectors, which can be achieved in the following manner: the computing unit of the computer network performs standard matrix operation (i.e., eigenvalue decomposition) on the covariance matrix, i.e., solves a special vector (i.e., eigenvector) and a corresponding scalar (i.e., eigenvalue), wherein each eigenvector satisfies that its direction remains unchanged after linear transformation represented by the covariance matrix, and the solving process finally obtains a plurality of eigenvectors which are orthogonal to each other and have the same dimension as the covariance matrix; finally, a plurality of low-rank eigenvectors are selected from all the eigenvectors to form a low-rank subspace, which can be achieved in the following manner: the computing unit of the computer network arranges all the eigenvectors in descending order according to their corresponding eigenvalues, and selects a specified number (default is 5) of eigenvectors from the front end of the sorted sequence as low-rank eigenvectors according to a predetermined rule (for example, selecting the first k eigenvectors so that the cumulative variance contribution rate exceeds a set threshold), and the change mode corresponding to the low-rank eigenvectors dominates the normal behavior of the network traffic, so that the matrix space formed by the set of low-rank eigenvectors which can efficiently represent most of the normal traffic behavior of the network with a low dimension is taken as the low-rank subspace.

[0060] In some embodiments, projecting the traffic data vector into the low-rank subspace to obtain the squared prediction error of the computer network at the current time can be achieved in the following steps:

[0061] Projecting the traffic data vector into the low-rank subspace to obtain a real-time reconstruction vector of the network traffic of the computer network at the current time;

[0062] Calculating the residual vector between the real-time reconstruction vector and the reconstruction vector of the low-rank subspace;

[0063] Determining the squared prediction error of the computer network at the current time according to the residual vector.

[0064] It should be noted that in this application, the squared prediction error represents the overall abnormality degree of the entire computer network at the current time; the real-time reconstruction vector is the numerical value of the traffic data at the current time under the ideal normal state; and the residual vector represents the deviation degree of the network traffic at the current time relative to the historical normal traffic mode.

[0065] In a specific implementation, first, the traffic data vector is projected to the low-rank subspace to obtain the real-time reconstruction vector of the network traffic of the computer network at the current time. This can be achieved by obtaining the traffic data vector collected at the current time, and performing matrix multiplication between the traffic data vector and the vector matrix in the low-rank subspace, so that the result of the matrix multiplication is taken as the result of the projection operation, that is, the real-time reconstruction vector of the network traffic of the computer network at the current time. Then, the residual vector between the real-time reconstruction vector and the reconstruction vector of the low-rank subspace can be achieved by performing subtraction operation between the original traffic data vector at the current time and the real-time reconstruction vector at corresponding positions to obtain a difference vector with the same dimension as the residual vector. Finally, the squared prediction error of the computer network at the current time can be determined according to the residual vector, which can be achieved by performing square operation on each element in the residual vector, summing all the squared results, and taking the final single numerical value as the squared prediction error of the computer network at the current time.

[0066] In step 103, if the squared prediction error exceeds the behavior security baseline of the computer network, the network traffic matrix within the current time granularity window is decomposed into a low-rank matrix and a sparse matrix, and then a plurality of abnormal source node and destination node pairs are screened based on the low-rank matrix and the sparse matrix, and the confidence features of the abnormal state in each abnormal source node and destination node pair are determined.

[0067] In some embodiments, the network traffic matrix within the current time granularity window can be decomposed into a low-rank matrix and a sparse matrix by the following steps:

[0068] Obtaining the network traffic matrix within the current time granularity window;

[0069] Performing model convergence solution on the network traffic matrix to obtain a low-rank matrix and a sparse matrix.

[0070] It should be noted that in the present application, the low-rank matrix carries the stable, relevant normal traffic component in the network, and the low-rank matrix can represent the dominant normal behavior pattern of the whole network; and the sparse matrix carries the sudden, isolated abnormal traffic component, and the sparse matrix can accurately record the position and amplitude of the abnormal event; in specific implementation, first, a network traffic matrix in a current time granularity window is obtained; then, the network traffic matrix is subjected to model convergence solving to obtain the low-rank matrix and the sparse matrix, which can be implemented in the following manner, that is, the network traffic matrix is input into a preset optimization model, the optimization model is based on matrix decomposition and imposes low-rank and sparsity constraints, an iterative algorithm is used to solve the optimization model, the numerical values of two to-be-solved matrices are updated for multiple times until a preset convergence condition is met, and then the calculation is stopped, so that the two result matrices finally output when the convergence condition is met, which respectively represent the normal traffic base and the abnormal traffic component, are taken as the low-rank matrix and the sparse matrix.

[0071] In some embodiments, a plurality of abnormal source node and destination node pairs are screened based on the low-rank matrix and the sparse matrix, and the abnormal source node and destination node pairs are determined by referring to Figure 2 The diagram is a flowchart for determining abnormal source node and destination node pairs in some embodiments of the present application, and the determination of the abnormal source node and destination node pairs in the present embodiment can be implemented in the following steps:

[0072] In step 1031, the relative abnormal contribution degree of each source node and destination node pair is calculated according to the low-rank matrix and the sparse matrix;

[0073] In step 1032, a node pair abnormal threshold in the computer network is determined by all the relative abnormal contribution degrees;

[0074] In step 1033, the abnormal source node and destination node pairs are screened from the computer network based on the node pair abnormal threshold, and a plurality of abnormal source node and destination node pairs are obtained.

[0075] In a specific implementation, firstly, the relative abnormal contribution of each source node and destination node pair can be calculated according to the low-rank matrix and the sparse matrix in the following manner: for each source node and destination node pair, the abnormal traffic sequence of the source node and the destination node pair is extracted from the sparse matrix, and the normal traffic sequence of the source node and the destination node pair is extracted from the low-rank matrix, and the ratio of the norm of the abnormal traffic sequence to the norm of the normal traffic sequence is calculated as the relative abnormal contribution of the source node and the destination node pair. Through the above manner, the relative abnormal contribution of each source node and destination node pair can be obtained, which represents the severity of the abnormal traffic of each source node and destination node pair. Then, the abnormal threshold of the node pair in the computer network can be determined in the following manner: all the relative abnormal contributions are sorted from large to small as an abnormal contribution sequence, and the element value at the 90% position in the abnormal contribution sequence is selected as the abnormal threshold of the node pair in the computer network. The abnormal threshold of the node pair is a quantitative standard for determining whether each source node and destination node pair is in an abnormal state. Finally, the abnormal source node and destination node pair in the computer network can be screened based on the abnormal threshold of the node pair in the following manner: the abnormal source node and destination node pair whose relative abnormal contribution is greater than the abnormal threshold of the node pair in the computer network is obtained, and a plurality of abnormal source node and destination node pairs can be obtained.

[0076] In some embodiments, the confidence feature of the abnormal state in each abnormal source node and destination node pair can be determined in the following steps:

[0077] For each abnormal source node and destination node pair, the abnormal intensity sequence of the source node and the destination node pair is obtained from the sparse matrix.

[0078] The average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity in the abnormal intensity sequence are calculated.

[0079] The confidence feature of the abnormal state in the source node and the destination node pair is determined by the average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity, and the confidence feature of the abnormal state in each abnormal source node and destination node pair is obtained.

[0080] It should be noted that in the present application, the confidence feature is a quantifiable abnormal behavior feature of each abnormal source node and destination node pair; the abnormal intensity sequence is a sequence recording the abnormal dynamic change process of the source node and the destination node pair during the entire detection window; the average abnormal intensity is an index for measuring the average severity level of abnormal traffic, the abnormal intensity variance is an index for measuring the fluctuation stability of abnormal traffic, and the maximum abnormal intensity is an index for measuring the most severe impact degree of abnormal traffic.

[0081] In a specific implementation, first, for each abnormal source node and destination node pair, the abnormal intensity sequence of the source node and the destination node pair can be obtained from the sparse matrix in the following manner: for each abnormal source node and destination node pair, the data processing unit of the computer network extracts the element values of all columns of the row corresponding to the source node and the destination node pair in the sparse matrix according to the row index of the source node and the destination node pair in the sparse matrix, and arranges all the element values in column order, so as to form a sequence of the abnormal traffic values of the source node and the destination node pair at each time within the entire time granularity window as the abnormal intensity sequence of the source node and the destination node pair; then, the average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity in the abnormal intensity sequence can be calculated in the following manner: the arithmetic mean of all elements in the abnormal intensity sequence is calculated as the average abnormal intensity, the average of the squares of the deviations of each element in the abnormal intensity sequence from the average abnormal intensity is calculated as the abnormal intensity variance, and the element with the largest absolute value in the abnormal intensity sequence is found to obtain the maximum abnormal intensity; finally, the confidence feature of the abnormal state in the source node and the destination node pair is determined by the average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity, and the confidence feature of the abnormal state in each abnormal source node and destination node pair can be obtained in the following manner: the set of the average abnormal intensity, the abnormal intensity variance and the maximum abnormal intensity is taken as the confidence feature of the abnormal state in the source node and the destination node pair, and the confidence feature of the abnormal state in each source node and destination node pair can be obtained by the above manner.

[0082] In step 104, alarm information containing specific abnormal addresses and attack types within the current time granularity window is generated according to the confidence feature of each abnormal source node and destination node pair, and further triggering the network controller to perform shaping processing on the abnormal traffic in the abnormal source node and destination node pair.

[0083] In some embodiments, the alarm information containing specific abnormal addresses and attack types within the current time granularity window can be generated according to the confidence feature of each abnormal source node and destination node pair in the following steps:

[0084] According to the confidence feature of each abnormal source node and destination node pair, the attack type within the current time granularity window is determined;

[0085] determining network addresses of each of the abnormal source node and destination node pairs;

[0086] generating alarm information containing specific abnormal addresses and attack types within the current time granularity window through the attack type and the network addresses.

[0087] In specific implementation, first, the attack type within the current time granularity window can be determined according to the confidence features of each abnormal source node and destination node pair, which can be implemented in the following manner: an analysis unit in the computer network inputs the confidence feature vector of each abnormal source node and destination node pair into a pre-set classification model; the classification model maps the input feature vector to a specified class label according to the pre-learned decision boundary, so as to take the output result of the classification model, i.e. the qualitative judgment of the nature of the abnormal behavior, as the attack type determined within the current time granularity window; then, the network addresses of each abnormal source node and destination node pair can be determined in the following manner: an address mapping unit in the computer network queries the pre-set network device registry or address mapping relationship according to the unique identifier of the source node and destination node pair that has been marked as abnormal, and obtains the source Internet Protocol address and the destination Internet Protocol address corresponding to the source node and destination node pair, so as to take the pair of logical identifiers that can accurately locate the abnormal communication endpoints as the network addresses of the abnormal source node and destination node pair; finally, the alarm information containing specific abnormal addresses and attack types within the current time granularity window can be generated through the attack type and the network addresses in the following manner: an alarm assembling unit in the computer network binds the attack type and the network address, and attaches the timestamp of the current time granularity window, fills in the corresponding data fields according to the pre-defined alarm information format, so as to take the complete record integrating the key elements of the abnormal event as the alarm information containing specific abnormal addresses and attack types within the current time granularity window.

[0088] In some embodiments, triggering the network controller to perform shaping processing on the abnormal traffic of the abnormal source node and destination node pair can be implemented in the following manner: an alarm response module in the computer network sends the alarm information and the control strategy to be executed to the network controller through a standard southbound interface protocol; the network controller parses the instruction, identifies the source network address and the destination network address corresponding to the abnormal traffic to be controlled, and generates specific flow table rules according to the strategy; the network controller sends the flow table rules to the underlying network switching devices involved in the abnormal traffic path, so as to take the mandatory management action for isolating or weakening the influence of the abnormal traffic, which is automatically implemented according to the detection result, as the shaping processing process of the abnormal traffic, thereby completing the shaping processing of the abnormal traffic of the abnormal source node and destination node pair.

[0089] In addition, another aspect of the present application, in some embodiments, the present application provides a computer-based network monitoring system, with reference to Figure 3 The figure is a structural schematic diagram of a computer-based network monitoring system according to some embodiments of the present application, which includes a collection module 201, a processing module 202 and an execution module 203, which are described as follows:

[0090] The collection module 201 is mainly used in the present application to collect historical traffic data between all source node and destination node pairs in the computer network with a preset time granularity window, to construct a network traffic matrix with rows representing source node and destination node pairs and columns representing time;

[0091] The processing module 202 is used in the present application to generate a low-rank subspace representing the normal traffic pattern of the network based on the network traffic matrix, monitor the traffic data vector in the computer network, project the traffic data vector to the low-rank subspace, and then obtain the squared prediction error of the computer network at the current time;

[0092] It should be noted that the processing module 202 is also used to decompose the network traffic matrix in the current time granularity window into a low-rank matrix and a sparse matrix if the squared prediction error exceeds the behavior security baseline of the computer network, and then filter a plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix, and then determine the confidence features of the abnormal state in each abnormal source node and destination node pair;

[0093] The execution module 203 is mainly used in the present application to generate alarm information containing specific abnormal addresses and attack types within the current time granularity window according to the confidence features of each abnormal source node and destination node pair, and then trigger the network controller to perform shaping processing on the abnormal traffic in the abnormal source node and destination node pair.

[0094] The above describes an example of the computer-based network monitoring system and method provided by the embodiments of the present application in detail. It can be understood that the corresponding device contains the hardware structure and / or software module corresponding to each function in order to realize the above functions. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0095] In some embodiments, the present application also provides a computer device, comprising a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the computer-based network monitoring method described above.

[0096] In some embodiments, referring to Figure 4 , the dashed line in the figure indicates that the unit or the module is optional, and the figure is a structural schematic diagram of a computer device for implementing the computer-based network monitoring method according to the embodiments of the present application. The computer-based network monitoring method described in the above embodiments can be implemented by the computer device shown in Figure 4 , which comprises at least one processor 301, a memory 302, and at least one communication unit 305, and the computer device can be a terminal device or a server or a chip.

[0097] The processor 301 can be a general-purpose processor or a special-purpose processor. For example, the processor 301 can be a central processing unit (CPU), which can be used to control the computer device, execute software programs, and process data of the software programs. The computer device can further comprise a communication unit 305 to realize input (reception) and output (transmission) of signals.

[0098] For example, the computer device can be a chip, and the communication unit 305 can be an input and / or output circuit of the chip, or the communication unit 305 can be a communication interface of the chip, and the chip can be a component of a terminal device or a network device or other devices.

[0099] For another example, the computer device can be a terminal device or a server, and the communication unit 305 can be a transceiver of the terminal device or the server, or the communication unit 305 can be a transceiver circuit of the terminal device or the server.

[0100] The computer device can comprise one or more memories 302, which have programs 304 stored thereon, and the programs 304 can be run by the processor 301 to generate instructions 303, so that the processor 301 executes the method described in the above method embodiments according to the instructions 303. Optionally, the memory 302 can also store data (such as a target audit model). Optionally, the processor 301 can also read the data stored in the memory 302, and the data can be stored in the same storage address as the program 304, or the data can be stored in a different storage address from the program 304.

[0101] The processor 301 and the memory 302 can be separately arranged or integrated together, for example, on a system on chip (SOC) of the terminal device.

[0102] It should be understood that each step of the above method embodiments can be completed by a logic circuit in the form of hardware or instructions in the form of software in the processor 301, and the processor 301 can be a CPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, for example, discrete gates or transistor logic devices, or discrete hardware components.

[0103] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0104] For example, in some embodiments, the present application also provides a computer-readable storage medium, which stores instructions or codes, when the instructions or codes are run on a computer, cause the computer to perform the above-mentioned computer-based network monitoring method.

[0105] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0106] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A computer-based network monitoring method, characterized by, The method comprises the following steps: collecting historical traffic data between all source node and destination node pairs in a computer network in a preset time granularity window, and constructing a network traffic matrix with rows representing source node and destination node pairs and columns representing time; generating a low-rank subspace representing normal traffic patterns of the network based on the network traffic matrix, monitoring a traffic data vector in the computer network, projecting the traffic data vector to the low-rank subspace, and obtaining a squared prediction error of the computer network at a current time; if the squared prediction error exceeds a behavior security baseline of the computer network, decomposing a network traffic matrix in a current time granularity window into a low-rank matrix and a sparse matrix, screening a plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix, and determining a confidence feature of an abnormal state in each abnormal source node and destination node pair, wherein the confidence feature is a quantitative feature of an abnormal behavior of each abnormal source node and destination node pair; generating alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence feature of each abnormal source node and destination node pair, and triggering a network controller to perform shaping processing on abnormal traffic in the abnormal source node and destination node pair; wherein the screening of the plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix specifically comprises: for each source node and destination node pair, extracting an abnormal traffic sequence of the source node and destination node pair from the sparse matrix, extracting a normal traffic sequence of the source node and destination node pair from the low-rank matrix, calculating a ratio of a norm of the abnormal traffic sequence to a norm of the normal traffic sequence as a relative abnormal contribution degree of the source node and destination node pair, and obtaining the relative abnormal contribution degree of each source node and destination node pair; determining a node pair abnormal threshold in the computer network through all the relative abnormal contribution degrees; screening abnormal source node and destination node pairs from the computer network based on the node pair abnormal threshold, and obtaining a plurality of abnormal source node and destination node pairs; wherein the determination of the confidence feature of the abnormal state in each abnormal source node and destination node pair specifically comprises: for each abnormal source node and destination node pair, obtaining an abnormal intensity sequence of the source node and destination node pair from the sparse matrix; calculating an average abnormal intensity, an abnormal intensity variance, and a maximum abnormal intensity in the abnormal intensity sequence; determining the confidence feature of the abnormal state in the source node and destination node pair through the average abnormal intensity, the abnormal intensity variance, and the maximum abnormal intensity, and obtaining the confidence feature of the abnormal state in each abnormal source node and destination node pair.

2. The method of claim 1, wherein, The generation of the low-rank subspace representing the normal traffic patterns of the network based on the network traffic matrix specifically comprises: performing principal component analysis on the network traffic matrix to obtain a covariance matrix of the computer network; performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvectors; screening a plurality of low-rank eigenvectors from all the eigenvectors to form a low-rank subspace.

3. The method of claim 1, wherein, The projecting the traffic data vector to the low-rank subspace further comprises: The projecting the traffic data vector to the low-rank subspace further comprises: The calculating the residual vector between the real-time reconstruction vector and the reconstruction vector of the low-rank subspace further comprises: The calculating the residual vector between the real-time reconstruction vector and the reconstruction vector of the low-rank subspace further comprises:

4. The method of claim 1, wherein, The calculating the residual vector between the real-time reconstruction vector and the reconstruction vector of the low-rank subspace further comprises: The decomposing the network traffic matrix in the current time granularity window into a low-rank matrix and a sparse matrix further comprises: The decomposing the network traffic matrix in the current time granularity window into a low-rank matrix and a sparse matrix further comprises:

5. The method of claim 1, wherein, The generating the alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence features of each abnormal source node and destination node pair further comprises: The generating the alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence features of each abnormal source node and destination node pair further comprises: The generating the alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence features of each abnormal source node and destination node pair further comprises: The network probe is used to collect historical traffic data between all source node and destination node pairs in the computer network.

6. The method of claim 1, wherein, The behavior security baseline is a threshold value between normal and abnormal computer network traffic.

7. The method of claim 1, wherein, The system comprises:

8. A computer-based network monitoring system employing the method of any one of claims 1 to 7 for network monitoring, characterized by A collection module is configured to collect historical traffic data between all source node and destination node pairs in the computer network in a preset time granularity window, and construct a network traffic matrix with rows representing source node and destination node pairs and columns representing time. A processing module is configured to generate a low-rank subspace representing normal network traffic patterns based on the network traffic matrix, monitor traffic data vectors in the computer network, project the traffic data vectors to the low-rank subspace, and further obtain a square prediction error of the computer network at the current time. The processing module is further configured to, if the square prediction error exceeds a behavior security baseline of the computer network, decompose a network traffic matrix in the current time granularity window into a low-rank matrix and a sparse matrix, and further screen a plurality of abnormal source node and destination node pairs based on the low-rank matrix and the sparse matrix, and further determine confidence features of abnormal states in each abnormal source node and destination node pair. An execution module is configured to generate alarm information containing specific abnormal addresses and attack types in the current time granularity window according to the confidence features of each abnormal source node and destination node pair, and further trigger a network controller to perform shaping processing on abnormal traffic in the abnormal source node and destination node pair. ​

Images