Abnormal behavior identification method and system

Abstract

The invention discloses an abnormal behavior recognition method and system, relates to the field of network security protection, and solves the problem of low recognition accuracy of abnormal behaviors in a network. According to the embodiment of the invention, through multi-dimensional feature modeling, the feature tensor for comprehensively describing network behaviors is constructed, the problem of one-sidedness of traditional single-dimensional features is solved, and the essential difference between normal and abnormal behaviors can be depicted more finely; normal behaviors are modeled through multi-dimensional Gaussian distribution, the probability density of the normal behaviors is accurately described, and false alarms caused by normal fluctuation can be reduced; and the decision boundary is adjusted in real time through Bayesian inference, so that the change of the network state can be adapted, and the problem that the accuracy of a traditional fixed threshold is reduced due to environment change is avoided. According to the whole method, a closed loop from accurate sensing and reliable modeling to intelligent decision making is formed, and high-accuracy and low-false-alarm detection of abnormal behaviors in a high-dynamic network environment is achieved.

Description

Technical Field

[0001] This invention relates to the field of network security protection technology, and in particular to a method and system for identifying abnormal behavior. Background Technology

[0002] In recent years, the rapid popularization and deep integration of cloud computing, the Internet of Things (IoT), and 5G communication technologies have accelerated the evolution of network architecture towards a distributed and ubiquitous direction, with the scale of connections between various terminal devices and network nodes growing exponentially. From industrial IoT terminals and smart home devices to mobile office terminals, the access of massive heterogeneous devices has made network scenarios increasingly complex, and data interaction behavior in the network exhibits significant fragmentation and dynamism. Specifically, fragmentation is manifested in the intermittency of data transmission, the irregularity of data packet size, and the diversification of communication protocols. The network behavior of a single user or device is no longer a continuous and stable data stream, but consists of multiple short-term, discrete interaction fragments. Rhythm is manifested in the real-time changes in network topology, the rapid switching of service types, and the continuous iteration of attack methods. Traditional static network behavior patterns are no longer suitable for the current network environment.

[0003] Against this technological backdrop, fields such as network security protection and user behavior analysis place higher demands on the accuracy and timeliness of perception technologies. In anomaly detection scenarios, single-dimensional perception relies solely on a single type of network feature for judgment, which is highly prone to misjudgment or missed judgment. Relying on this single dimension alone is insufficient to capture the characteristics of abnormal behavior at multiple levels, such as protocol interaction, data encryption methods, and node associations, resulting in a significant decrease in the accuracy of abnormal behavior identification and making it difficult to achieve timely early warning and interception of abnormal behavior.

[0004] Therefore, there is a need for a method and system for identifying abnormal behavior. Summary of the Invention

[0005] To address the low accuracy of existing technologies in identifying abnormal behavior in networks, this invention provides an abnormal behavior identification method and system, which can improve the accuracy of identifying abnormal behavior in networks. The specific technical solution is as follows: In a first aspect, embodiments of this application provide an abnormal behavior identification method, including: Acquire network traffic data; construct a multidimensional feature tensor for the global session based on the network traffic data, which includes temporal feature vectors, spatial feature vectors, and semantic feature vectors; model normal behavior in the network traffic data using a multidimensional Gaussian distribution based on the multidimensional feature tensor to obtain the probability density function of the normal behavior; calculate a target threshold based on the network traffic data using a Bayesian inference mechanism; and identify abnormal behavior in the network traffic data based on the probability density function of the normal behavior and the target threshold.

[0006] Preferably, after identifying the abnormal behavior in the network traffic data, the method further includes: acquiring new session traffic data; performing sparse low-rank decomposition on the multidimensional feature tensor to obtain a low-rank representation of the multidimensional feature tensor; converting the new session traffic data into a rank tensor with the same dimension as the low-rank representation; and incrementally updating the multidimensional feature tensor based on the rank tensor and the low-rank representation.

[0007] Preferably, the probability density function is a multidimensional Gaussian distribution function, and the covariance matrix in the multidimensional Gaussian distribution function is obtained by Ledoit-Wolf contraction estimation; the contraction coefficient in the Ledoit-Wolf contraction estimation is based on the variance of the elements of the sample covariance matrix, and the estimation formula for the variance of the elements of the sample covariance matrix includes: ; in, Let be the element in the i-th row and j-th column of the sample covariance matrix. This is a variance estimate for the elements of the sample covariance matrix, where n is the sample size and k is the sample index. , Let i be the observed values ​​of the i-th and j-th flow indicators in the k-th sample. , Let be the average of the i-th and j-th flow indicators.

[0008] Preferably, the formula for calculating the target threshold includes: ; ; in, This represents session traffic data within the current time window; Indicates the target threshold. This represents the probability density boundary threshold for determining anomalies. For the threshold prior probability distribution based on historical experience, This represents the probability of observing the current data at a given threshold. This is a threshold posterior probability distribution that combines historical experience with current observations.

[0009] Preferably, after identifying the abnormal behavior in the network traffic data, the method further includes: determining the attack type of the abnormal behavior based on the traffic data of the abnormal behavior.

[0010] Preferably, the construction of a multidimensional feature tensor for a global session based on network traffic data includes: calculating the time interval of a session event sequence based on the network traffic data, and analyzing the temporal correlation based on the time interval using an autocorrelation function to obtain the time feature vector; wherein, the session event sequence is a sequence formed by the creation time and end time of all sessions within a preset time window; calculating the interaction strength between all hosts corresponding to all sessions within the preset time window based on the network traffic data, and constructing a communication topology matrix based on the interaction strength to obtain the spatial feature vector; mapping the protocol semantics to a high-dimensional feature space through embedding vector learning to obtain the semantic feature vector; and combining the time feature vector, the spatial feature vector, and the semantic feature vector into the session feature vector through tensor decomposition.

[0011] Preferably, the acquisition of network traffic data includes: capturing raw data packets from the network interface using a zero-copy ring buffer; and partitioning the raw data packets based on the 5-tuple and application layer session identifier to obtain the network traffic data based on session aggregation.

[0012] Secondly, embodiments of this application provide an abnormal behavior recognition system, applied to the method described in the first aspect, the system comprising: The acquisition module retrieves network traffic data. The feature engineering module is used to construct a multidimensional feature tensor of the global session based on network traffic data. This multidimensional feature tensor includes a temporal feature vector, a spatial feature vector, and a semantic feature vector. The modeling module is used to model the normal behavior in the network traffic data based on the multidimensional feature tensor using a multidimensional Gaussian distribution, and obtain the probability density function of the normal behavior. The calculation module is used to calculate the target threshold based on the network traffic data through a Bayesian inference mechanism; The identification module is used to identify abnormal behavior in the network traffic data based on the probability density function of the normal behavior and the target threshold.

[0013] Thirdly, embodiments of this application provide a computing device, including: a memory for storing a program; and a processor for loading the program to execute the method as described in the first aspect.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method described in the first aspect.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: By constructing a feature tensor that comprehensively describes network behavior through multi-dimensional feature modeling, it overcomes the problem of the one-sidedness of traditional single-dimensional features and can more finely characterize the essential differences between normal and abnormal behavior; by modeling normal behavior through multi-dimensional Gaussian distribution, it accurately describes the probability density of normal behavior and can reduce false alarms caused by normal fluctuations; by adjusting the decision boundary in real time through Bayesian inference, it can adapt to changes in network state and avoid the problem of accuracy decline caused by environmental changes in traditional fixed thresholds. The entire method forms a closed loop from accurate perception and reliable modeling to intelligent decision-making, achieving high accuracy and low false alarm detection of abnormal behavior in highly dynamic network environments. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0017] Figure 1 A flowchart illustrating an abnormal behavior identification method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an abnormal behavior recognition system provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] To address the problem of low accuracy in identifying abnormal behavior in networks using traditional methods, this invention provides an abnormal behavior identification method and system that can improve the accuracy of identifying abnormal behavior in networks.

[0023] Please see Figure 1 , Figure 1 This application provides a flowchart illustrating an abnormal behavior identification method, which is applied to a computing device. Figure 1 As shown, the method includes: Step 101: The computing device acquires network traffic data.

[0024] The computing device can be a server, network device, or terminal, specifically a server, network device, or terminal within the target area network. It can directly or indirectly acquire network traffic data within the target area network. The target area network serves as a security protection zone, requiring anomaly detection and defense. The computing device can acquire network traffic data within this target area network, identify anomalies, and then execute corresponding defense strategies.

[0025] The computing devices can employ a distributed acquisition mechanism, combining mirror ports and distributed probes, to ensure data integrity in large-scale networks.

[0026] Preferably, the acquisition of network traffic data includes: capturing raw data packets from the network interface using a zero-copy ring buffer;

[0027] Specifically, for each network node within the target area network, a shared circular buffer can be established between its driver layer and the acquisition application. Captured data packets are directly stored in this buffer, and the acquisition program reads from this buffer, avoiding unnecessary memory copying, greatly improving acquisition efficiency and reducing CPU overhead.

[0028] Then, each network node, acting as a computing device, executes the method of the embodiments of this application based on the captured data packets to identify abnormal behavior; alternatively, data packets captured by some or all network nodes can be aggregated at the computing device for identifying abnormal behavior.

[0029] Not all traffic is valuable for anomaly detection. During capture, the computing device can perform preliminary filtering based on predefined whitelist rules or random sampling rates to reduce the load on subsequent processing modules and ensure stable system operation even under overload conditions.

[0030] The computing device can then identify and process invalid, erroneous, or redundant information in the raw data stream, ensuring the accuracy and consistency of the data.

[0031] The computing device can perform integrity checks on data packets, specifically by checking the CRC / FCS checksum of the data packets; at the same time, it verifies the header checksum and TCP / UDP checksum of the IP datagram to ensure that no bit errors occur during data transmission.

[0032] The computing device can check the fragmentation information of IP packets, the legal combination of TCP flags (e.g., whether there are illegal packets that set both SYN and FIN bits), and whether the IP address conforms to the RFC standard (e.g., whether it is a multicast address, broadcast address, etc.).

[0033] Then, the computing device can process data packets from different sources in a unified manner, converting fields such as IP address, MAC address, and protocol type into a standard format to obtain a data packet sequence that has been verified, has a unified format, and is logically sound.

[0034] Preferably, the computing device can divide the collected data packets into sessions, and each session is modeled as... Where i is the session index. For the source address, For the destination address, Timestamps for session creation and termination. This refers to the protocol and payload characteristics.

[0035] Specifically, the original data packet is divided based on the 5-tuple and the application layer session identifier to obtain the network traffic data based on session aggregation.

[0036] Step 102: The computing device constructs a multidimensional feature tensor of the global session based on network traffic data.

[0037] The multidimensional feature tensor includes a temporal feature vector, a spatial feature vector, and a semantic feature vector.

[0038] Preferably, the computing device can calculate the time interval of the session event sequence based on the network traffic data, and analyze the temporal correlation based on the time interval using an autocorrelation function to obtain the time feature vector; wherein, the session event sequence is a sequence formed by the creation time and end time of all sessions within a preset time window; based on the network traffic data, the interaction strength between all hosts corresponding to all sessions within the preset time window is calculated, and a communication topology matrix is ​​constructed based on the interaction strength to obtain the spatial feature vector; the protocol semantics are mapped to a high-dimensional feature space through embedding vector learning to obtain the semantic feature vector; the time feature vector, the spatial feature vector, and the semantic feature vector are combined into the session feature vector through tensor decomposition.

[0039] The computing device uses a preset time window to sort all session creation and termination events according to their timestamps, forming an ordered sequence of session events. Then, for each consecutive event pair, the time interval is calculated to obtain a time interval sequence. The periodicity and memorability of the time interval sequence are then quantified using an autocorrelation function, the calculation formula of which includes: ; in, Here, is the time lag parameter, representing the amount of time delay under consideration; N is the total length of the time interval sequence. The value of the autocorrelation function at lag τ reflects the correlation between the sequence and its delayed version. Let j be the j-th time interval in the time interval sequence. This coefficient measures the strength of the linear correlation between the current time interval and the time interval after a delay of τ. If the two are positively correlated (e.g., both increase or both decrease), the product is positive; if they are negatively correlated (one increases and the other decreases), the product is negative. This is used to average the summation results to obtain the average correlation strength under delay τ.

[0040] Then, the computing device can calculate the autocorrelation function for multiple τ values ​​(τ=1,2,...,L), forming a time feature vector: ; The computing device can also dynamically adjust the analysis window according to the network traffic density, using a longer window to ensure statistical significance during low traffic and a shorter window to capture rapid changes during high traffic.

[0041] The computing device can extract the IP addresses of all unique hosts in the network and construct a host set. Then, for each pair of hosts in the host set, the interaction strength between them within a time window can be calculated. This interaction strength is then normalized and used as element values ​​to construct a communication topology matrix, which serves as a spatial feature vector.

[0042] Preferably, the interaction strength can be the number of sessions, the number of bytes of traffic, or the session duration.

[0043] Computing devices can map communication semantics to a high-dimensional feature space through protocol parsing and introduce embedded vector representations as semantic feature vectors.

[0044] The computing device can first extract protocol-related information for each session from the session data, including transport layer protocols, application layer protocols, port number semantics, and payload characteristics (such as encryption type and data mode); then construct a protocol set P, which contains all observed protocol types, and learn an embedding vector for each protocol. .

[0045] Specifically, computing devices can employ context-based embedding learning. If two protocols frequently appear in the same network session sequence, their embedding vectors will be trained to be closer in a high-dimensional space, allowing the computer to determine the semantic similarity of the protocols through vector distance.

[0046] Then, the computing device can concatenate or perform other summarization processes based on the embedding vectors corresponding to all protocols in the protocol set P to obtain semantic feature vectors.

[0047] After obtaining the three feature vectors mentioned above, the computing device can standardize these three feature vectors to eliminate the influence of dimensions, and then construct a multidimensional feature vector from the three standardized feature vectors.

[0048] The expressions for constructing multidimensional feature vectors include: ; Where X is the generated third-order tensor, that is, the multidimensional feature vector; These are respectively time feature vector, spatial feature vector, and semantic feature vector. is the weight factor of the k-th component, reflecting the importance of the pattern; K is the rank of the decomposition, controlling the complexity of the representation; This is the tensor outer product operator.

[0049] Preferably, the computing device can introduce attention weights to dynamically adjust the importance of different dimensions in a specific context, and the formula is extended as follows: ; in, , , These are learnable attention weights.

[0050] In this way, the computing device generates a unified and comprehensive multidimensional feature tensor for a time window using all session data within that time window. This tensor includes internal temporal patterns (temporal dimension), group interaction structure (spatial dimension), and semantic information of the services used (semantic dimension).

[0051] Step 103: The computing device models the normal behavior in the network traffic data based on the multidimensional feature tensor using a multidimensional Gaussian distribution, and obtains the probability density function of the normal behavior.

[0052] The computing device can flatten or decompose a multidimensional feature tensor into behavior vectors x. Each behavior vector represents the comprehensive features of a network behavior instance in the temporal, spatial, and semantic dimensions.

[0053] Specifically, a behavior vector is a low-dimensional, dense numerical vector that is a comprehensive and quantitative summary of the behavior of all network sessions within a specific time window (e.g., the past 5 minutes). It is not a vector representing a single session, but rather a fingerprint or profile representing the overall behavioral pattern of a behavioral entity (such as a host, a subnet, or the entire network egress) over a specific time period. It is obtained by concatenating the temporal feature vector, spatial feature vector, and semantic feature vector corresponding to a time window.

[0054] Assuming that normal behavior in network traffic data follows a multidimensional Gaussian distribution, computing devices can construct the probability density function of normal behavior: ; Where x is the behavior vector, This is a mean vector, representing the central tendency of normal behavior; The covariance matrix characterizes the correlation structure between features; The determinant of the covariance matrix reflects the overall volume of the characteristic space; The square of the Mahalanobis distance is used to measure the degree to which the feature vector deviates from the normal pattern.

[0055] To avoid the singularity problem of the covariance matrix in high-dimensional scenarios (where the feature dimension d is large and the sample size n is relatively small), the computing device can use Ledoit-Wolf shrinkage estimation to obtain the estimated covariance matrix.

[0056] Among these options, the computing device can select a simple diagonal matrix as the shrinkage target: Where F is a diagonal matrix, Let d be the element in the i-th row and j-th column of the sample covariance matrix S, and d be the number of dimensions of the sample covariance matrix.

[0057] Choosing a diagonal matrix as the shrinkage target means retaining only the variance of each variable and eliminating the covariance between variables (i.e., the correlation between variables). In small-sample, high-dimensional scenarios, the estimation of covariance between variables is often noisy and unreliable, while the estimation of their own variance is relatively more stable. By shrinking the diagonal matrix to contain only their own variance, we can retain the fluctuation information of the variables themselves while suppressing unreliable covariance information, thus making the estimation of the covariance matrix more robust.

[0058] The contraction formula is as follows: ; in, The covariance matrix is ​​the estimate of the contraction. This is the contraction factor, with a value range of [0,1].

[0059] The computing device can calculate the optimal solution that minimizes the mean square error based on the Ledoit-Wolf theorem. : ; in, The variance of the elements of the sample covariance matrix. Let be the covariance of the sample covariance and the covariance of the corresponding elements of the target matrix. The elements are in the diagonal matrix; the numerator in the formula reflects the estimation error, and the denominator reflects the model complexity.

[0060] Then, considering the characteristics of network traffic data, the computing device can use the following formula to estimate the variance of the elements of the sample covariance matrix: ; in, This is a variance estimate for the elements of the sample covariance matrix, where n is the sample size and k is the sample index. , Let i be the observed values ​​of the i-th and j-th flow indicators in the k-th sample. , Let be the average of the i-th and j-th flow indicators.

[0061] In this formula, by incorporating the difference between the sample covariance elements and the true covariance elements into the variance estimation, the essence is to model the variance of the covariance estimation error. This improvement allows the variance estimation to not only reflect the fluctuations of the data itself, but also to capture the error fluctuations in the covariance estimation process, thereby improving the stability and accuracy of the covariance matrix.

[0062] Variance estimation Substitute the contraction factor The formula can be used to calculate the approximately optimal shrinkage factor. .

[0063] Network traffic data has characteristics such as non-stationarity, burstiness, and high-dimensional correlation. The improved formula mentioned above adapts to this dynamic nature through the residual term. Even if the data distribution changes, the variance estimate can be adjusted in real time by the difference between the sample covariance and the true covariance, avoiding bias caused by distribution drift.

[0064] When traffic surges, traditional variance estimation can fluctuate significantly due to "extreme values". The improved formula constrains extreme values ​​by using the sum of squares of the residuals, reducing the interference of sudden traffic on covariance estimation and ensuring that it can still stably characterize the relationship between variables in traffic surge scenarios.

[0065] Network traffic typically involves dozens or even hundreds of metrics. Traditional variance estimation is prone to causing the covariance matrix to become irreversible in high-dimensional scenarios due to the curse of dimensionality. The embodiments of this application enhance the robustness of high-dimensional covariance matrices through shrinkage estimation and error-oriented variance modeling, making them more reliable in multi-metric correlation analysis.

[0066] The computing device can use network traffic data that is identified as normal behavior, and model the probability density function of normal behavior through the above process to obtain the probability density function of normal behavior.

[0067] The multidimensional Gaussian distribution modeling in this step, by providing multidimensional feature vectors, not only sets filters on multiple independent dimensions, but more importantly, models how these dimensions "should be normally correlated." Abnormal behavior can be identified more accurately precisely because it either deviates significantly in a single dimension or more subtly disrupts the normal correlation patterns between multiple dimensions. The detection method based on multidimensional joint probability in this application embodiment can more accurately identify abnormal behavior.

[0068] Step 104: The computing device calculates the target threshold based on the network traffic data using a Bayesian inference mechanism.

[0069] Among them, network traffic data refers to the observation dataset within the current time window, and the target threshold refers to the threshold used to identify abnormal behavior.

[0070] Preferably, the computing device can construct a formula for calculating the target threshold based on Bayes' theorem, specifically including: ; ; in, This represents network traffic data within the current time window; Indicates the target threshold. This represents the probability density boundary threshold for determining anomalies. For the threshold prior probability distribution based on historical experience, This represents the probability of observing the current data at a given threshold. To integrate historical experience and the posterior probability distribution of the threshold after current observation, The probability of observing the current data.

[0071] The computing device can update the threshold prior probability distribution based on continuously growing datasets of normal and abnormal behaviors. This allows for the dynamic updating of the posterior distribution of the normal and abnormal probability density boundary thresholds, thereby dynamically updating the target threshold.

[0072] Step 105: The computing device identifies abnormal behavior in the network traffic data based on the probability density function of the normal behavior and the target threshold.

[0073] Specifically, when the probability density of a certain behavior vector in the network traffic data is less than the target threshold, the behavior vector is determined to be the vector corresponding to abnormal behavior.

[0074] Preferably, after identifying the abnormal behavior in the network traffic data, the computing device can acquire the new session traffic data; perform sparse low-rank decomposition on the multidimensional feature tensor to obtain the low-rank representation of the multidimensional feature tensor; convert the new session traffic data into a rank tensor with the same dimension as the low-rank representation; and perform incremental updates on the multidimensional feature tensor based on the rank tensor and the low-rank representation.

[0075] The calculation formula for sparse low-rank decomposition includes: ; Where X is a multidimensional feature tensor, K is the theoretical maximum rank of the original tensor, which is equal to its minimum dimension value; and r is the rank of the approximate decomposition.

[0076] For new session data Convert it to a rank-one tensor: ; is the conversion factor.

[0077] The formulas for incremental updates include: . This is the multidimensional feature vector after incremental updates.

[0078] Preferably, after identifying the abnormal behavior in the network traffic data, the computing device can determine the attack type of the abnormal behavior based on the traffic data of the abnormal behavior.

[0079] The computing device can extract the features of the traffic data of the abnormal behavior and match them with a preset attack feature library to determine the attack type of the abnormal behavior; it can also determine the attack type of the abnormal behavior based on a machine learning classification model; or it can determine the attack type of the abnormal behavior based on the correlation analysis of traffic semantics and context.

[0080] In this embodiment, multidimensional feature modeling constructs a feature tensor that comprehensively describes network behavior, overcoming the limitations of traditional single-dimensional features and enabling a more refined depiction of the essential differences between normal and abnormal behavior. Modeling normal behavior using a multidimensional Gaussian distribution accurately describes its probability density, reducing false alarms caused by normal fluctuations. Real-time adjustment of the decision boundary through Bayesian inference adapts to changes in network state, avoiding the accuracy decline caused by environmental changes in traditional fixed thresholds. The entire method forms a closed loop from accurate perception and reliable modeling to intelligent decision-making, achieving high accuracy and low false alarm detection of abnormal behavior in highly dynamic network environments.

[0081] The method of this application will be described below with reference to a specific embodiment: To verify the effectiveness of the multi-dimensional network behavior perception framework proposed in this invention, a corporate LAN environment was selected as the experimental subject, and network session data was collected over a week: 1. Data Preparation In this network environment, 2000 session records were collected, including: Normal office traffic is 1800 transactions (including HTTP / HTTPS web page access, SMTP email sending and receiving, FTP file transfer, etc.). 200 abnormal attack traffic entries (including port scans, slow DoS attacks, and covert channel communication).

[0082] After data cleaning and session segmentation, all sessions were processed into feature vectors using a standardized method. Each session was extracted with 20 features, including time interval distribution, mean and variance of packet length, and protocol semantic embedding vector.

[0083] 2. Construction of Multidimensional Tensors Based on the number of nodes and time slices, construct a three-dimensional tensor: ; Here, 300 represents the number of communication hosts, and 100 represents the time slice division.

[0084] Principal components were extracted using sparse tensor decomposition. With the decomposition rank r=8, the first 8 feature components were obtained, which explained a cumulative variance of 93%, significantly reducing feature redundancy.

[0085] 3. Probabilistic Modeling and Parameter Setting The extracted tensor feature vectors are modeled using a multidimensional Gaussian distribution. ; The dimension d=20.

[0086] To avoid singularity in the covariance matrix, the Ledoit-Wolf contraction estimation is used: Where α = 0.15, S is the sample covariance, and F is the diagonal matrix.

[0087] 4. Dynamic threshold detection Within a time window of t=50, the probability distribution of all sessions is statistically analyzed, and the threshold is dynamically estimated using a Bayesian update mechanism. The final threshold converges to .

[0088] The test results show that: The average probability density of normal flow is ; The average probability density of abnormal attack traffic is .

[0089] Therefore, sessions below the threshold are marked as abnormal.

[0090] 5. Performance Comparison Traditional method (based on port number + traffic size rules): detection rate 78%, false alarm rate 12%; The method proposed in this application (multidimensional tensor modeling + Gaussian probability detection) has a detection rate of 96% and a false alarm rate of 3%.

[0091] Traditional methods are almost incapable of detecting slow scanning attacks and covert channels, while the solution of this invention can accurately identify more than 95% of covert communication behaviors. Therefore, the method of this application can effectively detect complex abnormal behaviors such as scanning attacks, slow DoS attacks, and covert channels, with both detection rate and accuracy superior to traditional methods.

[0092] In this embodiment, by jointly modeling multi-dimensional features such as time, space, and semantics, the network behavior characteristics can be comprehensively depicted, effectively avoiding the limitations of single-dimensional detection methods. By adopting a dynamic threshold update strategy based on Bayesian inference, adaptive adjustments can be made according to changes in network state, improving the flexibility and accuracy of anomaly detection. An improved covariance matrix shrinkage estimation method is used to solve the singularity problem in high-dimensional feature modeling, ensuring the stability and robustness of the probability model. Through sparse tensor decomposition and incremental update algorithms, the computational complexity is significantly reduced, improving real-time processing capabilities while ensuring detection accuracy.

[0093] The method provided in the embodiments of this application has been described above. The system provided in the embodiments of this application will be described below.

[0094] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an abnormal behavior recognition system provided in an embodiment of this application, as shown below. Figure 2 As shown, the system 20 includes: Module 201 retrieves network traffic data; Feature engineering module 202 is used to construct a multidimensional feature tensor of the global session based on network traffic data. The multidimensional feature tensor includes a temporal feature vector, a spatial feature vector, and a semantic feature vector. Modeling module 203 is used to model the normal behavior in the network traffic data based on the multidimensional feature tensor through a multidimensional Gaussian distribution, and obtain the probability density function of the normal behavior. Calculation module 204 is used to calculate the target threshold based on the network traffic data through a Bayesian inference mechanism; The identification module 205 is used to identify abnormal behavior in the network traffic data based on the probability density function of the normal behavior and the target threshold.

[0095] Preferably, the system further includes an update module 206, which is used to acquire new session traffic data; perform sparse low-rank decomposition on the multidimensional feature tensor to obtain a low-rank representation of the multidimensional feature tensor; convert the new session traffic data into a rank tensor with the same dimension as the low-rank representation; and perform incremental updates on the multidimensional feature tensor based on the rank tensor and the low-rank representation.

[0096] Preferably, the probability density function is a multidimensional Gaussian distribution function, and the covariance matrix in the multidimensional Gaussian distribution function is obtained by Ledoit-Wolf contraction estimation; the contraction coefficient in the Ledoit-Wolf contraction estimation is based on the variance of the elements of the sample covariance matrix, and the estimation formula for the variance of the elements of the sample covariance matrix includes: ; in, Let be the element in the i-th row and j-th column of the sample covariance matrix. This is a variance estimate for the elements of the sample covariance matrix, where n is the sample size and k is the sample index. , Let i be the observed values ​​of the i-th and j-th flow indicators in the k-th sample. , Let be the average of the i-th and j-th flow indicators.

[0097] Preferably, the formula for calculating the target threshold includes: ; ; in, This represents session traffic data within the current time window; Indicates the target threshold. This represents the probability density boundary threshold for determining anomalies. For the threshold prior probability distribution based on historical experience, This represents the probability of observing the current data at a given threshold. This is a threshold posterior probability distribution that combines historical experience with current observations.

[0098] Preferably, the identification module 205 is also used to determine the attack type of the abnormal behavior based on the traffic data of the abnormal behavior.

[0099] Preferably, the feature engineering module 202 is specifically used to calculate the time interval of the session event sequence based on the network traffic data, and analyze the temporal correlation based on the time interval using an autocorrelation function to obtain the time feature vector; wherein, the session event sequence is a sequence formed by the creation time and end time of all sessions within a preset time window; based on the network traffic data, calculate the interaction strength between all hosts corresponding to all sessions within the preset time window, and construct a communication topology matrix based on the interaction strength to obtain the spatial feature vector; map the protocol semantics to a high-dimensional feature space through embedding vector learning to obtain the semantic feature vector; and combine the time feature vector, the spatial feature vector, and the semantic feature vector into the session feature vector through tensor decomposition. Preferably, the acquisition module 201 is specifically used to capture raw data packets from the network interface using a zero-copy ring buffer; and to divide the raw data packets based on the five-tuple and application layer session identifier to obtain the network traffic data based on session aggregation.

[0100] The abnormal behavior recognition system provided in this application can be understood by referring to the relevant content in the foregoing method embodiment section, and will not be repeated here.

[0101] like Figure 3 As shown, Figure 3 This is a schematic diagram of a possible logical structure of a computing device provided in an embodiment of this application. The computing device 30 includes a processor 301, a communication interface 302, a memory 303, and a bus 304. The processor 301, the communication interface 302, and the memory 303 are interconnected via the bus 304. In an embodiment of this application, the processor 301 is used to control and manage the operation of the computing device 30. For example, the processor 301 is used to execute... Figure 1The steps in the embodiments and / or other processes used in the techniques described herein. Communication interface 302 is used to support communication by computing device 30. Memory 303 is used to store program code and data of computing device 30.

[0102] The processor 301 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc. The bus 304 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0103] In another embodiment of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the above-described... Figure 1 The method described in the embodiments.

[0104] Those skilled in the art will recognize that the units of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0105] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0106] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for identifying abnormal behavior, characterized in that, The method includes: Obtain network traffic data; A multidimensional feature tensor for global sessions is constructed based on network traffic data. The multidimensional feature tensor includes a temporal feature vector, a spatial feature vector, and a semantic feature vector. By using a multidimensional Gaussian distribution, the normal behavior in the network traffic data is modeled based on the multidimensional feature tensor to obtain the probability density function of the normal behavior; The target threshold is calculated based on the network traffic data using a Bayesian inference mechanism. Based on the probability density function of the normal behavior and the target threshold, abnormal behavior in the network traffic data is identified.

2. The method according to claim 1, characterized in that, After identifying anomalous behavior in the network traffic data, the method further includes: Get new session traffic data; The multidimensional feature tensor is subjected to sparse low-rank decomposition to obtain the low-rank representation of the multidimensional feature tensor. Convert the newly added session traffic data into a rank tensor with the same rank representation dimension as the low-rank representation. Based on the rank tensor and the low-rank representation, the multidimensional feature tensor is incrementally updated.

3. The method according to claim 1, characterized in that, The probability density function is a multidimensional Gaussian distribution function, and the covariance matrix in the multidimensional Gaussian distribution function is obtained through Ledoit-Wolf contraction estimation; the contraction coefficient in the Ledoit-Wolf contraction estimation is based on the variance of the elements of the sample covariance matrix, and the estimation formula for the variance of the elements of the sample covariance matrix includes: ； in, Let be the element in the i-th row and j-th column of the sample covariance matrix. This is a variance estimate for the elements of the sample covariance matrix, where n is the sample size and k is the sample index. , Let i be the observed values ​​of the i-th and j-th flow indicators in the k-th sample. , Let be the average of the i-th and j-th flow indicators.

4. The method according to claim 1, characterized in that, The formula for calculating the target threshold includes: ； ； in, This represents network traffic data within the current time window; Indicates the target threshold. This represents the probability density boundary threshold for determining anomalies. For the threshold prior probability distribution based on historical experience, This represents the probability of observing the current data at a given threshold. This is a threshold posterior probability distribution that combines historical experience with current observations.

5. The method according to claim 4, characterized in that, After identifying anomalous behavior in the network traffic data, the method further includes: Based on the traffic data of the abnormal behavior, the attack type of the abnormal behavior is determined.

6. The method according to any one of claims 1-5, characterized in that, The multidimensional feature tensor for constructing a global session based on network traffic data includes: Based on the network traffic data, the time interval of the session event sequence is calculated, and the time series correlation is analyzed by autocorrelation function based on the time interval to obtain the time feature vector; wherein, the session event sequence is a sequence formed by all session creation times and session end times within a preset time window; Based on the network traffic data, the interaction strength between all hosts corresponding to all sessions within the preset time window is calculated, and a communication topology matrix is ​​constructed based on the interaction strength to obtain the spatial feature vector. The semantic feature vector is obtained by mapping the protocol semantics to a high-dimensional feature space through embedding vector learning. The session feature vector is constructed based on the time feature vector, the spatial feature vector, and the semantic feature vector.

7. The method according to any one of claims 1-5, characterized in that, The acquisition of network traffic data includes: Use a zero-copy circular buffer to capture raw data packets from the network interface; The original data packet is segmented based on the 5-tuple and application layer session identifier to obtain the network traffic data based on session aggregation.

8. An abnormal behavior recognition system, characterized in that, The system, applied to the method of any one of claims 1-7, comprises: The acquisition module retrieves network traffic data. The feature engineering module is used to construct a multidimensional feature tensor of the global session based on network traffic data. The multidimensional feature tensor includes a temporal feature vector, a spatial feature vector, and a semantic feature vector. The modeling module is used to model the normal behavior in the network traffic data based on the multidimensional feature tensor using a multidimensional Gaussian distribution, and to obtain the probability density function of the normal behavior. The calculation module is used to calculate the target threshold based on the network traffic data using a Bayesian inference mechanism; The identification module is used to identify abnormal behavior in the network traffic data based on the probability density function of the normal behavior and the target threshold.

9. A computing device, characterized in that, include: Memory, used to store programs; A processor for loading the program to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method of any one of claims 1-7.

Images