Network anomaly monitoring method, device and program product based on traffic fingerprint learning

Abstract

The present disclosure belongs to the technical field of network security, and particularly relates to a network anomaly monitoring method, device and program product based on traffic fingerprint learning, which comprises the following steps: extracting metadata irrelevant to encryption of a target network; the metadata comprises five-tuple, timestamp, length and protocol flag information of a data packet; the metadata is aggregated with a source IP address as a primary key to define a network entity; session feature extraction is performed on a session of the target network entity and other network entities; the extracted session features comprise timing features, operation sequence features and scale distribution features; the operation sequence features comprise Shannon entropy of a continuous request and response type pair; an action fingerprint of the target network entity is constructed according to the session features; and whether the target network and the target network entity are abnormal is judged according to a deviation degree of a current action fingerprint of the target network entity and a fingerprint baseline. The present disclosure can realize accurate identification of network anomalies.

Description

Technical Field

[0001] This disclosure belongs to the field of network security technology, and specifically relates to a network anomaly monitoring method, device, and program product based on traffic fingerprint learning. Background Technology

[0002] As networks grow in scale and become more complex, the network management plane, consisting of network management systems (NMS) and software-defined networking (SDN) controllers, has become the core command center of the entire network. It is responsible for critical tasks such as configuration distribution, performance monitoring, and fault handling. Its northbound interface, internal microservice communication, and southbound protocol channels (such as NETCONF and gNMI) carry high-privilege and sensitive operation commands and status information. Security directly determines the stability and reliability of the network.

[0003] Currently, its security protection mainly faces two major challenges: external attack penetration and internal abnormal behavior (deviations from normal operation by users or system components with legitimate privileges, such as configuration tampering, sensitive data sniffing, and the establishment of hidden backdoors). Traditional methods are significantly inadequate for the latter.

[0004] Detection methods based on fixed rules are outdated, rely on known attack signature databases or static policies (such as ACLs), are difficult to deal with zero-day attacks or internally mutated slow malicious behaviors, and lack the ability to detect unknown threats.

[0005] Analysis based on resource access logs is coarse-grained. It can detect anomalies by accessing server resources (such as API call counts and CPU utilization), but it cannot distinguish between normal configuration queries and low-frequency data theft disguised as normal queries. Abnormal behavior is hidden in the sequence of legitimate business operations.

[0006] Existing traffic learning technologies suffer from significant interference or poor applicability. Active injection probing introduces additional loads that interfere with highly sensitive services, making it unsuitable for production environments. General network traffic anomaly detection focuses on the data plane, whose traffic model differs greatly from the highly structured, protocol-driven, and interactively complex traffic characteristics of the management plane, resulting in poor application performance. While deep packet inspection (DPI) and load analysis improve accuracy, they face the challenge of widespread encrypted traffic and suffer from high computational overhead and difficulty in guaranteeing real-time performance. Summary of the Invention

[0007] To address the aforementioned issues, this disclosure provides a network anomaly monitoring method based on traffic fingerprinting, which has the beneficial effect of accurately identifying network anomalies. The method includes: extracting encryption-independent metadata of the target network; the metadata includes: a 5-tuple of data packets, timestamp, length, and protocol identifier information; aggregating the metadata using the source IP address as the primary key to define a network entity; extracting session features from the sessions between the target network entity and other network entities, the extracted session features including: temporal features, operation sequence features, and scale distribution features; wherein, the operation sequence features include: Shannon entropy of consecutive request and response type pairs; constructing a behavioral fingerprint of the target network entity based on the session features; and determining whether the target network and the target network entity are abnormal based on the deviation of the target network entity's current behavioral fingerprint from the fingerprint baseline.

[0008] Furthermore, session feature extraction is performed on the session between the target network entity and other network entities, including: obtaining the request packet arrival interval sequence of the target network entity; performing discrete wavelet transform on the request packet arrival interval sequence, and using the sum of the energy squares of the first few approximate coefficients as the temporal feature.

[0009] Furthermore, the session feature extraction for the session between the target network entity and other network entities also includes: obtaining the continuous request and response type pair symbol sequence of the target network entity within the target session; calculating the Shannon entropy and two-character combination frequency vector of the continuous request and response type pair symbol sequence; repeating the above method to calculate the Shannon entropy and two-character combination frequency vector corresponding to all sessions of the target network entity; and using the statistics of the Shannon entropy and the statistics of the two-character combination frequency vector as the operation sequence feature.

[0010] Furthermore, the session feature extraction for the session between the target network entity and other network entities also includes: using the length sequence statistics of the packets sent by the target network entity as the scale distribution feature.

[0011] Furthermore, the method for constructing the fingerprint baseline includes: obtaining a set of normal fingerprints from the historical normal periods of the target network entity; calculating the mean vector of the normal fingerprint set in the original feature space; performing incremental principal component analysis on the normal fingerprint set to construct a dimension-reduced mapping matrix; projecting the historical normal fingerprints in the normal fingerprint set onto the principal subspace based on the mean vector and the dimension-reduced mapping matrix, and calculating the distribution parameters of all historical normal fingerprints; and constructing a fingerprint baseline based on the distribution parameters of all historical normal fingerprints.

[0012] Furthermore, the method for constructing the fingerprint baseline also includes: training an LSTM-Autoencoder network based on a normal fingerprint set; and constructing a fingerprint baseline based on the maximum reconstruction error of the LSTM-Autoencoder network.

[0013] Further, determining whether the target network and the target network entity are abnormal based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline includes: calculating the Mahalanobis distance between the projection of the current behavioral fingerprint in the principal subspace and the projection of the fingerprint baseline in the principal subspace based on the distribution parameters of the fingerprint baseline, and determining whether the target network and the target network entity are abnormal based on the Mahalanobis distance; and / or inputting the current behavioral fingerprint into the LSTM-Autoencoder network, calculating the reconstruction error of the current behavioral fingerprint, and determining whether the target network and the target network entity are abnormal based on the deviation between the reconstruction error of the current behavioral fingerprint and the maximum reconstruction error.

[0014] Furthermore, after determining whether the target network and the target network entity are abnormal based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, the method further includes: adding normal fingerprints that are falsely reported as abnormal to the normal fingerprint set; updating the fingerprint baseline when the increment of the normal fingerprint set is greater than a threshold; and / or constructing an abnormal fingerprint set based on the abnormal fingerprints, performing cluster analysis on the abnormal fingerprint set to determine the abnormal behavior corresponding to each cluster, and constructing a detector for the corresponding abnormal behavior pattern based on each cluster; the detector is used for anomaly analysis of behavioral fingerprints.

[0015] This disclosure also proposes a network anomaly monitoring system based on traffic fingerprinting learning, comprising: a data acquisition module for extracting encryption-independent metadata of the target network; the metadata including: a 5-tuple of data packets, timestamp, length, and protocol identifier information; an entity definition module for aggregating the metadata with the source IP address as the primary key to define network entities; a feature extraction module for extracting session features from the sessions between the target network entity and other network entities, the extracted session features including: temporal features, operation sequence features, and scale distribution features; wherein, the operation sequence features include: Shannon entropy of consecutive request and response type pairs; a fingerprint construction module for constructing a behavioral fingerprint of the target network entity based on the session features; and an anomaly judgment module for judging whether the target network and the target network entity are abnormal based on the deviation of the current behavioral fingerprint of the target network entity from the fingerprint baseline.

[0016] This disclosure also proposes a computer program product stored in a computer-readable storage medium, which, when executed by a processor, is used to at least implement the above-described network anomaly monitoring method based on traffic fingerprinting.

[0017] Compared with the prior art, this disclosure has the following advantages:

[0018] (1) Bypassing encryption restrictions: By extracting encryption-independent metadata for analysis, the technical difficulties of encrypted traffic parsing can be bypassed without relying on encrypted content;

[0019] (2) Precise entity definition: The source IP address is used as the primary key to aggregate metadata to define network entities, and the scattered metadata is associated with specific entities, which can more accurately locate the subject of abnormal behavior.

[0020] (3) Comprehensive and in-depth feature coverage: The session features cover temporal features, operation sequence features and scale distribution features, which not only capture the interaction patterns in the time dimension, but also quantify the complexity and randomness of the operation sequence through the Shannon entropy of continuous request and response type pairs, thereby reducing the probability of missing anomalies.

[0021] (4) Stable behavior modeling: Based on session features, behavioral fingerprints are constructed to transform dynamic behaviors into comparable and stable representations, avoiding accidental interference from single operations; by judging anomalies by deviation from the fingerprint baseline, it can sensitively identify variant behaviors that deviate from the normal pattern and improve the ability to perceive unknown threats.

[0022] Other features and advantages of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A network anomaly detection method based on traffic fingerprinting learning according to an embodiment is shown;

[0025] Figure 2 A network anomaly detection system based on traffic fingerprinting learning according to an embodiment is shown. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0027] Figure 1 The network anomaly detection method based on traffic fingerprinting provided in this disclosure includes:

[0028] Extract encryption-independent metadata from the target network; the metadata includes: the 5-tuple of the data packet, timestamp, length, and protocol identifier information;

[0029] The metadata is aggregated using the source IP address as the primary key to define network entities;

[0030] Session features are extracted from the conversations between the target network entity and other network entities. The extracted session features include: temporal features, operation sequence features, and scale distribution features; wherein, the operation sequence features include: Shannon entropy of consecutive request and response type pairs.

[0031] Construct the behavioral fingerprint of the target network entity based on the session characteristics;

[0032] Based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, it is determined whether the target network and the target network entity are abnormal.

[0033] According to some embodiments of this disclosure, the network anomaly monitoring method based on traffic fingerprinting proposed in this disclosure deploys probes at key network nodes to collect original network packets in a mirrored manner, extracts encryption-independent metadata, including: the packet's 5-tuple, timestamp, length, and protocol identifier information, aggregates metadata with the source IP address as the primary key, defines a set of network entities, and defines the interactions between network entities as sessions. Based on the extracted session features (time-sequence features, operation sequence features, and scale distribution features, etc.), it constructs behavioral fingerprints of network entities, and determines whether a network entity or network is abnormal by comparing the deviation between the current behavioral fingerprint and the fingerprint baseline (based on the set of normal behavioral fingerprints of network entities during historical normal periods, established through a dual model including incremental principal component analysis and autoencoder reconstruction error). The process of metadata extraction and network entity and session definition is as follows:

[0034] 1. Deploy probes at key nodes in the network management plane to collect raw network packet sequences in a mirrored manner:

[0035] P={p1,p2,...,pn}

[0036] 2. For each package p i Extract its encryption-independent metadata to form a vector m. i :

[0037] m i =(t i src_ip i dst_ip i src_port i dst_port i ,proto i ,len i ,flags i )

[0038] Among them, t i For packet arrival timestamps (accurate to microseconds), src_ip i and dst_ip i For the source and destination IP addresses, src_port i and dst_port i For source and destination ports, proto i For transport layer protocol identifiers, len i For packet length (bytes), flags i This is a set of TCP flags (such as SYN, ACK, PSH, FIN).

[0039] 3. Output metadata sequence M={m1,m2,...,m n}

[0040] 4. Define the network entity set E = {E1, E2, ..., E...} K}, will entity E k Within the time window ΔT, the specific target (dst_ip) j dst_port j The interaction sequence of S is defined as a session. k,j .

[0041] Furthermore, session feature extraction is performed on the sessions between the target network entity and other network entities, including:

[0042] Obtain the request packet arrival interval sequence of the target network entity;

[0043] The energy sum of the first few approximate coefficients obtained by performing discrete wavelet transform on the arrival interval sequence of the request packets is used as the time series feature.

[0044] According to some embodiments of this disclosure, timing feature fk (t) The extraction process is as follows: calculate the arrival time of all request packets initiated by the target network entity, and construct the request packet arrival time interval sequence I. k , to I k Perform Discrete Wavelet Transform (DWT) and take the sum of the squares of the energies of the first c approximation coefficients as the time series characteristic value.

[0045] Furthermore, extracting session features from the sessions between the target network entity and other network entities also includes:

[0046] Obtain the sequence of consecutive request and response type pairs of the target network entity within the target session;

[0047] Calculate the Shannon entropy and two-character combination frequency vector of the consecutive request and response type pairs of symbol sequences;

[0048] Repeat the above method to calculate the Shannon entropy and two-character combination frequency vector for all sessions of the target network entity;

[0049] The statistics of the Shannon entropy and the statistics of the two-character combination frequency vector are used as the features of the operation sequence.

[0050] According to some embodiments of this disclosure, the operation sequence feature f k (s) The extraction process is as follows: For any session S k,j Encode consecutive request-response type pairs into symbolic sequences O k,j Calculate the Shannon entropy H(Ok,j) and the frequency vectors of all two-character combinations. Repeat the above method to calculate the Shannon entropy and the frequency vectors of all two-character combinations for all sessions. Take the statistics (such as mean and maximum) of the Shannon entropy and the frequency vectors of all two-character combinations for all sessions, and use these statistics as the features of the operation sequence.

[0051] Furthermore, extracting session features from the sessions between the target network entity and other network entities also includes:

[0052] The length sequence statistics of packets sent by the target network entity are used as the scale distribution feature.

[0053] According to some embodiments of this disclosure, the scale distribution feature f k (l) The extraction process is as follows: Calculate the length sequence L of the packets sent by the target network entity. k The statistic is used as a scale distribution characteristic; for example, its mean μ(L) is used. k ), standard deviation σ(L) k ), skewness γ (L k () as a scale distribution feature.

[0054] Furthermore, the method for constructing the fingerprint baseline includes:

[0055] Obtain the set of normal fingerprints from the historical normal periods of the target network entity;

[0056] Calculate the mean vector of the normal fingerprint set in the original feature space;

[0057] Incremental principal component analysis is performed on the normal fingerprint set to construct a dimension-reduced mapping matrix;

[0058] Based on the mean vector and the dimensionality reduction mapping matrix, the historical normal fingerprints in the set of normal fingerprints are projected onto the principal subspace, and the distribution parameters of all historical normal fingerprints are calculated.

[0059] A fingerprint baseline is constructed based on the distribution parameters of all historical normal fingerprints.

[0060] Furthermore, the method for constructing the fingerprint baseline also includes:

[0061] Train an LSTM-Autoencoder network based on a normal fingerprint set;

[0062] A fingerprint baseline is constructed based on the maximum reconstruction error of the LSTM-Autoencoder network.

[0063] According to some embodiments of this disclosure, the process of constructing a fingerprint baseline includes:

[0064] (1) Collect the fingerprint set F of entities during normal historical periods. normal , of which F normal ={F1,F2,...,F K}, Fk=[f k (t) ,f k (s) ,f k (l) ] T Where T represents transpose;

[0065] (2) Calculate F normal The mean vector in the original feature space:

[0066]

[0067] (3) For F normal Incremental principal component analysis is applied to learn the dimensionality-reduced mapping matrix W;

[0068] (4) Project all historical normal fingerprints to the master subspace:

[0069] Fpc ={W T (F-μ)∣F∈Fnormal}

[0070] (5) Calculate the distribution parameters μ projected onto the principal subspace. pc (mean) and Σ pc (Covariance matrix);

[0071] (6) Calculate F pc The 95th quantile of the Mahalanobis distance of all samples is used as the threshold θ. D ;

[0072] (7) Use F normal Train an LSTM-Autoencoder and compute its maximum reconstruction error on the validation set as a threshold θ. E .

[0073] Further, based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, it is determined whether the target network and the target network entity are abnormal, including:

[0074] Based on the distribution parameters of the fingerprint baseline, calculate the Mahalanobis distance between the projection of the current behavior fingerprint in the principal subspace and the projection of the fingerprint baseline in the principal subspace; and determine whether the target network and the target network entity are abnormal based on the Mahalanobis distance; and / or

[0075] The current behavior fingerprint is input into the LSTM-Autoencoder network, the reconstruction error of the current behavior fingerprint is calculated, and the deviation between the reconstruction error of the current behavior fingerprint and the maximum reconstruction error is used to determine whether the target network and the target network entity are abnormal.

[0076] According to some embodiments of this disclosure, the anomaly detection process based on fingerprint baseline includes:

[0077] (1) For the fingerprint F to be judged new Project it onto the main subspace:

[0078] F pc =W T (F new -μ)

[0079] (2) Calculate fingerprint F in the principal subspace new Mahalanobis distance from the fingerprint baseline:

[0080]

[0081] (3) F new The reconstruction error is calculated by inputting the trained LSTM-Autoencoder network.

[0082]

[0083] in, Output for the model.

[0084] According to some embodiments of this disclosure, in anomaly detection based on fingerprint baseline, context association is used for determination: if D M >θ D or E r >θ E If this occurs, an initial alert is triggered, and the operation context set C related to the event is queried, including operation logs of plan change orders, fault work orders, and other related entities within the time period Δt before and after the event. If C = If there is no reasonable business explanation, it is judged as hidden abnormal behavior.

[0085] Furthermore, after determining whether the target network and the target network entity are abnormal based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, the method further includes:

[0086] Add normal fingerprints that falsely report anomalies to the normal fingerprint set; update the fingerprint baseline when the increment of the normal fingerprint set exceeds a threshold; and / or

[0087] An abnormal fingerprint set is constructed based on the abnormal fingerprints. Cluster analysis is performed on the abnormal fingerprint set to determine the abnormal behavior corresponding to each cluster. A detector for the corresponding abnormal behavior pattern is constructed based on each cluster. The detector is used for abnormal analysis of behavioral fingerprints.

[0088] According to some embodiments of this disclosure, the network anomaly detection method based on traffic fingerprinting proposed in this disclosure can also achieve online self-evolution, and the technical concept is as follows:

[0089] 1. Add fingerprint samples of normal behavior that are confirmed as false alarms to F normal The behavioral fingerprint samples confirmed as genuine anomalies will be added to the anomaly database F. anormal When F normal The increment reaches the set size N update At that time, update the parameters (W, μ, μ) involved in the anomaly detection according to the method described above. pc ,Σ pc ,θ D ,θ E ).

[0090] 2. Regularly check F anormal Perform cluster analysis to generate new anomalous pattern clusters {Cluster} new}, and use its central feature vector as a new monitoring pattern prototype to enhance subsequent anomaly identification; regularly update the anomaly sample database F anormal ={F a1 ,F a2 Density-based clustering analysis (such as the DBSCAN algorithm) is performed on the clusters. This algorithm does not require a preset number of clusters and can automatically discover dense groups of samples with similar characteristics. The clustering is based on the Euclidean distance between behavioral fingerprint vectors, and the algorithm outputs a set of clusters {Cluster1, Cluster2, ..., Cluster}. m}, where each cluster represents a newly discovered, recurring suspicious behavior pattern; for significant clusters satisfying the minimum size threshold Nmin, the cluster is defined as Cluster. j Calculate its centroid vector c j and distribution radius r j (The average distance from the sample within the cluster to the centroid). (c) j ,r j The binary tuple is defined as a new exception pattern prototype. This prototype quantifies the core characteristics and fluctuation range of such anomalous behavior from the data. Each new prototype... It is packaged as a lightweight detector. For future fingerprint testing... test Calculate its relationship with c j Distance d(F) test ,c j If d(F) test ,c j ) <r j k (where k is an adjustable relaxation coefficient) can then be used to determine F. test If the anomaly pattern is matched, a specific alert is triggered directly. This is equivalent to adding a pattern matching detection layer based on specific threat patterns on top of the original general statistical detection model (the mathematical model corresponding to the anomaly judgment based on the fingerprint baseline mentioned above), enabling the model to remember and identify known types of variant attacks more quickly and accurately.

[0091] The above solution will be further explained below with reference to the embodiments:

[0092] This embodiment takes a network management plane containing an SDN controller (IP: 10.0.0.1) and two operation and maintenance management terminals (normal operation and maintenance personnel 10.0.1.100 and potentially abnormal terminal 10.0.1.101) as an example to demonstrate the specific implementation and calculation process of the method disclosed herein.

[0093] S1 & S2: Data Acquisition and Fingerprint Modeling

[0094] Data Acquisition (S1): Mirror traffic at the controller inlet and continuously collect data for T=300 seconds. Taking entity E_normal (10.0.1.100) as an example, in a configuration query session within 1 second, 5 interaction packets are captured. Its metadata (simplified to timestamp t and length len) sequence is: M=[(0.000,120),(0.215,580),(0.328,80),(1.102,1250),(1.118,90)].

[0095] Feature extraction calculation (S2):

[0096] Temporal characteristics (f_t): Calculate the packet interval sequence I = [0.215, 0.113, 0.774, 0.016] seconds. Perform a first-order discrete Haar wavelet transform on it to obtain approximate coefficients cA1 = [0.232, 0.559]. Calculate the energy f_t = ||cA1||^2 = (0.232)^2 + (0.559)^2 = 0.368.

[0097] Operational entropy feature (f_s): Abstracting a session as a request (REQ) - response (RES) sequence: [REQ,RES,REQ,RES,RES]. Calculating its information entropy: There are two notations, P(REQ) = 0.4, P(RES) = 0.6, H = -(0.4 * ... log2(0.4) + 0.6* log2(0.6))≈0.971.

[0098] Scale feature (f_l): Calculate the mean μ_l=100 and standard deviation σ_l=28.28 of the request packet length sequence [120,80].

[0099] Fingerprint construction: Based on a large number of historical sessions, the typical fingerprint of E_normal can be summarized as F_normal=[f_t:0.35±0.1,H:0.95±0.2,μ_l:102±30,σ_l:25±10] (in the form of mean ± standard deviation).

[0100] S3: Anomaly Detection and Verification

[0101] Baseline modeling (S3.1): Collect 1000 historical normal fingerprints of E_normal, calculate the global mean μ and covariance matrix, and after dimensionality reduction by PCA, determine the Mahalanobis distance threshold θ_D=3.0 and the reconstruction error threshold θ_E=1.5.

[0102] Anomaly deviation calculation (S3.2):

[0103] Suppose that the suspicious entity E_abnormal (10.0.1.101) generates a fingerprint late at night: F_test=[f_t:0.05,H:1.8,μ_l:150,σ_l:5].

[0104] Calculate the Mahalanobis distance: D_M = sqrt((F_test - μ)) Σ^{-1} *(F_test-μ)^T)=5.2.

[0105] Calculate the reconstruction error: The pre-trained LSTM-Autoencoder outputs E_r=2.1.

[0106] D_M(5.2)>θ_D(3.0) and E_r(2.1)>θ_E(1.5), triggering a preliminary alarm.

[0107] Context Validation (S3.3): Query the context C within 15 minutes before and after the event: No planned change orders, no associated fault work orders, and no other operations personnel activity. C= Based on the high deviation, it was ultimately determined to be a hidden anomaly (consistent with low-frequency, slow-speed scanning or data snooping characteristics).

[0108] S4: System Self-Evolution

[0109] Clustering findings: The accumulated F_anomaly library contains 50 anomalous fingerprints. Using DBSCAN clustering (parameters eps=0.5, min_samples=3), a significant cluster, Cluster_1, was found, containing 10 similar fingerprints.

[0110] Prototype quantization: Calculate the centroid c_1 and radius r_1 of Cluster_1:

[0111] c_1=[f_t:0.08,H:1.7,μ_l:145,σ_l:8] (mean of each feature dimension)

[0112] r_1 = Mean Euclidean distance(c_1, each sample point) = 0.3

[0113] Model enhancement: (c_1, r_1) is registered as a new threat pattern "P1: Low-frequency scanning probe". Afterward, any new fingerprint F_new that satisfies distance(F_new, c_1) < 0.5 (relaxation coefficient k = 1.67) will directly match pattern P1 and trigger an alert, without requiring further complex statistical model calculations.

[0114] Verification Conclusion: This embodiment fully verifies, through specific mathematical calculations, the quantization mapping from raw traffic to behavioral fingerprints, anomaly detection based on statistical distance and context, and the self-evolutionary process of transforming discrete anomalies into reusable detection knowledge (prototype (c_1, r_1)). This method can effectively identify well-disguised low-frequency threats that traditional methods miss, and ensure continuously evolving security capabilities.

[0115] Based on the same technological concept, such as Figure 2 As shown, this disclosure also proposes a network anomaly monitoring system based on traffic fingerprinting learning, the system comprising:

[0116] The data acquisition module is used to extract encryption-independent metadata of the target network; the metadata includes: the five-tuple of the data packet, timestamp, length and protocol identifier information;

[0117] The entity definition module is used to aggregate the metadata using the source IP address as the primary key to define network entities;

[0118] The feature extraction module is used to extract session features from the conversations between the target network entity and other network entities. The extracted session features include: temporal features, operation sequence features, and scale distribution features; wherein, the operation sequence features include: Shannon entropy of consecutive request and response type pairs.

[0119] The fingerprint construction module is used to construct the behavioral fingerprint of the target network entity based on the session characteristics;

[0120] The anomaly detection module is used to determine whether the target network and the target network entity are abnormal based on the deviation between the current behavior fingerprint of the target network entity and the fingerprint baseline.

[0121] This disclosure also proposes a computer program product stored in a computer-readable storage medium, which, when executed by a processor, is used to at least implement the above-described network anomaly monitoring method based on traffic fingerprinting.

[0122] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A network anomaly detection method based on traffic fingerprinting, characterized in that, include: Extract encryption-independent metadata from the target network; The metadata includes: the 5-tuple of the data packet, timestamp, length, and protocol identifier information; The metadata is aggregated using the source IP address as the primary key to define network entities; Session features are extracted from the conversations between the target network entity and other network entities. The extracted session features include: temporal features, operation sequence features, and scale distribution features; wherein, the operation sequence features include: Shannon entropy of consecutive request and response type pairs. Construct the behavioral fingerprint of the target network entity based on the session characteristics; Based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, determine whether the target network and the target network entity are abnormal; Session feature extraction is performed on the conversations between the target network entity and other network entities, including: Obtain the request packet arrival interval sequence of the target network entity; The energy sum of the first few approximate coefficients obtained by performing discrete wavelet transform on the arrival interval sequence of the request packets is used as the time series feature; Extracting session features from the conversations between the target network entity and other network entities also includes: Obtain the sequence of consecutive request and response type pairs of the target network entity within the target session; Calculate the Shannon entropy and two-character combination frequency vector of the consecutive request and response type pairs of symbol sequences; Repeat the above method to calculate the Shannon entropy and two-character combination frequency vector for all sessions of the target network entity; The statistics of the Shannon entropy and the statistics of the two-character combination frequency vector are used as the features of the operation sequence. Extracting session features from the conversations between the target network entity and other network entities also includes: The length sequence statistics of packets sent by the target network entity are used as the scale distribution feature.

2. The network anomaly monitoring method based on traffic fingerprinting as described in claim 1, characterized in that, The method for constructing the fingerprint baseline includes: Obtain the set of normal fingerprints from the historical normal periods of the target network entity; Calculate the mean vector of the normal fingerprint set in the original feature space; Incremental principal component analysis is performed on the normal fingerprint set to construct a dimension-reduced mapping matrix; Based on the mean vector and the dimensionality reduction mapping matrix, the historical normal fingerprints in the set of normal fingerprints are projected onto the principal subspace, and the distribution parameters of all historical normal fingerprints are calculated. A fingerprint baseline is constructed based on the distribution parameters of all historical normal fingerprints.

3. The network anomaly monitoring method based on traffic fingerprinting as described in claim 2, characterized in that, The method for constructing the fingerprint baseline further includes: Train an LSTM-Autoencoder network based on a normal fingerprint set; A fingerprint baseline is constructed based on the maximum reconstruction error of the LSTM-Autoencoder network.

4. The network anomaly monitoring method based on traffic fingerprinting as described in claim 3, characterized in that, Based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, determine whether the target network and the target network entity are abnormal, including: Based on the distribution parameters of the fingerprint baseline, calculate the Mahalanobis distance between the projection of the current behavior fingerprint in the principal subspace and the projection of the fingerprint baseline in the principal subspace; and determine whether the target network and the target network entity are abnormal based on the Mahalanobis distance; and / or The current behavior fingerprint is input into the LSTM-Autoencoder network, the reconstruction error of the current behavior fingerprint is calculated, and the deviation between the reconstruction error of the current behavior fingerprint and the maximum reconstruction error is used to determine whether the target network and the target network entity are abnormal.

5. The network anomaly monitoring method based on traffic fingerprinting as described in claim 4, characterized in that, After determining whether the target network and the target network entity are abnormal based on the deviation between the current behavioral fingerprint of the target network entity and the fingerprint baseline, the method further includes: Add normal fingerprints that falsely report anomalies to the normal fingerprint set; update the fingerprint baseline when the increment of the normal fingerprint set exceeds a threshold; and / or An abnormal fingerprint set is constructed based on the abnormal fingerprints. Cluster analysis is performed on the abnormal fingerprint set to determine the abnormal behavior corresponding to each cluster. A detector for the corresponding abnormal behavior pattern is constructed based on each cluster. The detector is used for abnormal analysis of behavioral fingerprints.

6. A network anomaly monitoring system based on traffic fingerprinting, characterized in that, include: The data acquisition module is used to extract encryption-independent metadata from the target network; The metadata includes: the 5-tuple of the data packet, timestamp, length, and protocol identifier information; The entity definition module is used to aggregate the metadata using the source IP address as the primary key to define network entities; The feature extraction module is used to extract session features from the sessions between the target network entity and other network entities. The extracted session features include: temporal features, operation sequence features, and scale distribution features; wherein, the operation sequence features include: Shannon entropy of consecutive request and response type pairs. The fingerprint construction module is used to construct the behavioral fingerprint of the target network entity based on the session characteristics; The anomaly detection module is used to determine whether the target network and the target network entity are abnormal based on the deviation between the current behavior fingerprint of the target network entity and the fingerprint baseline. Session feature extraction is performed on the conversations between the target network entity and other network entities, including: Obtain the request packet arrival interval sequence of the target network entity; The energy sum of the first few approximate coefficients obtained by performing discrete wavelet transform on the arrival interval sequence of the request packets is used as the time series feature; Extracting session features from the conversations between the target network entity and other network entities also includes: Obtain the sequence of consecutive request and response type pairs of the target network entity within the target session; Calculate the Shannon entropy and two-character combination frequency vector of the consecutive request and response type pairs of symbol sequences; Repeat the above method to calculate the Shannon entropy and two-character combination frequency vector for all sessions of the target network entity; The statistics of the Shannon entropy and the statistics of the two-character combination frequency vector are used as the features of the operation sequence. Extracting session features from the conversations between the target network entity and other network entities also includes: The length sequence statistics of packets sent by the target network entity are used as the scale distribution feature.

7. A computer program product, said computer program product being stored in a computer-readable storage medium, characterized in that, When the computer program product is executed by a processor, it is used to implement at least the network anomaly monitoring method based on traffic fingerprinting as described in any one of claims 1-5.

Images