A deep learning-based network traffic anomaly detection method

Abstract

The application discloses a network traffic anomaly detection method based on deep learning, and belongs to the technical field of computer network security, and comprises the following steps: S1, collecting network traffic data and host state data; S2, determining a traffic anomaly degree of the network traffic data and a host state severity of the host state data; S3, based on the traffic anomaly degree and the host state severity, calculating a potential harmfulness score; and S4, dynamically adjusting a detection threshold for subsequent network traffic data anomaly judgment according to the potential harmfulness score.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer network security, in particular to a network traffic anomaly detection method based on deep learning. BACKGROUND

[0002] The detection method in the prior art establishes a baseline model by learning the normal traffic pattern, and determines that the new traffic is abnormal when it deviates from the baseline in statistical characteristics; the core defect of such method is that it focuses on the statistical abnormality of traffic pattern, and cannot evaluate the substantive harmfulness of the anomaly to the system. In complex environments such as cloud native, traffic generated by normal business activities such as service update and elastic scaling may be misjudged as abnormal, leading to alarm fatigue, so that real threats may be ignored.

[0003] To solve the problem that the prior art cannot evaluate the real harmfulness of abnormal events, the present application provides a method that can distinguish between malicious attacks and benign business fluctuations; the goal of this method is to change from a recognizer of pattern deviation to an evaluator and manager of systematic risk.

[0004] The above information disclosed in the above BACKGROUND section is only used to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present application is to provide a network traffic anomaly detection method based on deep learning to solve the problems raised in the above BACKGROUND.

[0006] The technical solution of the present application is as follows, the specific steps include:

[0007] S1, collecting network traffic data and host state data;

[0008] S2, determining the traffic anomaly degree of the network traffic data and the host state severity of the host state data;

[0009] S3, calculating the potential harmfulness score based on the traffic anomaly degree and the host state severity;

[0010] S4, dynamically adjusting the detection threshold for subsequent network traffic data anomaly determination according to the potential harmfulness score.

[0011] Preferably, S2 includes:

[0012] The network traffic data is converted into a network traffic feature vector, and the network traffic feature vector is input into a preset autoencoder model to calculate the traffic anomaly degree;

[0013] The host state data is converted into a host state feature vector, and the host state feature vector is evaluated according to a preset rule set to calculate a host state severity.

[0014] Preferably, the step of determining the traffic anomaly degree comprises:

[0015] The reconstruction error of the network traffic feature vector after reconstruction via the autoencoder model is calculated to obtain an original anomaly degree;

[0016] The original anomaly degree is normalized to generate the traffic anomaly degree.

[0017] Preferably, the step of determining the host state severity comprises:

[0018] The host state feature vector is evaluated according to a preset rule set based on a service level agreement to obtain an original severity;

[0019] The original severity is normalized to generate the host state severity.

[0020] Preferably, S3 comprises:

[0021] The causal attribution confidence is calculated based on the time sequence relationship between the network traffic data and the host state data;

[0022] The traffic anomaly degree, the host state severity, and the causal attribution confidence are fused through a preset weighting model to generate a potential harm score.

[0023] Preferably, S4 comprises:

[0024] A key field is extracted from the current network traffic data to determine the type of event to which it belongs;

[0025] A current detection threshold corresponding to the type of event is retrieved from a preset threshold mapping table that stores the correspondence between the type of event and the detection threshold;

[0026] The traffic anomaly degree is compared with the current detection threshold to obtain a comparison result.

[0027] Preferably, S4 further comprises:

[0028] In response to the comparison result being that the traffic anomaly degree is greater than the current detection threshold, an updated detection threshold is calculated based on the potential harm score and a preset reference harm threshold through a feedback control loop formula;

[0029] In response to the comparison result being that the traffic anomaly degree is not greater than the current detection threshold, the current detection threshold is taken as the updated detection threshold.

[0030] Preferably, S4 further comprises:

[0031] The updated detection threshold replaces the current detection threshold in the threshold mapping table, and the updated detection threshold is used for abnormality determination on subsequently received network traffic data belonging to the same event type.

[0032] Preferably, the event type is determined by the source IP address, destination port and protocol of the network traffic data.

[0033] The present application provides a deep learning-based network traffic anomaly detection method, which has the following improvements and advantages compared with the prior art:

[0034] 1. The technical solution constructs a complete and adaptive detection system from data collection, quantitative evaluation, risk fusion to closed-loop feedback; instead of analyzing statistical anomalies of network traffic in isolation, it realizes two-dimensional data input by synchronously collecting network traffic data and host state data; the fusion of such multi-modal data lays a solid data foundation for transforming from pure pattern deviation identification to comprehensive evaluation and management of systematic risks;

[0035] 2. The present solution realizes independent quantification and subsequent effective fusion of different dimensional risks; for network traffic, the reconstruction error is calculated using a preset autoencoder model to obtain the traffic anomaly degree; this method can deeply mine the internal pattern of normal traffic in an unsupervised learning manner, and has high sensitivity in detecting unknown and new abnormal patterns; for host state, a preset rule set containing service level agreement or expert knowledge is used to evaluate the host state feature vector to obtain the host state severity; this method ensures the business relevance and accuracy of host risk evaluation; through subsequent normalization processing, two indicators with different sources and dimensions are converted into standardized and comparable risk scales, which is a key prerequisite for scientific fusion of multi-modal data;

[0036] 3. The present solution deeply understands the time correlation between traffic anomalies and host state deterioration, and innovatively introduces the calculation of causal attribution confidence; this confidence is used to quantify the possibility of causal correlation between an identified network traffic anomaly and a host state deterioration event that occurs immediately after it; this enables the system to deeply analyze the internal logic between the two, effectively distinguishing between benign business fluctuations that are only statistically abnormal but do not cause actual damage, and malicious attacks that directly lead to system state deterioration; finally, a preset weighted model is used to fuse the traffic anomaly degree, host state severity and causal attribution confidence to generate a comprehensive potential harm score; this score is no longer a simple numerical superposition, but a deep and accurate evaluation of the real harm of abnormal events;

[0037] 4. The most technically advanced part of this scheme is its adaptive closed-loop feedback mechanism; the system dynamically adjusts the detection threshold for subsequent network traffic anomaly judgment according to the calculated potential harm score; this adjustment is based on the fine management of event types, i.e. determining the event type of the traffic according to the source IP address, destination port and protocol, and updating the detection threshold of the event; for events with high assessed harm, the system will automatically lower the detection threshold for similar events, improving the detection sensitivity; conversely, for events assessed as benign fluctuations, the threshold will be raised to enhance the system's tolerance; this mechanism simulates the memory and adaptability of the immune system, enabling the detection system to continuously learn from the environment and automatically optimize its detection strategy, thus maintaining a dynamic balance between detection sensitivity and accuracy in the face of constantly changing business patterns and attack methods, significantly reducing false positives and allowing security operations resources to focus more on real network threats. BRIEF DESCRIPTION OF DRAWINGS

[0038] The application will be further explained below in conjunction with the accompanying drawings and examples:

[0039] Figure 1 is a flow chart of a network traffic anomaly detection method based on deep learning. DETAILED DESCRIPTION

[0040] To make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with specific examples.

[0041] Example 1

[0042] Please refer to Figure 1 The present application provides a network traffic anomaly detection method based on deep learning, the specific steps of which include:

[0043] S1, collecting network traffic data and host state data;

[0044] S2, determining the traffic anomaly degree of the network traffic data and the host state severity of the host state data;

[0045] S3, calculating the potential harm score based on the traffic anomaly degree and the host state severity;

[0046] S4, dynamically adjusting the detection threshold for subsequent network traffic data anomaly judgment according to the potential harm score;

[0047] This embodiment discloses a network traffic anomaly detection method based on deep learning, which constructs a complete and adaptive detection closed loop.

[0048] Step S1, collecting network traffic data and host state data, is a step of constructing the data basis of the method; the purpose is to obtain two different dimension data sources required for risk assessment; network traffic data refers to raw packet information captured in real time through network probes or switch port mirroring, containing source IP address, destination IP address, source port, destination port, protocol type and data payload; host state data refers to system performance indicators and running state logs collected from servers or virtual machines running network services through proxy programs, such as central processor usage, memory consumption, disk I / O, network connection number and running state of key processes; the synchronous collection of these two heterogeneous data provides comprehensive context information for subsequent correlation analysis and hazard assessment;

[0049] Step S2, determining the traffic anomaly degree of network traffic data and the host state severity of host state data, is the core quantitative evaluation step of the method; the goal is to convert the original multi-dimensional data into standardized and comparable risk indicators; traffic anomaly degree is a quantitative indicator to measure the degree of deviation of the current network traffic pattern from the normal baseline; host state severity is another quantitative indicator to assess the risk level of the current host running state deterioration; independent quantitative evaluation of network behavior and host state lays the foundation for subsequent fusion analysis;

[0050] Step S3, based on the traffic anomaly degree and the host state severity, calculating the potential hazard score, is the risk fusion decision step of the method; the innovation of this step is that it does not consider the statistical anomaly of network traffic in isolation, but correlates it with the actual impact of host state, so as to evaluate the real harm of an abnormal event; the potential hazard score is a comprehensive risk score that combines the degree of traffic deviation and the degree of host system deterioration that may be caused thereby, aiming to judge whether the abnormal traffic constitutes a substantial threat to the system;

[0051] Step S4, according to the potential hazard score, dynamically adjusting the detection threshold for subsequent network traffic data anomaly determination, is the adaptive closed-loop feedback step of the method; the purpose of this step is to enable the detection system to have learning and adaptation ability; the detection threshold is the judgment threshold for determining whether the network traffic is abnormal; by adjusting the threshold in the reverse direction according to the result of the potential hazard score, the system can realize intelligent control of detection sensitivity; for events with high evaluated hazard, the system will lower the detection threshold for similar events, thereby improving the detection sensitivity; on the contrary, for events evaluated as benign fluctuations, the threshold will be raised to enhance the tolerance of the system;

[0052] The embodiment constructs a complete technical solution from data collection, double-dimension quantification, risk fusion evaluation to closed-loop feedback adjustment through the above steps; overcomes the defect that the prior art only focuses on flow statistical characteristics and cannot evaluate substantial hazards, greatly improves the accuracy of abnormal event qualification by fusing host state data; and the dynamic threshold adjustment mechanism enables the system to continuously learn and adapt to dynamic changes in the business environment, effectively filters out benign flow fluctuations caused by service updates, elastic scaling and other normal business activities, significantly reduces the false positive rate, and enables the security operation team to focus more on real network threats.

[0053] S2 comprises:

[0054] The network flow data is converted into a network flow feature vector, and the network flow feature vector is input into a preset autoencoder model to calculate and obtain a flow anomaly degree;

[0055] The host state data is converted into a host state feature vector, and the host state feature vector is evaluated according to a preset rule set to calculate and obtain a host state severity;

[0056] In the embodiment, the calculation methods of the flow anomaly degree and the host state severity in step S2 are limited;

[0057] To calculate the flow anomaly degree, the network flow data is converted into a network flow feature vector, and the network flow feature vector is input into a preset autoencoder model; the implementation is to extract key statistical features from the original network flow data, such as packet rate, byte rate, new connection number, protocol distribution per unit time, etc., combine and standardize these numerical features to form a network flow feature vector; the preset autoencoder model is a deep learning neural network, and the working principle is to compress the input vector into a low-dimensional latent representation through an encoder, and then try to reconstruct the original input vector from the latent representation through a decoder; the model needs to be trained on a large number of network flow feature vector datasets in normal business periods, so that it can accurately learn the internal mode of normal flow; when a new network flow feature vector is input, the model will calculate the difference between the reconstruction output and the original input, which is the reconstruction error, directly reflecting the deviation of the current flow from the normal mode, and serving as the basis for the flow anomaly degree;

[0058] In the embodiment, the autoencoder model specifically adopts a deep neural network composed of fully connected layers, and the structure is symmetrical. The encoder part contains three layers: the number of nodes of the input layer is the same as the dimension of the network traffic feature vector, for example, 64 dimensions, followed by two hidden layers with 32 and 16 nodes respectively. The decoder part is mirror-symmetrical with the encoder and contains two hidden layers with 32 and 64 nodes respectively. All the hidden layers use the rectified linear unit as the activation function, and the output layer uses the Sigmoid activation function to ensure that the numerical range of the reconstructed vector is consistent with the normalized input vector.

[0059] The training of the model is based on a 30-day normal business traffic data set that does not contain any known attacks; the goal of the training is to minimize the reconstruction error, and the loss function uses the mean square error, which is:

[0060] ;

[0061] wherein, : loss function, in the embodiment, the mean square error is used; : input network traffic feature vector; : reconstruction function of the autoencoder; : output vector reconstructed by the autoencoder; : dimension of the network traffic feature vector, i.e., the number of features; : dimension index in the feature vector; : i-th component of the input feature vector : i-th component of the reconstructed vector : i-th component of the reconstructed vector : i-th component of the reconstructed vector

[0062] We use the Adam optimizer for gradient descent, with a learning rate of , a batch size of 256, and a total of 100 cycles of training until the loss function converges;

[0063] To calculate the host state severity, the host state data is converted into a host state feature vector, and the host state feature vector is evaluated according to a pre-set rule set. The implementation is as follows: core performance indicators such as CPU usage, memory occupancy, and active process number are extracted from the collected host state data to form a host state feature vector; the pre-set rule set is a set of machine-readable logical judgment conditions formulated according to the business service level agreement or the experience of operation and maintenance experts; the working principle is that the rule set contains a series of if-then rules, for example, if the CPU usage exceeds 95% for 5 consecutive minutes, the severity is assigned a value of 0.9; the system matches and evaluates the host state feature vector in the rule set as input, thereby obtaining an initial severity score;

[0064] The rule set is a structured set of judgment logic, which sets a clear severity level for different host state indicators or their combinations; for example, a simplified rule set can be defined as follows:

[0065] Rule 1, high CPU load: if the average CPU usage in the past 5 minutes exceeds 90% and the duration exceeds 2 minutes, the base severity is ;

[0066] Rule 2, memory depletion: if the available memory is less than 5% of the total memory, the base severity is ;

[0067] Rule 3, suspicious process: if an unauthorized or process name in the known malware library appears, the base severity is ;

[0068] Rule 4, connection surge: if the total number of TCP connections grows by more than 300% in 1 minute and the total number is greater than 1000, the base severity is ;

[0069] When the host state feature vector meets multiple rules at the same time, the initial severity score takes the highest score in all triggered rules to reflect the most urgent state. For example, if rule 1 and rule 4 are met at the same time, the initial severity score is ;

[0070] This embodiment provides a specific and feasible technical means for the calculation of traffic anomaly degree and host state severity by using the autoencoder model and the SLA-based rule set; the use of autoencoder can automatically capture the complex patterns of normal traffic in an unsupervised learning manner, achieving high sensitivity detection of unknown abnormal types; the severity evaluation method based on the rule set directly integrates the domain knowledge and business requirements of operation and maintenance experts into the evaluation system, ensuring the accuracy and business relevance of host state risk assessment, making the entire detection scheme more automated and intelligent.

[0071] Embodiment 2

[0072] The step of determining the traffic anomaly degree includes:

[0073] The reconstruction error of the network traffic feature vector after reconstruction via the autoencoder model is calculated to obtain the original anomaly degree;

[0074] The original anomaly degree is normalized to generate the traffic anomaly degree;

[0075] The step of determining the host state severity includes:

[0076] The host state feature vector is evaluated according to a preset rule set based on a service level agreement to obtain an original severity;

[0077] The original severity is normalized to generate a host state severity;

[0078] In this embodiment, the generation process of the traffic anomaly degree and the host state severity from the original value to the final standardized score is further clarified;

[0079] The step of determining the traffic anomaly degree specifically includes: calculating the reconstruction error of the network traffic feature vector after reconstruction via the autoencoder model to obtain an original anomaly degree; the calculation is realized by the following formula:

[0080] ;

[0081] Wherein, : original anomaly degree; : input feature vector composed of statistical features extracted from traffic data; : autoencoder reconstruction function; the original anomaly degree here is the Euclidean distance between the input vector and the output vector after reconstruction of the input vector via the autoencoder function , and the numerical size directly reflects the degree of deviation of the input data from the normal mode; : calculation of Euclidean distance (L2 norm);

[0082] The original anomaly degree is normalized to generate a traffic anomaly degree; the normalization aims to eliminate the dimension influence and map the anomaly degree to a unified and comparable interval, for example; the processing is completed by a preset normalization function , which can be constructed based on the maximum and minimum values or statistical distribution of historical data, and its role is to convert the unbounded reconstruction error into a standardized traffic anomaly degree ;

[0083] In this embodiment, the min-max normalization is used as the normalization function , which linearly maps the original anomaly degree to the interval , and the calculation formula is:

[0084] ;

[0085] Wherein, and are the minimum value and the maximum value of the reconstruction error calculated from the normal traffic data set in the model training stage; for any new error value less than , the normalized result is 0; for any value greater than The new error value, whose normalized result is set to 1, is used to indicate an extreme anomaly; : The default normalization function, which is the min-max normalization function in this case;

[0086] The steps for determining the severity of a host's state are similar. Based on a pre-defined set of rules according to the service level agreement (SSPA), the host's state feature vector is evaluated to obtain the raw severity. This process can be represented as an evaluation function. Based on business rules, the host status feature vector Its source is a performance metric extracted from host status data and mapped to a raw severity level. ;

[0087] The original severity scores are normalized to generate host state severity scores; to ensure consistency in subsequent calculations, a pre-defined normalization function is used. Original severity Mapping to intervals generates the final host state severity. ;

[0088] Due to the original severity It is directly assigned by the rule set and located in The values ​​within the interval, therefore the normalization function here. It can be an identity mapping, that is To maintain the intuitiveness of its business semantics. If the rule set generates... If the range is uncertain, then it can be compared with... The same min-max normalization method;

[0089] By normalizing the original anomaly and severity, this embodiment ensures that two indicators from different sources and with different dimensions can be fairly integrated in the subsequent hazard score calculation. This standardized processing method is a key prerequisite for the effective integration of multimodal data. It improves the stability and reliability of the comprehensive scoring model, making the final potential hazard score more scientific and interpretable.

[0090] S3 includes:

[0091] Based on the temporal relationship between network traffic data and host status data, the causal attribution confidence score is calculated.

[0092] By using a pre-defined weighted model, the potential hazard score is generated by integrating traffic anomaly degree, host status severity, and causal attribution confidence.

[0093] In this embodiment, the calculation process of the potential hazard score in step S3 is described in detail;

[0094] The process calculates a causal attribution confidence based on the temporal relationship between the network traffic data and the host state data; the causal attribution confidence is a quantitative indicator to measure the likelihood of a causal relationship between the identified network traffic anomaly and a subsequent host state deterioration event; to obtain the confidence, when a network traffic is preliminarily determined as abnormal, the system analyzes the host state data in a very short time window after the abnormality; for example, the confidence is quantified by the conditional probability that the host state severity significantly increases after the traffic anomaly degree exceeds a threshold value within a preset time window of a specific event type ;

[0095] The very short time window is set to 30 seconds after the traffic anomaly event occurs; the host state severity significantly increases is defined as: within the time window, the value of the host state severity at least increases by 50% compared to the beginning of the window, and the peak value exceeds a preset high threshold value, for example 1000; therefore, the formula for calculating the causal attribution confidence is:

[0096]

[0097] wherein, : the causal attribution confidence, a quantitative indicator; : the calculation of the conditional probability; : the host state severity; : the time when the traffic anomaly occurs; : the preset very short time window, which is 60 seconds in this embodiment; : indicates that the host state severity significantly increases within the time window after the abnormality occurs; : the traffic anomaly degree; : the traffic anomaly degree at the time t; : the detection threshold value; : the specific event type; : represents the current time, indicating that the threshold value is dynamically changed; : the detection threshold value for the event type at the current time ; This probability value is obtained based on the statistics of historical data, i.e., the conditional probability that the host state severity significantly increases after the traffic anomaly degree exceeds the threshold value within a preset time window of all types

[0098] ​​​​​​​​​​​​The proportion of events satisfying the host state deterioration condition in the traffic anomaly event;

[0099] The preset weighting model is a linear combination model, and the linear model provides a simple and effective fusion manner for the embodiment. In other embodiments, a gradient boosting decision tree, a neural network or other nonlinear models can be used for fusion to capture more complex interaction relationships between factors, so as to more accurately depict the dose-effect relationship of the risk. The purpose is to assign corresponding weights to different factors according to their importance, so as to calculate a comprehensive risk score. The calculation formula of the model is as follows:

[0100] ;

[0101] wherein, : potential harm score; : traffic anomaly degree from step S2; : host state severity from step S2; : causal attribution confidence; : dimensionless weight coefficient; the weight coefficient here is set in advance by regression analysis training on historical security event data or according to the experience of security experts, and reflects the relative importance of the abnormal degree of traffic itself, the actual impact on the host, and the correlation strength between the two in the final harm assessment;

[0102] To determine these weights, the embodiment uses a logistic regression analysis based on historical data. The security operation team will review all events that triggered a high harm score in the past period, for example, 3 months, and label each event as a real threat or benign fluctuation. The triplet of each event is used as an input feature, and the event label is used as an output variable to train a logistic regression model. The goal of model training is to find a set of optimal weights so that the potential harm score can best distinguish between the two types of events; for example, by maximizing the value of TP events while minimizing the value of FP events. After the model is trained, the model coefficients are normalized to obtain the final weight coefficients; for example, a set of typical weight values can be ;

[0103] The embodiment introduces causal attribution confidence and uses a weighted model to fuse three elements, which greatly improves the accuracy of potential harm score. It is no longer simply adding traffic anomalies and host anomalies, but deeply analyzes the causal relationship between the two. This enables the system to effectively distinguish between benign fluctuations that are only statistically abnormal but do not cause actual damage, and malicious attacks that directly lead to system state deterioration, thereby achieving a deeper and more accurate assessment of risk.

[0104] Embodiment 3

[0105] S4 comprises extracting key fields from current network traffic data to determine the type of event to which it belongs;

[0106] From a preset threshold mapping table that stores the correspondence between event types and detection thresholds, the current detection threshold corresponding to the event type is retrieved;

[0107] The traffic anomaly degree is compared with the current detection threshold to obtain a comparison result;

[0108] The event type is determined by the source IP address, destination port and protocol of the network traffic data, or according to the actual application scenario, the destination IP address, application layer protocol features and other fields can be further combined to form a more refined event type identifier;

[0109] In this embodiment, the first half of the process of dynamically adjusting the detection threshold in step S4 is described, and the event type involved is clearly defined;

[0110] To achieve differentiated detection, key fields are extracted from current network traffic data to determine the type of event to which it belongs; the event type is an identifier for classifying and aggregating network traffic, and its purpose is to implement refined detection strategies for different services or communication patterns; according to the above definition, the event type is determined by the source IP address, destination port and protocol of the network traffic data, forming a three-tuple:

[0111] ;

[0112] This definition can uniquely identify a specific network service or communication session mode;

[0113] Based on the determined event type, the corresponding current detection threshold is retrieved from a preset threshold mapping table that stores the correspondence between event types and detection thresholds; the threshold mapping table is a key-value storage structure that is dynamically maintained, and stores a dedicated detection threshold for each identified event type ​; when traffic arrives and is determined to be of event type Afterwards, the system queries the table to obtain the current threshold value for determining whether the traffic of this type is abnormal , representing the current time; for the event type that occurs, an initial threshold value set according to the system security baseline can be used;

[0114] After obtaining the threshold value, the traffic anomaly degree is compared with the current detection threshold value to obtain a comparison result; this step is to compare the calculated traffic anomaly degree with the current detection threshold value retrieved from the mapping table; if the comparison result is abnormal, and a subsequent hazard assessment and threshold value updating process is triggered; if , the comparison result is normal;

[0115] By introducing the concept of event type and maintaining independent detection threshold values for each type, the embodiment realizes highly differentiated and refined anomaly detection; it allows the system to use different detection sensitivities for different services, avoiding the high false positive or high false negative problems caused by one-size-fits-all detection; this threshold value management mechanism based on event type is the basis for realizing precise adaptive adjustment function, significantly improving the pertinence and effectiveness of the detection strategy.

[0116] Embodiment 4

[0117] S4 also includes, in response to the comparison result being that the traffic anomaly degree is greater than the current detection threshold value, generating an updated detection threshold value based on the potential hazard score and a preset reference hazard threshold value through a feedback control loop formula calculation;

[0118] in response to the comparison result being that the traffic anomaly degree is not greater than the current detection threshold value, taking the current detection threshold value as the updated detection threshold value;

[0119] S4 also includes replacing the updated detection threshold value in the threshold value mapping table with the current detection threshold value, and the updated detection threshold value is used for abnormality determination on subsequent received network traffic data belonging to the same event type;

[0120] In this embodiment, the latter half of the detection threshold value dynamic adjustment in step S4, i.e., the specific updating and application mechanism, is described in detail;

[0121] When the comparison result is that the traffic anomaly degree is greater than the current detection threshold value, the system will generate an updated detection threshold value based on the potential hazard score and a preset reference hazard threshold value through a feedback control loop formula calculation; the preset reference hazard threshold value is a reference value set by security experts according to the acceptable risk level; the feedback control loop formula is a mathematical model derived from the first-order feedback control theory, which is used to adjust the control parameters according to the error between the system output and the desired target; the calculation is as follows:

[0122] ;

[0123] wherein, : updated threshold value; : current threshold value; : a dimensionless learning rate representing the adjustment rate; : hazard score from step S3; : baseline hazard; is a preset threshold lower limit greater than zero, to prevent the threshold from being excessively reduced to an invalid value such as a negative number or zero after an extreme hazard event, ensuring the basic sensitivity of the detection system;

[0124] Learning rate is a key hyperparameter that controls the speed of threshold adjustment; in this embodiment, the value of is limited to the interval A larger value, such as , will make the threshold adjustment very fast, allowing the system to adapt to changes in threats more quickly, but it may also cause the system to oscillate when facing occasional benign fluctuations; a smaller value, such as , will make the system adjustment smoother and more stable, but the adaptation speed will be slower; generally, the initial value of ;

[0125] Baseline hazard threshold represents the normal hazard level that the system can tolerate without adjusting the detection sensitivity; its setting is not entirely dependent on subjective experience, but can be obtained in a data-driven manner; the method is to analyze all events labeled as benign fluctuations in the aforementioned historical event data set, calculate the average value or the 95th percentile of the potential hazard scores of these events, and take this value as ; for example, if the analysis shows that the average hazard score of FP events is , then can be set; this means that only when the hazard of a new event is significantly higher than the average level of historical benign fluctuations, the system will consider that it needs to become more vigilant, i.e., to lower the threshold;

[0126] The internal mechanism of this formula is that if the calculated hazard score is higher than the baseline If the difference is positive, the new threshold is increased If the difference is positive, the new threshold is increased , thus making the system more sensitive to similar events; conversely, if If the difference is positive, the new threshold is increased , thus making the system more sensitive to similar events; conversively, if

[0127] If the difference is not greater than the current threshold, the current threshold is taken as the updated threshold, i.e. , to maintain system stability

[0128] According to the above description, the update operation is performed, and the updated threshold is replaced with the current threshold associated with the event type in the threshold mapping table ; ;

[0129] The updated threshold is used to determine the abnormality of the network traffic data received subsequently and belonging to the same event type, forming a complete adaptive adjustment closed loop

[0130] The embodiment introduces a feedback control loop to construct a closed-loop, adaptive threshold adjustment system; the system can simulate the memory and adaptability of the immune mechanism to threats, i.e., maintaining high vigilance to confirmed high-risk threats and gradually developing tolerance to harmless foreign objects; this mechanism enables the detection system to continuously learn from the environment and automatically optimize its detection strategy to adapt to changing business patterns and attack methods, ultimately achieving a dynamic balance between detection sensitivity and accuracy.

[0131] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A deep learning-based network traffic anomaly detection method, characterized in that, The specific steps include: S1, collecting network traffic data and host state data; S2, determining the traffic anomaly degree of the network traffic data and the host state severity of the host state data; S3, based on the traffic anomaly degree and the host state severity, calculating the potential harm score; S4, according to the potential harm score, dynamically adjusting the detection threshold for subsequent network traffic data anomaly determination; S4 includes: Extracting key fields from the current network traffic data to determine the event type to which it belongs; From the preset threshold mapping table that stores the correspondence between event type and detection threshold, retrieve the current detection threshold corresponding to the event type; Compare the traffic anomaly degree with the current detection threshold to obtain a comparison result; S4 also includes: In response to the comparison result that the traffic anomaly degree is greater than the current detection threshold, based on the potential harm score and the preset reference harm threshold, an updated detection threshold is calculated by a feedback control loop formula; In response to the comparison result that the traffic anomaly degree is not greater than the current detection threshold, the current detection threshold is taken as the updated detection threshold.

2. The network traffic anomaly detection method based on deep learning according to claim 1, characterized in that S2 Including: Convert the network traffic data into a network traffic feature vector, and input the network traffic feature vector into a preset autoencoder model to calculate the traffic anomaly degree; Convert the host state data into a host state feature vector, and evaluate the host state feature vector according to a preset rule set to calculate the host state severity.

3. The method of claim 2, wherein, The steps for determining the traffic anomaly degree include: Calculate the reconstruction error of the network traffic feature vector after reconstruction via the autoencoder model to obtain the original anomaly degree; Normalize the original anomaly degree to generate the traffic anomaly degree.

4. The method of claim 2, wherein, The steps for determining the host state severity include: According to the preset rule set based on the service level agreement, evaluate the host state feature vector to obtain the original severity; Normalize the original severity to generate the host state severity.

5. The method of claim 1, wherein, S3 includes: Based on the time sequence relationship between the network traffic data and the host state data, calculate the causal attribution confidence; Through a preset weighting model, integrate the traffic anomaly degree, the host state severity and the causal attribution confidence to generate the potential harm score.

6. The method of claim 5, wherein, S4 also includes: Replace the current detection threshold with the updated detection threshold in the threshold mapping table, and the updated detection threshold is used for anomaly determination on subsequent received network traffic data belonging to the same event type.

7. The method of claim 6, wherein, The event type is determined by the source IP address, destination port and protocol of the network traffic data.

Images