Dynamic model detection method based on AI algorithm

Abstract

The invention discloses a dynamic model detection method based on an AI algorithm, and relates to the field of AI algorithms, and the method can monitor and collect network abnormal data in real time by installing a network monitoring tool in a monitoring network environment. And preprocessing the network abnormal data to generate a standardized data set, and storing the standardized data set in a MySQL relational database. The data sets are extracted in real time through SQL query, and a network anomaly feature vector Ft is generated by using a feature extraction formula. Then constructing a network flow dynamic model, inputting the historical network anomaly feature vector Ft into a long short-term memory (LSTM) network of an AI algorithm for preliminary training to obtain a network flow dynamic model M0, and dynamically updating the input of the network anomaly feature vector Ft extracted in real time to obtain an updated network flow model Mt; the dynamic model can capture tiny changes in network traffic, and potential abnormal behaviors can be effectively identified.

Description

Technical Field

[0001] The present invention relates to the field of AI algorithm technology, and in particular to a dynamic model detection method based on AI algorithm. Background Art

[0002] In the modern digital communication environment, real-time monitoring and anomaly detection of network traffic are important means to ensure network security. As the core technology, network traffic anomaly detection aims to identify and respond to various network attacks and abnormal behaviors, such as DDoS attacks, packet injections, and unauthorized access. These attacks may seriously affect the normal operation of the network, so timely and accurate detection and response to these abnormal behaviors are crucial to maintaining the stability and security of the network.

[0003] At present, network traffic anomaly detection technology mainly relies on predefined rules and static models. These methods often seem to be powerless when facing complex and changeable network environments. Static models are difficult to adapt to the dynamic changes of network traffic patterns, which can easily lead to false positives and false negatives, affecting the accuracy of detection. In addition, predefined rules are often unable to cope with new attacks and unknown threats, and attackers can evade detection by circumventing these rules. Therefore, existing detection methods have problems such as untimely response, insufficient accuracy, and poor adaptability when dealing with network anomalies. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a dynamic model detection method based on an AI algorithm, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention is implemented by the following technical solutions: A dynamic model detection method based on AI algorithm comprises the following steps:

[0006] S1. By installing a network monitoring tool in the monitoring network environment, real-time monitoring and collection of network abnormality data are performed, and the network abnormality data is pre-processed to obtain a standard data set, and then the standard data set is stored in a database;

[0007] S2, extracting the standard data set in the database in real time through SQL query, and extracting features from the standard data set to obtain the network anomaly feature vector Ft;

[0008] S3. Build a network traffic model through a long short-term memory network LSTM, and use the historical abnormal feature vector Ftl to preliminarily train the network traffic model to obtain a preliminary network traffic dynamic model M0, then input the real-time processed network abnormal feature vector Ft into the network traffic model, and dynamically update the network traffic model through an AI algorithm to obtain an updated network traffic model Mt;

[0009] S4. Construct a weighted Euclidean distance algorithm formula, then input the network anomaly feature vector Ft and the updated network traffic model Mt into the weighted Euclidean distance algorithm formula, calculate and output the deviation value Dt, and preset the first deviation threshold F1 and the second deviation threshold F2 to make an abnormal judgment on the deviation value Dt, obtain the network traffic detection result At, and trigger the response mechanism according to the evaluation result;

[0010] S5. Based on the preliminary comparative evaluation results, the error rate E of the network traffic detection result At is calculated, and the weight of the weighted Euclidean distance algorithm formula is dynamically adjusted based on the error rate E to obtain a new weight value Wi, and then the output optimized deviation value Dt2 and the first deviation threshold F1 and the second deviation threshold F2 are recalculated to perform a secondary comparative evaluation to optimize the network traffic anomaly and generate an iterative optimization mechanism.

[0011] Preferably, said S1 includes S11, S12 and S13;

[0012] S11. Monitor and collect network abnormal data in real time by installing network monitoring tools in the monitoring network environment;

[0013] The network monitoring tools include data packet capture tools, network analysis tools, protocol analysis tools and log analysis tools;

[0014] The abnormal data includes network delay L, number of times the network data packet protocol is used Xyc, total number of network data packet protocols Zcs, sequence number Seq, timestamp of connection establishment t start , the timestamp t of the connection closing end and the number of packets N for atypical ports non-typical ;

[0015] The network delay L is obtained by capturing network traffic data packets through a data packet capture tool, and then calculating and analyzing the delay of each network traffic data packet using a network analysis tool;

[0016] The network data packet protocol usage times Xyc and the network data packet protocol sum Zcs are obtained by analyzing the protocol layer information of each network traffic data packet through a protocol analysis tool;

[0017] The sequence number Seq is obtained by analyzing the log records of the firewall through a log analysis tool and extracting the sequence number of the network traffic data packet;

[0018] The timestamp of the connection establishment t start and the connection closed timestamp t end Obtain the connection establishment timestamp and connection closing timestamp of each session recorded by the firewall and load balancer devices through the work log;

[0019] The number of packets N of the atypical port non-typical Use log analysis tools to analyze firewall and intrusion monitoring system (IDS) logs, record and analyze the ports used in traffic, and mark and count the number of packets from atypical ports;

[0020] S12, preprocessing the collected network abnormal data to obtain a standard data set, wherein the preprocessing method includes data cleaning, data conversion and normalization processing;

[0021] S121, the data cleaning includes missing value processing, duplicate data removal and outlier processing;

[0022] The data conversion includes unified processing of data formats;

[0023] The normalization process converts the network abnormal data into a standard normal distribution with a mean of 0 and a standard deviation of 1;

[0024] The missing value handling is done by filling in missing data using mean and median values;

[0025] The duplicate data removal is performed by matching all fields of each record in the network abnormal data, identifying records with all fields being exactly the same, and removing these duplicates;

[0026] The outlier processing is performed by defining outliers based on business logic and then smoothing them using interpolation;

[0027] The data format unification processing is performed by using a data quality check tool to automatically detect and report inconsistent network abnormal data, and then using Pandas to unify the formatting of the network abnormal data in terms of time format, IP address format, numerical field, unified classification field, and text field;

[0028] S13. Build a MySQL relational database and set up a database connection pool, write the standard data set into the MySQL relational database in real time for storage, and extract the standard data set through SQL query.

[0029] Preferably, said S2 includes S21;

[0030] S21, extracting the standard data set in the MySQL relational database in real time through SQL query, performing feature extraction, and obtaining the network anomaly feature vector Ft, wherein the network anomaly feature vector Ft={L t , Fproto t , SeqVR t , SST t ,NTPU t}

[0031] The Lt represents the average network delay per unit time t, and the feature extraction formula is: Where n represents the total number of packets captured in unit time t, and Li represents the network delay of the i-th packet;

[0032] The Fproto t It represents the frequency of packet protocol layer interaction within unit time t. The feature extraction formula is: In the formula, Xyc i Indicates the number of times the i-th network packet protocol is used, Zcs t represents the total number of all protocols in the same time period t;

[0033] The SeqVR t It represents the rate of change of the data packet sequence number within the unit time t. The feature extraction formula is: In the formula, Seq i Indicates the sequence number of the i-th data packet, Seq i+1 Indicates the sequence number of the i+1th data packet;

[0034] The SST t It represents the survival time of the network session within the unit time t. The feature extraction formula is SSTt=t end -t start ;

[0035] The NTPU t It represents the usage rate of atypical ports in unit time t. The feature extraction formula is: Where N total Indicates the total number of packets for atypical ports.

[0036] Preferably, said S3 includes S31 and S32;

[0037] S31. Use the long short-term memory network LSTM in the AI ​​algorithm to build a network traffic model, and perform preliminary training on the network traffic model based on the historical network anomaly feature vector Ftl to obtain a preliminary network traffic dynamic model M0;

[0038] The preliminary network traffic dynamic model M0 is obtained by training with the following AI algorithm formula;

[0039] M0=f(Ftl);

[0040] Where f represents the LSTM function in the AI ​​algorithm, and the historical network anomaly feature vector Ftl is obtained by extracting the historical standard data set stored in the MySQL relational database for feature extraction. The historical network anomaly feature vector Ft={L l , Fprotol , SeqVR l , SST l ,NTPU l}.

[0041] Preferably, S32, inputting the network anomaly feature vector Ft obtained in real time into the preliminary network traffic dynamic model M0, and dynamically updating the preliminary network traffic dynamic model M0 through an AI algorithm to obtain an updated network traffic model Mt;

[0042] The updated network traffic model Mt is obtained by calculation using the following AI algorithm;

[0043] Mt=Mt-1+γ 1 ·g 1 (Ft-Ft-1)+γ 2 ·g 2 (h(Ft));

[0044] In the formula, γ 1 represents the first-level learning rate, γ 2 represents the secondary learning rate, g 1 represents the adjustment function of the linear feature, g 2 represents the nonlinear feature adjustment function, h(Ft) represents the nonlinear feature transformation function, and Ft-1 represents the network abnormality feature vector at time t-1.

[0045] Preferably, said S4 includes S41 and S42;

[0046] S41, by constructing a weighted Euclidean distance algorithm formula, and then calculating and outputting a deviation value Dt between the current network anomaly feature vector Ft and the predicted value of the network traffic model M at time t based on the obtained updated network traffic model Mt and the network anomaly feature vector Ft;

[0047] The deviation value Dt is calculated by the following weighted Euclidean distance algorithm formula:

[0048]

[0049] In the formula, Indicates the average network delay deviation value. Indicates the packet protocol layer interaction frequency deviation value, Indicates the packet sequence number change rate deviation value, Indicates the survival time deviation value of the network session. Indicates the deviation of the usage rate of the atypical port, w1, w2, w3, w4 and w5 respectively represent the network delay deviation value, the packet protocol layer interaction frequency deviation value, the packet sequence number change rate deviation value, the network session survival time deviation value and the preset weight value of the deviation of the usage rate of the atypical port, and the specific value is set by the user;

[0050] It represents the predicted value of the average network delay at time t by the network traffic model;

[0051] It represents the predicted value of the interaction frequency of the packet protocol layer at time t by the network traffic model;

[0052] It represents the predicted value of the rate of change of the packet sequence number at time t by the network traffic model;

[0053] It represents the predicted value of the survival time of the network session at time t by the network traffic model;

[0054] It represents the predicted value of the usage rate of atypical ports at time t by the network traffic model.

[0055] Preferably, S42, the user presets a first deviation threshold F1 and a second deviation threshold F2, and then performs a preliminary comparative evaluation with the obtained deviation value Dt, obtains the network traffic monitoring result At, and generates a response mechanism, and the specific evaluation content is as follows;

[0056]

[0057] The response mechanism triggers the response measures according to the abnormal level of the network traffic monitoring result At. The specific contents are as follows:

[0058] When the network traffic detection result At is a level 1 abnormality, all network connections are immediately blocked and S5 is executed;

[0059] When the network traffic monitoring result At is a level 2 abnormality, an abnormality log is recorded, monitoring continues, and S5 is executed;

[0060] When the network traffic monitoring result At is normal traffic, continue normal monitoring.

[0061] Preferably, the S5 includes S51, S52 and S53;

[0062] S51, when the network flow detection result At is abnormal, triggering the error rate analysis of the network flow detection result At, the network flow detection result At being abnormal includes the first-level abnormality and the second-level abnormality, according to the obtained network flow detection result At, calculating the error rate E of the network flow detection result At, and judging the accuracy of the network flow model Mt;

[0063] The error rate E is calculated by the following algorithm formula:

[0064]

[0065] Where, At s represents the actual network traffic detection result, and A represents the total number of samples detected, that is, the number of network traffic data points observed in the entire time period t.

[0066] Preferably, S52, according to the error rate E obtained, the weight value wi of the weighted Euclidean distance algorithm formula is adjusted by a gradient descent method in combination with the change direction of the error rate E to obtain a new weight value Wi;

[0067] The new weight value Wi is obtained by the following algorithm formula:

[0068]

[0069] Wherein, Wi represents the i-th adjusted new weight value, specifically the new network delay deviation weight value W1, the new data packet protocol layer interaction frequency deviation weight value W2, the new data packet sequence number change rate deviation weight value W3, the new network session survival time deviation weight value W4 and the new atypical port usage rate deviation weight value W5, wi represents the i-th old weight value, specifically wi = {w1, w2, w3, w4, w5}, η represents the weight adjustment learning rate, represents the partial derivative, Represents the partial derivative of the error rate with respect to the weight wi.

[0070] Preferably, S53, the obtained new weight value Wi is input into the weighted Euclidean distance algorithm formula for a second time, and the optimized deviation value Dt2 is calculated and output, and then the output optimized deviation value Dt2 is compared and evaluated with the first deviation threshold F1 and the second deviation threshold F2 for a second time, and the optimized network traffic detection result At2 is analyzed, and according to the network traffic detection result At, an iterative early warning strategy is executed, and the specific evaluation content is as follows;

[0071]

[0072] The iterative early warning strategy is triggered when the network traffic monitoring result At2 is abnormal, and the specific content is as follows;

[0073] When the network traffic monitoring result At2 is a first-level abnormality, S5 is iteratively executed for 3 times. If it is still abnormal after 3 iterations, an early warning message is generated to prompt the network administrator.

[0074] When the network traffic detection result At2 is a first-level abnormality, S5 is iteratively executed for 5 times. If it is still abnormal after 5 iterations, an early warning message is generated to prompt the network administrator.

[0075] Beneficial Effects

[0076] The present invention provides a dynamic model detection method based on AI algorithm, which has the following beneficial effects:

[0077] (1) This method can monitor and collect network anomaly data in real time by installing a network monitoring tool in the monitoring network environment. The network anomaly data is preprocessed to generate a standardized data set and stored in a MySQL relational database. These data sets are extracted in real time through SQL queries, and the network anomaly feature vector Ft is generated using the feature extraction formula. Subsequently, a network traffic dynamic model is constructed, and the historical network anomaly feature vector Ft is input into the long short-term memory network LSTM of the AI ​​algorithm for preliminary training to obtain the network traffic dynamic model M0, and the input of the real-time extracted network anomaly feature vector Ft is dynamically updated to obtain the updated network traffic model Mt. This dynamic model can capture subtle changes in network traffic, effectively identify potential abnormal behaviors, and significantly improve the accuracy of real-time anomaly detection, especially when facing complex and changeable network attacks.

[0078] (2) This method constructs a weighted Euclidean distance algorithm to calculate the deviation value Dt between the network anomaly feature vector Ft and the real-time updated network traffic model Mt, and performs an abnormality judgment on the deviation value through the preset first deviation threshold F1 and second deviation threshold F2 to obtain the network traffic detection result At. This judgment mechanism based on multi-level thresholds can effectively distinguish between normal traffic, minor anomalies and serious anomalies. In addition, the system also calculates the error rate E based on the detection results, and dynamically adjusts the weight parameter wi in the Euclidean distance algorithm through the gradient descent method to obtain a new weight value Wi, and then recalculates the optimized deviation value Dt2. Through this feedback and iterative optimization mechanism, the model can continuously adapt to new network traffic patterns, improve the accuracy and stability of detection, reduce false positives and false negatives, and ensure that the system can still operate stably in a complex network environment.

[0079] (3) When the network traffic monitoring result At is judged to be abnormal, the system will automatically trigger the corresponding response mechanism. For example, for the first-level anomaly, the system will immediately block all network connections to ensure that the threat does not spread; for the second-level anomaly, the system will record the abnormal log and continue to monitor to prevent the problem from further deteriorating. At the same time, the method also supports error rate E analysis of the network traffic detection result At, and based on the analysis results, the gradient descent method is used to optimize the weight parameters, thereby further improving the accuracy of detection. If anomalies are still detected after multiple iterations of optimization, the system will automatically generate warning information to prompt the network administrator to take further measures. This automated response and warning system not only improves the initiative and intelligence level of network security protection, but also can respond quickly when anomalies occur in the network environment, greatly reducing the harm caused by potential threats and ensuring the continuous and stable operation of the network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a schematic diagram of the steps of the dynamic model detection method based on AI algorithm of the present invention. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] Example 1

[0083] The present invention provides a dynamic model detection method based on AI algorithm, please refer to Figure 1 , including the following steps:

[0084] S1. By installing a network monitoring tool in the monitoring network environment, real-time monitoring and collection of network abnormality data are performed, and the network abnormality data is pre-processed to obtain a standard data set, and then the standard data set is stored in a database;

[0085] S2, extracting the standard data set in the database in real time through SQL query, and extracting features from the standard data set to obtain the network anomaly feature vector Ft;

[0086] S3. Build a network traffic model through a long short-term memory network LSTM, and use the historical abnormal feature vector Ftl to preliminarily train the network traffic model to obtain a preliminary network traffic dynamic model M0, then input the real-time processed network abnormal feature vector Ft into the network traffic model, and dynamically update the network traffic model through an AI algorithm to obtain an updated network traffic model Mt;

[0087] S4. Construct a weighted Euclidean distance algorithm formula, then input the network anomaly feature vector Ft and the updated network traffic model Mt into the weighted Euclidean distance algorithm formula, calculate and output the deviation value Dt, and preset the first deviation threshold F1 and the second deviation threshold F2 to make an abnormal judgment on the deviation value Dt, obtain the network traffic detection result At, and trigger the response mechanism according to the evaluation result;

[0088] S5. Based on the preliminary comparative evaluation results, the error rate E of the network traffic detection result At is calculated, and the weight of the weighted Euclidean distance algorithm formula is dynamically adjusted based on the error rate E to obtain a new weight value Wi, and then the output optimized deviation value Dt2 and the first deviation threshold F1 and the second deviation threshold F2 are recalculated to perform a secondary comparative evaluation to optimize the network traffic anomaly and generate an iterative optimization mechanism.

[0089] In this embodiment, the method can monitor and collect network abnormality data in real time by installing a network monitoring tool in a monitoring network environment. These data are preprocessed to generate a standard data set and stored in a MySQL relational database, providing a high-quality and consistent data basis for subsequent analysis. Secondly, the standard data set in the database is extracted in real time through SQL query, and feature extraction is performed to obtain the network abnormality feature vector Ft. The network traffic model is constructed by a long short-term memory network LSTM, and the network traffic model is preliminarily trained using the historical abnormal feature vector Ftl to generate a preliminary network traffic dynamic model M0, and then the real-time processed network abnormality feature vector Ft is input into the network traffic model, and the network traffic model is dynamically updated by an AI algorithm to obtain an updated network traffic model Mt, ensuring that the network traffic model can adapt to the ever-changing network environment and capture potential abnormal behaviors. Further, by constructing a weighted Euclidean distance algorithm, the network abnormality feature vector Ft is compared with the predicted value of the updated network traffic model Mt to calculate the deviation value Dt. The deviation value is judged as abnormal through the preset first deviation threshold F1 and second deviation threshold F2, and the network traffic detection result At is generated. The appropriate response mechanism is triggered according to the detection result, such as immediately blocking the connection or recording the abnormal log. Finally, the detection result is evaluated based on the error rate EEE, and the weight parameter wi in the weighted Euclidean distance algorithm is dynamically adjusted through the gradient descent method to obtain the optimized weight value Wi, and then the optimized deviation value Dt2 is calculated to achieve a secondary comparative evaluation. Through this iterative optimization mechanism, the system can continuously improve the detection accuracy and response sensitivity.

[0090] Example 2

[0091] This embodiment is explained in Example 1, please refer to Figure 1, specifically: S1 includes S11, S12 and S13;

[0092] S11. Monitor and collect network abnormal data in real time by installing network monitoring tools in the monitoring network environment;

[0093] Network monitoring tools include packet capture tools, network analysis tools, protocol analysis tools, and log analysis tools;

[0094] Abnormal data includes network delay L, number of times the network packet protocol is used Xyc, total number of network packet protocols Zcs, sequence number Seq, and timestamp of connection establishment t start , the timestamp t of the connection closing end and the number of packets N for atypical ports non-typical ;

[0095] The network delay L is obtained by capturing network traffic packets through a packet capture tool, and then using a network analysis tool to calculate and analyze the delay of each network traffic packet;

[0096] The number of times the network packet protocol is used Xyc and the total number of network packet protocols Zcs are obtained by analyzing the protocol layer information of each network traffic data packet through a protocol analysis tool;

[0097] The sequence number Seq is obtained by analyzing the log records of the firewall through log analysis tools and extracting the sequence number of the network traffic data packet;

[0098] Connection establishment timestamp t start and the connection closed timestamp t end Obtain the connection establishment timestamp and connection closing timestamp of each session recorded by the firewall and load balancer devices through the work log;

[0099] Number of packets from atypical ports N non-typical Use log analysis tools to analyze firewall and intrusion monitoring system (IDS) logs, record and analyze the ports used in traffic, and mark and count the number of packets from atypical ports;

[0100] S12, preprocessing the collected network abnormal data to obtain a standard data set, wherein the preprocessing method includes data cleaning, data conversion and normalization processing;

[0101] S121. Data cleaning includes missing value processing, duplicate data removal and outlier processing;

[0102] Data conversion includes unified processing of data formats;

[0103] Normalization is done by converting network anomaly data into a standard normal distribution with a mean of 0 and a standard deviation of 1;

[0104] Missing value handling was done by filling in missing data using mean and median values;

[0105] Duplicate data removal matches all fields of each record in the network anomaly data, identifies records with exactly the same fields, and removes these duplicates;

[0106] Outlier processing is done by defining outliers based on business logic and then using interpolation to smooth them out;

[0107] Data format unification processing: Use data quality check tools to automatically detect and report inconsistent network anomaly data, and then use Pandas to unify the format of time format, IP address format, numeric fields, unified classification fields, and text fields.

[0108] S13. Build a MySQL relational database and set up a database connection pool, write the standard data set into the MySQL relational database in real time for storage, and extract the standard data set through SQL query.

[0109] In this embodiment, the method ensures the comprehensiveness and accuracy of the data by using a network monitoring tool to monitor and collect network abnormal data in real time. Subsequently, these abnormal data are preprocessed into a standard data set through data cleaning, data conversion and normalization, eliminating noise, abnormal values ​​and format inconsistency in the data, and improving the quality and consistency of the data. These high-quality data are stored in a MySQL relational database and extracted in real time through SQL queries, providing a solid data foundation for subsequent model training and anomaly detection.

[0110] Example 3

[0111] This embodiment is explained in Example 2. Please refer to Figure 1 , specifically: S2 includes S21;

[0112] S21, extract the standard data set in the MySQL relational database in real time through SQL query, perform feature extraction, and obtain the network anomaly feature vector Ft, the network anomaly feature vector Ft={L t , Fproto t , SeqVR t , SST t ,NTPU t}

[0113] L t represents the average network delay per unit time t, and the feature extraction formula is: Where n represents the total number of packets captured in unit time t, and Li represents the network delay of the i-th packet;

[0114] Fproto t It represents the frequency of packet protocol layer interaction within unit time t. The feature extraction formula is: In the formula, Xyc i Indicates the number of times the i-th network packet protocol is used, Zcs t represents the total number of all protocols in the same time period t;

[0115] SeqVR t It represents the rate of change of the data packet sequence number within the unit time t. The feature extraction formula is: In the formula, Seq i Indicates the sequence number of the i-th data packet, Seq i+1 Indicates the sequence number of the i+1th data packet;

[0116] SST t It represents the survival time of the network session within the unit time t. The feature extraction formula is SSTt=t end -t start ;

[0117] NTPU t It represents the usage rate of atypical ports in unit time t. The feature extraction formula is: Where N total Indicates the total number of packets for atypical ports.

[0118] In this embodiment, the method extracts standard data sets from the MySQL relational database in real time through SQL queries, and the system can obtain the latest network traffic data in a timely manner. Then, by extracting the features of these data sets, a network anomaly feature vector Ft is generated. These feature vectors fully reflect the key behavioral characteristics of network traffic and provide strong data support for subsequent anomaly detection. This method accurately quantifies complex network traffic behaviors into analyzable feature vectors through a systematic feature extraction process, greatly improving the model's ability to identify abnormal behaviors. In addition, through the careful design of feature extraction formulas, the system can capture more subtle abnormal changes in network traffic and reduce the probability of false positives and false negatives. Overall, this method improves the detection accuracy while also enhancing the system's real-time response capability, making network security protection more proactive and efficient. Such improvements not only make network traffic anomaly detection more accurate, but also lay a solid foundation for responding to new network threats.

[0119] Example 4

[0120] This embodiment is explained in Example 4. Please refer to Figure 1 , specifically: S3 includes S31 and S32;

[0121] S31. Use the long short-term memory network LSTM in the AI ​​algorithm to build a network traffic model, and perform preliminary training on the network traffic model based on the historical network anomaly feature vector Ftl to obtain a preliminary network traffic dynamic model M0;

[0122] The preliminary network traffic dynamic model M0 is obtained through the following AI algorithm formula training;

[0123] M0=f(Ftl);

[0124] Where f represents the LSTM function in the AI ​​algorithm, and the historical network anomaly feature vector Ftl is obtained by extracting the historical standard data set stored in the MySQL relational database for feature extraction. The historical network anomaly feature vector Ft = {L l , Fproto l , SeqVR l , SST l ,NTPU l}.

[0125] S32, inputting the network anomaly feature vector Ft obtained in real time into the preliminary network traffic dynamic model M0, and dynamically updating the preliminary network traffic dynamic model M0 through an AI algorithm to obtain an updated network traffic model Mt;

[0126] The updated network traffic model Mt is obtained by calculation using the following AI algorithm;

[0127] Mt=Mt-1+γ 1 ·g 1 (Ft-Ft-1)+γ 2 ·g 2 (h(Ft));

[0128] In the formula, γ 1 represents the first-level learning rate, γ 2 represents the secondary learning rate, g 1 represents the adjustment function of the linear feature, g 2 represents the nonlinear feature adjustment function, h(Ft) represents the nonlinear feature transformation function, and Ft-1 represents the network abnormality feature vector at time t-1.

[0129] In this embodiment, the method uses the long short-term memory network LSTM algorithm, and the system performs preliminary training on the network traffic model based on the historical network anomaly feature vector Ft1 to generate a preliminary network traffic dynamic model M0. This model can capture the complex time series characteristics in network traffic, thus laying a solid foundation for anomaly detection. Then, by inputting the real-time acquired network anomaly feature vector Ft into the preliminary network traffic dynamic model M0, and dynamically updating the model through the AI ​​algorithm, a real-time updated network traffic model Mt is generated. This continuous model update mechanism ensures that the model can quickly adapt to changes in the network environment, thereby more accurately identifying and responding to abnormal traffic. This method significantly enhances the model's learning ability and processing ability for time series data by introducing the long short-term memory network LSTM and the AI ​​algorithm dynamic update mechanism. Specifically, the long short-term memory network LSTM can effectively memorize abnormal behavior patterns over a long span of time, avoiding information forgetting and detection lag problems, while the dynamic update mechanism ensures that the model remains sensitive and responsive to emerging network threats.

[0130] Example 5

[0131] This embodiment is explained in Example 4. Please refer to Figure 1 , specifically: S4 includes S41 and S42;

[0132] S41, by constructing a weighted Euclidean distance algorithm formula, and then calculating and outputting a deviation value Dt between the current network anomaly feature vector Ft and the predicted value of the network traffic model M at time t based on the obtained updated network traffic model Mt and the network anomaly feature vector Ft;

[0133] The deviation value Dt is calculated by the following weighted Euclidean distance algorithm formula;

[0134]

[0135] In the formula, Indicates the average network delay deviation value. Indicates the packet protocol layer interaction frequency deviation value, Indicates the packet sequence number change rate deviation value, Indicates the survival time deviation value of the network session. Indicates the deviation of the usage rate of the atypical port, w1, w2, w3, w4 and w5 respectively represent the network delay deviation value, the packet protocol layer interaction frequency deviation value, the packet sequence number change rate deviation value, the network session survival time deviation value and the preset weight value of the deviation of the usage rate of the atypical port, and the specific value is set by the user;

[0136] M t LIt represents the predicted value of the average network delay at time t by the network traffic model;

[0137] It represents the predicted value of the interaction frequency of the packet protocol layer at time t by the network traffic model;

[0138] It represents the predicted value of the rate of change of the packet sequence number at time t by the network traffic model;

[0139] It represents the predicted value of the survival time of the network session at time t by the network traffic model;

[0140] It represents the predicted value of the usage rate of atypical ports at time t by the network traffic model.

[0141] S42, the user presets a first deviation threshold F1 and a second deviation threshold F2, and then performs a preliminary comparative evaluation with the obtained deviation value Dt, obtains the network traffic monitoring result At, and generates a response mechanism. The specific evaluation content is as follows;

[0142]

[0143] The response mechanism triggers the response measures according to the abnormal level of the network traffic monitoring result At. The specific contents are as follows:

[0144] When the network traffic detection result At is a level 1 abnormality, all network connections are immediately blocked and S5 is executed;

[0145] When the network traffic monitoring result At is a level 2 abnormality, an abnormality log is recorded, monitoring continues, and S5 is executed;

[0146] When the network traffic monitoring result At is normal traffic, continue normal monitoring.

[0147] In this embodiment, the method can accurately calculate the deviation value Dt between the network anomaly feature vector Ft and the predicted value of the updated network traffic model Mt by utilizing the constructed weighted Euclidean distance algorithm. The deviation value Dt reflects the difference between the actual network behavior and the model prediction. Combined with the preset weight value set by the user, the system can effectively identify subtle anomalies in the traffic. Subsequently, through the preset first deviation threshold F1 and the second deviation threshold F2, the system performs a preliminary comparative evaluation of the deviation value Dt and generates a network traffic monitoring result At. Depending on the detection results, the system triggers the corresponding response mechanism, such as immediately blocking the network connection when a first-level anomaly is detected to ensure network security; recording an abnormal log and continuing monitoring when a second-level anomaly is detected to reduce potential risks; and maintaining regular monitoring for normal traffic to avoid unnecessary interference.

[0148] Example 6

[0149] This embodiment is explained in Example 5. Please refer to Figure 1 , specifically: S5 includes S51, S52 and S53;

[0150] S51, when the network flow detection result At is abnormal, trigger the error rate analysis of the network flow detection result At, the abnormal network flow detection result At includes the first-level abnormality and the second-level abnormality, calculate the error rate E of the network flow detection result At according to the obtained network flow detection result At, and judge the accuracy of the network flow model Mt;

[0151] The error rate E is calculated using the following algorithm formula:

[0152]

[0153] Where, At s represents the actual network traffic detection result, and A represents the total number of samples detected, that is, the number of network traffic data points observed in the entire time period t.

[0154] S52, according to the error rate E obtained, the weight value wi of the weighted Euclidean distance algorithm formula is adjusted by the gradient descent method in combination with the change direction of the error rate E to obtain a new weight value Wi;

[0155] The new weight value Wi is obtained through the following algorithm formula;

[0156]

[0157] Wherein, Wi represents the i-th adjusted new weight value, specifically the new network delay deviation weight value W1, the new data packet protocol layer interaction frequency deviation weight value W2, the new data packet sequence number change rate deviation weight value W3, the new network session survival time deviation weight value W4 and the new atypical port usage rate deviation weight value W5, wi represents the i-th old weight value, specifically wi = {w1, w2, w3, w4, w5}, η represents the weight adjustment learning rate, represents the partial derivative, Represents the partial derivative of the error rate with respect to the weight wi.

[0158] S53, the obtained new weight value Wi is input into the weighted Euclidean distance algorithm formula for a second time, and the optimized deviation value Dt2 is calculated and output, and then the output optimized deviation value Dt2 is compared and evaluated with the first deviation threshold F1 and the second deviation threshold F2 for a second time, and the optimized network traffic detection result At2 is analyzed, and according to the network traffic detection result At, an iterative early warning strategy is executed, and the specific evaluation content is as follows;

[0159]

[0160] The iterative warning strategy is triggered when the network traffic monitoring result At2 is abnormal. The specific contents are as follows;

[0161] When the network traffic monitoring result At2 is a first-level abnormality, S5 is iteratively executed for 3 times. If it is still abnormal after 3 iterations, an early warning message is generated to prompt the network administrator;

[0162] When the network traffic detection result At2 is a first-level abnormality, S5 is iteratively executed for 5 times. If it is still abnormal after 5 iterations, an early warning message is generated to prompt the network administrator.

[0163] In this embodiment, when the network traffic detection result At is determined to be abnormal, the system will immediately calculate the error rate E and evaluate the accuracy of the current network traffic model Mt. Based on the analysis results of the error rate, the system dynamically adjusts the weight parameter wi of the weighted Euclidean distance algorithm through the gradient descent method to generate a new weight value Wi, thereby optimizing the sensitivity of the model to different network features. This step ensures that the model can self-correct and gradually improve the accuracy of detection. Then, the system inputs the updated weight value into the weighted Euclidean distance algorithm again, calculates the optimized deviation value Dt2, and compares and evaluates it with the preset first deviation threshold F1 and the second deviation threshold F2 for a second time, and generates an iterative optimization mechanism. This iterative optimization mechanism enables the system to continuously improve the adaptability and accuracy of the model during the detection process, ensuring that the results of anomaly detection are more reliable. This method not only enables the model to be adaptively optimized during operation by introducing the error rate E analysis and the gradient descent method, but also ensures that the system can issue a warning in time when facing persistent anomalies through an iterative warning strategy, prompting the network administrator to take further measures. This intelligent adjustment and feedback mechanism greatly improves the robustness and response speed of the detection system, reduces the occurrence of false alarms and missed alarms, and ultimately makes network security protection more accurate and efficient, and better able to deal with complex and changing network threats.

[0164] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic model detection method based on AI algorithm, characterized by: The following steps are involved: S1. By installing a network monitoring tool in the monitoring network environment, real-time monitoring and collection of network abnormality data are performed, and the network abnormality data is pre-processed to obtain a standard data set, and then the standard data set is stored in a database; S2, extracting the standard data set in the database in real time through SQL query, and extracting features from the standard data set to obtain the network anomaly feature vector Ft; S3. Build a network traffic model through a long short-term memory network LSTM, and use the historical abnormal feature vector Ftl to preliminarily train the network traffic model to obtain a preliminary network traffic dynamic model M0, then input the real-time processed network abnormal feature vector Ft into the network traffic model, and dynamically update the network traffic model through an AI algorithm to obtain an updated network traffic model Mt; S4, construct a weighted Euclidean distance algorithm formula, then input the network anomaly feature vector Ft and the predicted value of the updated network traffic model Mt into the weighted Euclidean distance algorithm formula, calculate and output the deviation value Dt, and preset the first deviation threshold F1 and the second deviation threshold F2 to make an abnormal judgment on the deviation value Dt, obtain the network traffic detection result At, and trigger the response mechanism according to the evaluation result; S5. Based on the preliminary comparative evaluation results, the error rate E of the network traffic detection result At is calculated, and the weight of the weighted Euclidean distance algorithm formula is dynamically adjusted based on the error rate E to obtain a new weight value Wi, and then the output optimized deviation value Dt2 and the first deviation threshold F1 and the second deviation threshold F2 are recalculated to perform a secondary comparative evaluation to optimize the network traffic anomaly and generate an iterative optimization mechanism.

2. The dynamic model detection method based on AI algorithm according to claim 1, characterized in that: Said S1 includes S11, S12 and S13; S11. Monitor and collect network abnormal data in real time by installing network monitoring tools in the monitoring network environment; The network monitoring tools include data packet capture tools, network analysis tools, protocol analysis tools and log analysis tools; The abnormal data includes network delay L, number of times the network data packet protocol is used Xyc, total number of network data packet protocols Zcs, sequence number Seq, timestamp of connection establishment t start , the timestamp t of the connection closing end and the number of packets N for atypical ports non-typical ; S12, preprocessing the collected network abnormal data to obtain a standard data set, wherein the preprocessing method includes data cleaning, data conversion and normalization processing; S121, the data cleaning includes missing value processing, duplicate data removal and outlier processing; The data conversion includes unified processing of data formats; The normalization process converts the network abnormal data into a standard normal distribution with a mean of 0 and a standard deviation of 1; S13. Build a MySQL relational database and set up a database connection pool, write the standard data set into the MySQL relational database in real time for storage, and extract the standard data set through SQL query.

3. The dynamic model detection method based on AI algorithm according to claim 1, characterized in that: Said S2 includes S21; S21, extracting the standard data set in the MySQL relational database in real time through SQL query, performing feature extraction, and obtaining the network anomaly feature vector Ft, wherein the network anomaly feature vector Ft={L t , Fproto t , SeqVR t , SST t ,NTPU t } The L t represents the average network delay per unit time t, and the feature extraction formula is: Where n represents the total number of packets captured in unit time t, and Li represents the network delay of the i-th packet; The Fproto t It represents the frequency of packet protocol layer interaction within unit time t. The feature extraction formula is: In the formula, Xyc i Indicates the number of times the i-th network packet protocol is used, Zcs t represents the total number of all protocols in the same time period t; The SeqVR t It represents the rate of change of the data packet sequence number within the unit time t. The feature extraction formula is: In the formula, Seq i Indicates the sequence number of the i-th data packet, Seq i+1 Indicates the sequence number of the i+1th data packet; The SST t It represents the survival time of the network session within the unit time t. The feature extraction formula is SSTt=t end -t start ; The NTPU t It represents the usage rate of atypical ports in unit time t. The feature extraction formula is: Where N total Indicates the total number of packets for atypical ports.

4. The dynamic model detection method based on AI algorithm according to claim 3 is characterized in that: The S3 includes S31 and S32; S31. Use the long short-term memory network LSTM in the AI ​​algorithm to build a network traffic model, and perform preliminary training on the network traffic model based on the historical network anomaly feature vector Ftl to obtain a preliminary network traffic dynamic model M0; The preliminary network traffic dynamic model M0 is obtained by training with the following AI algorithm formula; M0=f(Ftl); Where f represents the LSTM function in the AI ​​algorithm, and the historical network anomaly feature vector Ftl is obtained by extracting the historical standard data set stored in the MySQL relational database for feature extraction. The historical network anomaly feature vector Ft={L l , Fproto l , SeqVR l , SST l ,NTPU l }.

5. The dynamic model detection method based on AI algorithm according to claim 4 is characterized in that: S32, inputting the network anomaly feature vector Ft obtained in real time into the preliminary network traffic dynamic model M0, and dynamically updating the preliminary network traffic dynamic model M0 through an AI algorithm to obtain an updated network traffic model Mt; The updated network traffic model Mt is obtained by calculation using the following AI algorithm; Mt=Mt-1+γ1·g1(Ft-Ft-1)+γ2·g2(h(Ft)); In the formula, γ1 represents the first-level learning rate, γ2 represents the second-level learning rate, g1 represents the adjustment function of linear features, g2 represents the nonlinear feature adjustment function, h(Ft) represents the nonlinear feature transformation function, and Ft-1 represents the network abnormal feature vector at time t-1.

6. The dynamic model detection method based on AI algorithm according to claim 4 is characterized in that: The S4 includes S41 and S42; S41, by constructing a weighted Euclidean distance algorithm formula, and then calculating and outputting a deviation value Dt between the current network anomaly feature vector Ft and the predicted value of the network traffic model M at time t based on the obtained updated network traffic model Mt and the network anomaly feature vector Ft; The deviation value Dt is calculated by the following weighted Euclidean distance algorithm formula: In the formula, Indicates the average network delay deviation value. Indicates the packet protocol layer interaction frequency deviation value, Indicates the packet sequence number change rate deviation value, Indicates the survival time deviation value of the network session. Indicates the deviation of the usage rate of the atypical port, w1, w2, w3, w4 and w5 respectively represent the network delay deviation value, the packet protocol layer interaction frequency deviation value, the packet sequence number change rate deviation value, the network session survival time deviation value and the preset weight value of the deviation of the usage rate of the atypical port, and the specific value is set by the user; It represents the predicted value of the average network delay at time t by the network traffic model; It represents the predicted value of the interaction frequency of the packet protocol layer at time t by the network traffic model; It represents the predicted value of the rate of change of the packet sequence number at time t by the network traffic model; It represents the predicted value of the survival time of the network session at time t by the network traffic model; It represents the predicted value of the usage rate of atypical ports at time t by the network traffic model.

7. The dynamic model detection method based on AI algorithm according to claim 6 is characterized in that: S42, the user presets a first deviation threshold F1 and a second deviation threshold F2, and then performs a preliminary comparison and evaluation with the obtained deviation value Dt, obtains a network traffic detection result At, and generates a response mechanism. The specific evaluation content is as follows; The response mechanism triggers the response measures according to the abnormal level of the network traffic monitoring result At. The specific contents are as follows: When the network traffic detection result At is a level 1 abnormality, all network connections are immediately blocked and S5 is executed; When the network traffic monitoring result At is a level 2 abnormality, an abnormality log is recorded, monitoring continues, and S5 is executed; When the network traffic monitoring result At is normal traffic, continue normal monitoring.

8. The dynamic model detection method based on AI algorithm according to claim 7 is characterized in that: The S5 includes S51, S52 and S53; S51, when the network flow detection result At is abnormal, triggering the error rate analysis of the network flow detection result At, the abnormal network flow detection result At includes primary abnormality and secondary abnormality, and calculating the error rate E of the network flow detection result At according to the obtained network flow detection result At, and judging the accuracy of the network flow model Mt; The error rate E is calculated by the following algorithm formula: Where, At s represents the actual network traffic detection result, and A represents the total number of samples detected, that is, the number of network traffic data points observed in the entire time period t.

9. The dynamic model detection method based on AI algorithm according to claim 8, characterized in that: S52, according to the error rate E obtained, the weight value wi of the weighted Euclidean distance algorithm formula is adjusted by the gradient descent method in combination with the change direction of the error rate E to obtain a new weight value Wi; The new weight value Wi is obtained by the following algorithm formula: Wherein, Wi represents the i-th adjusted new weight value, specifically the new network delay deviation weight value W1, the new data packet protocol layer interaction frequency deviation weight value W2, the new data packet sequence number change rate deviation weight value W3, the new network session survival time deviation weight value W4 and the new atypical port usage rate deviation weight value W5, wi represents the i-th old weight value, specifically wi = {w1, w2, w3, w4, w5}, η represents the weight adjustment learning rate, represents the partial derivative, Represents the partial derivative of the error rate with respect to the weight wi.

10. The dynamic model detection method based on AI algorithm according to claim 9, characterized in that: S53, the obtained new weight value Wi is input into the weighted Euclidean distance algorithm formula for a second time, and the optimized deviation value Dt2 is calculated and output, and then the output optimized deviation value Dt2 is compared and evaluated with the first deviation threshold F1 and the second deviation threshold F2 for a second time, and the optimized network traffic detection result At2 is analyzed, and according to the network traffic detection result At, an iterative early warning strategy is executed, and the specific evaluation content is as follows; The iterative early warning strategy is triggered when the network traffic monitoring result At2 is abnormal, and the specific content is as follows; When the network traffic monitoring result At2 is a first-level abnormality, S5 is iteratively executed for 3 times. If it is still abnormal after 3 iterations, an early warning message is generated to prompt the network administrator. When the network traffic detection result At2 is a first-level abnormality, S5 is iteratively executed for 5 times. If it is still abnormal after 5 iterations, an early warning message is generated to prompt the network administrator.

Images