Intelligent early warning system based on machine learning

Through an intelligent early warning system based on machine learning, network data analysis and early warning problems are solved, accurate early warning and visual expression of network data are achieved, and network security is enhanced.

CN118972102AInactive Publication Date: 2024-11-15JIANGSU VOCATIONAL COLLEGE OF BUSINESS

Patent Information

Application Number
CN202410962424.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art cannot effectively analyze and warn of network data, resulting in the inability to accurately predict the network security status, and thus cannot effectively protect the security.

Method used

An intelligent early warning system based on machine learning was designed, including preprocessing module, feature extraction module, index prediction module, index comparison module and visual display module. By collecting log information, network feature data is extracted, and pre-trained network early warning model is used. Make predictions and generate warning information.

Benefits of technology

It realizes accurate warning and visual expression of network data, enhances the security of network data, and is of great significance to the development of network security technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972102A_ABST
    Figure CN118972102A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of data processing, and discloses an intelligent early warning system based on machine learning, a preprocessing module is used for collecting log information, obtaining initial network data from the log information, and preprocessing the initial network data to obtain to-be-analyzed network data; the feature extraction module is used for extracting data feature information of to-be-analyzed network data to obtain network feature data; the index prediction module is used for inputting the network feature data into a pre-trained network early warning model so as to predict a network state condition through the network early warning model, and outputting a network prediction result; the index comparison module is used for determining a network prediction index of the current network based on the network prediction result, comparing the network prediction index with a preset network security index, and when the network prediction index is greater than the network security index, determining that the network state is abnormal and generating early warning information; according to the invention, the security of network data is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent early warning system based on machine learning. Background Art

[0002] Network security issues are a key factor affecting computer network applications, and also the most important influencing factors. Solving network security issues is the basis for ensuring better application of computer networks. With the continuous development of the current big data era, people are paying more and more attention to network security issues. Better solving network security issues has become a necessary task and requirement, which has very important significance and value. During the use of network data, network data cannot be analyzed and predicted, and network security cannot be accurately warned, so network data cannot be protected. Therefore, studying an intelligent early warning system based on machine learning is of great significance for network data security protection. Summary of the invention

[0003] The purpose of the present invention is to solve the above problems and design an intelligent early warning system based on machine learning.

[0004] The first aspect of the present invention provides an intelligent early warning system based on machine learning, wherein the intelligent early warning system based on machine learning includes a preprocessing module, a feature extraction module, an indicator prediction module, an indicator comparison module and a visualization display module, wherein:

[0005] A preprocessing module, used to collect log information, obtain initial network data from the log information, and preprocess the initial network data to obtain network data to be analyzed;

[0006] A feature extraction module is used to extract data feature information from the network data to be analyzed to obtain network feature data;

[0007] An indicator prediction module, used for inputting the network feature data into a pre-trained network early warning model, so as to predict the network status through the network early warning model and output the network prediction result;

[0008] An indicator comparison module, used to determine a network prediction indicator of the current network based on the network prediction result, compare the network prediction indicator with a preset network security indicator, and when the network prediction indicator is greater than the network security indicator, determine that the network state is abnormal and generate warning information;

[0009] The visual display module is used to visually display the warning information and issue a warning to the monitoring personnel in the form of signal voice.

[0010] Optionally, in a first implementation of the first aspect of the present invention, the preprocessing module includes a data segmentation processing submodule, a data missing value processing submodule, a data outlier processing submodule and a data normalization processing submodule, wherein:

[0011] The data segmentation processing submodule is used to obtain the network traffic packet of the initial network data, and to segment and extract the hexadecimal text captured in the network traffic packet in combination with the protocol and the structure and field of the data packet and datagram to obtain a data set to be processed;

[0012] A data missing value processing submodule is used to process missing values ​​of the data in the to-be-processed data set by adopting a mean interpolation method to obtain a first data set;

[0013] A data outlier processing submodule, used to perform outlier detection on the first data set using the median absolute difference method, and judge the flow reliability of the detected outlier. If it is reliable, the current outlier is regarded as a normal value; if it is unreliable, the current outlier is processed as a missing value;

[0014] The data normalization processing submodule is used to obtain a second data set after processing the data outliers, and to perform data normalization processing on the data in the second data set using the min-max method to obtain the network data to be analyzed.

[0015] Optionally, in a second implementation of the first aspect of the present invention, the feature extraction module includes a decision tree generation submodule, a noise addition submodule and an importance assessment submodule, wherein:

[0016] A decision tree generation submodule is used to initialize a decision tree, generate a training data set and an out-of-bag data set from the network data to be analyzed using a Bootstrap resampling method, and generate a decision tree on the training data set;

[0017] Add a noise submodule to predict the out-of-bag data based on the generated decision tree, add noise to the feature values ​​in the out-of-bag data, and obtain a new out-of-bag data set;

[0018] The importance evaluation submodule is used to use the generated decision tree to predict the new out-of-bag data set, count the number of correctly predicted data, repeatedly iterate to evaluate the feature importance of the network data to be analyzed, and obtain network feature data.

[0019] Optionally, in a third implementation of the first aspect of the present invention, the indicator prediction module includes a training set input submodule, an initialization parameter submodule, a parameter optimization submodule, a test evaluation submodule and a network prediction submodule, wherein:

[0020] The training set input submodule is used to obtain historical network data, generate a training set based on the historical network data, initialize the training set after data preprocessing, and use the preprocessed training set as a training set input sample of the BP neural network;

[0021] The initialization parameter submodule is used to obtain the number of hidden layer nodes of the BP neural network and use the PSO algorithm to optimize the initial parameters of the BP neural network corresponding to each prediction indicator;

[0022] The parameter optimization sub-submodule is used to calculate the error of the BP neural network, take the error of the BP neural network as the input of the particle update process, optimize by setting the factor parameters of PSO, and assign the optimal solution to the network, wherein the factor parameters at least include the number of iterations, weights and learning factors;

[0023] The test and evaluation submodule is used to train and predict the network using gradient descent as the training algorithm, after setting the training parameters, evaluate the performance of the algorithm through the error of the prediction results, and perform test and evaluation to obtain the network early warning model;

[0024] The network prediction submodule is used to input the network feature data into the network early warning model, predict the network status, and output the network prediction result.

[0025] Optionally, in a fourth implementation of the first aspect of the present invention, the parameter optimization sub-submodule includes a setting unit, a sorting unit, an updating unit, a comparison unit, an invariant unit and an iteration unit, wherein:

[0026] The setting unit is used to initialize parameters, input the initial parameters of the BP neural network and set the initial parameters of the WSPSO algorithm, and use the Tent chaotic map to initialize the particle population;

[0027] The sorting unit is used to calculate the fitness value of each particle and sort them by size;

[0028] The updating unit is used to apply the WSPSO algorithm to calculate the cost function, update all particle speeds and position values, and then calculate the objective function value, find the optimal position and the optimal fitness value of the particle, and compare the mutation rate with the random number in [0, 1] to determine the optimal position and the optimal objective function value;

[0029] The iteration unit is used to determine whether the iteration termination conditions are met. If so, the loop is exited and the optimal solution is output. If not, the iteration continues until the conditions are met and the optimal solution is output.

[0030] Optionally, in a fifth implementation of the first aspect of the present invention, the updating unit includes a comparing unit and an invariant unit, wherein:

[0031] The comparison unit is used to perform polynomial mutation on the particles at the optimal position when the mutation rate is greater than a random number in [0, 1], calculate the function value of the mutated particles, compare and obtain the optimal position and the best objective function value;

[0032] The invariant unit is used to keep the optimal position and the optimal objective function value of the particle population without mutation when the mutation rate is less than or equal to the random number in [0, 1].

[0033] Optionally, in a sixth implementation of the first aspect of the present invention, the indicator comparison module includes a weight assignment submodule, a classification training submodule and an iterative training submodule, wherein:

[0034] The weight assignment submodule is used to obtain the assessment information in multiple assessment schemes of the network status and assign the same weight to each assessment information;

[0035] The classification training submodule is used to train the weak classifier according to the weighted assessment information. When a training sample can be classified by the weak classifier, the weight of the training sample is reduced when constructing the next assessment data set. If a training sample is misclassified, the weight of the training sample is increased.

[0036] The iterative training submodule is used to train the next weak classifier with the assessment data set whose weights have been updated. The whole process is iterative, and the classifiers learned based on each assessment data set are combined into a strong classifier. The network security indicators corresponding to each assessment information are obtained according to the strong classifier.

[0037] Optionally, in a seventh implementation of the first aspect of the present invention, the method for operating the intelligent early warning system based on machine learning comprises the following steps:

[0038] Collecting log information, obtaining initial network data from the log information, and preprocessing the initial network data to obtain network data to be analyzed;

[0039] Extracting data feature information from the network data to be analyzed to obtain network feature data;

[0040] Inputting the network feature data into a pre-trained network early warning model to predict the network status through the network early warning model and outputting the network prediction result;

[0041] Determine a network prediction index of the current network based on the network prediction result, compare the network prediction index with a preset network security index, and when the network prediction index is greater than the network security index, determine that the network state is abnormal, and generate warning information;

[0042] The warning information is displayed visually, and a warning is issued to the monitoring personnel in the form of a signal voice.

[0043] Optionally, in an eighth implementation manner of the first aspect of the present invention, collecting log information, obtaining initial network data from the log information, and preprocessing the initial network data to obtain the network data to be analyzed includes:

[0044] Obtaining a network traffic packet of the initial network data, segmenting and extracting the hexadecimal text captured in the network traffic packet in combination with the protocol and the structure and field of the data packet and datagram to obtain a data set to be processed;

[0045] Using mean interpolation method to process missing values ​​in the data set to be processed to obtain a first data set;

[0046] Use the median absolute difference method to detect outliers on the first data set, and judge the reliability of the flow of the detected outliers. If it is reliable, the current outlier is regarded as a normal value; if it is unreliable, the current outlier is processed as a missing value;

[0047] After the data outliers are processed, a second data set is obtained, and the data in the second data set is normalized using the min-max method to obtain the network data to be analyzed.

[0048] Optionally, in a ninth implementation of the first aspect of the present invention, the inputting the network feature data into a pre-trained network early warning model to predict the network status through the network early warning model and outputting the network prediction result includes:

[0049] Acquire historical network data, generate a training set based on the historical network data, initialize the training set after data preprocessing, and use the preprocessed training set as a training set input sample of the BP neural network;

[0050] Obtain the number of hidden layer nodes of the BP neural network, and use the PSO algorithm to optimize the initial parameters of the BP neural network corresponding to each prediction indicator;

[0051] Calculate the error of the BP neural network, use the error of the BP neural network as the input of the particle update process, optimize by setting the factor parameters of PSO, and assign the optimal solution to the network, wherein the factor parameters at least include the number of iterations, weights, and learning factors;

[0052] Using gradient descent as the training algorithm, after setting the training parameters, the network is trained and predicted. The performance of the algorithm is evaluated by the error of the prediction results, and the network early warning model is obtained by testing and evaluation.

[0053] The network characteristic data is input into the network early warning model, the network status is predicted, and the network prediction result is output.

[0054] In the technical solution provided by the present invention, the system includes a preprocessing module, a feature extraction module, an indicator prediction module, an indicator comparison module and a visualization display module. By collecting log information, initial network data is obtained from the log information, and the initial network data is preprocessed to obtain the network data to be analyzed; data feature information is extracted from the network data to be analyzed to obtain network feature data; the network feature data is input into a pre-trained network early warning model to predict the network status through the network early warning model, and the network prediction result is output; based on the network prediction result, the network prediction index of the current network is determined, and the network prediction index is compared with the preset network security index. When the network prediction index is greater than the network security index, the network status is determined to be abnormal, and a warning information is generated; the warning information is visualized, and a warning is issued to the monitoring personnel in the form of a signal voice; the present invention accurately warns and visualizes the network data, thereby enhancing the security of the network data, which is of great significance to the development of network security technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0056] Figure 1 A schematic diagram of the structure of an intelligent early warning system based on machine learning provided by an embodiment of the present invention;

[0057] Figure 2 Another structural schematic diagram of an intelligent early warning system based on machine learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 A schematic diagram of the structure of an intelligent early warning system based on machine learning provided by an embodiment of the present invention, the system includes a preprocessing module, a feature extraction module, an indicator prediction module, an indicator comparison module and a visualization display module, wherein:

[0060] The preprocessing module 101 is used to collect log information, obtain initial network data from the log information, and preprocess the initial network data to obtain network data to be analyzed;

[0061] The feature extraction module 102 is used to extract data feature information from the network data to be analyzed to obtain network feature data;

[0062] The indicator prediction module 103 is used to input the network feature data into the pre-trained network early warning model to predict the network status through the network early warning model and output the network prediction result;

[0063] The indicator comparison module 104 is used to determine the network prediction indicator of the current network based on the network prediction result, compare the network prediction indicator with the preset network security indicator, and when the network prediction indicator is greater than the network security indicator, determine that the network state is abnormal and generate warning information;

[0064] The visualization display module 105 is used to visualize the warning information and issue a warning to the monitoring personnel in the form of signal voice.

[0065] In this embodiment, the operation method of the system includes the following steps: collecting log information, obtaining initial network data from the log information, preprocessing the initial network data, and obtaining network data to be analyzed; extracting data feature information of the network data to be analyzed to obtain network feature data; inputting the network feature data into a pre-trained network early warning model to predict the network status through the network early warning model, and outputting the network prediction result; determining the network prediction index of the current network based on the network prediction result, comparing the network prediction index with the preset network security index, and when the network prediction index is greater than the network security index, determining that the network status is abnormal, and generating warning information; visually displaying the warning information, and issuing a warning to the monitoring personnel in the form of signal voice.

[0066] In this embodiment, a network traffic packet of initial network data is obtained, and the hexadecimal text captured in the network traffic packet is combined with the protocol and the structure and field of the data packet and datagram to be segmented and extracted to obtain a data set to be processed; the mean interpolation method is used to process missing values ​​of the data in the data set to be processed to obtain a first data set; the absolute value difference median method is used to detect outliers in the first data set, and the traffic reliability of the detected outliers is judged. If it is reliable, the current outlier is used as a normal value; if it is unreliable, the current outlier is processed as a missing value; after the data outlier processing, a second data set is obtained, and the data in the second data set is normalized using the min-max method to obtain the network data to be analyzed.

[0067] In this embodiment, historical network data is obtained, and a training set is generated based on the historical network data, the training set after data preprocessing is initialized, and the preprocessed training set is used as a training set input sample of the BP neural network; the number of hidden layer nodes of the BP neural network is obtained, and the PSO algorithm is used to optimize the initial parameters of the BP neural network corresponding to each prediction indicator; the error of the BP neural network is calculated, and the error of the BP neural network is used as the input of the particle update process, and the network is optimized by setting the factor parameters of the PSO, and the optimal solution is assigned to the network, wherein the factor parameters include at least the number of iterations, weights, and learning factors; gradient descent is used as the training algorithm, and after setting the training parameters, the network is trained and predicted, the performance of the algorithm is evaluated by the error of the prediction result, and the test and evaluation are performed to obtain a network early warning model; the network feature data is input into the network early warning model, the network status is predicted, and the network prediction result is output.

[0068] In an embodiment of the present invention, by collecting log information, initial network data is obtained from the log information, and the initial network data is preprocessed to obtain network data to be analyzed; data feature information is extracted from the network data to be analyzed to obtain network feature data; the network feature data is input into a pre-trained network early warning model to predict the network status through the network early warning model, and a network prediction result is output; a network prediction index of the current network is determined based on the network prediction result, and the network prediction index is compared with a preset network security index. When the network prediction index is greater than the network security index, the network status is determined to be abnormal, and a warning information is generated; the warning information is visualized, and a warning is issued to the monitoring personnel in the form of a signal voice; the present invention accurately warns and visualizes the network data, thereby enhancing the security of the network data, which is of great significance to the development of network security technology.

[0069] See also Figure 2 Another structural diagram of an intelligent early warning system based on machine learning in an embodiment of the present invention includes:

[0070] The intelligent early warning system based on machine learning includes a preprocessing module 101, a feature extraction module 102, an indicator prediction module 103, an indicator comparison module 104 and a visualization display module 105;

[0071] In this embodiment, the preprocessing module 101 includes a data segmentation processing submodule, a data missing value processing submodule, a data abnormal value processing submodule and a data normalization processing submodule, wherein:

[0072] The data segmentation processing submodule 1011 is used to obtain the network traffic packet of the initial network data, and to segment and extract the hexadecimal text captured in the network traffic packet in combination with the protocol and the structure and fields of the data packet and datagram to obtain a data set to be processed;

[0073] A data missing value processing submodule 1012 is used to process missing values ​​of data in the processing data set by using a mean interpolation method to obtain a first data set;

[0074] The data outlier processing submodule 1013 is used to perform outlier detection on the first data set using the median absolute difference method, and judge the flow reliability of the detected outlier. If it is reliable, the current outlier is regarded as a normal value; if it is unreliable, the current outlier is processed as a missing value;

[0075] The data normalization processing submodule 1014 is used to obtain a second data set after processing the data outliers, and to perform data normalization processing on the data in the second data set using the min-max method to obtain the network data to be analyzed.

[0076] In this embodiment, the feature extraction module 102 includes a decision tree generation submodule, a noise addition submodule and an importance assessment submodule, wherein:

[0077] The decision tree generation submodule 1021 is used to initialize the decision tree, use the Bootstrap resampling method to generate a training data set and an out-of-bag data set from the network data to be analyzed, and generate a decision tree on the training data set;

[0078] A noise adding submodule 1022 is used to predict the out-of-bag data based on the generated decision tree, add noise to the feature values ​​in the out-of-bag data, and obtain a new out-of-bag data set;

[0079] The importance evaluation submodule 1023 is used to use the generated decision tree to predict the new out-of-bag data set, count the number of correctly predicted data, repeatedly iterate to evaluate the feature importance of the network data to be analyzed, and obtain network feature data.

[0080] In this embodiment, if the accuracy of the out-of-bag data set is greatly reduced after noise is added to the eigenvalue, it means that the eigenvalue plays a great role in classification, indicating that the importance of the eigenvalue is very high. Conversely, if the accuracy of the out-of-bag data is almost the same as the accuracy of the original out-of-bag data after noise is added to the eigenvalue, it means that the eigenvalue plays a small role in classification, indicating that the importance of the eigenvalue in the data set is very low.

[0081] In this embodiment, the indicator prediction module 103 includes a training set input submodule, an initialization parameter submodule, a parameter optimization submodule, a test evaluation submodule and a network prediction submodule, wherein:

[0082] The training set input submodule 1031 is used to obtain historical network data, generate a training set according to the historical network data, initialize the training set after data preprocessing, and use the preprocessed training set as a training set input sample of the BP neural network;

[0083] Initialization parameter submodule 1032, used to obtain the number of hidden layer nodes of the BP neural network, and optimize the initial parameters of the BP neural network corresponding to each prediction indicator using the PSO algorithm;

[0084] The parameter optimization sub-submodule 1033 is used to calculate the error of the BP neural network, use the error of the BP neural network as the input of the particle update process, optimize by setting the factor parameters of PSO, and assign the optimal solution to the network, wherein the factor parameters at least include the number of iterations, weights, and learning factors;

[0085] The test and evaluation submodule 1034 is used to train and predict the network using gradient descent as the training algorithm, after setting the training parameters, evaluate the performance of the algorithm through the error of the prediction result, and perform test and evaluation to obtain a network early warning model;

[0086] The network prediction submodule 1035 is used to input the network feature data into the network early warning model, predict the network status, and output the network prediction result.

[0087] In this embodiment, the parameter optimization sub-submodule includes a setting unit, a sorting unit, an updating unit, a comparison unit, an invariant unit and an iteration unit, wherein:

[0088] The setting unit is used to initialize parameters, input the initial parameters of the BP neural network and set the initial parameters of the WSPSO algorithm, and use the Tent chaotic map to initialize the particle population;

[0089] The sorting unit is used to calculate the fitness value of each particle and sort them by size;

[0090] The updating unit is used to apply the WSPSO algorithm to calculate the cost function, update all particle speeds and position values, and then calculate the objective function value, find the optimal position and the optimal fitness value of the particle, and compare the mutation rate with the random number in [0, 1] to determine the optimal position and the optimal objective function value;

[0091] The iteration unit is used to determine whether the iteration termination conditions are met. If so, the loop is exited and the optimal solution is output. If not, the iteration continues until the conditions are met and the optimal solution is output.

[0092] In this embodiment, by introducing the dynamic nonlinear time factor K t Improve the search range, and in the later stage with K t The local search capability is enhanced.

[0093]

[0094] K t =k0+cos(πk / 2K)

[0095] Where: μ is the shrinkage factor; is the velocity of particle i at the kth iteration; is the position of particle i at the kth iteration; is the local optimal position of particle i at the kth iteration; is the global optimal position of particle i at the kth iteration; r1 and r2 are two uniformly distributed random numbers in the range [0, 1]; K is the maximum number of iterations; k0 is a random number in the range [0.5, 1]; i = 1, 2, 3, ..., n.

[0096]

[0097] Where: τ is the acceleration coefficient, which controls the exploration behavior of the algorithm.

[0098]

[0099] Where: p min , p max They are the initial velocity and subordinate velocity for realizing particle motion respectively; p1 and p2 are the forces between particles, p1 and p2 control the velocity update of particles and make a certain balance in the global and local search of the algorithm, and p1 and p2 are variable during the iteration process.

[0100] In this embodiment, the updating unit includes a comparing unit and an invariant unit, wherein:

[0101] The comparison unit is used to perform polynomial mutation on the particles at the optimal position when the mutation rate is greater than a random number in [0, 1], calculate the function value of the mutated particles, compare and obtain the optimal position and the best objective function value;

[0102] The invariant unit is used to keep the optimal position and the optimal objective function value of the particle population without mutation when the mutation rate is less than or equal to the random number in [0, 1].

[0103] In this embodiment, the indicator comparison module 104 includes a weight assignment submodule, a classification training submodule and an iterative training submodule, wherein:

[0104] The weight assignment submodule 1041 is used to obtain the assessment information in multiple assessment schemes of the network status and assign the same weight to each assessment information;

[0105] The classification training submodule 1042 is used to train the weak classifier according to the weighted assessment information. When a training sample can be classified by the weak classifier, the weight of the training sample is reduced when constructing the next assessment data set. If a training sample is misclassified, the weight of the training sample is increased.

[0106] The iterative training submodule 1043 is used to train the next weak classifier using the assessment data set with updated weights. The whole process is iterative, and the classifiers learned based on each assessment data set are combined into a strong classifier. The network security indicators corresponding to each assessment information are obtained based on the strong classifier.

[0107] Through the implementation of the above scheme, the system includes a preprocessing module, a feature extraction module, an indicator prediction module, an indicator comparison module and a visualization display module; the present invention accurately warns and visualizes network data, thereby enhancing the security of network data, which is of great significance to the development of network security technology.

[0108] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An intelligent early warning system based on machine learning, characterized in that: The intelligent early warning system based on machine learning includes a preprocessing module, a feature extraction module, an indicator prediction module, an indicator comparison module and a visualization display module, wherein: A preprocessing module, used to collect log information, obtain initial network data from the log information, and preprocess the initial network data to obtain network data to be analyzed; A feature extraction module is used to extract data feature information from the network data to be analyzed to obtain network feature data; An indicator prediction module, used for inputting the network feature data into a pre-trained network early warning model, so as to predict the network status through the network early warning model and output the network prediction result; An indicator comparison module, used to determine a network prediction indicator of the current network based on the network prediction result, compare the network prediction indicator with a preset network security indicator, and when the network prediction indicator is greater than the network security indicator, determine that the network state is abnormal and generate warning information; The visual display module is used to visually display the warning information and issue a warning to the monitoring personnel in the form of signal voice.

2. The intelligent early warning system based on machine learning as claimed in claim 1, characterized in that: The preprocessing module includes a data segmentation processing submodule, a data missing value processing submodule, a data abnormal value processing submodule and a data normalization processing submodule, wherein: The data segmentation processing submodule is used to obtain the network traffic packet of the initial network data, and to segment and extract the hexadecimal text captured in the network traffic packet in combination with the protocol and the structure and field of the data packet and datagram to obtain a data set to be processed; A data missing value processing submodule is used to process missing values ​​of the data in the to-be-processed data set by adopting a mean interpolation method to obtain a first data set; A data outlier processing submodule, used to perform outlier detection on the first data set using the median absolute difference method, and judge the flow reliability of the detected outlier. If it is reliable, the current outlier is regarded as a normal value; if it is unreliable, the current outlier is processed as a missing value; The data normalization processing submodule is used to obtain a second data set after processing the data outliers, and to perform data normalization processing on the data in the second data set using the min-max method to obtain the network data to be analyzed.

3. The intelligent early warning system based on machine learning as claimed in claim 1, characterized in that: The feature extraction module includes a decision tree generation submodule, a noise addition submodule and an importance assessment submodule, wherein: A decision tree generation submodule is used to initialize a decision tree, generate a training data set and an out-of-bag data set from the network data to be analyzed using a Bootstrap resampling method, and generate a decision tree on the training data set; Add a noise submodule to predict the out-of-bag data based on the generated decision tree, add noise to the feature values ​​in the out-of-bag data, and obtain a new out-of-bag data set; The importance evaluation submodule is used to use the generated decision tree to predict the new out-of-bag data set, count the number of correctly predicted data, repeatedly iterate to evaluate the feature importance of the network data to be analyzed, and obtain network feature data.

4. The intelligent early warning system based on machine learning as claimed in claim 1, characterized in that: The indicator prediction module includes a training set input submodule, an initialization parameter submodule, a parameter optimization submodule, a test evaluation submodule and a network prediction submodule, wherein: The training set input submodule is used to obtain historical network data, generate a training set based on the historical network data, initialize the training set after data preprocessing, and use the preprocessed training set as a training set input sample of the BP neural network; The initialization parameter submodule is used to obtain the number of hidden layer nodes of the BP neural network and use the PSO algorithm to optimize the initial parameters of the BP neural network corresponding to each prediction indicator; The parameter optimization sub-submodule is used to calculate the error of the BP neural network, take the error of the BP neural network as the input of the particle update process, optimize by setting the factor parameters of PSO, and assign the optimal solution to the network, wherein the factor parameters at least include the number of iterations, weights and learning factors; The test and evaluation submodule is used to train and predict the network using gradient descent as the training algorithm, after setting the training parameters, evaluate the performance of the algorithm through the error of the prediction results, and perform test and evaluation to obtain the network early warning model; The network prediction submodule is used to input the network feature data into the network early warning model, predict the network status, and output the network prediction result.

5. The intelligent early warning system based on machine learning as claimed in claim 4, characterized in that: The parameter optimization sub-submodule includes a setting unit, a sorting unit, an updating unit, a comparison unit, an invariant unit and an iteration unit, wherein: The setting unit is used to initialize parameters, input the initial parameters of the BP neural network and set the initial parameters of the WSPSO algorithm, and use the Tent chaotic map to initialize the particle population; The sorting unit is used to calculate the fitness value of each particle and sort them by size; The updating unit is used to apply the WSPSO algorithm to calculate the cost function, update all particle speeds and position values, and then calculate the objective function value, find the optimal position and the optimal fitness value of the particle, and compare the mutation rate with the random number in [0, 1] to determine the optimal position and the optimal objective function value; The iteration unit is used to determine whether the iteration termination conditions are met. If so, the loop is exited and the optimal solution is output. If not, the iteration continues until the conditions are met and the optimal solution is output.

6. The intelligent early warning system based on machine learning as claimed in claim 5, characterized in that: The updating unit includes a comparison unit and an invariant unit, wherein: The comparison unit is used to perform polynomial mutation on the particles at the optimal position when the mutation rate is greater than a random number in [0, 1], calculate the function value of the mutated particles, compare and obtain the optimal position and the best objective function value; The invariant unit is used to keep the optimal position and the optimal objective function value of the particle population without mutation when the mutation rate is less than or equal to the random number in [0, 1].

7. The intelligent early warning system based on machine learning as claimed in claim 1, characterized in that: The indicator comparison module includes a weight assignment submodule, a classification training submodule and an iterative training submodule, wherein: The weight assignment submodule is used to obtain the assessment information in multiple assessment schemes of the network status and assign the same weight to each assessment information; The classification training submodule is used to train the weak classifier according to the weighted assessment information. When a training sample can be classified by the weak classifier, the weight of the training sample is reduced when constructing the next assessment data set. If a training sample is misclassified, the weight of the training sample is increased. The iterative training submodule is used to train the next weak classifier with the assessment data set whose weights have been updated. The whole process is iterative, and the classifiers learned based on each assessment data set are combined into a strong classifier. The network security indicators corresponding to each assessment information are obtained according to the strong classifier.

8. The intelligent early warning system based on machine learning as claimed in claim 1, characterized in that: The operation method of the intelligent early warning system based on machine learning comprises the following steps: Collecting log information, obtaining initial network data from the log information, and preprocessing the initial network data to obtain network data to be analyzed; Extracting data feature information from the network data to be analyzed to obtain network feature data; Inputting the network feature data into a pre-trained network early warning model to predict the network status through the network early warning model and outputting the network prediction result; Determine a network prediction index of the current network based on the network prediction result, compare the network prediction index with a preset network security index, and when the network prediction index is greater than the network security index, determine that the network state is abnormal, and generate warning information; The warning information is displayed visually, and a warning is issued to the monitoring personnel in the form of a signal voice.

9. The intelligent early warning system based on machine learning as claimed in claim 1, characterized in that: The collecting log information, obtaining initial network data from the log information, and preprocessing the initial network data to obtain the network data to be analyzed include: Obtaining a network traffic packet of the initial network data, segmenting and extracting the hexadecimal text captured in the network traffic packet in combination with the protocol and the structure and field of the data packet and datagram to obtain a data set to be processed; Using mean interpolation method to process missing values ​​in the data set to be processed to obtain a first data set; Use the median absolute difference method to detect outliers on the first data set, and judge the reliability of the flow of the detected outliers. If it is reliable, the current outlier is regarded as a normal value; if it is unreliable, the current outlier is processed as a missing value; After the data outliers are processed, a second data set is obtained, and the data in the second data set is normalized using the min-max method to obtain the network data to be analyzed.

10. The intelligent early warning system based on machine learning according to claim 1, characterized in that: The step of inputting the network feature data into a pre-trained network early warning model to predict the network status through the network early warning model and outputting a network prediction result includes: Acquire historical network data, generate a training set based on the historical network data, initialize the training set after data preprocessing, and use the preprocessed training set as a training set input sample of the BP neural network; Obtain the number of hidden layer nodes of the BP neural network, and use the PSO algorithm to optimize the initial parameters of the BP neural network corresponding to each prediction indicator; Calculate the error of the BP neural network, use the error of the BP neural network as the input of the particle update process, optimize by setting the factor parameters of PSO, and assign the optimal solution to the network, wherein the factor parameters at least include the number of iterations, weights, and learning factors; Using gradient descent as the training algorithm, after setting the training parameters, the network is trained and predicted. The performance of the algorithm is evaluated by the error of the prediction results, and the network early warning model is obtained by testing and evaluation. The network characteristic data is input into the network early warning model, the network status is predicted, and the network prediction result is output.

Citation Information

Patent Citations

  • Data filter processing method based on LTE (Long Term Evolution) signaling

    CN105306296A

  • Internet financial credit evaluation method based on PSO-BP neural network

    CN112037012A

  • Construction method of model for reducing octane number loss

    CN112687349A

  • Gearbox fault diagnosis method and device based on improved PSO-BP neural network

    CN113188794A

  • Battery management system fault prediction method and device

    CN114527389A

Cited By

  • Market supervision dynamic monitoring and intelligent risk early warning system and method based on big data

    CN120013234A