A network security early warning method and system
Through multi-level data analysis and neural network evaluation model, the problem of inefficient identification of threats in complex network environments is solved, the accuracy and real-time nature of network security warnings are improved, and the security protection capabilities of the data platform are enhanced.
Patent Information
- Application Number
- CN202410714990.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-04
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-06-04
AI Technical Summary
Existing network security early warning technologies are difficult to identify unknown threats in a timely manner when dealing with complex network environments, and are inefficient, with high false alarm rates and missed response rates. They lack effective comprehensive analysis methods, resulting in being unable to handle modern complex network security threats.
By acquiring and preprocessing the data transmission and storage of data information on the data platform, performing multi-level data analysis and clustering, combining security assessment models and neural networks for error assessment and model adjustment, improving the identification ability and early warning accuracy of potential threats.
It significantly improves the accuracy, real-time and comprehensiveness of network security warnings, enhances the security protection capabilities of the data platform, and can more effectively identify and deal with potential threats in complex network environments.
Smart Images

Figure CN118646569B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network security early warning, and in particular to a network security early warning method and system. Background Art
[0002] At present, with the rapid development and widespread application of information technology, data platforms are playing an increasingly important role in all walks of life. However, the security issues of data platforms are also becoming increasingly prominent, especially in the process of data transmission and storage, which are vulnerable to various network attacks and data leakage threats. Existing network security early warning technologies mainly rely on traditional intrusion detection systems and firewalls, which usually identify potential security threats through rule matching and feature extraction. However, these methods have certain limitations, such as the inability to detect unknown threats in a timely manner, insufficient ability to analyze multi-level data in complex network environments, and high false alarm and missed alarm rates. When processing massive data, existing technologies often face challenges such as low efficiency, difficulty in real-time warning, and lack of effective comprehensive analysis methods, which makes them incapable of dealing with modern complex network security threats.
[0003] There is a need for an improved network security early warning method that improves the ability to identify potential threats in complex network environments through multi-level and multi-category data processing and analysis, effectively solving the shortcomings of the existing technology. Summary of the invention
[0004] The purpose of the present invention is to provide a network security early warning method and system to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0005] On the one hand, the present application provides a network security early warning method, comprising:
[0006] Obtain the transmission data information, storage data information and environmental parameter information of the data platform corresponding to each piece of data information;
[0007] Preprocessing the transmission data information of the data platform to obtain transmission data information of at least two levels;
[0008] Performing cluster analysis on the stored data information to obtain at least two categories of stored data information;
[0009] Sending all levels of transmission data information and all categories of storage data information to a preset security assessment model for security factor assessment to obtain the security factor of each level of transmission data and each category of storage data;
[0010] Comprehensively analyze all safety factors and preset historical safety events to determine the combination of factors affecting the safety factors of all data, and send the environmental parameter information of the data platform corresponding to each data information and the combination of factors affecting the data to the safety warning model for processing to obtain safety warning information for each data information;
[0011] The security warning information of each data information is evaluated for errors, the preset security warning model is adjusted based on the error evaluation result, and the corresponding final security warning information of each data information is generated based on the adjusted security warning model.
[0012] On the other hand, the present application also provides a network security early warning system, including:
[0013] An acquisition unit, used to acquire transmission data information, storage data information of the data platform and environmental parameter information of the data platform corresponding to each piece of data information;
[0014] A first processing unit, configured to perform data preprocessing on the transmission data information of the data platform to obtain transmission data information of at least two levels;
[0015] An analysis unit, configured to perform cluster analysis on the stored data information to obtain at least two categories of stored data information;
[0016] An evaluation unit, used to send the transmission data information of all levels and the storage data information of all categories to a preset security evaluation model for security factor evaluation, and obtain the security factor of the transmission data of each level and the storage data of each category;
[0017] The second processing unit is used to perform a comprehensive analysis based on all safety factors and preset historical safety events, determine the combination of factors affecting the safety factors of all data, and send the environmental parameter information of the data platform corresponding to each data information and the combination of factors affecting the data to the safety warning model for processing to obtain safety warning information for each data information;
[0018] The third processing unit is used to perform error evaluation on the security warning information of each data information, adjust the preset security warning model based on the error evaluation result, and generate the corresponding final security warning information of each data information based on the adjusted security warning model.
[0019] The beneficial effects of the present invention are:
[0020] The present invention enhances the ability to identify potential threats in complex network environments through multi-level data preprocessing and cluster analysis, and solves the deficiencies in the prior art. The method improves the accuracy and effectiveness of data processing by removing data noise and redundancy; uses clustering algorithms and the Laida criterion to distinguish between normal and abnormal storage data, thereby improving the accuracy of anomaly detection; combines the Pearson correlation coefficient and the logistic regression model for security assessment, thereby improving the accuracy of the security assessment model; performs dynamic correlation analysis through the spatiotemporal feature matrix and the vector autoregression model, optimizes security influencing factors, and improves the pertinence and effectiveness of early warning; uses BP neural networks and LSTM neural networks for error assessment and model adjustment, thereby enhancing the accuracy and real-time nature of early warning. Overall, the present invention significantly improves the accuracy, real-time nature, and comprehensiveness of network security early warnings, and effectively enhances the security protection capabilities of the data platform.
[0021] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 A schematic diagram of the process flow of the network security early warning method described in an embodiment of the present invention;
[0024] Figure 2 This is a topological diagram of the network security early warning system described in an embodiment of the present invention.
[0025] In the figure: 701, acquisition unit; 702, first processing unit; 703, analysis unit; 704, evaluation unit; 705, second processing unit; 706, third processing unit. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0027] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0028] Embodiment 1:
[0029] This embodiment provides a network security early warning method. Figure 1 , the figure shows that the method includes step S1, step S2, step S3, step S4, step S5 and step S6:
[0030] Step S1, obtaining the transmission data information, storage data information and environmental parameter information of the data platform corresponding to each piece of data information;
[0031] In this step, obtaining the data platform's transmission data information, storage data information, and the environmental parameter information of the data platform corresponding to each data information is the starting step of the network security early warning method. In actual operation, this step involves collecting and recording various key information of the data platform for subsequent data processing and analysis. Transmission data information includes various key parameters in the data transmission process, such as transmission rate, transmission path, transmission time, etc.; storage data information includes the storage status, storage location, storage time, etc. of the data on the storage medium; and the environmental parameter information of the data platform covers various characteristics and parameters of the environment in which the data platform is located, such as network topology, hardware configuration, software environment, etc.
[0032] Step S2: preprocess the transmission data information of the data platform to obtain at least two levels of transmission data information, so as to better understand the structure and association relationship of the data, help discover hidden rules and potential associations between the data, and thus improve the accuracy and reliability of subsequent data analysis and early warning;
[0033] Specifically, step S2 includes step S21, step S22, step S23, step S24 and step S25:
[0034] Step S21: Clean the transmission data information of the data platform, wherein the data cleaning includes removing missing values and detecting anomalies to obtain cleaned transmission data information;
[0035] Therefore, data cleaning can improve the quality and accuracy of transmitted data information, reduce noise and interference, and ensure the reliability of subsequent data analysis and early warning.
[0036] Step S22, sorting the cleaned transmission data information according to their corresponding transmission time, and obtaining the transmission relationship between each transmission data information according to the sorting order, wherein the transmission relationship is the output data corresponding to its input data;
[0037] This step establishes a transmission relationship diagram between the cleaned transmission data information, so as to better understand the process and characteristics of data transmission, and provide an important basis for subsequent data analysis and security warning. By establishing the transmission relationship, problems and risks in the data transmission process can be identified, and timely measures can be taken to deal with and prevent them, thereby improving the security and reliability of data transmission.
[0038] Step S23, performing pairwise analysis on the transmission relationship between each transmission data information, wherein two transmission relationships are analyzed to determine whether one transmission relationship is a subset of another transmission relationship, if so, the other transmission relationship is deleted, if not, the two transmission relationships are retained to obtain a basic transmission relationship, wherein the basic transmission relationship is a transmission relationship at the highest level;
[0039] Therefore, this step simplifies and optimizes the transmission relationship diagram to extract the highest-level basic transmission relationship, reduce redundant and repeated information, and make the data transmission relationship clearer and easier to understand.
[0040] Step S24: Repeat the analysis of deleted transmission relations in pairs until there is no transmission relation that can be deleted, and obtain at least one hierarchical transmission relation, so as to ensure that the obtained hierarchical transmission relation is more concise and accurate. The transmission relation diagram obtained in this way is more structured and hierarchical, which helps to understand the relationship and rules in the data transmission process.
[0041] Step S25: construct a tree-like transmission network based on the transmission relationships of all levels according to their corresponding transmission data, thereby obtaining transmission data information of at least two levels.
[0042] By converting the transmission relationship into a tree-like transmission network, the transmission data information of at least two levels obtained can be used as the basis for subsequent analysis and early warning, thereby clearly showing the hierarchical relationship of data transmission, which helps to deeply understand the structure and characteristics of the data transmission process.
[0043] Step S3, performing cluster analysis on the stored data information to obtain at least two categories of stored data information;
[0044] This step divides the stored data information into at least two categories through cluster analysis, thereby revealing the inherent relationship and structure between the data and helping to discover potential patterns and anomalies in the data.
[0045] Step S4: Send the transmission data information of all levels and the storage data information of all categories to the preset security assessment model for security factor assessment, and obtain the security factor of each level of transmission data and each category of storage data. This can reveal the potential security risks and hazards that may exist during the transmission and storage of data, and help to take effective security measures in a timely manner.
[0046] And step S5, conduct a comprehensive analysis based on all security factors and preset historical security events, determine the combination of factors affecting the security factors of all data, and send the environmental parameter information of the data platform corresponding to each data information and the combination of factors affecting the data to the security warning model for processing to obtain security warning information for each data information;
[0047] By performing comprehensive analysis and security warning model processing in step S5, the security of the data can be comprehensively evaluated, and potential security risks and threats can be discovered in a timely manner, which helps to protect the security and integrity of the data and reduce the possibility of security incidents.
[0048] Further, the step S5 includes step S51, step S52, step S53 and step S54:
[0049] Step S51, performing data conversion processing on the security coefficients of the transmission data of each level and the storage data of each category to obtain a spatiotemporal feature matrix;
[0050] This step collects the security factors of the transmission data at each level and the storage data of each category. These security factors have been calculated in the previous steps and have been normalized and standardized. Next, the security factors are appropriately converted according to the time attributes and spatial attributes of the data. Time attributes may include transmission time or storage time, and spatial attributes may include transmission path or storage location, etc. Data conversion processing involves operations such as data aggregation, dimensionality reduction, and feature selection to extract the most representative and informative spatiotemporal features. Then, the converted security factors are combined into a spatiotemporal feature matrix. Each row of the spatiotemporal feature matrix represents a data point, and each column represents a feature, which may include spatiotemporal attributes such as transmission time, storage location, and transmission path, as well as the corresponding security factors.
[0051] Step S52: performing time series analysis on the spatiotemporal feature matrix according to a preset vector autoregression model, and obtaining a dynamic correlation pattern by analyzing the category structure of the data information and the dynamic change pattern of the safety coefficient over time;
[0052] The vector autoregression model is a statistical model used to analyze time series data, which can describe the dynamic relationship and change trend between data. The parameters of the model can be trained and adjusted according to historical data. Then, the spatiotemporal feature matrix is used as input, and the preset vector autoregression model is used for time series analysis and processing, which includes analyzing the category structure of the data, and modeling and predicting the dynamic change pattern of the safety factor over time. Therefore, this step uses the vector autoregression model to perform time series analysis and processing, which can reveal the dynamic relationship and change trend between data, provide a deeper and more comprehensive perspective for security analysis, and help to timely discover security risks and take corresponding countermeasures.
[0053] Step S53: performing correlation analysis according to the dynamic correlation mode, and obtaining comprehensive factors affecting the safety coefficient by mining the correlation and mutual influence between preset historical safety events and the safety coefficient;
[0054] The above-mentioned use of dynamic correlation mode to conduct correlation analysis on preset historical security events and security factors includes identifying the correlation between historical security events and security factors and the patterns and laws of mutual influence. Then, based on the results of the correlation analysis, the comprehensive factors affecting the security factor are determined. These factors include network environment, data transmission process, storage facilities, etc., as well as external factors related to security, such as attack behaviors, system vulnerabilities, etc. This step can deeply explore the key factors affecting the security factor through correlation analysis and comprehensive evaluation, provide a more comprehensive and accurate basis for the evaluation and management of data security, and help to timely discover and respond to potential security risks and threats.
[0055] Step S54: Optimizing the comprehensive factors using a genetic algorithm, screening out the combination of factors that have the greatest impact on the change in the safety factor by simulating a natural selection process, and obtaining a combination of factors that influence the change in the safety factor.
[0056] Therefore, by optimizing the factors affecting safety through genetic algorithms, the combination of factors that have the greatest impact on the change of the safety factor can be effectively screened out, which improves the accuracy and reliability of safety analysis and helps to more accurately evaluate and predict the safety of data. Among them, the comprehensive factors are converted into the genetic coding form of the genetic algorithm. Each comprehensive factor can be encoded into a gene sequence, and each gene represents a possible influencing factor. Next, a suitable genetic algorithm optimization framework is designed. This includes selecting appropriate genetic algorithm parameters, such as population size, crossover probability, mutation probability, etc., as well as determining the fitness function and stop conditions. Then, the genetic algorithm is used for optimization. Through continuous iteration, the natural selection process is simulated to screen out the combination of factors that have the greatest impact on the change of the safety factor. In each generation, the fitness of each individual is evaluated according to the fitness function, and new individuals are generated according to operations such as selection, crossover and mutation.
[0057] Step S55, divide the environmental parameter information of the data platform corresponding to the preset historical data information, the combination of influencing factors and the preset historical security warning information into a training set and a verification set, thereby making full use of the historical data information to train and verify the security warning model, improve the generalization ability and prediction accuracy of the model, and help to effectively warn of possible security risks and threats.
[0058] Step S56: sending the training set to the BP neural network model for training, wherein the environmental parameter information of the data platform corresponding to the preset historical data information and the combination of the influencing factors are matched according to a preset matching calculation formula to obtain a matching value between each environmental parameter information and the combination of the influencing factors;
[0059] This step uses the BP neural network model and matching degree calculation formula to comprehensively consider the environmental parameter information and influencing factor combination of the data platform, improve the accuracy and reliability of security warning information, and help to timely discover and respond to potential security threats and risks. Among them, the preset matching degree calculation formula includes:
[0060]
[0061] Where A is the matching value of the environmental parameter information of the data platform corresponding to each preset historical data information and the combination of the influencing factors, f(x i ) represents the number of the i-th parameter in the environmental parameter information, f(y i) represents the number of the i-th influencing factor in the influencing factor combination, n represents the total number of parameters and influencing factors, f 2 (x i ) means f(x i ) squared, f 2 (y i ) represents f(y i ) squared.
[0062] Step S57: All matching values are screened based on the decision tree algorithm, and the historical data information corresponding to the matching values greater than the preset threshold obtained by screening is used as the data information that needs security warning, and security warning information of the corresponding category of the historical data information is constructed;
[0063] Therefore, this step can automatically identify historical data information with higher security risks by screening the matching values using the decision tree algorithm, providing an effective auxiliary means for security warnings, and helping to timely discover and respond to potential security threats and risks. Among them, firstly, all matching values are modeled and trained using the decision tree algorithm. The decision tree is a commonly used supervised learning algorithm that segments the features of the training data and constructs a series of decision rules to achieve data classification or prediction. Secondly, all matching values are predicted or classified based on the trained decision tree model. By taking the matching value as input and using the decision tree model for prediction, the classification result of whether each historical data information needs a security warning can be obtained.
[0064] Step S58: verify the security warning information of the verification set and the corresponding category of historical data information. If the error between the security warning information of the verification set and the corresponding category of historical data information is less than a preset threshold, the environmental parameter information and influencing factor combination of the data platform corresponding to each data information is sent to the trained security warning model for processing to obtain the security warning information corresponding to each data information.
[0065] By verifying the security warning information of the corresponding categories of the validation set and historical data information, the prediction accuracy of the model can be evaluated, and when the accuracy meets the requirements, the effective generation of security warning information can be achieved, which helps to improve the accuracy and credibility of the warning so that corresponding safety protection measures can be taken in time.
[0066] And step S6, performing error evaluation on the security warning information of each data information, adjusting the preset security warning model based on the error evaluation result, and generating the corresponding final security warning information of each data information based on the adjusted security warning model.
[0067] This step can continuously optimize the security warning model through error assessment and model adjustment of security warning information, improve its accurate prediction ability and warning effect of security incidents, thereby effectively responding to various security threats and risks and ensuring the safe operation of the data platform.
[0068] Embodiment 2:
[0069] The difference between this embodiment and embodiment 1 is that step S3 includes step S31, step S32, step S33 and step S34:
[0070] Step S31, clustering the stored data information based on a distance-based clustering algorithm to obtain at least two clusters, wherein the clusters contain at least one category of stored data information, thereby revealing the intrinsic relationship and structure between the data, helping to discover potential patterns and anomalies in the data, and improving the understanding and management level of the data storage status.
[0071] Step S32: all the clusters are processed based on the Laida criterion, wherein a threshold range corresponding to each cluster is calculated, thereby improving the grasp and understanding of data categories and features, and facilitating subsequent data classification and anomaly detection.
[0072] Step S33, analyze all the threshold ranges, and use the minimum threshold range formed by all the threshold ranges as the abnormal threshold range for judging the stored data information; the threshold range can be used as a basis for abnormality detection, which is conducive to timely discovery and processing of abnormal situations in the stored data information.
[0073] Step S34: based on the abnormal threshold range, the stored data information is divided into normal category stored data information and abnormal category stored data information, wherein the normal category stored data information is classified according to the threshold range corresponding to each cluster cluster, and at least two normal categories of stored data information are obtained, thereby achieving the effect of more accurately identifying normal data and abnormal data, and providing strong support for subsequent data analysis and security warnings.
[0074] Embodiment 3:
[0075] The difference between this embodiment and embodiment 1 or 2 is that step S4 includes step S41, step S42, step S43 and step S44:
[0076] Step S41: extract features of all levels of transmission data information and all categories of storage data information to obtain security feature information corresponding to each data;
[0077] Feature extraction is the process of converting raw data into a representative feature vector or feature set. These features can be statistical properties of data, spectral distribution, time series characteristics, etc., aiming to capture the key information and characteristics of the data. Then, for each data information, the corresponding security feature information is extracted. These security feature information include the degree of encryption of the data, access control permissions, data transmission rate, etc., which are important indicators for evaluating data security. After obtaining the security feature information corresponding to each data, it can be used for subsequent security assessment and early warning analysis. These security feature information will help to comprehensively evaluate the security of the data, identify potential security risks and take corresponding security measures. Therefore, this embodiment can convert raw data into representative security feature information through feature extraction, which helps to comprehensively evaluate the security and risk level of the data, and can more accurately identify potential security threats.
[0078] Step S42: Calculate the Pearson correlation coefficient according to the security feature information corresponding to each data, and obtain a correlation value by calculating the linear correlation between the security feature information and the preset historical safety accidents;
[0079] This step quantifies the correlation between safety feature information and historical safety accidents through Pearson correlation coefficient calculation, making it possible to more accurately assess the impact of safety features on safety accidents, providing a more reliable basis for safety assessment and early warning.
[0080] Step S43: Processing the correlation value based on a preset logistic regression model, establishing a quantitative relationship between the safety feature information and historical safety accidents, and obtaining a safety coefficient corresponding to each data;
[0081] Logistic regression is a statistical model used to establish the relationship between independent variables and dependent variables, and is usually used to explain classification problems. In this embodiment, the logistic regression model will be used to establish a quantitative relationship between security feature information and historical safety accidents. Next, the relationship between the correlation value and the historical safety accidents is fitted by the logistic regression model to obtain the degree of influence of each security feature information on the occurrence of historical safety accidents. This process will establish a quantitative relationship between security feature information and historical safety accidents, thereby quantitatively describing the impact of security features on safety accidents. Finally, based on the established quantitative relationship, the safety factor corresponding to each data is calculated. The safety coefficient reflects the risk level of the data relative to historical safety accidents. The higher the value, the higher the security of the data, and vice versa, the greater the risk level. The calculation formula of the safety coefficient is as follows:
[0082]
[0083] Among them, S nIt represents the safety factor corresponding to the nth data, e is the base of the natural logarithm, β0, β1, ....β n are the parameters of the logistic regression model, X1,X2,...,X n It is the corresponding security feature information.
[0084] Step S44: normalize and standardize the security coefficients corresponding to each data according to different categories and different levels to obtain the security coefficients of the transmission data of each level and the storage data of each category.
[0085] Through normalization and standardization, the measurement standards of safety factors of different categories and levels can be unified to make them comparable, providing a more accurate basis for safety assessment and risk management.
[0086] Embodiment 4:
[0087] The difference between this embodiment and any one of embodiments 1-3 is that step S6 includes step S61, step S62, step S63, step S64 and step S65:
[0088] Step S61: marking the transmission data sent to each level and the storage data of each category based on the security warning information of each data information, and marking based on the position corresponding to the security warning information to obtain the position information corresponding to each security warning information;
[0089] By marking the transmitted data and stored data and combining it with location information, the source and scope of impact of security warning information can be accurately identified, which helps to take targeted security measures in a timely manner and effectively prevent and respond to potential security threats and risks.
[0090] Step S62: convert the position information corresponding to each safety warning information into a feature vector to obtain a feature vector set for each marking point;
[0091] By converting location information into feature vectors, safety warning information and other related information can be effectively integrated and represented, thus providing a richer and more comprehensive data foundation for subsequent safety analysis and processing.
[0092] Step S63: input the preset feature vector set of historical mark points and the training deviation value as a training data set into the preset initial network model to obtain a prediction deviation value, obtain a loss function according to the training deviation value and the prediction deviation value, and obtain a trained LSTM neural network model according to the loss function;
[0093] By training the LSTM neural network model, the relationship between the features and deviation values in the historical data can be effectively learned, and the model can be tuned according to the loss function, thereby improving the prediction ability and accuracy of the model, and then more accurately predicting security warning information, which helps to timely discover and respond to potential security threats and risks. The loss function includes:
[0094]
[0095] Among them, Y represents the set of true deviation values, represents the set of prediction deviation values, N represents the number of samples, a b represents the bth true deviation value, Represents the b-th prediction deviation value.
[0096] Step S64, inputting the feature vector set of each marking point into the trained LSTM neural network model to obtain the corresponding position deviation value of each marking point;
[0097] This step uses the trained LSTM neural network model to effectively process and analyze the feature vector of each marking point, thereby predicting the position deviation value of each marking point, which helps to improve the accuracy and reliability of location information, and thus improve the performance and effectiveness of the overall safety warning system.
[0098] Step S65: adjusting the fitness function of the BP neural network in the safety warning model based on the corresponding position deviation value of each marking point until the corresponding position deviation value of each marking point is less than a preset threshold.
[0099] In this step, the position deviation value of each marking point is used as part of the fitness function of the BP neural network. Furthermore, the goal of network training is to reduce the position deviation value of each marking point as much as possible so that it is close to the preset threshold. Then, the parameters of the neural network are adjusted according to the position deviation value of each marking point through the back propagation algorithm. Specifically, the neural network will calculate the position deviation value of each marking point according to the current parameter settings, and adjust the parameters through the gradient descent method so that the position deviation value is gradually reduced. Then, repeat the above steps and continuously optimize the parameters of the neural network until the position deviation value of each marking point is less than the preset threshold. Finally, when the position deviation value of each marking point is less than the preset threshold, it means that the neural network has converged and the training process of the model is completed.
[0100] This step dynamically adjusts the fitness function of the neural network so that the model can be gradually optimized, and the position deviation value of each marking point gradually approaches the preset threshold to improve the accuracy and stability of the model, making the safety warning system more reliable and effective.
[0101] Embodiment 5:
[0102] like Figure 2 As shown, this embodiment provides a network security early warning system, see Figure 2 The system includes an acquisition unit 701 , a first processing unit 702 , an analysis unit 703 , an evaluation unit 704 , a second processing unit 705 , and a third processing unit 706 .
[0103] The acquisition unit 701 is used to acquire the transmission data information, storage data information and environment parameter information of the data platform corresponding to each piece of data information;
[0104] The first processing unit 702 is used to perform data preprocessing on the transmission data information of the data platform to obtain transmission data information of at least two levels;
[0105] An analyzing unit 703 is used to perform cluster analysis on the stored data information to obtain at least two categories of stored data information;
[0106] An evaluation unit 704 is used to send the transmission data information of all levels and the storage data information of all categories to a preset security evaluation model for security coefficient evaluation, and obtain the security coefficient of the transmission data of each level and the storage data of each category;
[0107] The second processing unit 705 is used to perform a comprehensive analysis based on all security factors and preset historical security events, determine the combination of influencing factors on the security factors of all data, and send the environmental parameter information of the data platform corresponding to each data information and the combination of influencing factors to the security warning model for processing to obtain security warning information for each data information;
[0108] The third processing unit 706 is used to perform error evaluation on the security warning information of each data information, adjust the preset security warning model based on the error evaluation result, and generate the corresponding final security warning information of each data information based on the adjusted security warning model.
[0109] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
[0111] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A network security early warning method, It is characterized in that include: Obtain the transmission data information, storage data information and environmental parameter information of the data platform corresponding to each piece of data information; Preprocessing the transmission data information of the data platform to obtain transmission data information of at least two levels; Performing cluster analysis on the stored data information to obtain at least two categories of stored data information; Sending all levels of transmission data information and all categories of storage data information to a preset security assessment model for security factor assessment to obtain the security factor of each level of transmission data and each category of storage data; Comprehensively analyze all safety factors and preset historical safety events to determine the combination of factors affecting the safety factors of all data, and send the environmental parameter information of the data platform corresponding to each data information and the combination of factors affecting the data to the safety warning model for processing to obtain safety warning information for each data information; Performing an error evaluation on the security warning information of each data information, adjusting the preset security warning model based on the error evaluation result, and generating the corresponding final security warning information of each data information based on the adjusted security warning model; Among them, a comprehensive analysis is conducted based on all security factors and preset historical security events to determine the combination of factors affecting the security factor of all data, including: The security coefficients of the transmission data of each level and the storage data of each category are converted into a spatiotemporal feature matrix; Performing time series analysis on the spatiotemporal feature matrix according to a preset vector autoregression model, and obtaining a dynamic correlation pattern by analyzing the category structure of data information and the dynamic change pattern of the safety coefficient over time; Performing correlation analysis according to the dynamic correlation mode, and obtaining comprehensive factors affecting the safety factor by mining the correlation and mutual influence between preset historical safety events and the safety factor; The comprehensive factors are optimized by using a genetic algorithm, and the combination of factors that have the greatest impact on the change of the safety factor is screened out by simulating a natural selection process, thereby obtaining a combination of factors that affect the change of the safety factor; The environmental parameter information and influencing factors of the data platform corresponding to each data information are combined and sent to the security warning model for processing to obtain the security warning information of each data information, including: Divide the environmental parameter information of the data platform corresponding to the preset historical data information, the combination of influencing factors and the preset historical safety warning information into a training set and a verification set; The training set is sent to the BP neural network model for training, wherein the environmental parameter information of the data platform corresponding to the preset historical data information and the combination of the influencing factors are matched according to a preset matching calculation formula to obtain a matching value of each environmental parameter information and the combination of the influencing factors; Based on the decision tree algorithm, all matching values are screened, and the historical data information corresponding to the matching values greater than the preset threshold obtained by screening is used as the data information that needs security warning, and security warning information corresponding to the category of the historical data information is constructed; Verify the security warning information of the verification set and the corresponding category of historical data information. If the error between the security warning information of the verification set and the corresponding category of historical data information is less than a preset threshold, send the environmental parameter information and influencing factor combination of the data platform corresponding to each data information to the trained security warning model for processing to obtain the security warning information corresponding to each data information; The security warning information of each data information is subjected to error evaluation, and the preset security warning model is adjusted based on the error evaluation result, including: Marking the transmission data sent to each level and the storage data of each category based on the security warning information of each data information, and marking based on the position corresponding to the security warning information to obtain the position information corresponding to each security warning information; The location information corresponding to each safety warning information is converted into a feature vector to obtain a feature vector set for each marking point; Inputting a preset feature vector set of historical mark points and a training deviation value as a training data set into a preset initial network model to obtain a prediction deviation value, obtaining a loss function according to the training deviation value and the prediction deviation value, and obtaining a trained LSTM neural network model according to the loss function; Input the feature vector set of each marking point into the trained LSTM neural network model to obtain the corresponding position deviation value of each marking point; The fitness function of the BP neural network in the safety warning model is adjusted based on the corresponding position deviation value of each marking point until the corresponding position deviation value of each marking point is less than a preset threshold.
2. The network security early warning method according to claim 1, Features , performing data preprocessing on the transmission data information of the data platform to obtain at least two levels of transmission data information, including: Cleaning the transmission data information of the data platform, wherein the data cleaning includes removing missing values and detecting anomalies, to obtain cleaned transmission data information; Sorting the cleaned transmission data information according to their corresponding transmission time, and obtaining the transmission relationship between each transmission data information according to the sorting order, wherein the transmission relationship is the output data corresponding to its input data; Perform pairwise analysis on the transmission relationship between each transmission data information, wherein two transmission relationships are analyzed to determine whether one transmission relationship is a subset of another transmission relationship, if so, delete the other transmission relationship, if not, retain both transmission relationships, and obtain a basic transmission relationship, wherein the basic transmission relationship is a transmission relationship at the highest level; Repeat the pairwise analysis of the deleted transmission relations until there are no transmission relations that can be deleted, and obtain at least one hierarchical transmission relation; A tree-like transmission network is constructed based on the transmission relationships of all levels according to their corresponding transmission data, thereby obtaining transmission data information of at least two levels.
3. The network security early warning method according to claim 1, Features , clustering analysis is performed on the stored data information to obtain at least two categories of stored data information, including: Clustering the stored data information based on a distance-based clustering algorithm to obtain at least two clusters, wherein the clusters contain at least one category of stored data information; Processing all the clusters based on the Laida criterion, wherein a threshold range corresponding to each cluster is calculated; Analyze all the threshold ranges, and use the minimum threshold range formed by all the threshold ranges as the abnormal threshold range for determining the stored data information; The storage data information is divided into normal category storage data information and abnormal category storage data information based on the abnormal threshold range, wherein the normal category storage data information is classified according to the threshold range corresponding to each cluster cluster to obtain at least two normal category storage data information.
4. The network security early warning method according to claim 1, Features , the transmission data information of all levels and the storage data information of all categories are sent to the preset security assessment model for security factor assessment, including: Extract features from all levels of transmission data information and all categories of storage data information to obtain security feature information corresponding to each data; Calculate the Pearson correlation coefficient based on the safety feature information corresponding to each data, and obtain the correlation value by calculating the linear correlation between the safety feature information and the preset historical safety accidents; The correlation value is processed based on a preset logistic regression model to establish a quantitative relationship between the safety feature information and historical safety accidents, and obtain a safety coefficient corresponding to each data; The security coefficient corresponding to each data is normalized and standardized according to different categories and levels to obtain the security coefficient of the transmission data at each level and the storage data of each category.
5. The network security early warning method according to claim 1, Features ,The preset matching degree calculation formula includes: ; Wherein, A is the matching value of the combination of the environmental parameter information of the data platform corresponding to each preset historical data information and the influencing factor, Indicates the number of the i-th parameter in the environment parameter information. represents the number of the i-th influencing factor in the influencing factor combination, n represents the total number of parameters and influencing factors, express The square of express The square of .
6. The network security early warning method according to claim 1, Features ,The loss function includes: ; Among them, Y represents the set of true deviation values, represents the set of prediction deviation values, N represents the number of samples, represents the bth true deviation value, Represents the b-th prediction deviation value.
7. A network security early warning system, It is characterized in that include: An acquisition unit, used to acquire transmission data information, storage data information of the data platform and environmental parameter information of the data platform corresponding to each piece of data information; A first processing unit, configured to perform data preprocessing on the transmission data information of the data platform to obtain transmission data information of at least two levels; An analysis unit, configured to perform cluster analysis on the stored data information to obtain at least two categories of stored data information; An evaluation unit, used to send the transmission data information of all levels and the storage data information of all categories to a preset security evaluation model for security factor evaluation, and obtain the security factor of the transmission data of each level and the storage data of each category; The second processing unit is used to perform a comprehensive analysis based on all safety factors and preset historical safety events, determine the combination of factors affecting the safety factors of all data, and send the environmental parameter information of the data platform corresponding to each data information and the combination of factors affecting the data to the safety warning model for processing to obtain safety warning information for each data information; A third processing unit is used to perform error evaluation on the security warning information of each data information, adjust the preset security warning model based on the error evaluation result, and generate corresponding final security warning information of each data information based on the adjusted security warning model; Among them, a comprehensive analysis is conducted based on all security factors and preset historical security events to determine the combination of factors affecting the security factor of all data, including: The security coefficients of the transmission data of each level and the storage data of each category are converted into a spatiotemporal feature matrix; Performing time series analysis on the spatiotemporal feature matrix according to a preset vector autoregression model, and obtaining a dynamic correlation pattern by analyzing the category structure of data information and the dynamic change pattern of the safety coefficient over time; Performing correlation analysis according to the dynamic correlation mode, and obtaining comprehensive factors affecting the safety factor by mining the correlation and mutual influence between preset historical safety events and the safety factor; The comprehensive factors are optimized by using a genetic algorithm, and the combination of factors that have the greatest impact on the change of the safety factor is screened out by simulating a natural selection process, thereby obtaining a combination of factors that affect the change of the safety factor; The environmental parameter information and influencing factors of the data platform corresponding to each data information are combined and sent to the security warning model for processing to obtain the security warning information of each data information, including: Divide the environmental parameter information of the data platform corresponding to the preset historical data information, the combination of influencing factors and the preset historical safety warning information into a training set and a verification set; The training set is sent to the BP neural network model for training, wherein the environmental parameter information of the data platform corresponding to the preset historical data information and the combination of the influencing factors are matched according to a preset matching calculation formula to obtain a matching value of each environmental parameter information and the combination of the influencing factors; Based on the decision tree algorithm, all matching values are screened, and the historical data information corresponding to the matching values greater than the preset threshold obtained by screening is used as the data information that needs security warning, and security warning information corresponding to the category of the historical data information is constructed; Verify the security warning information of the verification set and the corresponding category of historical data information. If the error between the security warning information of the verification set and the corresponding category of historical data information is less than a preset threshold, send the environmental parameter information and influencing factor combination of the data platform corresponding to each data information to the trained security warning model for processing to obtain the security warning information corresponding to each data information; The security warning information of each data information is subjected to error evaluation, and the preset security warning model is adjusted based on the error evaluation result, including: Marking the transmission data sent to each level and the storage data of each category based on the security warning information of each data information, and marking based on the position corresponding to the security warning information to obtain the position information corresponding to each security warning information; The location information corresponding to each safety warning information is converted into a feature vector to obtain a feature vector set for each marking point; Inputting a preset feature vector set of historical mark points and a training deviation value as a training data set into a preset initial network model to obtain a prediction deviation value, obtaining a loss function according to the training deviation value and the prediction deviation value, and obtaining a trained LSTM neural network model according to the loss function; Input the feature vector set of each marking point into the trained LSTM neural network model to obtain the corresponding position deviation value of each marking point; The fitness function of the BP neural network in the safety warning model is adjusted based on the corresponding position deviation value of each marking point until the corresponding position deviation value of each marking point is less than a preset threshold.
