Network security monitoring system and method thereof

Through the security monitoring model trained by deep learning technology, the inefficiency and false alarm problems of traditional network security monitoring systems in the face of complex environments and new threats are solved, and efficient, precise monitoring and rapid response of network security are achieved.

CN118944974AInactive Publication Date: 2024-11-12XUZHOU ZUOGONGYOUJIANG INTELLIGENT TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411296774.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When traditional network security monitoring systems face complex and changing network environments and new threats, their evaluation efficiency is low, the rule base update is lagging, making it difficult to identify unknown threats, and are costly, which is prone to false alarms.

Method used

Using deep learning technology, by obtaining network traffic data, processing and storing, training security monitoring models based on distributed databases, judging whether there are security events in the network data flow in real time, and generating alarm information, improving monitoring efficiency and accuracy, and reducing false alarms and missed reports, quickly update the rule base.

Benefits of technology

It has achieved comprehensive and accurate monitoring of network security, reduced resource consumption and costs, enhanced the ability to identify unknown threats, timely discovered new threats, reduced false alarms and missed reports, and was highly adaptable.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118944974A_ABST
    Figure CN118944974A_ABST
Patent Text Reader

Abstract

The invention discloses a network security monitoring system and method, and belongs to the technical field of network security, and the method comprises the steps: obtaining network flow data, processing the network flow data to obtain distributed data, and storing the distributed data in a distributed database; based on a distributed database, training the distributed data through a deep learning algorithm to obtain a safety monitoring model; collecting a network data flow, judging whether the network data flow has a network security event through the security monitoring model, and outputting a judgment result; classifying judgment results to obtain a normal network data flow and an abnormal network data flow; and generating corresponding alarm information according to the network security event corresponding to the abnormal network data flow. The network security monitoring efficiency can be improved, the resource consumption is reduced, the cost is reduced, the method adapts to a complex and changeable network environment, comprehensive and accurate monitoring of the network security is realized, the unknown threat identification capability is relatively strong, and the false alarm condition can be effectively reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a network security monitoring system and method thereof. Background Art

[0002] With the rapid advancement of information technology, cyberspace has become a cornerstone of global connectivity, driving the comprehensive operation and innovation of economic systems, social structures, and cultural ecosystems. However, this progress is also accompanied by increasingly severe cybersecurity challenges, manifested in the diversification of cyberattack methods, the frequent occurrence of data leaks, and the widespread existence of system security vulnerabilities. These security threats pose unprecedented risks and pressures to personal privacy protection, the stability of corporate operations, and the strategic bottom line of national security. It is urgent to adopt professional technical means and strategies to build a comprehensive, multi-layered cybersecurity protection system.

[0003] In the existing technology, traditional network security monitoring systems, such as rule-based scanning systems and manual penetration testing systems, can reveal and quantify the security situation of network systems to a certain extent, but they will show significant limitations when dealing with increasingly complex and changing network ecosystems and attack strategies. These monitoring systems are highly dependent on manual analysis and matching of preset rules, which leads to low evaluation efficiency. Especially when dealing with large-scale network systems, the overall process is lengthy and it is difficult to achieve comprehensive data coverage. When faced with emerging attack methods and dynamically changing network environments, traditional network security monitoring systems are limited by the lag in updating the rule base and lack of flexibility and adaptability, making it difficult to quickly capture new security threats. At the same time, because they are built based on known vulnerabilities and attack patterns, their accuracy is limited, and their ability to identify unknown threats is weak, which can easily lead to false alarms. In addition, large-scale network security monitoring requires a lot of manpower and computing resources, which is costly.

[0004] Therefore, it is necessary to provide a network security monitoring system and method thereof to solve the above technical problems. Summary of the Invention

[0005] In order to solve the above problems, the present invention proposes a network security monitoring system and method, which can improve the monitoring efficiency of network security through deep learning technology, reduce resource consumption, and lower costs. It can also adapt to complex and changeable network environments to achieve comprehensive and accurate monitoring of network security. At the same time, it has a strong ability to identify unknown threats, which can effectively reduce false alarms and missed alarms. The system rule base is updated quickly and has strong adaptability, so it can detect new threats in a timely manner.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] A network security monitoring system of the present invention comprises:

[0008] An acquisition module, configured to acquire network traffic data, process the network traffic data to obtain distribution data, and store the distribution data in a distributed database;

[0009] A training module, configured to train the distributed data using a deep learning algorithm based on the distributed database to obtain a security monitoring model;

[0010] A judgment module, configured to collect network data streams, determine whether a network security incident exists in the network data streams through the security monitoring model, and output a judgment result;

[0011] A classification module, configured to classify the judgment result to obtain normal network data flow and abnormal network data flow;

[0012] The alarm module is used to generate corresponding alarm information according to the network security event corresponding to the abnormal network data flow.

[0013] A further improvement of the present invention is that the acquisition module includes:

[0014] A monitoring unit, configured to monitor a data communication port and a data communication protocol of the network traffic data through a network sniffing device;

[0015] An extraction unit, configured to extract key data elements of the network traffic data through a data analysis tool;

[0016] a processing unit, configured to process the key data elements according to preset rules to generate the distribution data;

[0017] The uploading storage unit is used to upload and store the distributed data into the distributed database through a data uploading tool.

[0018] A further improvement of the present invention is that the training module includes:

[0019] a preprocessing unit, configured to obtain the distributed data in the distributed database and preprocess the distributed data to obtain target distributed data;

[0020] a marking unit, configured to mark the target distribution data according to historical network attack data to obtain marked distribution data;

[0021] A generating unit, configured to select three different deep learning algorithms for training based on the labeled distribution data to obtain three different distribution models;

[0022] An adjustment unit, configured to adjust the parameter combination of each distribution model to an optimal parameter combination through a cross-validation algorithm;

[0023] A verification unit is used to verify the prediction effect of each distribution model based on the marked distribution data, and determine the optimal distribution model, that is, the safety monitoring model, based on the prediction effect.

[0024] A further improvement of the present invention is that the step of obtaining the distributed data in the distributed database and preprocessing the distributed data to obtain target distributed data specifically includes:

[0025] Obtaining distribution data in the distributed database, removing noise data from the distribution data, and obtaining denoised distribution data;

[0026] Select any field in the denoised distribution data, count the abnormal values ​​in the field, and when the abnormal value exceeds a preset threshold, determine the abnormal data point corresponding to the field, and perform corresponding processing on the abnormal data point to obtain abnormality-processed data;

[0027] Feature selection is performed on the data after the abnormality processing to obtain the target distribution data.

[0028] A further improvement of the present invention is that: selecting any field in the denoised distribution data, counting abnormal values ​​in the field, determining abnormal data points corresponding to the field when the abnormal value exceeds a preset threshold, and performing corresponding processing on the abnormal data points to obtain abnormally processed data, specifically includes:

[0029] Setting a statistical standard value for the field and determining a reasonable value range within the field based on the statistical data distribution of the field;

[0030] For the field, anomaly judgment is performed based on a preset standard deviation coefficient threshold and variance, data points that deviate from the standard deviation by more than one value are marked as abnormal data points, and the abnormal data points are processed accordingly;

[0031] If the proportion of the abnormal data points in the total field exceeds a preset proportion, the field is removed.

[0032] A further improvement of the present invention is that: the label distribution data includes a label distribution training set and a label distribution verification set, three different deep learning algorithms are selected to train the label distribution training set to obtain three different distribution models, and the prediction effect of each distribution model is verified by the label distribution verification set.

[0033] A further improvement of the present invention is that the judgment module includes:

[0034] A collection unit, configured to collect the network data stream;

[0035] A filtering unit, configured to deploy a data flow filter at a data flow entry point of the security monitoring model, and remove network data flows irrelevant to intrusion detection from the network data flow through the data flow filter to obtain a target network data flow;

[0036] The determination unit is used to obtain the characteristic parameters of the target network data flow, calculate the matching probability of the characteristic parameters through the security monitoring model, judge whether there is a network security incident in the target network data flow according to the matching probability, and generate a corresponding judgment result.

[0037] A further improvement of the present invention is that: the characteristic parameters of the target network data flow are obtained, the matching probability of the characteristic parameters is calculated by the security monitoring model, and whether the target network data flow has a network security incident is determined based on the matching probability, and a corresponding judgment result is generated, and the judgment is performed using the following intrusion detection algorithm:

[0038] Assuming the matching probability to be P and the first threshold to be T, calculating the deviation B=|PT| between the matching probability P and the first threshold T;

[0039] If the deviation degree B is less than or equal to the first threshold T, it is determined that there is no network security incident in the target network data flow;

[0040] If the deviation B is greater than the first threshold T and the matching probability P is greater than the first threshold T, it is necessary to determine whether the matching probability P is associated with the first threshold T:

[0041] Set the second threshold to X and calculate the correlation Wherein, P is the matching probability, T is the first threshold and H is the correlation degree. If the correlation degree H is greater than the second threshold X, it is determined that a network security incident exists in the target network data flow.

[0042] A further improvement of the present invention is that the classification module also includes counting the number of abnormal network data flows, the larger the number of abnormal network data flows, the higher the alarm level of the corresponding alarm information, and the categories of network security events corresponding to the abnormal network data flows are different, and the corresponding alarm categories of the alarm information are different.

[0043] A network security monitoring method of the present invention comprises:

[0044] Acquire network traffic data, process the network traffic data to obtain distribution data, and store the distribution data in a distributed database;

[0045] Based on the distributed database, the distributed data is trained by a deep learning algorithm to obtain a security monitoring model;

[0046] Collecting network data streams, determining whether there are network security incidents in the network data streams through the security monitoring model, and outputting the determination result;

[0047] Classifying the judgment results to obtain normal network data flows and abnormal network data flows;

[0048] Generate corresponding alarm information based on the network security event corresponding to the abnormal network data flow.

[0049] The beneficial effects of the present invention are: the present invention proposes a network security monitoring system and method thereof, which can obtain network traffic data through an acquisition module, a training module, a judgment module, a classification module and an alarm module, process the network traffic data to obtain distribution data, and store the distribution data in a distributed database; based on the distributed database, the distribution data is trained by a deep learning algorithm to obtain a security monitoring model; the network data stream is collected, and the network data stream is judged by the security monitoring model whether there is a network security incident, and the judgment result is output; the judgment result is classified to obtain normal network data streams and abnormal network data streams; according to the network security incidents corresponding to the abnormal network data streams, corresponding alarm information is generated, so that the network security monitoring efficiency can be improved through deep learning technology, resource consumption can be reduced, and costs can be reduced, and it can adapt to complex and changeable network environments to achieve comprehensive and accurate monitoring of network security. At the same time, it has a strong ability to identify unknown threats, which can effectively reduce false alarms and missed reports, and the system rule base is updated quickly and has strong adaptability, so new threats can be discovered in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the structure of a network security monitoring system provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the structure of an acquisition module of a network security monitoring system provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the structure of a training module of a network security monitoring system provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of the structure of a judgment module of a network security monitoring system provided by an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of a flow chart of a network security monitoring method provided by an embodiment of the present invention;

[0055] Figure 6 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0057] See also Figure 1 , is a schematic diagram of the structure of a network security monitoring system provided by an embodiment of the present invention, the monitoring system comprising:

[0058] An acquisition module, configured to acquire network traffic data, process the network traffic data to obtain distribution data, and store the distribution data in a distributed database;

[0059] A training module, configured to train the distributed data using a deep learning algorithm based on the distributed database to obtain a security monitoring model;

[0060] A judgment module, configured to collect network data streams, determine whether a network security incident exists in the network data streams through the security monitoring model, and output a judgment result;

[0061] A classification module, configured to classify the judgment result to obtain normal network data flow and abnormal network data flow;

[0062] The alarm module is used to generate corresponding alarm information according to the network security event corresponding to the abnormal network data flow.

[0063] It's understandable that network packet capture technology and traffic collection mechanisms can be used to continuously capture raw traffic data from the network. This data can then be converted into structured distributed data through techniques such as data cleaning, format conversion, and feature extraction. This distributed data can reflect the real-time state of network traffic, including its spatial distribution and temporal evolution. This distributed data can then be efficiently stored in a distributed database. Leveraging the scalability, high availability, and fault tolerance of distributed storage, data integrity and efficient access can be ensured.

[0064] Based on a distributed database and incorporating deep learning algorithms, the training module enables in-depth learning and pattern recognition of distributed data. By constructing a multi-layer neural network model, the module automatically learns the normal behavior patterns of network traffic and the characteristics of potential security threats. Through multiple rounds of iterative training and optimization, it generates a highly accurate security monitoring model. This model can effectively identify known network attacks and has good generalization capabilities for new attack patterns.

[0065] The judgment module collects network data flows in real time and rapidly analyzes them using a trained security monitoring model. By comparing data flow characteristics with the security baseline in the model, the module quickly determines whether potential network security incidents exist within the network data flow and instantly outputs the judgment results. The classification module categorizes the judgment results, generating corresponding normal and abnormal network data flows.

[0066] In practice, the alarm module automatically generates corresponding alarm information based on the output of the classification module. This alarm information includes key information such as the type of security incident, time of occurrence, scope of impact, and recommended emergency response measures. Through pre-defined alarm channels such as email, SMS, and system notifications, the alarm module ensures that the security team receives alarm information immediately, allowing them to quickly initiate emergency response processes and effectively control the development of network threats.

[0067] Among them, see Figure 2 , is a schematic diagram of the structure of an acquisition module of a network security monitoring system provided by an embodiment of the present invention, wherein the acquisition module 1 includes a monitoring unit 11, an extraction unit 12, a processing unit 13, and an upload storage unit 14:

[0068] A monitoring unit, configured to monitor a data communication port and a data communication protocol of the network traffic data through a network sniffing device;

[0069] An extraction unit, configured to extract key data elements of the network traffic data through a data analysis tool;

[0070] a processing unit, configured to process the key data elements according to preset rules to generate the distribution data;

[0071] The uploading storage unit is used to upload and store the distributed data into the distributed database through a data uploading tool.

[0072] It should be noted that the network sniffing device installed in the monitoring unit can use deep packet inspection (DPI) and traffic analysis technology to monitor network traffic data in real time. It can monitor the status of data communication ports in real time, such as the usage of TCP / UDP ports, and can also deeply analyze data communication protocols such as HTTP, HTTPS, FTP, etc., thereby comprehensively capturing various data in network activities and promptly discovering and tracking abnormal network traffic behavior.

[0073] The extraction unit can use data parsing tools to process the filtered network traffic data. These tools are based on regular expression matching, XML / JSON parsing and protocol-specific parsing algorithms. They can accurately extract key data elements in the network traffic data, such as user behavior information, session data and transaction records, thereby ensuring the integrity and accuracy of the data.

[0074] After obtaining the key data elements, the processing unit can process these data according to preset rules to generate distributed data with a specific format. These preset rules may include data cleaning (noise removal, error correction), data conversion (format unification, standardization), data analysis (statistical calculation, pattern recognition), and other aspects.

[0075] Finally, the upload storage unit uses data upload tools to quickly and securely transfer the processed distributed data to the distributed database system. This process fully considers the reliability, security, and efficiency of data transmission, using technologies such as breakpoint resuming, data encryption, and load balancing to ensure that data is accurately stored on each node of the distributed database.

[0076] Through the above methods, centralized management and efficient access to data can be achieved, facilitating subsequent data analysis and application.

[0077] Among them, see Figure 3 , is a schematic diagram of the structure of a training module of a network security monitoring system provided by an embodiment of the present invention. The training module 2 includes a preprocessing unit 21, a marking unit 22, a generating unit 23, an adjusting unit 24, and a verifying unit 25:

[0078] a preprocessing unit, configured to obtain the distributed data in the distributed database and preprocess the distributed data to obtain target distributed data;

[0079] a marking unit, configured to mark the target distribution data according to historical network attack data to obtain marked distribution data;

[0080] A generating unit, configured to select three different deep learning algorithms for training based on the labeled distribution data to obtain three different distribution models;

[0081] An adjustment unit, configured to adjust the parameter combination of each distribution model to an optimal parameter combination through a cross-validation algorithm;

[0082] A verification unit is used to verify the prediction effect of each distribution model based on the marked distribution data, and determine the optimal distribution model, that is, the safety monitoring model, based on the prediction effect.

[0083] In practical applications, the preprocessing unit can retrieve and aggregate widely distributed heterogeneous data from distributed databases. This process extracts and integrates the data, and then cleans, denoises, standardizes, and normalizes the integrated data. This eliminates data redundancy, corrects errors, and unifies data representation, generating clearly structured target distribution data to ensure efficient training and accurate predictions for subsequent models.

[0084] The labeling unit can use a historical cyberattack database as a reference to classify and label target distribution data using machine learning algorithms or expert rule systems. This process transforms unsupervised raw data into supervised learning samples, or labeled distribution data, where each sample is assigned a clear attack type label.

[0085] The generation unit introduces three different deep learning algorithms: convolutional neural networks (CNNs), recurrent neural networks (RNNs) and their variant, long short-term memory networks (LSTMs), and graph neural networks (GNNs). These algorithms can be customized for training based on the characteristics of labeled distribution data and construct three different distribution models. These algorithms have different advantages, such as CNN's ability to extract local features, RNN's ability to model time series, and GNN's ability to learn complex network structures.

[0086] To further improve model performance, the tuning unit can employ a cross-validation strategy, systematically adjusting the parameter configurations of each distribution model through multiple rounds of iteration. This process involves optimizing hyperparameters such as the learning rate and batch size, as well as fine-tuning the model's internal structural parameters. This approach avoids overfitting while achieving a globally optimal parameter combination, resulting in a model with stronger generalization capabilities and higher prediction accuracy.

[0087] Finally, the verification unit uses an independent validation set to comprehensively and objectively evaluate the predictive performance of each distribution model based on metrics such as precision, recall, and F1 score. Through comparative analysis, the verification unit scientifically identifies the best-performing model, known as the security monitoring model. This model not only effectively identifies known types of cyberattacks but also possesses a certain degree of generalization capability, enabling it to address new and unknown threats and enhance network security protection capabilities.

[0088] The obtaining of the distributed data in the distributed database and preprocessing the distributed data to obtain target distributed data specifically includes:

[0089] Obtaining distribution data in the distributed database, removing noise data from the distribution data, and obtaining denoised distribution data;

[0090] Select any field in the denoised distribution data, count the abnormal values ​​in the field, and when the abnormal value exceeds a preset threshold, determine the abnormal data point corresponding to the field, and perform corresponding processing on the abnormal data point to obtain abnormality-processed data;

[0091] Feature selection is performed on the data after the abnormality processing to obtain the target distribution data.

[0092] It's important to note that distributed data can first be acquired from multiple nodes in a distributed database, either in parallel or serially. This data typically consists of raw records from various sources, interspersed with noise data—inaccurate and incomplete data points resulting from equipment failures, human error, and transmission anomalies. To improve data quality, this noise data can be removed. Data cleaning techniques, such as detection and correction methods based on statistical rules, pattern recognition, and machine learning algorithms, can effectively remove noise from raw distributed data, generating denoised distributed data.

[0093] By calculating statistics on field values, such as the mean, standard deviation, and median, and comparing them against preset thresholds, we can accurately identify abnormal values ​​that fall outside the normal range of fluctuation. Once we confirm the presence of abnormal values ​​in a field and their number exceeds the preset threshold, we can further locate the specific abnormal data points. For these abnormal data points, we can adopt various processing strategies, such as smoothing, interpolation, direct deletion, or marking them as abnormal, to reduce their interference with the overall data analysis results and obtain data after abnormality processing.

[0094] To optimize data processing efficiency and model training effectiveness, feature selection can be performed on the anomaly-processed data. This involves selecting the most valuable feature subset from a large set of features for the target task (such as classification, regression, and clustering). This process typically involves techniques such as feature evaluation, feature search, and feature validation. Feature evaluation techniques can include correlation analysis and information gain calculation; feature search techniques can include forward selection, backward deletion, and random search; and feature validation techniques can include cross-validation.

[0095] Through the above method, the target distribution data for subsequent analysis and modeling can be obtained, which significantly improves the representativeness and effectiveness of the data, ensures the accuracy and reliability of the data, and greatly improves the efficiency of data processing and the accuracy of subsequent analysis results.

[0096] The step of selecting any field in the denoised distribution data, counting abnormal values ​​in the field, and determining abnormal data points corresponding to the field when the abnormal value exceeds a preset threshold, and performing corresponding processing on the abnormal data points to obtain abnormally processed data specifically includes:

[0097] Setting a statistical standard value for the field and determining a reasonable value range within the field based on the statistical data distribution of the field;

[0098] For the field, anomaly judgment is performed based on a preset standard deviation coefficient threshold and variance, data points that deviate from the standard deviation by more than one value are marked as abnormal data points, and the abnormal data points are processed accordingly;

[0099] If the proportion of the abnormal data points in the total field exceeds a preset proportion, the field is removed.

[0100] Specifically, you can first set a corresponding baseline value for the field, which can serve as a reference point for subsequent analysis. By analyzing the distribution of values ​​within the field and using statistical methods (such as histogram analysis and normal distribution hypothesis testing), you can determine a reasonable range of values ​​that includes most normal data points while excluding extreme or outlier data points.

[0101] When detecting abnormal values, a judgment mechanism based on standard deviation can be used. By calculating the standard deviation of the field data and combining it with a preset standard deviation coefficient threshold, which is usually 1, 2 or 3 times the standard deviation and can be adjusted according to data sensitivity and abnormal tolerance, data points that deviate significantly from the overall distribution, i.e., potential abnormal data points, can be identified. For these data points, a variety of strategies can be used for processing, including data replacement (such as using mean and median filling), data smoothing (such as reducing the impact of abnormalities through neighborhood averaging) or direct elimination, among which data replacement includes mean and median filling, etc.; data smoothing can reduce the impact of abnormalities through neighborhood averaging.

[0102] To assess the overall impact of outliers, calculate the proportion of outlier data points in a field. If this proportion exceeds a pre-set threshold, which is determined based on the specific nature of the dataset and business logic, the data quality of the field may be severely affected. In this case, the entire field can be removed and replaced with a new one from the data source, ensuring the accuracy and reliability of subsequent analysis.

[0103] Through the above method, specific fields in the denoised distribution data can be subjected to detailed statistics and analysis. Combined with the preset threshold judgment and exception handling strategy, outliers in the data can be effectively identified and processed, thereby improving the cleanliness and usability of the overall data set, facilitating subsequent data mining, modeling and analysis.

[0104] Among them, the label distribution data includes a label distribution training set and a label distribution verification set. Three different deep learning algorithms are selected to train the label distribution training set to obtain three different distribution models, and the prediction effect of each distribution model is verified by the label distribution verification set.

[0105] Among them, see Figure 4 , is a schematic diagram of the structure of a judgment module of a network security monitoring system provided by an embodiment of the present invention, wherein the judgment module 3 comprises a collection unit 31, a filtering unit 32 and a determination unit 33:

[0106] A collection unit, configured to collect the network data stream;

[0107] A filtering unit, configured to deploy a data flow filter at a data flow entry point of the security monitoring model, and remove network data flows irrelevant to intrusion detection from the network data flow through the data flow filter to obtain a target network data flow;

[0108] The determination unit is used to obtain the characteristic parameters of the target network data flow, calculate the matching probability of the characteristic parameters through the security monitoring model, judge whether there is a network security incident in the target network data flow according to the matching probability, and generate a corresponding judgment result.

[0109] Furthermore, the acquisition unit utilizes network packet capture technology, such as the Libpcap or WinPcap libraries, to capture data packets in transit at the network interface in real time, ensuring comprehensive and timely data. Furthermore, the acquisition unit must comprehensively consider network load and performance impacts, and employ intelligent scheduling algorithms to optimize resource allocation and avoid unnecessary interference with normal network communications.

[0110] The filtering unit deploys efficient data flow filters at the entry point of the data stream into the security monitoring system. These filters, based on predefined rule sets or machine learning algorithms, accurately identify and eliminate network data flows not directly relevant to the intrusion detection task. This process effectively reduces the data processing workload of subsequent determination units, improving the overall system's response speed and accuracy.

[0111] Using feature extraction technology, the determination unit extracts key characteristic parameters from the target data stream, including IP address, port number, protocol type, packet size, and transmission frequency. These characteristic parameters are processed using a security monitoring model and their matching probability with known security threat patterns is calculated. Based on the calculated matching probability, the determination unit intelligently determines whether the target network data stream contains network security incidents, such as DDoS attacks, SQL injections, and malware propagation, and generates corresponding judgment report results. These reports include key information such as incident type, severity, and impact, providing network administrators with timely and effective security warnings and response recommendations.

[0112] The method comprises obtaining characteristic parameters of the target network data flow, calculating a matching probability of the characteristic parameters using the security monitoring model, determining whether a network security incident occurs in the target network data flow based on the matching probability, and generating a corresponding judgment result. The judgment is performed using the following intrusion detection algorithm:

[0113] Assuming the matching probability to be P and the first threshold to be T, calculating the deviation B=|PT| between the matching probability P and the first threshold T;

[0114] If the deviation degree B is less than or equal to the first threshold T, it is determined that there is no network security incident in the target network data flow;

[0115] If the deviation B is greater than the first threshold T and the matching probability P is greater than the first threshold T, it is necessary to determine whether the matching probability P is associated with the first threshold T:

[0116] Set the second threshold to X and calculate the correlation Wherein, P is the matching probability, T is the first threshold and H is the correlation degree. If the correlation degree H is greater than the second threshold X, it is determined that a network security incident exists in the target network data flow.

[0117] Among them, the classification module also includes counting the number of the abnormal network data flows. The larger the number of the abnormal network data flows, the higher the alarm level of the corresponding alarm information. In addition, the categories of network security events corresponding to the abnormal network data flows are different, and the corresponding alarm categories of the alarm information are different.

[0118] In practical applications, the classification module can also integrate a function to count abnormal network data flows, dynamically monitoring and recording the number of abnormal data flow instances in the network environment. This statistical indicator is set based on the urgency and priority of subsequent alarm information. Specifically, when the number of abnormal network data flows detected shows a significant growth trend, the system can automatically increase the alarm level of the corresponding alarm information based on preset algorithm logic, ensuring that the network security team can respond promptly and prioritize security incidents with greater potential threats or wider impacts.

[0119] Furthermore, the classification module can distinguish between different cybersecurity incident categories corresponding to abnormal network data flows. Given the diversity and complexity of cybersecurity incidents, each category may require different response strategies and processes. Therefore, the classification module can categorize abnormal data flows into corresponding cybersecurity incident categories based on key factors such as their characteristics and behavioral patterns, and generate information with clear alarm categories.

[0120] Through the above methods, the pertinence and accuracy of alarm information can be improved, which greatly promotes the efficient management and rapid handling of security incidents.

[0121] Among them, see Figure 5 , is a flow chart of a network security monitoring method provided by an embodiment of the present invention, the monitoring method comprising:

[0122] S1, obtaining network traffic data, processing the network traffic data to obtain distribution data, and storing the distribution data in a distributed database;

[0123] S2, based on the distributed database, training the distributed data through a deep learning algorithm to obtain a security monitoring model;

[0124] S3, collecting network data flow, determining whether there is a network security incident in the network data flow through the security monitoring model, and outputting the determination result;

[0125] S4, classifying the judgment results to obtain normal network data flows and abnormal network data flows;

[0126] S5, generating corresponding alarm information according to the network security event corresponding to the abnormal network data flow.

[0127] It should be noted that Figure 5The execution subject of the method shown may be a software and / or hardware device. The execution subject of the present application may include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, the user equipment may include but is not limited to computers, smart phones, personal digital assistants (PDAs) and the electronic devices mentioned above. Network equipment may include but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this.

[0128] See also Figure 6 , is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. The electronic device 60 includes: a processor 61, a memory 62 and a computer program; wherein:

[0129] The memory 62 is used to store the computer program, which may also be a flash memory. The computer program is, for example, an application program or a functional module for implementing the above method.

[0130] The processor 61 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant description in the above method embodiment.

[0131] Optionally, the memory 62 may be independent or integrated with the processor 61 .

[0132] When the memory 62 is a device independent of the processor 61, the device may further include:

[0133] The bus 63 is used to connect the memory 62 and the processor 61 .

[0134] The present invention also provides a readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the methods provided in the various embodiments described above.

[0135] Among them, the readable storage medium can be a computer storage medium or a communication medium. Communication media include any medium that facilitates the transmission of computer programs from one place to another. Computer storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application-specific integrated circuit (ASIC). In addition, the ASIC can be located in a user device. Of course, the processor and the readable storage medium can also exist in a communication device as discrete components. The readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0136] The present invention also provides a program product, which includes execution instructions stored in a readable storage medium. At least one processor of a device can read the execution instructions from the readable storage medium, and at least one processor executes the execution instructions so that the device implements the methods provided in the various embodiments described above.

[0137] In the embodiments of the above-mentioned devices, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0138] Through the introduction of the above embodiments, the present invention can obtain network traffic data through the network security monitoring system and its method through the acquisition module, training module, judgment module, classification module and alarm module, process the network traffic data to obtain distribution data, and store the distribution data in a distributed database; based on the distributed database, the distribution data is trained by a deep learning algorithm to obtain a security monitoring model; the network data flow is collected, and the network data flow is judged by the security monitoring model whether there is a network security incident, and the judgment result is output; the judgment result is classified to obtain normal network data flow and abnormal network data flow; according to the network security incident corresponding to the abnormal network data flow, corresponding alarm information is generated, so that the network security monitoring efficiency can be improved through the deep learning algorithm, resource consumption can be reduced, and costs can be reduced. It can also adapt to complex and changeable network environments to achieve comprehensive and accurate monitoring of network security. At the same time, it has a strong ability to identify unknown threats, which can effectively reduce false alarms and missed reports, and the system rule base is updated quickly and has strong adaptability, so new threats can be discovered in a timely manner.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network security monitoring system, It is characterized in that The system comprises: An acquisition module, used for acquiring network flow data, processing the network flow data to obtain distribution data, and storing the distribution data in a distributed database; A training module, used to train the distributed data through a deep learning algorithm based on the distributed database to obtain a security monitoring model; A judgment module, used for collecting network data flow, judging whether there is a network security incident in the network data flow through the security monitoring model, and outputting the judgment result; A classification module, used to classify the judgment result to obtain normal network data flow and abnormal network data flow; The alarm module is used to generate corresponding alarm information according to the network security event corresponding to the abnormal network data flow.

2. A network security monitoring system according to claim 1, It is characterized in that The acquisition module comprises: A monitoring unit, used for monitoring the data communication port and data communication protocol of the network traffic data through a network sniffing device; An extraction unit, used for extracting key data elements of the network traffic data through a data analysis tool; A processing unit, used to process the key data element according to a preset rule to generate the distribution data; The upload storage unit is used to upload and store the distributed data into the distributed database through a data upload tool.

3. A network security monitoring system according to claim 1, It is characterized in that The training module comprises: A preprocessing unit, used for acquiring the distributed data in the distributed database, and preprocessing the distributed data to obtain target distributed data; A marking unit, used for marking the target distribution data according to the historical network attack data to obtain marked distribution data; A generating unit, configured to select three different deep learning algorithms for training according to the labeled distribution data to obtain three different distribution models; An adjustment unit, used to adjust the parameter combination of each distribution model to an optimal parameter combination through a cross-validation algorithm; A verification unit is used to verify the prediction effect of each distribution model according to the marked distribution data, and determine the optimal distribution model, that is, the safety monitoring model, according to the prediction effect.

4. A network security monitoring system according to claim 3, It is characterized in that The obtaining of the distributed data in the distributed database and preprocessing the distributed data to obtain target distributed data specifically includes: Acquire the distributed data in the distributed database, remove the noise data in the distributed data, and obtain denoised distributed data; Select any field in the denoised distribution data, count the abnormal values ​​in the field, and when the abnormal value exceeds a preset threshold, determine the abnormal data point corresponding to the field, and perform corresponding processing on the abnormal data point to obtain data after abnormal processing; Feature selection is performed on the data after the abnormal processing to obtain the target distribution data.

5. A network security monitoring system according to claim 4, It is characterized in that The selecting any field in the denoised distribution data, counting the abnormal value in the field, and when the abnormal value exceeds a preset threshold, determining the abnormal data point corresponding to the field, and performing corresponding processing on the abnormal data point to obtain data after abnormal processing, specifically includes: Setting a statistical standard value on the field, and determining a reasonable value range within the field according to the statistical data distribution of the field; For the field, anomaly judgment is made based on the preset standard deviation coefficient threshold and variance, data points that deviate from the standard deviation by more than one degree are marked as abnormal data points, and the abnormal data points are processed accordingly; if the proportion of the abnormal data points in the total field exceeds the preset proportion, the field is eliminated.

6. A network security monitoring system according to claim 3, It is characterized in that The marker distribution data includes a marker distribution training set and a marker distribution verification set. Three different deep learning algorithms are selected to train the marker distribution training set to obtain three different distribution models, and the prediction effect of each distribution model is verified by the marker distribution verification set.

7. A network security monitoring system according to claim 1, It is characterized in that The judging module comprises: A collection unit, used for collecting the network data stream; A filtering unit, configured to deploy a data flow filter at a data flow entry point of the security monitoring model, and remove network data flows irrelevant to intrusion detection from the network data flows through the data flow filter to obtain a target network data flow; The determination unit is used to obtain the characteristic parameters of the target network data flow, calculate the matching probability of the characteristic parameters through the security monitoring model, judge whether there is a network security incident in the target network data flow according to the matching probability, and generate a corresponding judgment result.

8. A network security monitoring system according to claim 7, It is characterized in that The characteristic parameters of the target network data flow are obtained, the matching probability of the characteristic parameters is calculated by the security monitoring model, and the target network data flow is judged whether there is a network security incident according to the matching probability, and a corresponding judgment result is generated, and the judgment is performed by the following intrusion detection algorithm: Assuming the matching probability as P and the first threshold as T, calculating the deviation B=|PT| between the matching probability P and the first threshold T; If the deviation degree B is less than or equal to the first threshold value T, it is determined that there is no network security incident in the target network data flow; If the deviation B is greater than the first threshold T and the matching probability P is greater than the first threshold T, it is necessary to determine whether the matching probability P is associated with the first threshold T: Set the second threshold to X and calculate the correlation Wherein, P is the matching probability, T is the first threshold and H is the correlation degree. If the correlation degree H is greater than the second threshold X, it is determined that a network security incident exists in the target network data flow.

9. A network security monitoring system according to claim 1, It is characterized in that The classification module also includes counting the number of abnormal network data flows. The larger the number of abnormal network data flows, the higher the alarm level of the corresponding alarm information. In addition, the categories of network security events corresponding to the abnormal network data flows are different, and the corresponding alarm categories of the alarm information are different.

10. A monitoring method for a network security monitoring system according to any one of claims 1 to 9, It is characterized in that The method comprises: Acquire network flow data, process the network flow data to obtain distribution data, and store the distribution data in a distributed database; Based on the distributed database, the distributed data is trained by a deep learning algorithm to obtain a security monitoring model; Collect network data flow, determine whether there is a network security incident in the network data flow through the security monitoring model, and output the determination result; The judgment results are classified to obtain normal network data flows and abnormal network data flows; and corresponding alarm information is generated according to network security events corresponding to the abnormal network data flows.

Citation Information

Patent Citations

  • Network security situation sensing system and method

    CN104348829A

  • Visualization-based security database management system

    CN117763618A

Cited By

  • Method and system for network traffic abnormity supervision and protection

    CN120729630A

  • A method and system for network traffic anomaly supervision protection

    CN120729630B