Method and system for network security situation awareness of big science device

Abstract

The invention discloses a big science device-oriented network security situation awareness method and system, and relates to the technical field of network security. The method comprises the following steps: carrying out data acquisition on a plurality of big science devices to obtain initial security information; preprocessing the initial security information based on Kafka according to a defined structured format; storing the preprocessed security information; modeling and detecting the preprocessed safety information from different angles based on a plurality of behavior analysis algorithms to generate alarm information; processing the alarm information, automatically generating response information, and sending the response information to safety probes of all big science devices; and the overall safety condition is directly presented based on front and rear end modules and a visualization technology. According to the invention, global situation awareness and automatic response are realized, a security protection system supporting joint defense and linkage among a plurality of big science devices is supported, and the network security protection capability and defense effect are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of network security technology, and specifically relates to a method and system for network security situation awareness for large scientific facilities. Background Art

[0002] With the rapid development of information technology and the popularization of the Internet, network security issues are increasingly becoming a major challenge for various industries. As a major scientific research infrastructure, large scientific facilities undertake important scientific research and experimental tasks, and their network security and stability directly affect the smooth progress of scientific research. Therefore, building a comprehensive, efficient, and real-time network security situation awareness system is crucial to ensuring the network security of large scientific facilities.

[0003] The network structure of large scientific facilities is usually complex and large, involving the interconnection of multiple scientific research institutions, laboratories and equipment. A large amount of data is generated during the operation of the equipment, and they work together through the network to support large-scale data collection, analysis and transmission. Therefore, these large scientific facilities not only face traditional network security threats such as malware, network attacks, identity theft and other problems, but may also face more complex and diverse forms of attacks. How to monitor and respond to various security threats in real time in a complex network environment has become an urgent problem to be solved.

[0004] However, traditional network security protection measures often have some shortcomings. Many firewalls and intrusion detection systems can only provide protection at a specific network level and cannot provide global security situation awareness capabilities across devices and systems, making it difficult to fully grasp the overall security status of large scientific facilities. On the other hand, traditional security protection technologies usually rely on manual analysis and response, making it difficult to achieve rapid and automated handling of security incidents, increasing the time delay in responding to sudden security incidents. At the same time, due to the wide distribution and strong heterogeneity of large scientific facilities, there are often barriers to security protection between single devices, making it difficult to achieve effective joint defense and linkage, resulting in local protection measures being unable to cover overall security needs. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for network security situation awareness for large scientific facilities, which solves the problems of being unable to provide global security situation awareness across devices and systems, lacking automated response, and not supporting joint defense and linkage among multiple large scientific facilities.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for network security situation awareness for large scientific facilities, comprising the following steps:

[0008] Step (1): Use the security probe of each large scientific facility to collect data on the network equipment, security equipment and related assets of the large scientific facility and the open source network related to security to obtain initial security information;

[0009] Step (2): according to the defined structured format, the initial security information is parsed, associated and enriched in real time based on Kafka to obtain pre-processed security information;

[0010] Step (3): storing the pre-processed security information;

[0011] Step (4): Based on a variety of behavior analysis algorithms, model and detect the pre-processed security information from different angles to generate alarm information;

[0012] Step (5): Process the alarm information and automatically generate response information, and send the response information to the safety probes of all large scientific facilities;

[0013] Step (6): Directly present the overall security status based on front-end and back-end modules and visualization technology.

[0014] On the other hand, the present invention provides a system for network security situation awareness for large scientific facilities, which is used to implement the aforementioned method for network security situation awareness for large scientific facilities, including a site module and a central server module; the site module includes a security probe module; the central server module includes a security information preprocessing module, a security information storage module, a security information analysis module, a security information application module, and a security situation visualization module;

[0015] The security probe module is used to collect data on the network equipment, security equipment and related assets of the large scientific facility and the open source network related to security by using the security probe of each large scientific facility to obtain initial security information;

[0016] The security information preprocessing module is used to parse, associate and enrich the initial security information in real time based on Kafka according to a defined structured format to obtain preprocessed security information;

[0017] The security information storage module is used to store the pre-processed security information;

[0018] The security information analysis module is used to model and detect the pre-processed security information from different angles based on a variety of behavior analysis algorithms to generate alarm information;

[0019] The safety information application module is used to process the alarm information and automatically generate response information, and send the response information to the safety probes of all large scientific facilities;

[0020] The security situation visualization module is used to directly present the overall security situation based on the front-end and back-end modules and visualization technology.

[0021] In a third aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for network security situation awareness for large scientific facilities.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned method for network security situation awareness for large scientific facilities.

[0023] The beneficial effects of the present invention are:

[0024] The present invention realizes global security situation awareness across devices and systems, can collect and process security information from multiple large scientific facilities in real time, and comprehensively grasp the overall network security situation through behavioral analysis and multi-angle data association; the designed automated response mechanism can quickly respond to security alarms, reduce the need for manual intervention, and improve the speed of emergency disposal, thereby effectively reducing the potential risks brought by network attacks; it supports joint defense and linkage among multiple large scientific facilities, ensuring coordinated protection among the major scientific facilities, and can quickly share security information and take consistent defense measures when threats occur, thereby enhancing the overall defense capabilities against network attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a method for network security situation awareness for large scientific facilities according to the present invention;

[0026] Figure 2 This is a system structure diagram of the present invention for network security situation awareness of large scientific facilities. DETAILED DESCRIPTION

[0027] It should be noted that in the embodiments of the present invention, the terms "configuration" and "setting" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are consistent. The terms "data", "security data", "information" and "security information" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are consistent. The terms "behavior", "activity" and "mode" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0028] It should also be noted that, in the embodiments of the present invention, the term "extract" may refer to obtaining information directly from a certain location through a preset index, or may refer to obtaining information indirectly from a certain location through a certain statistical or computational method.

[0029] It is also necessary to emphasize that in the embodiments of the present invention, the term "comprising" describes the main components of modules, steps, technologies or components, but does not exclude the existence of secondary components. In the absence of further restrictions, the elements defined by the sentence "comprising..." do not exclude the existence of other elements in the modules, steps, technologies or components of the elements.

[0030] Finally, it should be noted that the following describes the preferred implementation of the present invention. It should be pointed out that although the preferred implementation of the present invention has been described, once the basic creative concept of the present invention is known, a number of improvements and modifications can be made by those skilled in the art without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred implementation and all changes and modifications that fall within the scope of the implementation of the present invention.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] like Figure 1 As shown, the embodiment of the present invention provides a method for network security situation awareness for large scientific facilities, which specifically includes the following steps: Step (1): Use the security probe of each large scientific facility to collect information from network devices, security devices and their related assets, and security-related open source blogs, news, and posts of multiple large scientific facilities to obtain initial security information; In some embodiments of the present invention, the step (1) collects initial security information from network devices, security devices and related assets of multiple large scientific facilities and security-related open source blogs, news and posts, and specifically includes: Step (1-1): Collect the export mirror traffic from the export router to obtain relevant flow data information; Step (1-2): Collect mailbox server logs and DNS server logs from the application server to obtain relevant behavior data information; Steps (1-3): Collect computing node login logs and unified authentication login logs from key business systems to obtain relevant behavior data information; Steps (1-4): Collect network intrusion detection system logs, host intrusion detection system logs, Web application firewall logs, and intrusion prevention system logs from security devices to obtain relevant security event information; Steps (1-5): Collect website asset data and host asset data for websites and hosts to obtain relevant asset information; Steps (1-6): Extract threat information from security-related open source blogs, news, and posts, collect threatening IP addresses and domain names from the threat information and add them to the blacklist to obtain threat data information; Step (1-7): Use Filebeat to encrypt the security information collected in the above steps, and based on the security probe, transmit the encrypted information collected in steps (1-1) to step (1-5) to the security information preprocessing module of the central server module, and transmit the encrypted information collected in step (1-6) to the security information storage module of the central server module.

[0033] Furthermore, the step (1-1) collects the egress mirrored traffic from the egress router to obtain relevant flow data information, including the following specific steps: Step (1-1-1): Configure the router mirror port so that network traffic data can be copied and transmitted to the data acquisition device or system; Step (1-1-2): Preliminarily screen the collected network traffic data and remove irrelevant traffic; Step (1-1-3): Use Zeek to perform protocol analysis and integrity check on the filtered data to obtain various network protocol logs containing information such as source IP, target IP, port, request, response content, etc. Step (1-1-4): Transmit the network protocol log to the security information preprocessing module; Furthermore, the step (1-2) collects the mailbox server log and the DNS server log from the application server to obtain relevant behavior data information, including the following specific steps: Step (1-2-1): Configure the logging function of the application server to ensure that the logs of the email server and DNS server can completely and accurately record all request and response information; Step (1-2-2): Transmit the collected data to the security information preprocessing module; Furthermore, the steps (1-3) collect computing node login logs and unified authentication login logs from the key business system to obtain relevant behavior data information, including the following specific steps: Step (1-3-1): Configure the logging function of the computing nodes and the unified authentication system to record all user login information, including login time, login location, device information, etc. Step (1-3-2): Transmit the collected data to the security information preprocessing module; Furthermore, in the steps (1-4), the security device collects network intrusion detection system logs, host intrusion detection system logs, Web application firewall logs and intrusion prevention system logs to obtain relevant security event information, including the following specific steps: Step (1-4-1): Configure the log collection function of each security device to ensure that all intrusion detection and prevention events can be recorded in detail; Step (1-4-2): Transmit the collected data to the security information preprocessing module; Furthermore, the steps (1-5) collect website asset data and host asset data for the website and the host to obtain relevant asset information, including the following specific steps: Step (1-5-1): Automated scanning of websites and hosts to extract basic information about assets, including server IP addresses, operating system types, application versions, etc. Step (1-5-2): Update the asset library regularly to ensure that asset information can reflect newly added or offline devices and applications in a timely manner; Step (1-5-3): Transmit the collected data to the security information preprocessing module; Furthermore, the steps (1-6) extract threat information from security-related open source blogs, news and posts, collect threatening IP addresses and domain names from the threat information and add them to the blacklist to obtain threat data information, including the following specific steps: Step (1-6-1): Configure an external threat information collector to regularly capture information from security-related websites such as open source blogs, news, and forums; Step (1-6-2): Perform a preliminary analysis of the collected text information through natural language processing technology to automatically identify malicious IP addresses, domain names and other related identifiers; Step (1-6-3): automatically add the identified malicious IP addresses and domain names to the blacklist, and transfer the collected data to the security information storage module for storage; Furthermore, the step (1-7) uses Filebeat to encrypt the security information collected in the above steps, and based on the security probe, transmits the encrypted information collected in steps (1-1) to step (1-5) to the security information preprocessing module of the central server module, and transmits the encrypted information collected in step (1-6) to the security information storage module of the central server module, including the following specific steps: Step (1-7-1): Configure the Filebeat client to automatically forward various logs and data from the specified directory; Step (1-7-2): Enable the encryption function of Filebeat to encrypt the transmitted data; Step (1-7-3): Configure encrypted communication between the security probe and the central server to ensure that data can be stably and securely transmitted to the security information preprocessing module and the security information storage module; Step (2): parsing, associating and enriching the initial security information in real time based on Kafka according to a well-defined structured format to obtain pre-processed security information; In some embodiments of the present invention, the step (2) performs real-time parsing, association and enrichment of the initial security information based on Kafka according to a well-defined structured format to obtain pre-processed security information, specifically comprising: Step (2-1): Use Logstash to receive and decrypt the initial security information sent by all large scientific facility sites (i.e., the encrypted information sent in step (1-7)) to obtain the decrypted security information; Step (2-2): Logstash outputs the decrypted security information to Kafka; Step (2-3): Define different structural formats of security information according to the requirements of the behavior analysis algorithm; Step (2-4): Read the decrypted security information from Kafka, parse the decrypted security information into structured data according to the type of the decrypted security information, and obtain the pre-processed original structured data; Step (2-5): performing an association operation on the preprocessed original structured data according to a preset rule to form preprocessed behavior structured data, wherein the preset rule refers to a specific logic or condition that can be used to perform an association operation on the same type of security information, for example, requests from the same IP address that are adjacent in time can be classified as a security event according to time window matching; Step (2-6): enrich the preprocessed behavioral structured data by associating it with an external data source to obtain preprocessed security information; wherein the external data source includes third-party data services such as geographic IP databases, time synchronization services, and third-party API data that contain valuable information that can enrich the structured data.

[0076] Step (3): storing the pre-processed security information;

[0077] In some embodiments of the present invention, the step (3) of storing the pre-processed security information specifically includes:

[0078] Step (3-1): Establish a relational database MySQL, a non-relational database Elasticsearch, and an in-memory database Redis on the security information storage module of the central server module;

[0079] Step (3-2): receiving the pre-processed security information, classifying it according to its content, function and other characteristics, and storing it in different databases established in step (3-1);

[0080] Furthermore, the step (3-1) establishes a relational database MySQL, a non-relational database Elasticsearch, and an in-memory database Redis on the central server, including the following specific steps:

[0081] Step (3-1-1): Configure the relational database MySQL to store structured data, such as user information, behavioral events, attack events, etc., to ensure that the stored data can be flexibly queried and analyzed;

[0082] Step (3-1-2): Configure the non-relational database Elasticsearch to store massive log data and behavior analysis results, and provide efficient search and real-time statistics functions;

[0083] Step (3-1-3): Configure the memory database Redis to cache frequently accessed hot data and temporary data to improve the overall data loading speed;

[0084] Furthermore, the step (3-2) receives the pre-processed security information, classifies it according to its content, function and other characteristics, and stores it in different databases respectively, including the following specific steps:

[0085] Step (3-2-1): Store the pre-processed security information, such as behavior analysis results, security event statistics, etc., in the MySQL database;

[0086] Step (3-2-2): Store the pre-processed security information, such as network traffic data, alarm data, etc., in the Elasticsearch database;

[0087] Step (3-2-3): Store the pre-processed security information, such as hot data and temporary data that need to be frequently accessed during the current system processing, into the Redis database;

[0088] Step (3-2-4): Configure the storage management strategy of each database to ensure the accuracy, completeness and efficient query of data storage;

[0089] Furthermore, the step (3-2-4) configures the storage management strategy of each database to ensure the accuracy, completeness and efficient query of data storage, including the following specific steps:

[0090] Step (3-2-4-1): Perform a deduplication check on the stored data to ensure that the data stored in the database is not stored repeatedly due to repeated collection;

[0091] Step (3-2-4-2): Back up the stored data regularly and store the backup data in a high-security environment;

[0092] Step (3-2-4-3): Create a query index in the non-relational database Elasticsearch to ensure efficient query of large amounts of log data.

[0093] Step (4): Based on a variety of behavior analysis algorithms, various types of security information are modeled and detected from different angles to generate alarm information;

[0094] In some embodiments of the present invention, the step (4) is based on a variety of behavior analysis algorithms to model and detect various types of security information from different angles to generate alarm information, specifically including:

[0095] Step (4-1): receiving the security information output from different databases established in step (3-1) from the security information storage module, and performing different data analysis according to the type of the security information; including:

[0096] Perform DNS data analysis;

[0097] Conduct abnormal account login and brute force analysis;

[0098] Conduct network traffic data analysis;

[0099] Conduct safety equipment data analysis;

[0100] Step (4-2): For behavioral analysis algorithms that are not included but need to be added, integrate them into the security analysis engine in the form of plug-ins;

[0101] Furthermore, the DNS data analysis includes flood attack detection and abnormal IP detection;

[0102] The abnormal account login and brute force cracking analysis includes login node account detection, SSO login account detection and email login account detection;

[0103] The network traffic data analysis includes abnormal link number detection, unknown malicious traffic detection and malicious traffic detection based on coarse-grained labels;

[0104] The security device data analysis includes Web application firewall alarm analysis, network intrusion detection system alarm analysis, host intrusion detection system alarm analysis and intrusion prevention system alarm analysis;

[0105] Furthermore, the flood attack detection includes the following specific steps:

[0106] Step (I): Extract the traffic of DNS requests from the Elasticsearch database;

[0107] Step (II): count the number of different domain names queried by each query source IP in real time, and determine whether it exceeds a predetermined threshold;

[0108] Step (III): When the number of request queries exceeds a predetermined threshold, it is considered that the query source IP has launched a flood attack;

[0109] Step (IV): Generate alarm information for flood attack behavior and submit it to the security information application module;

[0110] Furthermore, the abnormal IP detection includes the following specific steps:

[0111] Step (I): Extract DNS query logs from the Elasticsearch database;

[0112] Step (II): Extract the source IP address from the DNS query log, build a DNS fingerprint feature based on the IP address, and extract relevant behavior features, such as the number of queries, the number of query failures, the number of abnormal domain names queried, the number of PTR queries, etc.

[0113] Step (III): Use unsupervised anomaly detection algorithms (including isolation forest and local outlier factor algorithms) to analyze the extracted DNS behavior features and calculate the global and local behavior anomaly scores for each IP address;

[0114] Step (IV): Identify IP addresses with abnormal behavior based on the anomaly score ranking of IP addresses. If the anomaly score of an IP exceeds the set threshold, it is considered an abnormal behavior IP;

[0115] Step (V): Generate alarm information for abnormal behavior IP and submit it to the security information application module;

[0116] Furthermore, the login node account detection includes the following specific steps:

[0117] Step (I): Extract node login logs from the MySQL database;

[0118] Step (II): extract information such as login time, login IP, login user name, login result (success or failure) from the node login log;

[0119] Step (III): For successful logins, check whether there are abnormal login behaviors such as logins during abnormal time periods or logins from abnormal IP addresses;

[0120] Step (IV): For login failures, count the number of login failures using the same login IP address and the number of login failures using different user names to detect whether there is a behavior that exceeds the threshold. If it exceeds the predetermined threshold, there is a brute force attack on the login node.

[0121] Step (V): Generate alarm information for abnormal login behavior and brute force cracking behavior, and submit it to the security information application module;

[0122] Furthermore, the SSO login account detection includes the following specific steps:

[0123] Step (I): extract unified authentication login log from MySQL database;

[0124] Step (II): extract the user login time, login IP, login user name, login application, and login result (success or failure) from the unified authentication login log;

[0125] Step (III): For successful logins, detect whether there are abnormal SSO login behaviors such as logins during abnormal time periods or logins from abnormal IP addresses;

[0126] Step (IV): For login failures, count the number of login failures using the same login IP address and the number of login failures using different user names, and detect whether there is a behavior where the number of login failures exceeds a threshold. If it exceeds the predetermined threshold, it means that SSO brute force cracking behavior has occurred;

[0127] Step (V): Generate alarm information for abnormal SSO login behavior and brute force cracking behavior, and submit it to the security information application module;

[0128] Furthermore, the email login account detection includes the following specific steps:

[0129] Step (I): Extract the mailbox server log from the MySQL database;

[0130] Step (II): Extract login time, login IP, login geographic location, login email address, login email protocol, login result (including success or failure) and other information from the email server log;

[0131] Step (III): For successful logins, detect whether there are abnormal login IP addresses with specific login protocols or abnormal email logins with abnormal login locations;

[0132] Step (IV): For login failures, check whether multiple login attempts to multiple accounts have failed within a short period of time. If so, it is a brute force attack on the email account.

[0133] Step (V): Generate alert information for abnormal email login and brute force cracking behavior, and submit it to the security information application module;

[0134] Furthermore, the abnormal link number detection includes the following specific steps:

[0135] Step (I): extracting stream data information from the Elasticsearch database;

[0136] Step (II): extract the communication records of the source IP address and the target IP address from the network traffic log, and construct a directed graph model to represent the connection between the internal host and the external host;

[0137] Step (III): Monitor each internal host and count its in-connection degree (ICD) and out-connection degree (OCD) within a certain time window. The in-connection degree indicates the number of external hosts pointing to the internal host, and the out-connection degree indicates the number of external hosts pointing from the internal host.

[0138] Step (IV): Calculate the Symmetry Degree of Internal Host (SDI) of each internal host, which is the ratio of the incoming connection degree to the outgoing connection degree. If the SDI value of an internal host is greater than three standard deviations (i.e., exceeds the fluctuation range of normal traffic), the connection behavior of the internal host is considered abnormal.

[0139] Step (V): Generate alarm information for the abnormally connected IP and submit it to the security information application module;

[0140] Furthermore, the unknown malicious traffic detection includes the following specific steps:

[0141] Step (I): extracting stream data information from the Elasticsearch database;

[0142] Step (II): preprocess the flow data information, including traffic splitting, data cleaning and image conversion, to convert the flow data into a format suitable for deep learning processing;

[0143] Step (III): Use multiple deep learning models (including AlexNet, VGG16, and LSTM) to extract the spatiotemporal characteristics of traffic, including data transmission volume, transmission frequency, and IP addresses of both communicating parties;

[0144] Step (IV): Screen the extracted features using the Minimum Redundancy Maximum Relevance (MRMR) algorithm to remove redundant features and ensure that the features used for classification have the maximum amount of information;

[0145] Step (V): Use the trained Support Vector Machine (SVM) multi-class classifier to classify known traffic and identify unknown traffic through the density ratio clustering algorithm. When a new type of traffic is detected, it is marked as unknown traffic and classified into a new category in real-time classification;

[0146] Step (VI): Further analyze the unknown traffic to determine whether it contains malicious behavior or malicious code, cluster the newly identified traffic using the density ratio clustering method, and combine it with known traffic information to analyze whether there may be abnormal encryption certificates or suspicious encryption protocol versions;

[0147] Step (VII): Generate alarm information for unknown traffic with abnormal phenomena detected, and submit it to the security information application module;

[0148] Furthermore, the malicious traffic detection based on coarse-grained labels includes the following specific steps:

[0149] Step (I): extracting stream data information from the Elasticsearch database;

[0150] Step (II): perform coarse-grained labeling on the flow data information at the time slice level to determine whether there is malicious traffic in each time slice;

[0151] Step (III): Use the Multi-Instance Learning (MIL) method to perform cluster analysis on the traffic in each time slice and estimate the maliciousness score of each instance. By clustering multiple subspaces, the preliminary maliciousness score of each instance is obtained;

[0152] Step (IV): Analyze the ratios among ports, protocols, and IP addresses of the traffic data based on the estimated malicious scores and traffic characteristics, and dynamically adjust the malicious scores of the traffic data based on the estimated malicious scores;

[0153] Step (V): Generate alarm information for traffic whose malicious score exceeds a preset threshold and submit it to the security information application module;

[0154] Furthermore, the Web application firewall alarm analysis includes the following specific steps:

[0155] Step (I): Extract the Web Application Firewall log from the MySQL database;

[0156] Step (II): Extract information such as source IP address, request URL, request parameters, interception reason, interception rule, risk level, etc. from the web application firewall log;

[0157] Step (III): Count different interception rules and risk levels for the source IP, set false positive filtering rules based on interception and risk levels according to expert experience, and determine whether it is a high-risk attack;

[0158] Step (IV): Generate alert information for high-risk attacks and submit it to the security information application module;

[0159] Furthermore, the network intrusion detection system alarm analysis includes the following specific steps:

[0160] Step (I): Extract the network intrusion detection system log from the Elasticsearch database;

[0161] Step (II): extract the traffic characteristics, attack characteristics, network intrusion types, detected malicious IPs and victim IPs of the network intrusion behaviors detected in the network intrusion detection system log;

[0162] Step (III): Determine whether the alarm is a false alarm based on the alarm noise reduction rules set by expert experience;

[0163] Step (IV): Generate alarm information for non-false alarms and submit it to the security information application module;

[0164] Furthermore, the host intrusion detection system alarm analysis includes the following specific steps:

[0165] Step (I): extract the host intrusion detection system log from the MySQL database;

[0166] Step (II): extracting access information, process startup information, system call information, host intrusion type and other information of the host intrusion behavior detected in the host intrusion detection system log;

[0167] Step (III): Determine whether the alarm is a false alarm based on the alarm noise reduction rules set by expert experience;

[0168] Step (IV): Generate alarm information for non-false alarms and submit it to the security information application module;

[0169] Furthermore, the intrusion prevention system alarm analysis includes the following specific steps:

[0170] Step (I): Extract the intrusion prevention system logs from the Elasticsearch database;

[0171] Step (II): Extract the intercepted attack source IP, victim IP, attack type, threat level, defense action and other information from the intrusion prevention system log;

[0172] Step (III): Count the attack types and threat levels of the attack source IPs;

[0173] Step (IV): Set a threshold based on expert experience to determine whether the attack type, threat level, and other quantities of the attack source IP exceed the threshold;

[0174] Step (V): Generate alarm information for attack behaviors that exceed the threshold and submit it to the security information application module.

[0175] Step (5): Processing the alarm information detected by data analysis and automatically generating response information, and sending the response information to the safety probes of all large scientific facilities;

[0176] In some embodiments of the present invention, the step (5) processes the alarm information detected by data analysis and automatically generates a response, and sends the response information to the safety probes of all large scientific facilities, specifically comprising:

[0177] Step (5-1): automatically responding to the alarm information according to a predefined threshold, and if the alarm information meets the predefined threshold and does not match the white list, entering the alarm information into the black list;

[0178] Step (5-2): Manage the data in the whitelist according to the whitelist management policy;

[0179] Step (5-3): When the blacklist is updated, the updated content is sent to the security probes of all large scientific facilities;

[0180] Step (5-4): After receiving the update content, the security probe of the large scientific facility will extract the IP address and domain name of the alarm information and perform dynamic blocking operations in the IPS, router and firewall;

[0181] Furthermore, the step (5-2) manages the data in the whitelist according to the whitelist management policy, including the following specific steps:

[0182] Step (5-2-1): System operation and maintenance personnel can manually add or remove IP addresses and domain names from the whitelist in the whitelist library;

[0183] Step (5-2-2): Automatically collect whitelist data through CDN address analysis;

[0184] Step (5-2-3): Classify and manage the IP addresses and domain names in the whitelist, including scientific research institutions, intranet addresses, cooperative institutions, public services, etc.

[0185] Furthermore, the step (5-2-2) automatically collects whitelist data through CDN address analysis, including the following specific steps:

[0186] Step (5-2-2-1): Extract DNS query logs from the database;

[0187] Step (5-2-2-2): Analyze the query domain name and query source IP according to the DNS query log, especially the domain name related to CDN service;

[0188] Step (5-2-2-3): Set up a known CDN domain name database, use the query domain name to compare with the known CDN domain name database to determine whether the request is provided by the CDN service. If so, extract the query source IP address from the DNS query log and add it to the whitelist;

[0189] Furthermore, in step (5-4), after receiving the update content, the security probe of the large scientific facility will extract its IP address and domain name, and perform dynamic blocking operations in the IPS, router and firewall, including the following specific steps:

[0190] Step (5-4-1): After receiving the updated content of the blacklist, the security probe of the large scientific facility extracts its IP address and domain name;

[0191] Step (5-4-2): According to the severity of the attack type corresponding to the IP address and domain name, set 6 blocking levels, namely one hour, one day, one week, one month, one year and permanent;

[0192] Step (5-4-3): According to the number of times the IP address and domain name have been blocked, increase the blocking level when it is blocked again;

[0193] Step (5-4-4): Perform blocking operations in IPS, routers, and firewalls according to the blocking level of IP addresses and domain names.

[0194] Step (6): Directly present the overall security status based on front-end and back-end modules and visualization technology;

[0195] In some embodiments of the present invention, the step (6) directly presents the overall security status based on the front-end and back-end modules and visualization technology, specifically including:

[0196] Step (6-1): Establish a control center at each large scientific facility site;

[0197] Step (6-2): In the back-end module of the central server module, the threat quantity, threat trend, threat source and other data of each control center are counted respectively to obtain real-time threat data, and the data is sent to the front-end module;

[0198] Step (6-3): The front-end module visualizes the real-time threat data and provides a page for system operation and maintenance personnel to manage;

[0199] Step (6-4): The control center obtains the visualized real-time threat data of the site from the front-end module and displays it on the display screen of the control center;

[0200] Step (6-5): Set up access control policies in the central server to ensure that the control center of one site cannot view the security status of other sites.

[0201] Furthermore, the step (6-2) counts the threat quantity, threat trend, threat source and other data of each control center in the back-end module of the central server, obtains real-time threat data, and sends the data to the front-end module, including the following specific steps:

[0202] Step (6-2-1): Build a Django backend module to automatically extract and count the security information of different sites from the security information storage module, and summarize it by site, threat type, threat quantity, and IP address of the threat source;

[0203] Step (6-2-2): For each site, collect statistics on threat type, threat quantity, number of automated responses, threat source, etc.

[0204] Step (6-2-3): Set the access control permissions for different site login users. Each site can only see the statistics of this site.

[0205] Step (6-2-4): Transmit real-time threat data to the Vue.js front-end module;

[0206] Furthermore, in step (6-3), the front-end module visualizes the real-time statistical data and provides a page for management for system operation and maintenance personnel, including the following specific steps:

[0207] Step (6-3-1): Build the Vue.js front-end module, create the login page, situation visualization page, asset page, and blacklist and whitelist page, and enable it to interact with the Django back-end module to receive real-time threat data;

[0208] Step (6-3-2): Use Echarts visualization technology to visualize the real-time threat data and display the results on the situation visualization page;

[0209] Furthermore, the step (6-3-2) uses Echarts visualization technology to visualize the real-time threat data and displays the results on the situation visualization page, including the following specific steps:

[0210] Step (6-3-2-1): Query the number of high-risk, medium-risk, and low-risk threats, and use Echarts line chart to display the threat trend in the past 24 hours;

[0211] Step (6-3-2-2): Query the real-time blocking number, and use Echarts line chart to visualize the real-time blocking number in the past 24 hours;

[0212] Step (6-3-2-3): Query the four attack types with the largest number of attacks on that day, and use Echarts pie chart to visualize the distribution of attack types on that day;

[0213] Step (6-3-2-4): Query the source IP address of the attack behavior, and use the Echarts bar chart to visualize the top five attack sources in terms of the number of attacks on that day;

[0214] Step (6-3-2-5): Query the security data collection performance, and use Echarts bar charts to visualize the real-time network traffic, server logs and other types of security data collection rates;

[0215] Step (6-3-2-6): Query real-time attack events and use Echarts map to visualize the attack trajectory;

[0216] Step (6-3-2-7): Query the real-time automated response events and use Echarts carousel chart to visualize the real-time attack handling situation;

[0217] Step (6-3-3): Obtain the site's asset information and blacklist and whitelist database through the Django backend module, and display them on the asset page and blacklist and whitelist page respectively, and allow system operation and maintenance personnel to view, search, delete, and add operations after logging in with the administrator account and corresponding password on the login page;

[0218] Furthermore, the step (6-4) in which the control center obtains the visualized real-time threat data of the site from the front-end module and presents it on the display screen of the control center includes the following specific steps:

[0219] Step (6-4-1): The control center enters the account and corresponding password of this site on the login page to log in;

[0220] Step (6-4-2): After successful login, jump to the situation visualization page of this site;

[0221] Step (6-4-3): Display the situation visualization page of this site on the display screen of the control center;

[0222] Furthermore, the step (6-5) sets an access control policy in the central server to ensure that the control center of one site cannot view the security status of other sites, including the following specific steps:

[0223] Step (6-5-1): Issue the site account and corresponding password for each site's control center to log in on the login page, and provide the administrator account and corresponding password to the system operation and maintenance personnel;

[0224] Step (6-5-2): Set up a jump mechanism after a successful login to the login page. After logging in with the site account and the corresponding password, you will be redirected to the situation visualization page corresponding to the site. After logging in with the administrator account and the corresponding password, you can freely select the page you want to view;

[0225] Step (6-5-3): Regularly update the site account and corresponding password, administrator account and corresponding password;

[0226] like Figure 2 As shown, an embodiment of the present invention further provides a system for network security situation awareness for large scientific facilities, which is used to implement a method for network security situation awareness for large scientific facilities provided by the present invention, and specifically includes a site module and a central server module;

[0227] Further, the site module includes a security probe module and a control center module;

[0228] Further, the central server module includes a security information preprocessing module, a security information storage module, a security information analysis module, a security information application module, a threat intelligence platform and a security situation visualization module;

[0229] Further, the security probe module includes a security information collection module and an automated response module;

[0230] Further, the control center module includes a security situation display module;

[0231] Furthermore, the security situation visualization module includes a front-end module module and a back-end module module.

[0232] Specifically, the security information collection module collects information from network devices, security devices and related assets of large scientific facilities to obtain initial security information;

[0233] Specifically, after receiving the updated blacklist content, the automated response module will extract its IP address and domain name, and perform dynamic blocking operations in the IPS, routers and firewalls;

[0234] Specifically, the security situation display module obtains the visualized real-time threat data of the site from the front-end module and presents it on the display screen of the control center;

[0235] Specifically, the security information preprocessing module performs real-time analysis, association and enrichment of the initial security information based on Kafka according to a well-defined structured format to obtain preprocessed security information;

[0236] Specifically, the security information storage module stores the pre-processed security information;

[0237] Specifically, the security information analysis module models and detects various types of security information from different angles based on a variety of behavior analysis algorithms to generate alarm information;

[0238] Specifically, the safety information application module processes the alarm information detected by data analysis and automatically generates response information, and sends the response information to safety probes of all large scientific facilities;

[0239] Specifically, the threat intelligence platform collects information from security-related open source blogs, news, and posts to obtain an initial security information block;

[0240] Specifically, the front-end module visualizes the real-time threat data and provides a page for system operation and maintenance personnel to manage;

[0241] Specifically, the back-end module counts the threat quantity, threat trend, threat source and other data of each control center respectively, obtains real-time threat data, and sends it to the front-end module after archiving and backing up.

[0242] In a third aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for network security situation awareness for large scientific facilities.

[0243] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned method for network security situation awareness for large scientific facilities.

[0244] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for network security situation awareness for large scientific facilities, characterized in that: The following steps are involved: Step (1): Use the security probe of each large scientific facility to collect data on the network equipment, security equipment and related assets of the large scientific facility and the open source network related to security to obtain initial security information; Step (2): according to the defined structured format, the initial security information is parsed, associated and enriched in real time based on Kafka to obtain pre-processed security information; Step (3): storing the pre-processed security information; Step (4): Based on a variety of behavior analysis algorithms, model and detect the pre-processed security information from different angles to generate alarm information; Step (5): Process the alarm information and automatically generate response information, and send the response information to the safety probes of all large scientific facilities; Step (6): Directly present the overall security status based on front-end and back-end modules and visualization technology.

2. A method for network security situation awareness for large scientific facilities according to claim 1, characterized in that: The step (1) comprises: Step (1-1): Collect the export mirror traffic from the export router to obtain relevant flow data information; Step (1-2): Collect mailbox server logs and DNS server logs from the application server to obtain relevant behavior data information; Steps (1-3): Collect computing node login logs and unified authentication login logs from key business systems to obtain relevant behavior data information; Steps (1-4): Collect network intrusion detection system logs, host intrusion detection system logs, Web application firewall logs, and intrusion prevention system logs from security devices to obtain relevant security event information; Steps (1-5): Collect website asset data and host asset data from the website and host to obtain relevant asset information; Steps (1-6): Extract threat information from security-related open source websites, collect threatening IP addresses and domain names and add them to the blacklist to obtain threat data information; Step (1-7): Use Filebeat to encrypt all security information collected from step (1-1) to step (1-6), and based on the security probe, transmit the encrypted information collected from step (1-1) to step (1-5) to the security information preprocessing module of the central server module, and transmit the encrypted information collected from step (1-6) to the security information storage module of the central server module.

3. The method for network security situation awareness for large scientific facilities according to claim 1, characterized in that: The step (2) comprises: Step (2-1): Use Logstash to receive and decrypt the initial security information sent by all large scientific facility sites to obtain the decrypted security information; Step (2-2): Output the decrypted security information to Kafka; Step (2-3): Define different structural formats of security information according to the requirements of the behavior analysis algorithm; Step (2-4): Read the decrypted security information from Kafka, parse it into structured data according to the type of the security information, and obtain the pre-processed original structured data; Step (2-5): performing an association operation on the preprocessed original structured data according to preset rules to form preprocessed behavioral structured data; Step (2-6): enrich the preprocessed behavioral structured data by associating it with external data sources to obtain preprocessed security information.

4. The method for network security situation awareness for large scientific facilities according to claim 1, characterized in that: The step (3) comprises: Step (3-1): Establish a relational database MySQL, a non-relational database Elasticsearch, and an in-memory database Redis on the security information storage module of the central server module; Step (3-2): receiving the pre-processed security information, and classifying the pre-processed security information according to its content and function, and storing them in different databases established in step (3-1).

5. The method for network security situation awareness for large scientific facilities according to claim 1, characterized in that: The step (4) comprises: Step (4-1): receiving security information output from different databases established in step (3-1) from the security information storage module, and performing data analysis according to the type of the security information, including DNS data analysis; account abnormal login and brute force cracking analysis; network traffic data analysis; and security device data analysis; Step (4-2): When additional behavioral analysis algorithms are required, they are integrated into the security analysis engine in the form of plug-ins.

6. A method for network security situation awareness for large scientific facilities according to claim 5, characterized in that: The DNS data analysis includes flood attack detection and abnormal IP detection; The abnormal account login and brute force cracking analysis includes login node account detection, SSO login account detection and email login account detection; The network traffic data analysis includes abnormal link number detection, unknown malicious traffic detection and unbalanced malicious traffic detection; The security device data analysis includes Web application firewall alarm analysis, network intrusion detection system alarm analysis, host intrusion detection system alarm analysis and intrusion prevention system alarm analysis.

7. The method for network security situation awareness for large scientific facilities according to claim 1, characterized in that: The step (5) comprises: Step (5-1): automatically responding to the alarm information according to a predefined threshold, and if the alarm information meets the predefined threshold and does not match the white list, entering the alarm information into the black list; Step (5-2): Manage the data in the whitelist according to the whitelist management policy; Step (5-3): When the blacklist is updated, the updated content is sent to the security probes of all large scientific facilities; Step (5-4): After receiving the updated blacklist, the security probe of the large scientific facility extracts the IP address and domain name of the alarm information and performs dynamic blocking operations in the IPS, router and firewall.

8. The method for network security situation awareness for large scientific facilities according to claim 1, characterized in that: The step (6) comprises: Step (6-1): Establish a control center at each large scientific facility site; Step (6-2): In the back-end module of the central server, the number of threats, threat trends and threat sources of each control center are counted respectively to obtain real-time threat data, which is then sent to the front-end module after archiving and backup; Step (6-3): The front-end module visualizes the real-time threat data and provides a page for system operation and maintenance personnel to manage; Step (6-4): The control center obtains the visualized real-time threat data of the site from the front-end module and displays it on the display screen of the control center; Step (6-5): Set up access control policies in the central server to ensure that the control center of one site cannot view the security status of other sites.

9. A system for network security situation awareness for large scientific facilities, used to implement the method for network security situation awareness for large scientific facilities as described in any one of claims 1 to 8, characterized in that: It includes multiple site modules and a central server module; each of the site modules includes a security probe module; the central server module includes a security information preprocessing module, a security information storage module, a security information analysis module, a security information application module, and a security situation visualization module; The security probe module is used to collect data on the network equipment, security equipment and related assets of the large scientific facility and the open source network related to security by using the security probe of each large scientific facility to obtain initial security information; The security information preprocessing module is used to parse, associate and enrich the initial security information in real time based on Kafka according to a defined structured format to obtain preprocessed security information; The security information storage module is used to store the pre-processed security information; The security information analysis module is used to model and detect the pre-processed security information from different angles based on a variety of behavior analysis algorithms to generate alarm information; The safety information application module is used to process the alarm information and automatically generate response information, and send the response information to the safety probes of all large scientific facilities; The security situation visualization module is used to directly present the overall security situation based on the front-end and back-end modules and visualization technology.

10. A system for network security situation awareness for large scientific facilities according to claim 9, characterized in that: The site module also includes a control center module, and the control center module includes a security situation display module; the security situation visualization module includes a front-end module and a back-end module; The security situation display module obtains the visualized real-time threat data of the site from the front-end module and presents it on the display screen of the control center; The front-end module visualizes the real-time threat data and provides a management page for system operation and maintenance personnel; The back-end module counts the threat quantity, threat trend and threat source data of each control center respectively, obtains real-time threat data, and sends it to the front-end module after archiving and backing up.

11. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; Among them, when one or more programs are executed by the one or more processors, the one or more processors implement the method for network security situation awareness for large scientific facilities as described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, which, when executed by a processor, enable the processor to implement a method for network security situation awareness for large scientific facilities as described in any one of claims 1-8.

Images