Internet data security monitoring system based on big data

Abstract

The invention discloses an internet data security monitoring system based on big data, particularly relates to the field of network security, and comprises a real-time monitoring and data acquisition module, a desensitization data preprocessing module, a security feature extraction module, a network situation analysis module, an attack detection module, a traceability tracking module and a security early warning response module. According to the Internet data security monitoring system based on big data, comprehensive acquisition of various data sources is realized through the real-time monitoring and data acquisition module, and the dynamic and comprehensive capability of the security monitoring system is improved; processing is carried out through a desensitization data preprocessing module according to a sensitive information classification standard, it is ensured that workers cannot directly make contact with original sensitive information in the follow-up processing process, and the safety and privacy of data are protected; third security information is obtained through the attack detection module, attack behaviors are judged more accurately, related details of attacks are determined, and potential security threats are found more timely and accurately.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and more specifically, to an Internet data security monitoring system based on big data. Background Art

[0002] With the rapid development of information technology, the Internet has been deeply integrated into all aspects of social life. With the explosive growth of data scale and the high complexity of data patterns, data security has become particularly important. Once data is subject to security threats such as leakage, tampering or malicious attacks, it will not only bring about privacy infringement and property loss, but may also cause serious negative impacts.

[0003] Existing big data security technologies have played an important role in reducing data security risks, assisting corporate compliance construction, and promoting data development and utilization. User and entity behavior anomaly detection based on generative adversarial networks has shown higher accuracy in experiments, better than the baseline model method and to a certain extent ensured the relative security of the network environment.

[0004] However, in actual use, it still has some shortcomings. For example, traditional security monitoring methods only detect known specific attack patterns and lack the ability to comprehensively, real-timely and dynamically monitor the entire Internet data ecosystem. It is difficult to efficiently extract valuable security-related information and conduct in-depth analysis, resulting in the discovery of potential security risks not being timely and accurate enough. The data is not desensitized, and staff are not protected from contact with sensitive data. In the event of a data leak, it is impossible to trace the source in a timely manner. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides an Internet data security monitoring system based on big data, which solves the problems raised in the above-mentioned background technology through the following scheme.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An Internet data security monitoring system based on big data, characterized in that it includes a system database, a system central processing module and a user information terminal, and also includes:

[0008] Real-time monitoring and data acquisition module: used for real-time monitoring through data acquisition tools to obtain first security information and transmit it to the desensitized data preprocessing module in real time;

[0009] Desensitized data preprocessing module: used for performing preprocessing operation on the first security information in the real-time monitoring and data acquisition module, the preprocessing operation is used for obtaining the first desensitized key feature corresponding to the first security information, and transmitting it to the security feature extraction module;

[0010] Security feature extraction module: used for performing feature recognition operation on the first desensitized key feature in the desensitized data preprocessing module, the feature recognition operation is used to obtain the second security information and transmit it to the network situation analysis module;

[0011] Network situation analysis module: used to obtain a network security situation analysis model, obtain a second key feature corresponding to the second security information through the second security information according to the network security situation analysis model, and transmit it to the attack detection module;

[0012] Attack detection module: used to make a preliminary judgment based on the second key feature corresponding to the second security information in the network situation analysis module, and obtain the intelligent attack detection model, obtain the third security information through the abnormal second key feature according to the network security situation analysis model, and transmit it to the tracing module;

[0013] Source tracing module: used to analyze the third security information in the attack detection module, use the source tracing algorithm to determine the third key features corresponding to the abnormal event, and transmit it to the security warning response module;

[0014] Security warning response module: used to start the security warning mechanism and make effective response actions in time based on the third key feature in the traceability module;

[0015] The system database includes all data texts of the Internet data security monitoring system and collects the information texts output by each module in real time. The system central processing module is used for the information text instructions output by each module in the central control system. The user information terminal is an information output device for receiving the Internet data security monitoring system.

[0016] Preferably, the real-time monitoring and data acquisition module is used to perform real-time monitoring through a data acquisition tool to obtain the first security information and transmit it to the desensitized data preprocessing module in real time;

[0017] Desensitized data preprocessing module: used for performing preprocessing operation on the first security information in the real-time monitoring and data acquisition module, the preprocessing operation is used for obtaining the first desensitized key feature corresponding to the first security information, and transmitting it to the security feature extraction module;

[0018] Security feature extraction module: used for performing feature recognition operation on the first desensitized key feature in the desensitized data preprocessing module, the feature recognition operation is used to obtain the second security information and transmit it to the network situation analysis module;

[0019] Network situation analysis module: used to obtain a network security situation analysis model, obtain a second key feature corresponding to the second security information through the second security information according to the network security situation analysis model, and transmit it to the attack detection module;

[0020] Attack detection module: used to make a preliminary judgment based on the second key feature corresponding to the second security information in the network situation analysis module, and obtain the intelligent attack detection model, obtain the third security information through the abnormal second key feature according to the network security situation analysis model, and transmit it to the tracing module;

[0021] Source tracing module: used to analyze the third security information in the attack detection module, use the source tracing algorithm to determine the third key features corresponding to the abnormal event, and transmit it to the security warning response module;

[0022] Security warning response module: used to start the security warning mechanism and make effective response actions in time based on the third key feature in the traceability module;

[0023] The system database includes all data texts of the Internet data security monitoring system and collects the information texts output by each module in real time. The system central processing module is used for the information text instructions output by each module in the central control system. The user information terminal is an information output device for receiving the Internet data security monitoring system.

[0024] Preferably, the real-time monitoring and data acquisition module obtains the first security information, specifically including:

[0025] Deploy network traffic collection tools at key network nodes. The network traffic collection tools capture all data packets entering and leaving the network, including the source IP address, destination IP address, port number, protocol type, packet size, and transmission time of each data packet;

[0026] Establish data connections with servers, operating systems, and applications associated with the Internet data security monitoring system to obtain system logs, which include system startup time, system shutdown time, user login records, user logout records, file access and modification records, and system error and warning information;

[0027] Data collection points are set up on the front-end and back-end servers of Internet applications to collect behavioral data during various Internet applications. The behavioral data includes users' web browsing history, online application usage behavior, and social media interaction behavior.

[0028] Preferably, the desensitized data preprocessing module has the following steps for preprocessing operations:

[0029] Data integrity check: Use hash check to check the received first security information to determine whether the integrity of the first security information is qualified;

[0030] Remove noise and erroneous data: Use data cleaning technology to identify and remove noise and erroneous data in the first security information;

[0031] Format unification processing: For structured data and semi-structured data, the data in different database tables are converted according to the specified field definitions and data types; for unstructured data, the text log contents in different formats are organized into text in a standard format;

[0032] Desensitization: Establish a sensitive information identification mechanism, conduct a comprehensive scan based on the sensitive information classification standards and identification rules stored in the preset security monitoring database, and use desensitization technology to process sensitive information;

[0033] In the preset security monitoring database, the desensitizing processing technology corresponding to the sensitive information classification standards and identification rules is obtained. The preset security monitoring database is used to store the correspondence between the sensitive information classification standards and identification rules and the desensitizing processing technology.

[0034] Preferably, in the desensitized data preprocessing module, the first desensitizing key feature includes IP address, network traffic ratio, user behavior, behavior operation frequency and usage logs of each application.

[0035] Preferably, the network situation analysis module obtains the second key feature corresponding to the second security information, specifically including:

[0036] The steps of obtaining the flow change rate in the second key feature by presetting the network security situation analysis model corresponding to the second security information in the security monitoring database are as follows:

[0037] Based on the current time t, extract the unit time period adjacent to time t, and calculate the size of the network traffic in the adjacent unit time period, which is specifically expressed as:

[0038] q1=-ln8×(q T-1 +q T-2 +q T ),

[0039] Among them, q1 represents the network traffic in the unit time period adjacent to the current time t, T represents the unit time period including the current time t, and q T-1 It is expressed as the network traffic in the first unit time period before the unit time period T, q T-2 It is expressed as the network traffic in the second unit time period before the unit time period T, q T It is expressed as the network traffic in the unit time period T;

[0040] q2=-ln8×(q T+1 +q T+2 +q T ),

[0041] Among them, q2 represents the network traffic in the unit time period adjacent to the current time t, T represents the unit time period including the current time t, and q T+1 It is expressed as the network traffic in the first unit time period after the unit time period T, q T+2 It is expressed as the network traffic in the second unit time period after the unit time period T, q T It is expressed as the network traffic in the unit time period T;

[0042] By comparing the size of network traffic in the previous and next unit time periods, the traffic change rate FCR at the current moment is calculated, which is specifically expressed as:

[0043] FCR=(|q2-q1| / q1)×100%,

[0044] Among them, q1 represents the network traffic in the adjacent unit time period to the left of the current time t, q2 represents the network traffic in the adjacent unit time period to the right of the current time t, and |q2-q1| represents the absolute value of the difference in network traffic in adjacent unit time periods.

[0045] Preferably, the attack detection module obtains the third security information, specifically including:

[0046] For the received second key features, a comparison analysis is performed one by one based on the judgment criteria;

[0047] After a single feature meets the benchmark, the interrelationships among multiple related features are comprehensively analyzed;

[0048] The second key feature that exceeds the normal range and does not conform to the normal behavior pattern is marked and extracted as the abnormal second key feature, input into the intelligent attack detection model to obtain the third security information, and transmitted to the security feature extraction module for training.

[0049] Preferably, the attack detection module obtains the third security information, specifically including:

[0050] According to the second key feature, an intelligent attack detection model corresponding to the second key feature in a preset security monitoring database is obtained, and the preset question-answer database is used to store the corresponding relationship between the second key feature and the intelligent attack detection model;

[0051] The third security information includes abnormal attack type, abnormal attack source, and abnormal attack situation;

[0052] The specific steps of obtaining the attack situation in the third security information by presetting the intelligent attack detection model corresponding to the second key feature in the security monitoring database are as follows:

[0053] Based on the total duration Ti of the time series and the observation value x at the corresponding time point in the time series t, calculate the average value xa of the time series data, which is specifically expressed as:

[0054]

[0055] Among them, ti represents the index of the time point in the time series;

[0056] Based on the average value xa of the time series data, the total duration Ti of the time series, and the observation value x at the corresponding time point in the time series t , calculate the attack situation index AT, which is specifically expressed as:

[0057]

[0058] Where ti represents the index of the time point in the time series, ta represents the average value of the time series, and its value is ((Ti+1) / 2);

[0059] When the AT value is greater than 0, the attack on the network is increasing, and emergency defense measures also need to be strengthened; when the AT value is equal to 0, the attack on the network tends to be stable. Emergency defense measures seize control; when the AT value is less than 0, the attack on the network is weakened, and emergency defense measures look for the source of intrusion.

[0060] Preferably, the tracing module determines the third key feature corresponding to the abnormal event, specifically including:

[0061] Inputting the third security information into a source tracing algorithm, wherein the source tracing algorithm includes an algorithm based on data association analysis, an algorithm based on network traffic analysis, and an algorithm based on log analysis;

[0062] The third key feature corresponding to the abnormal event is determined based on the algorithm of data association analysis, and the data set containing the third security information is set to D, and the multiple data elements contained in D are d i ;

[0063] Based on the set rules: When the attack type is a DDoS attack and the attack source is a specific IP address IP1, query the relevant network traffic changes and system log records, and compare them with the element data that meets the conditions in D;

[0064] The potential features of the network traffic change rate during the attack period and the abnormal events recorded in the system log are extracted. The potential features include the network connection status of a specific IP address before and after the attack, the resource utilization rate of the affected area in the network, and the change characteristics of the attack intensity over time.

[0065] To achieve the above object, the present invention provides the following technical solution: a method for Internet data security monitoring based on big data, implementing the above-mentioned Internet data security monitoring system based on big data, comprising:

[0066] S1: Real-time monitoring and collection: Real-time monitoring is performed through data collection tools to obtain the first safety information;

[0067] S2: Desensitization preprocessing: performing a preprocessing operation on the first security information, where the preprocessing operation is used to obtain a first desensitization key feature corresponding to the first security information;

[0068] S3: extract security features: perform feature recognition operation on the first desensitized key feature, and the feature recognition operation is used to obtain second security information;

[0069] S4: Analyze network situation: obtain a network security situation analysis model, and obtain a second key feature corresponding to the second security information through the second security information according to the network security situation analysis model;

[0070] S5: Attack detection: make a preliminary judgment based on the second key feature corresponding to the second security information, and obtain an intelligent attack detection model, and obtain third security information through the abnormal second key feature according to the network security situation analysis model;

[0071] S6: Trace the source of the abnormality: Analyze the third security information and use the traceability algorithm to determine the third key feature corresponding to the abnormal event;

[0072] S7: The safety warning mechanism takes response: Based on the third key feature, the safety warning mechanism is activated and effective response actions are taken in a timely manner.

[0073] Technical effects and advantages of the present invention:

[0074] 1. The present invention realizes comprehensive collection of multiple data sources through real-time monitoring and data acquisition modules, thereby improving the dynamic and comprehensive capabilities of the security monitoring system;

[0075] 2. The present invention processes the data according to the sensitive information classification standard through the desensitized data preprocessing module, ensuring that the staff cannot directly access the original sensitive information during the subsequent processing, thereby protecting the security and privacy of the data;

[0076] 3. The present invention obtains the third security information through the attack detection module, more accurately judges the attack behavior and determines the relevant details of the attack, thereby discovering potential security threats more timely and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a system flow chart of the present invention.

[0078] Figure 2 It is a system step diagram of the present invention.

[0079] Figure 3 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0082] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0083] As attached Figure 1 The Internet data security monitoring system based on big data shown in the figure includes a system database, a system central processing module and a user information terminal, and also includes a real-time monitoring and data acquisition module, a desensitized data preprocessing module, a security feature extraction module, a network situation analysis module, an attack detection module, a tracing module, and a security warning response module.

[0084] The system database includes all data texts of the Internet data security monitoring system and collects the information texts output by each module in real time. The system central processing module is used for the information text instructions output by each module in the central control system. The user information terminal is an information output device for receiving the Internet data security monitoring system.

[0085] The real-time monitoring and data acquisition module is used to perform real-time monitoring through data acquisition tools to obtain the first security information and transmit it to the desensitized data preprocessing module in real time.

[0086] Specifically, the Internet data security monitoring system monitors and collects data through corresponding monitoring and collection tools. The monitoring and collection tools include but are not limited to deploying network traffic collection tools at key network nodes, establishing connections with various servers and applications, and setting user behavior data collection points at the front and back ends of Internet applications, etc., to obtain the first security information. The first security information includes network traffic data, system log data, user behavior data, application data, and device information data.

[0087] This embodiment does not limit the type of monitoring and collection tools and the command input method. When performing real-time monitoring and data collection operations, multiple monitoring columns are constructed for different data types or monitoring areas, and corresponding collection and monitoring work is performed in each monitoring column. Each monitoring column exists independently, and the collection and monitoring operations in the same monitoring column will not affect the operations in other monitoring columns. In this embodiment, one monitoring column will be used as an example for explanation.

[0088] In a possible implementation, obtaining the first security information includes: deploying a network traffic collection tool at key network nodes, the key network nodes including but not limited to routers, firewalls, and core switches; the network traffic collection tool captures all data packets entering and leaving the network, including the source IP address, destination IP address, port number, protocol type, data packet size, and transmission time of each data packet;

[0089] Establish data connections with servers, operating systems, and applications associated with the Internet data security monitoring system to obtain system logs, which include system startup time, system shutdown time, user login records, user logout records, file access and modification records, and system error and warning information;

[0090] Data collection points are set up on the front-end and back-end servers of Internet applications to collect behavioral data during various Internet applications. The behavioral data includes users' web browsing history, online application usage behavior, and social media interaction behavior.

[0091] In one possible implementation, obtaining the first security information includes: analyzing the data type obtained by the monitoring and collection tool and setting corresponding monitoring thresholds and rules based on the historical data of a preset security monitoring database; when the data indicators obtained in real time exceed the threshold or violate the rules, the data obtained in real time is immediately transmitted to the desensitized data preprocessing module.

[0092] In this embodiment, the network traffic data is associated with the maximum allowed packet sending threshold of a specific IP address within a unit time; the user behavior data is associated with the maximum login number threshold of a single user account in a day.

[0093] In this embodiment, the violation of the rules includes comparing the current number of system log errors of the server with the average number of errors in the past week, and if there is a significant increase, the real-time acquired data is immediately transmitted to the desensitized data preprocessing module.

[0094] The desensitized data preprocessing module is used to perform preprocessing operations on the first security information in the real-time monitoring and data acquisition module. The preprocessing operation is used to obtain the first desensitized key feature corresponding to the first security information and transmit it to the security feature extraction module.

[0095] Specifically, after acquiring the first security information transmitted by the real-time monitoring and data acquisition module, the Internet data security monitoring system can obtain multiple key features corresponding to the first security information through preprocessing operations, namely, the first desensitized key feature corresponding to the first security information.

[0096] In a possible implementation, the steps of the preprocessing operation are as follows:

[0097] Data integrity check: Use hash check to check the received first security information to determine whether the integrity of the first security information is qualified;

[0098] In this embodiment, if the first security information fails to pass the data verification, a retransmission request is sent to the real-time monitoring and data acquisition module to supplement the missing data content;

[0099] Remove noise and erroneous data: Use data cleaning technology to identify and remove noise and erroneous data in the first security information;

[0100] In this embodiment, there are abnormal data packets in the network traffic data due to network waves, and the field values ​​in the data packets do not conform to normal network protocol specifications. The abnormal data is identified and removed based on the IP address format and port number range according to the data cleaning technology;

[0101] Format unification processing: For structured data and semi-structured data, the data in different database tables are converted according to the specified field definitions and data types; for unstructured data, the text log contents in different formats are organized into text in a standard format;

[0102] In this embodiment, when unifying the semi-structured data structure, XML format data is converted into JSON format, and then the JSON data of different structures are adjusted to a unified standard structure;

[0103] Desensitization: Establish a sensitive information identification mechanism, conduct a comprehensive scan based on the sensitive information classification standards and identification rules stored in the preset security monitoring database, and use desensitization technology to process sensitive information;

[0104] Specifically, in a preset security monitoring database, a desensitization processing technology corresponding to the sensitive information classification standard and identification rule is obtained, and the preset security monitoring database is used to store the corresponding relationship between the sensitive information classification standard and identification rule and the desensitization processing technology;

[0105] In this embodiment, a replacement method is used to randomly replace some positions in digital sensitive information with virtual values; a mask method is used to replace the text-type sensitive credit-specific symbol "*".

[0106] In one possible implementation, obtaining the first desensitized key feature corresponding to the first security information includes: using feature engineering technology to extract key indicators to obtain the first desensitized key feature corresponding to the first security information, the first desensitized key feature including IP address, network traffic ratio, user behavior, behavior operation frequency, and usage logs of each application.

[0107] The security feature extraction module is used to perform feature recognition operation on the first desensitizing key feature in the desensitizing data preprocessing module. The feature recognition operation is used to obtain the second security information and transmit it to the network situation analysis module.

[0108] Specifically, after obtaining the first desensitized key feature corresponding to the first security information transmitted by the desensitized data preprocessing module, the Internet data security monitoring system can obtain the second security information through a feature recognition operation. The second security information includes abnormal behavior of each IP address, abnormal operation of user accounts, and potential security vulnerability characteristics of each application.

[0109] In one possible implementation, obtaining the second security information includes: based on historical security monitoring data and various known security threats stored in a preset security monitoring database, constructing a corresponding feature recognition rule library for the first desensitizing key feature, the feature recognition rule library being a target feature recognition rule corresponding to the first desensitizing key feature, the target feature recognition rule including a decision tree learning model, a support vector machine learning model, and a neural network learning model; in the preset security monitoring database, obtaining a feature recognition rule library corresponding to the first desensitizing key feature, the preset security monitoring database being used to save the correspondence between the first desensitizing key feature and the feature recognition rule library.

[0110] In this embodiment, the characteristics of network traffic data are extracted based on identifying a specific IP address that continuously sends a large number of data packets of the same size in a short period of time; the characteristics of user behavior data are extracted by identifying user accounts that frequently log in and perform abnormal operations during abnormal time periods.

[0111] The network situation analysis module is used to obtain a network security situation analysis model, obtain a second key feature corresponding to the second security information through the second security information according to the network security situation analysis model, and transmit it to the attack detection module.

[0112] Specifically, the network security situation analysis model is a pre-constructed learning model. By inputting the second security information into the network security situation analysis model, the network security situation analysis model obtains the second key feature according to the second security information and sends the second key feature to the attack detection module.

[0113] In a possible implementation, obtaining the second key feature corresponding to the second security information includes: obtaining, according to the second security information, a network security situation analysis model corresponding to the second security information in a preset security monitoring database, wherein the preset question-and-answer database is used to store the corresponding relationship between the second security information and the network security situation analysis model;

[0114] Specifically, the second key features include traffic change rate, specific IP traffic pattern, protocol traffic ratio, abnormal operation frequency, abnormal behavior time series, abnormal login time and location, and abnormal system logs.

[0115] It should be noted that the specific steps of obtaining the flow change rate in the second key feature by presetting the network security situation analysis model corresponding to the second security information in the security monitoring database are as follows:

[0116] Based on the current time t, extract the unit time period adjacent to time t, and calculate the size of the network traffic in the adjacent unit time period, which is specifically expressed as:

[0117] q1=-ln8×(q T-1 +q T-2 +q T ),

[0118] Among them, q1 represents the network traffic in the unit time period adjacent to the current time t, T represents the unit time period including the current time t, and q T-1 It is expressed as the network traffic in the first unit time period before the unit time period T, q T-2 It is expressed as the network traffic in the second unit time period before the unit time period T, q T It is expressed as the network traffic in the unit time period T;

[0119] q2=-ln8×(q T+1 +q T+2 +q T ),

[0120] Among them, q2 represents the network traffic in the unit time period adjacent to the current time t, T represents the unit time period including the current time t, and q T+1 It is expressed as the network traffic in the first unit time period after the unit time period T, q T+2 It is expressed as the network traffic in the second unit time period after the unit time period T, q TIt is expressed as the network traffic in the unit time period T;

[0121] By comparing the size of network traffic in the previous and next unit time periods, the traffic change rate FCR at the current moment is calculated, which is specifically expressed as:

[0122] FCR=(|q2-q1| / q1)×100%,

[0123] Among them, q1 represents the network traffic in the adjacent unit time period to the left of the current time t, q2 represents the network traffic in the adjacent unit time period to the right of the current time t, and |q2-q1| represents the absolute value of the difference in network traffic in adjacent unit time periods.

[0124] The attack detection module is used to make a preliminary judgment based on the second key feature corresponding to the second security information in the network situation analysis module, and obtain the intelligent attack detection model. According to the network security situation analysis model, the third security information is obtained through the abnormal second key feature and transmitted to the tracing module.

[0125] Specifically, after acquiring the second key feature transmitted by the network situation analysis module, the Internet data security monitoring system extracts the abnormal second key feature through preliminary judgment.

[0126] In a possible implementation, obtaining the third security information includes: performing a comparison analysis one by one based on a judgment reference for the received second key features;

[0127] In this embodiment, taking the network traffic change rate as an example, if the traffic change rate received at time t is 20%, and the normal threshold range set by the judgment benchmark is [-10%, 10%], it is preliminarily judged that the network traffic situation corresponding to this feature may be abnormal;

[0128] After a single feature meets the benchmark, the interrelationships among multiple related features are comprehensively analyzed;

[0129] In this embodiment, when it is found that the user operation frequency increases and the behavior sequence also obviously does not conform to normal logic, it is preliminarily determined that an abnormal situation exists;

[0130] The second key feature that exceeds the normal range and does not conform to the normal behavior pattern is marked and extracted as the abnormal second key feature, input into the intelligent attack detection model to obtain the third security information, and transmitted to the security feature extraction module for training.

[0131] Specifically, the intelligent attack detection model is a pre-constructed learning model. By inputting the second key feature into the intelligent attack detection model, the intelligent attack detection model obtains the third security information according to the second key feature and sends the third security information to the attack detection module.

[0132] In a possible implementation, obtaining the third security information further includes: obtaining, according to the second key feature, an intelligent attack detection model corresponding to the second key feature in a preset security monitoring database, wherein the preset question-and-answer database is used to store a correspondence between the second key feature and the intelligent attack detection model;

[0133] Specifically, the third security information includes abnormal attack type, abnormal attack source, and abnormal attack situation;

[0134] It should be noted that the specific steps of obtaining the attack situation in the third security information by presetting the intelligent attack detection model corresponding to the second key feature in the security monitoring database are as follows:

[0135] Based on the total duration Ti of the time series and the observation value x at the corresponding time point in the time series t , calculate the average value xa of the time series data, which is specifically expressed as:

[0136]

[0137] Among them, ti represents the index of the time point in the time series;

[0138] Based on the average value xa of the time series data, the total duration Ti of the time series, and the observation value x at the corresponding time point in the time series t , calculate the attack situation index AT, which is specifically expressed as:

[0139]

[0140] Where ti represents the index of the time point in the time series, ta represents the average value of the time series, and its value is ((Ti+1) / 2);

[0141] It should be noted that when the AT value is greater than 0, the attack on the network is increasing, and emergency defense measures also need to be strengthened; when the AT value is equal to 0, the attack on the network tends to be stable. Emergency defense measures seize control; when the AT value is less than 0, the attack on the network is weakened, and emergency defense measures look for the source of intrusion.

[0142] The source tracing module is used to analyze the third security information in the attack detection module, use the source tracing algorithm to determine the third key feature corresponding to the abnormal event, and transmit it to the security warning response module.

[0143] In a possible implementation, determining the third key feature corresponding to the abnormal event includes: inputting the third security information into a source tracing algorithm, where the source tracing algorithm includes an algorithm based on data association analysis, an algorithm based on network traffic analysis, and an algorithm based on log analysis;

[0144] It should be noted that the third key feature corresponding to the abnormal event is determined based on the algorithm of data association analysis. Suppose the data set included in the third security information is D, and the multiple data elements included in D are d i (i=1, 2, ..., n);

[0145] Based on the set rules: When the attack type is a DDoS attack and the attack source is a specific IP address IP1, query the relevant network traffic changes and system log records, and compare them with the element data that meets the conditions in D; and extract the network traffic change rate during the attack time period and the potential characteristics of abnormal events recorded in the system log. The potential characteristics include but are not limited to the network connection status of the specific IP address before and after the attack, the resource utilization rate of the affected area in the network, and the change characteristics of the attack intensity over time.

[0146] The security warning response module is used to activate the security warning mechanism and make effective response actions in a timely manner based on the third key feature in the traceability module.

[0147] Specifically, based on the third key feature, the warning level of the abnormal event is determined; according to the warning level, corresponding warning information is generated, and the warning information includes the specific attack type of the security incident, the warning level, the affected area and application, and the attack situation analysis results; the warning information is released through various channels, and the channels for releasing the warning information include but are not limited to: internal notification of the system to network security management personnel, operation and maintenance personnel, sending emails or text messages to relevant persons in charge, and popping up a prominent warning box on the interface of the relevant monitoring system.

[0148] In this embodiment, different warning levels are set to correspond to different risk intervals. When the attack type is a high-risk DDoS attack, the attack source is clear and the affected range is wide, and the attack trend is on an upward trend, it is determined as a high-level warning; when the attack scale is small and the impact range is limited, it is determined as a low-level warning.

[0149] As attached Figure 2 The method for Internet data security monitoring based on big data shown includes: S1: real-time monitoring and collection, S2: desensitization preprocessing, S3: extraction of security features, S4: analysis of network situation, S5: attack detection, S6: tracing the source of abnormalities, and S7: security warning mechanism taking response.

[0150] S1: Real-time monitoring and collection: Real-time monitoring is performed through data collection tools to obtain the first safety information;

[0151] S2: Desensitization preprocessing: performing a preprocessing operation on the first security information, where the preprocessing operation is used to obtain a first desensitization key feature corresponding to the first security information;

[0152] S3: extract security features: perform feature recognition operation on the first desensitized key feature, and the feature recognition operation is used to obtain second security information;

[0153] S4: Analyze network situation: obtain a network security situation analysis model, and obtain a second key feature corresponding to the second security information through the second security information according to the network security situation analysis model;

[0154] S5: Attack detection: make a preliminary judgment based on the second key feature corresponding to the second security information, and obtain an intelligent attack detection model, and obtain third security information through the abnormal second key feature according to the network security situation analysis model;

[0155] S6: Trace the source of the abnormality: Analyze the third security information and use the traceability algorithm to determine the third key feature corresponding to the abnormal event;

[0156] S7: The safety warning mechanism takes response: Based on the third key feature, the safety warning mechanism is activated and effective response actions are taken in a timely manner.

[0157] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0158] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An Internet data security monitoring system based on big data, comprising a system database, a system central processing module and a user information terminal, characterized in that: Also includes: Real-time monitoring and data acquisition module: used for real-time monitoring through data acquisition tools to obtain first security information and transmit it to the desensitized data preprocessing module in real time; Desensitized data preprocessing module: used for performing preprocessing operation on the first security information in the real-time monitoring and data acquisition module, the preprocessing operation is used for obtaining the first desensitized key feature corresponding to the first security information, and transmitting it to the security feature extraction module; Security feature extraction module: used for performing feature recognition operation on the first desensitized key feature in the desensitized data preprocessing module, the feature recognition operation is used to obtain the second security information and transmit it to the network situation analysis module; Network situation analysis module: used to obtain a network security situation analysis model, obtain a second key feature corresponding to the second security information through the second security information according to the network security situation analysis model, and transmit it to the attack detection module; Attack detection module: used to make a preliminary judgment based on the second key feature corresponding to the second security information in the network situation analysis module, and obtain the intelligent attack detection model, obtain the third security information through the abnormal second key feature according to the network security situation analysis model, and transmit it to the tracing module; Source tracing module: used to analyze the third security information in the attack detection module, use the source tracing algorithm to determine the third key features corresponding to the abnormal event, and transmit it to the security warning response module; Security warning response module: used to start the security warning mechanism and make effective response actions in time based on the third key feature in the traceability module; The system database includes all data texts of the Internet data security monitoring system and collects the information texts output by each module in real time. The system central processing module is used for the information text instructions output by each module in the central control system. The user information terminal is an information output device for receiving the Internet data security monitoring system.

2. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The real-time monitoring and data acquisition module obtains the first security information, specifically including: Deploy network traffic collection tools at key network nodes. The network traffic collection tools capture all data packets entering and leaving the network, including the source IP address, destination IP address, port number, protocol type, packet size, and transmission time of each data packet; Establish data connections with servers, operating systems, and applications associated with the Internet data security monitoring system to obtain system logs, which include system startup time, system shutdown time, user login records, user logout records, file access and modification records, and system error and warning information; Data collection points are set up on the front-end and back-end servers of Internet applications to collect behavioral data during various Internet applications. The behavioral data includes users' web browsing history, online application usage behavior, and social media interaction behavior.

3. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The desensitized data preprocessing module has the following steps for preprocessing: Data integrity check: Use hash check to check the received first security information to determine whether the integrity of the first security information is qualified; Remove noise and erroneous data: Use data cleaning technology to identify and remove noise and erroneous data in the first security information; Format unification processing: For structured data and semi-structured data, the data in different database tables are converted according to the specified field definitions and data types; For unstructured data, organize text log contents in different formats into text in a standard format; Desensitization: Establish a sensitive information identification mechanism, conduct a comprehensive scan based on the sensitive information classification standards and identification rules stored in the preset security monitoring database, and use desensitization technology to process sensitive information; In the preset security monitoring database, the desensitizing processing technology corresponding to the sensitive information classification standards and identification rules is obtained. The preset security monitoring database is used to store the correspondence between the sensitive information classification standards and identification rules and the desensitizing processing technology.

4. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The desensitized data preprocessing module, the first desensitized key features include IP address, network traffic ratio, user behavior, behavior operation frequency and usage logs of each application.

5. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The network situation analysis module obtains the second key feature corresponding to the second security information, specifically including: The steps of obtaining the flow change rate in the second key feature by presetting the network security situation analysis model corresponding to the second security information in the security monitoring database are as follows: Based on the current time t, extract the unit time period adjacent to time t, and calculate the size of the network traffic in the adjacent unit time period, which is specifically expressed as: q1=-ln8×(q T-1 +q T-2 +q T ), Among them, q1 represents the network traffic in the unit time period adjacent to the current time t, T represents the unit time period including the current time t, and q T-1 It is expressed as the network traffic in the first unit time period before the unit time period T, q T-2 It is expressed as the network traffic in the second unit time period before the unit time period T, q T It is expressed as the network traffic in the unit time period T; q2=-ln8×(q T+1 +q T+2 +q T ), Among them, q2 represents the network traffic in the unit time period adjacent to the current time t, T represents the unit time period including the current time t, and q T+1 It is expressed as the network traffic in the first unit time period after the unit time period T, q T+2 It is expressed as the network traffic in the second unit time period after the unit time period T, q T It is expressed as the network traffic in the unit time period T; By comparing the size of network traffic in the previous and next unit time periods, the traffic change rate FCR at the current moment is calculated, which is specifically expressed as: FCR=(|q2-q1| / q1)×100%, Among them, q1 represents the network traffic in the adjacent unit time period to the left of the current time t, q2 represents the network traffic in the adjacent unit time period to the right of the current time t, and |q2-q1| represents the absolute value of the difference in network traffic in adjacent unit time periods.

6. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The attack detection module obtains the third security information, specifically including: For the received second key features, a comparison analysis is performed one by one based on the judgment criteria; After a single feature meets the benchmark, the interrelationships among multiple related features are comprehensively analyzed; The second key feature that exceeds the normal range and does not conform to the normal behavior pattern is marked and extracted as the abnormal second key feature, input into the intelligent attack detection model to obtain the third security information, and transmitted to the security feature extraction module for training.

7. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The attack detection module obtains the third security information, specifically including: According to the second key feature, an intelligent attack detection model corresponding to the second key feature in a preset security monitoring database is obtained, and the preset question-answer database is used to store the corresponding relationship between the second key feature and the intelligent attack detection model; The third security information includes abnormal attack type, abnormal attack source, and abnormal attack situation; The specific steps of obtaining the attack situation in the third security information by presetting the intelligent attack detection model corresponding to the second key feature in the security monitoring database are as follows: Based on the total duration Ti of the time series and the observation value x at the corresponding time point in the time series t , calculate the average value xa of the time series data, which is specifically expressed as: Among them, ti represents the index of the time point in the time series; Based on the average value xa of the time series data, the total duration Ti of the time series, and the observation value x at the corresponding time point in the time series t , calculate the attack situation index AT, which is specifically expressed as: Among them, ti represents the index of the time point in the time series, ta represents the average value of the time series, and its value is ((Ti+1) / 2); When the AT value is greater than 0, the attack on the network is increasing, and emergency defense measures also need to be strengthened; when the AT value is equal to 0, the attack on the network tends to be stable. Emergency defense measures seize control; when the AT value is less than 0, the attack on the network is weakened, and emergency defense measures look for the source of intrusion.

8. The Internet data security monitoring system based on big data according to claim 1 is characterized in that: The tracing module determines the third key feature corresponding to the abnormal event, specifically including: Inputting the third security information into a source tracing algorithm, the source tracing algorithm includes an algorithm based on data association analysis, an algorithm based on network traffic analysis, and an algorithm based on log analysis; The third key feature corresponding to the abnormal event is determined based on the algorithm of data association analysis, and the data set containing the third security information is set to D, and the multiple data elements contained in D are d i ; Based on the set rules: When the attack type is a DDoS attack and the attack source is a specific IP address IP1, query the relevant network traffic changes and system log records, and compare them with the element data that meets the conditions in D; The potential features of the network traffic change rate during the attack period and the abnormal events recorded in the system log are extracted. The potential features include the network connection status of a specific IP address before and after the attack, the resource utilization rate of the affected area in the network, and the change characteristics of the attack intensity over time.

Images