A network information security analysis method and system based on data analysis

Abstract

The present invention discloses a network information security analysis method and system based on data analysis, which relates to the field of network security technology. When the system is running, it realizes comprehensive monitoring and intelligent protection of network security, collects visitor's network traffic, logs, user behavior and system event data in real time, and forms a complete data set D. Through data denoising, filling missing values ​​and data format conversion preprocessing steps, the feature extraction module further extracts key features from network traffic, logs, user behavior and system events to form a feature data set X. features ,The behavior analysis module uses machine learning algorithms to establish a baseline model, ,obtains the matching result of the anomaly evaluation index S and the preset threshold Y through ,training and analysis of historical data and feature data sets, formulates ,visitor anomaly evaluation strategy, and implements corresponding access restrictions and ,notification measures to eliminate the protection blind spots of a single device and ,achieve comprehensive network monitoring.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a network information security analysis method and system based on data analysis. Background Art

[0002] Network security refers to the technology and practice of protecting computer systems and networks from information leakage, theft, damage and attack. Specifically, network security covers multiple aspects such as data protection, identity authentication, access control, intrusion detection and response. In the specific field of network security, behavior analysis and anomaly detection is an important research direction. This direction improves network security by monitoring and analyzing the behavior of users and devices and identifying abnormal activities.

[0003] In the specific application of behavior analysis and anomaly detection, existing methods mainly rely on predefined rules and signatures to identify abnormal behaviors. The disadvantage of this method is that it is difficult to deal with new and unknown threats because new threats usually do not match existing rules and signatures. In addition, the maintenance and updating of rules and signatures requires a lot of manpower and time, which limits the response speed and flexibility of the system.

[0004] Moreover, the current situation and shortcomings are mainly caused by the complexity of the network environment and the continuous evolution of attack methods. When the system cannot effectively detect and respond to abnormal behaviors, attackers may exploit these vulnerabilities to carry out malicious activities, resulting in serious consequences such as data leakage, service interruption and system damage. Especially in the face of persistent threat attacks, traditional detection methods are particularly weak. The high false alarm rate will cause the security team to be exhausted in dealing with a large number of irrelevant alarms, while the high false alarm rate means that the real threat may be ignored, posing a great hidden danger to system security. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies in the prior art, the present invention provides a network information security analysis method and system based on data analysis, which solves the problems mentioned in the background technology.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: a network information security analysis system based on data analysis, including a data acquisition module, a data preprocessing module, a feature extraction module, a behavior analysis module and an anomaly detection module;

[0009] The data collection module collects data in real time through system logs and security event management tools, including collecting visitor network traffic data, log data, user behavior data and system event data to form a collection data set D;

[0010] The data preprocessing module preprocesses the collected data set D, including data denoising preprocessing, missing value filling preprocessing and data format conversion preprocessing, to form a processed data set ;

[0011] The feature extraction module extracts data from the dataset Perform feature extraction, including network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, to form a feature dataset X features ;

[0012] The behavior analysis module uses a machine learning algorithm to establish a baseline model by combining the feature data set X features Input the baseline model for training and analysis to obtain the degree of deviation from the normal behavior pattern: the abnormality assessment index S;

[0013] The anomaly detection module matches the anomaly assessment index S with a preset anomaly assessment threshold Y, obtains the visitor anomaly assessment strategy scheme, and executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy scheme, including visitor access restrictions and relevant personnel notifications.

[0014] Preferably, the data collection module includes a log collection unit and a flow collection unit;

[0015] The log collection unit is responsible for collecting log data XL, user behavior data XU and system event data XE from system logs and security event management tools in real time, including real-time monitoring and collection of log data and centralized collection and management of security events by using log collection tools and configuring SIEM;

[0016] Among them, log data includes collecting server, application, operating system, security device and database log files;

[0017] User behavior data includes collecting users’ login time, operation behavior and access records;

[0018] System event data includes all operating events of the operating system and errors, warnings, and other related system events of applications;

[0019] The traffic collection unit collects the visitor's network traffic data XF in real time, including data packet size, transmission rate, connection duration and protocol type;

[0020] The data packet size is captured and collected through traffic mirroring and network probes, including statistics on the average, maximum, and minimum data packet size of each connection;

[0021] The collected data set D specifically includes event type frequency, event time distribution, event source IP, login frequency, operation type frequency, access resource type, system event type frequency, CPU and memory usage, error and warning events, data packet size, transmission rate, connection duration and protocol type.

[0022] Preferably, the data preprocessing module includes a denoising unit and a correction unit;

[0023] The denoising unit performs data denoising preprocessing on the collected data set D using filtering rules, statistical methods and machine learning detection, including log data denoising to filter out system messages and redundant records that are not related to network security; user behavior data denoising to remove misoperation and invalid access related operation records; system event data denoising to remove irrelevant system events, system warnings and system related event reminders; network traffic data denoising to filter out invalid data packets and duplicate data packets;

[0024] The correction unit performs missing value filling preprocessing and data format conversion preprocessing on the collected data set D after denoising preprocessing to form a processed data set , including a subset of network traffic data , Log data subset , User behavior data subset and a subset of system event data , specifically including packet average, transmission rate, protocol type frequency, event type frequency, event time distribution, login frequency, operation type frequency, system event type frequency, CPU usage, and error and warning event frequency;

[0025] Among them, data format conversion includes time format conversion, encoding conversion and data type conversion; filling missing values ​​includes mean filling, median filling, nearest neighbor filling and interpolation filling.

[0026] Preferably, the feature extraction module includes a network and log extraction unit and a behavior and event extraction unit;

[0027] The network and log extraction unit performs a Perform feature extraction, including network traffic feature extraction XF: packet average value ASize, transmission rate TRate and protocol type frequency PFReq; log feature extraction XL: event type frequency ETFreq and event time distribution ETDist;

[0028] The behavior and event extraction unit performs a Perform feature extraction, including behavior feature extraction XU: user login frequency LFreq and operation type frequency ATFreq; system event feature extraction XE: system event type frequency SETFreq, CPU usage CUsage, error and warning event frequency EWFreq;

[0029] The feature dataset X is composed by integrating the relevant features extracted by network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE. features .

[0030] Preferably, the data packet average value ASize is obtained by calculating the average value of all data packet sizes in the network connection; the transmission rate TRate is obtained by calculating the number of transmissions in a fixed time; the protocol type frequency PFReq is obtained by counting the usage frequency of different protocol types; the event type frequency ETFreq is obtained by counting the occurrence frequency of different types of events; and the event time distribution ETDist is obtained by counting the distribution frequency of events in each hour segment of a natural day.

[0031] Preferably, the user login frequency LFreq is obtained by counting the ratio of the number of logins on a natural day to the total number of logins; the operation type frequency ATFreq is obtained by counting the frequency of users executing different types of operation sequence numbers; the system event type frequency SETFreq is obtained by counting the occurrence frequency of different types of system event sequence numbers; the CPU utilization rate CUsage is obtained by counting the CPU utilization of the system in different time periods; the error and warning event frequency EWFreq is obtained by counting the frequency of error events and warning times occurring within a fixed period.

[0032] Preferably, the behavior analysis module includes a modeling unit;

[0033] The modeling unit establishes a baseline model M by using a machine learning algorithm, including selecting a mixed Gaussian model as the baseline model M by features Input the baseline model M for training and analysis to obtain the feature dataset X features The degree of deviation of several data points from the normal behavior pattern: the abnormal evaluation index S, which evaluates the state of the visitor's behavior deviation from the normal pattern;

[0034] Among them, the feature dataset X features Specifically: X features ={XF,XL,XU,XE};

[0035] The abnormality assessment index S is obtained by the following calculation formula:

[0036] ;

[0037] In the formula, S represents the anomaly evaluation index, which specifically represents the feature data set X features abnormal performance, Represents the feature dataset X features The probability under the baseline model M specifically represents the weighted combination of Gaussian distributions of the mixed Gaussian model to fit the data, and log represents the natural logarithm function.

[0038] Preferably, the formula for fitting the data using a weighted combination of Gaussian distributions is as follows:

[0039] ;

[0040] In the formula, Represents the feature dataset X features The probability distribution of K represents the number of Gaussian components, specifically the number of Gaussian distributions in the mixed Gaussian model. Represents the weight of the k-th Gaussian component, specifically the contribution ratio of the k-th Gaussian component in the entire mixed Gaussian model, satisfying , Represents the normal distribution of the kth Gaussian component, specifically used to describe the feature data set X features The probability density under the kth Gaussian component is, represents the mean vector of the k-th Gaussian component, specifically the center position of the k-th Gaussian component, Represents the covariance matrix of the k-th Gaussian component, specifically indicating the distribution range and direction of the k-th Gaussian component;

[0041] Among them, the normal distribution of the kth Gaussian component The formula is as follows:

[0042] ;

[0043] In the formula, d represents the feature dataset X features The dimension of the feature dataset X features The number of Represents the covariance matrix The determinant of , specifically the covariance matrix The scale of Represents the feature dataset X features Subtract the mean vector After the transposition, Represents the covariance matrix The inverse matrix of , exp represents the exponential function, specifically the exponential part of the Gaussian distribution of the mixed Gaussian model.

[0044] Preferably, the anomaly detection module includes a matching unit and a response unit;

[0045] The matching unit matches the preset relevant information with the required comparison value, including matching the abnormal evaluation index S with the preset abnormal evaluation threshold Y, to obtain the visitor abnormal evaluation strategy solution;

[0046] Abnormal evaluation index S < abnormal evaluation threshold Y, the visitor has no abnormal evaluation results, and the current visitor is not restricted or notified of monitoring related personnel;

[0047] Abnormal evaluation index S ≥ abnormal evaluation threshold Y, obtain the visitor abnormal evaluation results, restrict the current visitor and monitor the notification of relevant personnel;

[0048] Abnormal evaluation index S ≥ twice the abnormal evaluation threshold Y, obtain the visitor warning evaluation results, freeze the current visitor's behavior and operations, and execute relevant emergency security management personnel notifications;

[0049] The response unit executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy plan, including visitor access restrictions and relevant personnel notifications. Access restrictions include blocking visitor login, ending sessions, restricting access to resources, restricting access to directories, multi-factor authentication and restricting visitor permissions. Notifications include security administrator notifications and network monitoring team notifications.

[0050] A network information security analysis method based on data analysis includes the following steps:

[0051] Step 1: The data collection module collects data in real time through system logs and security event management tools, including collecting visitor network traffic data, log data, user behavior data and system event data to form a collection data set D;

[0052] Step 2: The data preprocessing module preprocesses the collected data set D, including data denoising preprocessing, missing value filling preprocessing and data format conversion preprocessing, to form a processed data set ;

[0053] Step 3: Feature extraction module for data set Perform feature extraction, including network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, to form a feature dataset X features ;

[0054] Step 4: The behavior analysis module uses machine learning algorithms to build a baseline model by features Input the baseline model for training and analysis to obtain the degree of deviation from the normal behavior pattern: the abnormality assessment index S;

[0055] Step 5: The anomaly detection module matches the anomaly assessment index S with the preset anomaly assessment threshold Y, obtains the visitor anomaly assessment strategy plan, and executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy plan, including visitor access restrictions and relevant personnel notifications.

[0056] (III) Beneficial effects

[0057] The present invention provides a network information security analysis method and system based on data analysis, which has the following beneficial effects:

[0058] (1) When the system is running, it realizes comprehensive monitoring and intelligent protection of network security. First, the system collects the visitor's network traffic, logs, user behavior and system event data in real time to form a complete data set D. Then, through data denoising, filling missing values ​​and data format conversion preprocessing steps, the quality and consistency of the data are improved. The feature extraction module further extracts key features from network traffic, logs, user behavior and system events to form a feature data set X. features ,The behavior analysis module uses machine learning algorithms to establish a baseline model, and obtains the ,abnormal evaluation index S through training and analysis of historical data and ,feature data sets. Finally, the anomaly detection module ,formulates visitor anomaly evaluation strategy based on the ,matching result between S and the preset threshold Y, and implements ,corresponding access restrictions and notification measures. Through multi-source data ,collection, the system provides a global security perspective, eliminates the ,protection blind spots of a single device, and realizes comprehensive network monitoring, ,and the feature extraction and behavior analysis modules ,precisely extract key features through intelligent algorithms, and establishes ,a behavioral baseline model, which can accurately identify and ,evaluate abnormal behaviors.

[0059] (2) By integrating the relevant features extracted by network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, a feature dataset X is formed. features , and by transforming the feature dataset X features Input the baseline model M for training and analysis to obtain the feature dataset X features The degree of deviation of several data points from the normal behavior pattern: the anomaly assessment index S, which improves the accuracy and sensitivity of anomaly detection and can accurately identify normal and abnormal behavior patterns.

[0060] (3) By matching the anomaly assessment index S with the preset anomaly assessment threshold Y, the visitor anomaly assessment strategy is obtained, and the specific implementation of the plan content, that is, the implementation of access restrictions and notification measures, ensures that potential threats can be responded to quickly and prevent the further spread of security incidents. The response measures cover multiple aspects from login blocking, session termination to resource access restrictions and permission control, providing comprehensive security protection and effectively preventing various types of security threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a schematic diagram of a flow chart of a network information security analysis system based on data analysis according to the present invention;

[0062] Figure 2 The present invention is a schematic diagram of the steps of a network information security analysis method based on data analysis. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0064] Example 1

[0065] The present invention provides a network information security analysis system based on data analysis, please refer to Figure 1 , including data acquisition module, data preprocessing module, feature extraction module, behavior analysis module and anomaly detection module;

[0066] The data collection module collects data in real time through system logs and security event management tools, including collecting visitor network traffic data, log data, user behavior data and system event data to form a collection data set D;

[0067] The data preprocessing module preprocesses the collected data set D, including data denoising preprocessing, missing value filling preprocessing and data format conversion preprocessing, to form a processed data set ;

[0068] The feature extraction module extracts data from the dataset Perform feature extraction, including network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, to form a feature dataset X features ;

[0069] The behavior analysis module uses a machine learning algorithm to establish a baseline model by combining the feature data set X features Input the baseline model for training and analysis to obtain the degree of deviation from the normal behavior pattern: the abnormality assessment index S;

[0070] The anomaly detection module matches the anomaly assessment index S with a preset anomaly assessment threshold Y, obtains the visitor anomaly assessment strategy scheme, and executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy scheme, including visitor access restrictions and relevant personnel notifications.

[0071] In this embodiment, comprehensive monitoring and intelligent protection of network security are achieved. First, the system collects the visitor's network traffic, logs, user behavior and system event data in real time to form a complete data set D. Then, through data denoising, filling missing values ​​and data format conversion preprocessing steps, the quality and consistency of the data are improved. The feature extraction module further extracts key features from network traffic, logs, user behavior and system events to form a feature data set X. features ,The behavior analysis module uses machine learning algorithms to establish a baseline model, and obtains the ,abnormal evaluation index S through training and analysis of historical data and ,feature data sets. Finally, the anomaly detection module ,formulates visitor anomaly evaluation strategy based on the ,matching result between S and the preset threshold Y, and implements ,corresponding access restrictions and notification measures. Through multi-source data ,collection, the system provides a global security perspective, eliminates the ,protection blind spots of a single device, and realizes comprehensive network monitoring, ,and the feature extraction and behavior analysis modules ,precisely extract key features through intelligent algorithms, and establishes ,a behavioral baseline model, which can accurately identify and ,evaluate abnormal behaviors.

[0072] Example 2

[0073] Please refer to Figure 1 ,Specifically: the data collection module includes a log collection unit and a flow collection unit;

[0074] The log collection unit is responsible for collecting log data XL, user behavior data XU and system event data XE from system logs and security event management tools in real time, including real-time monitoring and collection of log data and centralized collection and management of security events by using log collection tools and configuring SIEM;

[0075] Among them, log data includes collecting server, application, operating system, security device and database log files;

[0076] User behavior data includes collecting users’ login time, operation behavior and access records;

[0077] System event data includes all operating events of the operating system and errors, warnings, and other related system events of applications;

[0078] The traffic collection unit collects the visitor's network traffic data XF in real time, including data packet size, transmission rate, connection duration and protocol type;

[0079] The data packet size is captured and collected through traffic mirroring and network probes, including statistics on the average, maximum, and minimum data packet size of each connection;

[0080] The collected data set D specifically includes event type frequency, event time distribution, event source IP, login frequency, operation type frequency, access resource type, system event type frequency, CPU and memory usage, error and warning events, data packet size, transmission rate, connection duration and protocol type.

[0081] The data preprocessing module includes a denoising unit and a correction unit;

[0082] The denoising unit performs data denoising preprocessing on the collected data set D using filtering rules, statistical methods and machine learning detection, including log data denoising to filter out system messages and redundant records that are not related to network security; user behavior data denoising to remove misoperation and invalid access related operation records; system event data denoising to remove irrelevant system events, system warnings and system related event reminders; network traffic data denoising to filter out invalid data packets and duplicate data packets;

[0083] The correction unit performs missing value filling preprocessing and data format conversion preprocessing on the collected data set D after denoising preprocessing to form a processed data set , including a subset of network traffic data , Log data subset , User behavior data subset and a subset of system event data , specifically including packet average, transmission rate, protocol type frequency, event type frequency, event time distribution, login frequency, operation type frequency, system event type frequency, CPU usage, and error and warning event frequency;

[0084] Among them, data format conversion includes time format conversion, encoding conversion and data type conversion; filling missing values ​​includes mean filling, median filling, nearest neighbor filling and interpolation filling.

[0085] Example 3

[0086] Please refer to Figure 1 ,Specifically: the feature extraction module includes a network and log extraction unit and a behavior and event extraction unit;

[0087] The network and log extraction unit performs a Perform feature extraction, including network traffic feature extraction XF: packet average value ASize, transmission rate TRate and protocol type frequency PFReq; log feature extraction XL: event type frequency ETFreq and event time distribution ETDist;

[0088] The behavior and event extraction unit performs a Perform feature extraction, including behavior feature extraction XU: user login frequency LFreq and operation type frequency ATFreq; system event feature extraction XE: system event type frequency SETFreq, CPU usage CUsage, error and warning event frequency EWFreq;

[0089] The feature dataset X is composed by integrating the relevant features extracted by network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE. features .

[0090] Among them, the average packet value ASize is obtained by calculating the average size of all packet sizes in the network connection; the transmission rate TRate is obtained by calculating the number of transmissions in a fixed time; the protocol type frequency PFReq is obtained by counting the usage frequency of different protocol types; the event type frequency ETFreq is obtained by counting the occurrence frequency of different types of events; the event time distribution ETDist is obtained by counting the distribution frequency of events in each hour segment of a natural day.

[0091] Among them, the user login frequency LFreq is obtained by counting the ratio of the number of logins on a natural day to the total number of logins; the operation type frequency ATFreq is obtained by counting the frequency of users executing different types of operation sequence numbers; the system event type frequency SETFreq is obtained by counting the occurrence frequency of different types of system event sequence numbers; the CPU utilization rate CUsage is obtained by counting the CPU usage of the system in different time periods; the error and warning event frequency EWFreq is obtained by counting the frequency of error events and warning times occurring within a fixed period.

[0092] The behavior analysis module includes a modeling unit;

[0093] The modeling unit establishes a baseline model M by using a machine learning algorithm, including selecting a mixed Gaussian model as the baseline model M by features Input the baseline model M for training and analysis to obtain the feature dataset X features The degree of deviation of several data points from the normal behavior pattern: the abnormal evaluation index S, which evaluates the state of the visitor's behavior deviation from the normal pattern;

[0094] Among them, the feature dataset X features Specifically: X features ={XF,XL,XU,XE};

[0095] The abnormality assessment index S is obtained by the following calculation formula:

[0096] ;

[0097] In the formula, S represents the anomaly evaluation index, which specifically represents the feature data set X features abnormal performance, Represents the feature dataset X features The probability under the baseline model M specifically represents the weighted combination of Gaussian distributions of the mixed Gaussian model to fit the data, and log represents the natural logarithm function.

[0098] Among them, the formula for fitting data using a weighted combination of Gaussian distributions is as follows:

[0099] ;

[0100] In the formula, Represents the feature dataset X features The probability distribution of K represents the number of Gaussian components, specifically the number of Gaussian distributions in the mixed Gaussian model. Represents the weight of the k-th Gaussian component, specifically the contribution ratio of the k-th Gaussian component in the entire mixed Gaussian model, satisfying , Represents the normal distribution of the kth Gaussian component, specifically used to describe the feature data set X features The probability density under the kth Gaussian component is, represents the mean vector of the k-th Gaussian component, specifically the center position of the k-th Gaussian component, Represents the covariance matrix of the k-th Gaussian component, specifically indicating the distribution range and direction of the k-th Gaussian component;

[0101] Among them, the normal distribution of the kth Gaussian component The formula is as follows:

[0102] ;

[0103] In the formula, d represents the feature dataset X features The dimension of the feature dataset X features The number of Represents the covariance matrix The determinant of , specifically the covariance matrix The scale of Represents the feature dataset X features Subtract the mean vector After the transposition, Represents the covariance matrix The inverse matrix of , exp represents the exponential function, specifically the exponential part of the Gaussian distribution of the mixed Gaussian model.

[0104] In this embodiment, the relevant features extracted by network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE are integrated to form a feature data set X features , and by transforming the feature dataset X features Input the baseline model M for training and analysis to obtain the feature dataset X features The degree of deviation of several data points from the normal behavior pattern: the anomaly assessment index S, which improves the accuracy and sensitivity of anomaly detection and can accurately identify normal and abnormal behavior patterns.

[0105] Example 4

[0106] Please refer to Figure 1 ,Specifically: the anomaly detection module includes a matching unit and a response unit;

[0107] The matching unit matches the preset relevant information with the required comparison value, including matching the abnormal evaluation index S with the preset abnormal evaluation threshold Y, to obtain the visitor abnormal evaluation strategy solution;

[0108] Abnormal evaluation index S < abnormal evaluation threshold Y, the visitor has no abnormal evaluation results, and the current visitor is not restricted or notified of monitoring related personnel;

[0109] Abnormal evaluation index S ≥ abnormal evaluation threshold Y, obtain the visitor abnormal evaluation results, restrict the current visitor and monitor the notification of relevant personnel;

[0110] Abnormal evaluation index S ≥ twice the abnormal evaluation threshold Y, obtain the visitor warning evaluation results, freeze the current visitor's behavior and operations, and execute relevant emergency security management personnel notifications;

[0111] The response unit executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy plan, including visitor access restrictions and relevant personnel notifications. Access restrictions include blocking visitor login, ending sessions, restricting access to resources, restricting access to directories, multi-factor authentication and restricting visitor permissions. Notifications include security administrator notifications and network monitoring team notifications.

[0112] In this embodiment, the visitor anomaly assessment strategy plan is obtained by matching the anomaly assessment index S with the preset anomaly assessment threshold Y, and the specific implementation of the plan content, that is, executing access restrictions and notification measures, ensures that potential threats can be responded to quickly and prevents the further spread of security incidents. The response measures cover multiple aspects from login blocking, session termination to resource access restrictions and permission control, providing comprehensive security protection and effectively preventing various types of security threats.

[0113] Example 5

[0114] A network information security analysis method based on data analysis, please refer to Figure 2 , specifically: including the following steps:

[0115] Step 1: The data collection module collects data in real time through system logs and security event management tools, including collecting visitor network traffic data, log data, user behavior data and system event data to form a collection data set D;

[0116] Step 2: The data preprocessing module preprocesses the collected data set D, including data denoising preprocessing, missing value filling preprocessing and data format conversion preprocessing, to form a processed data set ;

[0117] Step 3: Feature extraction module for data set Perform feature extraction, including network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, to form a feature dataset X features ;

[0118] Step 4: The behavior analysis module uses machine learning algorithms to build a baseline model by features Input the baseline model for training and analysis to obtain the degree of deviation from the normal behavior pattern: the abnormality assessment index S;

[0119] Step 5: The anomaly detection module matches the anomaly assessment index S with the preset anomaly assessment threshold Y, obtains the visitor anomaly assessment strategy plan, and executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy plan, including visitor access restrictions and relevant personnel notifications.

[0120] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A network information security analysis system based on data analysis, characterized in that: It includes data acquisition module, data preprocessing module, feature extraction module, behavior analysis module and anomaly detection module; The data collection module collects data in real time through system logs and security event management tools, including collecting visitor network traffic data, log data, user behavior data and system event data to form a collection data set D; The data preprocessing module preprocesses the collected data set D, including data denoising preprocessing, missing value filling preprocessing and data format conversion preprocessing, to form a processed data set ; The feature extraction module extracts data from the dataset Perform feature extraction, including network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, to form a feature dataset X features ; The behavior analysis module uses a machine learning algorithm to establish a baseline model by combining the feature data set X features Input the baseline model for training and analysis to obtain the degree of deviation from the normal behavior pattern: the abnormality assessment index S; The behavior analysis module includes a modeling unit; The modeling unit establishes a baseline model M by using a machine learning algorithm, including selecting a mixed Gaussian model as the baseline model M by features Input the baseline model M for training and analysis to obtain the feature dataset X features The degree of deviation of several data points from the normal behavior pattern: the abnormal evaluation index S, which evaluates the state of the visitor's behavior deviation from the normal pattern; Among them, the feature dataset X features Specifically: X features ={XF,XL,XU,XE}; The abnormality assessment index S is obtained by the following calculation formula: ； In the formula, S represents the anomaly evaluation index, which specifically represents the feature data set X features abnormal performance, Represents the feature dataset X features The probability under the baseline model M specifically represents the weighted combination of Gaussian distributions of the mixed Gaussian model to fit the data acquisition, and log represents the natural logarithm function; Among them, the formula for fitting data using a weighted combination of Gaussian distributions is as follows: ； In the formula, Represents the feature dataset X features The probability distribution of K represents the number of Gaussian components, specifically the number of Gaussian distributions in the mixed Gaussian model. Represents the weight of the k-th Gaussian component, specifically the contribution ratio of the k-th Gaussian component in the entire mixed Gaussian model, satisfying , Represents the normal distribution of the kth Gaussian component, specifically used to describe the feature data set X features The probability density under the kth Gaussian component is, represents the mean vector of the k-th Gaussian component, specifically the center position of the k-th Gaussian component, Represents the covariance matrix of the k-th Gaussian component, specifically indicating the distribution range and direction of the k-th Gaussian component; Among them, the normal distribution of the kth Gaussian component The formula is as follows: ； In the formula, d represents the feature dataset X features The dimension of the feature dataset X features The number of Represents the covariance matrix The determinant of , specifically the covariance matrix The scale of Represents the feature dataset X features Subtract the mean vector After the transposition, Represents the covariance matrix The inverse matrix of , exp represents the exponential function, specifically the exponential part of the Gaussian distribution of the mixed Gaussian model; The anomaly detection module matches the anomaly assessment index S with a preset anomaly assessment threshold Y, obtains the visitor anomaly assessment strategy scheme, and executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy scheme, including visitor access restrictions and relevant personnel notifications.

2. A network information security analysis system based on data analysis according to claim 1, characterized in that: The data collection module includes a log collection unit and a flow collection unit; The log collection unit is responsible for collecting log data XL, user behavior data XU and system event data XE from system logs and security event management tools in real time, including real-time monitoring and collection of log data and centralized collection and management of security events by using log collection tools and configuring SIEM; Among them, log data includes collecting server, application, operating system, security device and database log files; User behavior data includes collecting users’ login time, operation behavior and access records; System event data includes all operating events of the operating system and errors, warnings, and other related system events of applications; The traffic collection unit collects the visitor's network traffic data XF in real time, including data packet size, transmission rate, connection duration and protocol type; The data packet size is captured and collected through traffic mirroring and network probes, including statistics on the average, maximum, and minimum data packet size of each connection; The collected data set D specifically includes event type frequency, event time distribution, event source IP, login frequency, operation type frequency, access resource type, system event type frequency, CPU and memory usage, error and warning events, data packet size, transmission rate, connection duration and protocol type.

3. The network information security analysis system based on data analysis according to claim 1 is characterized in that: The data preprocessing module includes a denoising unit and a correction unit; The denoising unit performs data denoising preprocessing on the collected data set D using filtering rules, statistical methods and machine learning detection, including log data denoising to filter out system messages and redundant records that are not related to network security; user behavior data denoising to remove misoperation and invalid access related operation records; system event data denoising to remove irrelevant system events, system warnings and system related event reminders; Network traffic data denoising filters invalid and duplicate packets; The correction unit performs missing value filling preprocessing and data format conversion preprocessing on the collected data set D after denoising preprocessing to form a processed data set , including a subset of network traffic data , Log data subset , User behavior data subset and a subset of system event data , specifically including packet average, transmission rate, protocol type frequency, event type frequency, event time distribution, login frequency, operation type frequency, system event type frequency, CPU usage, and error and warning event frequency; Among them, data format conversion includes time format conversion, encoding conversion and data type conversion; filling missing values ​​includes mean filling, median filling, nearest neighbor filling and interpolation filling.

4. The network information security analysis system based on data analysis according to claim 1 is characterized in that: The feature extraction module includes a network and log extraction unit and a behavior and event extraction unit; The network and log extraction unit performs a Perform feature extraction, including network traffic feature extraction XF: packet average value ASize, transmission rate TRate and protocol type frequency PFReq; log feature extraction XL: event type frequency ETFreq and event time distribution ETDist; The behavior and event extraction unit performs a Perform feature extraction, including behavior feature extraction XU: user login frequency LFreq and operation type frequency ATFreq; system event feature extraction XE: system event type frequency SETFreq, CPU usage CUsage, error and warning event frequency EWFreq; By integrating the relevant features extracted by network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, Composition feature dataset X features .

5. A network information security analysis system based on data analysis according to claim 4, characterized in that: in, The average packet value ASize is obtained by calculating the average size of all packet sizes in the network connection; the transmission rate TRate is obtained by calculating the number of transmissions in a fixed time; the protocol type frequency PFReq is obtained by counting the usage frequency of different protocol types; the event type frequency ETFreq is obtained by counting the occurrence frequency of different types of events; the event time distribution ETDist is obtained by counting the distribution frequency of events in each hour segment of a natural day.

6. A network information security analysis system based on data analysis according to claim 4, characterized in that: in, The user login frequency LFreq is obtained by counting the ratio of the number of logins on a natural day to the total number of logins; the operation type frequency ATFreq is obtained by counting the frequency of users executing different types of operation sequence numbers; the system event type frequency SETFreq is obtained by counting the occurrence frequency of different types of system event sequence numbers; the CPU utilization rate CUsage is obtained by counting the CPU usage of the system in different time periods; the error and warning event frequency EWFreq is obtained by counting the frequency of error events and warning times occurring within a fixed period.

7. The network information security analysis system based on data analysis according to claim 1 is characterized in that: The anomaly detection module includes a matching unit and a response unit; The matching unit matches the preset relevant information with the required comparison value, including matching the abnormal evaluation index S with the preset abnormal evaluation threshold Y, to obtain the visitor abnormal evaluation strategy solution; Abnormal evaluation index S < abnormal evaluation threshold Y, the visitor has no abnormal evaluation results, and the current visitor is not restricted or notified of monitoring related personnel; Abnormal evaluation index S ≥ abnormal evaluation threshold Y, obtain the visitor abnormal evaluation results, restrict the current visitor and monitor the notification of relevant personnel; Abnormal evaluation index S ≥ twice the abnormal evaluation threshold Y, obtain the visitor warning evaluation results, freeze the current visitor's behavior and operations, and execute relevant emergency security management personnel notifications; The response unit executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy plan, including visitor access restrictions and relevant personnel notifications. Access restrictions include blocking visitor login, ending sessions, restricting access to resources, restricting access to directories, multi-factor authentication and restricting visitor permissions. Notifications include security administrator notifications and network monitoring team notifications.

8. A network information security analysis method based on data analysis, comprising a network information security analysis system based on data analysis as claimed in any one of claims 1 to 7, characterized in that: The following steps are involved: Step 1: The data collection module collects data in real time through system logs and security event management tools, including collecting visitor network traffic data, log data, user behavior data and system event data to form a collection data set D; Step 2: The data preprocessing module preprocesses the collected data set D, including data denoising preprocessing, missing value filling preprocessing and data format conversion preprocessing, to form a processed data set ; Step 3: Feature extraction module for data set Perform feature extraction, including network traffic feature extraction XF, log feature extraction XL, behavior feature extraction XU and system event feature extraction XE, to form a feature dataset X features ; Step 4: The behavior analysis module uses machine learning algorithms to build a baseline model by features Input the baseline model for training and analysis to obtain the degree of deviation from the normal behavior pattern: the abnormality assessment index S; Step 5: The anomaly detection module matches the anomaly assessment index S with the preset anomaly assessment threshold Y, obtains the visitor anomaly assessment strategy plan, and executes corresponding measures on the visitor according to the content of the visitor anomaly assessment strategy plan, including visitor access restrictions and relevant personnel notifications.

Images