Method and system for performing network information security analysis by using big data

Abstract

The invention discloses a method and system for performing network information security analysis by using big data, and relates to the technical field of network security, and the method comprises the following steps: obtaining network data in a given time window; preprocessing the network data; calculating the access frequency of the to-be-tested resource and the number of wrong login attempts by using the preprocessed network data; calculating a correlation coefficient between the access frequency and the number of wrong login attempts, and identifying a first security / threat signal; the network data is constructed into a graph structure, nodes represent users or resources, edges represent interaction relations, a GraphSage model is used for graph embedding and anomaly detection, and a second security / threat signal is identified; and automatically triggering corresponding measures according to the first security / threat signal and the second security / threat signal. According to the invention, the network can be effectively protected from various security threats, and a more reliable security guarantee is provided for enterprises and individual users.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and specifically to a method and system for performing network information security analysis using big data. Background Art

[0002] With the rapid development of the Internet, network security threats have become increasingly complex and diverse. Traditional security protection measures (such as firewalls and intrusion detection systems) mainly rely on predefined rules and signatures to identify known attack patterns, but these methods are often powerless when faced with new or unknown security threats. In addition, the amount of data in modern network environments is growing exponentially, and traditional analysis tools have difficulty processing such a large data set, resulting in slow response to security incidents and low accuracy.

[0003] In this context, the use of big data technology and graph neural network (GNN) models for network information security analysis has become a new trend. By integrating multi-source data (such as network logs, application logs, user behavior data, etc.) and applying advanced machine learning algorithms, potential security threats can be more effectively identified and corresponding protective measures can be taken in a timely manner.

[0004] In view of this, it is necessary to provide a method and system for network information security analysis using big data. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent, by proposing a method and system for network information security analysis using big data, thereby solving the problems of how to identify hidden attack patterns and abnormal behaviors in a complex network environment, how to achieve rapid security incident detection and response while ensuring high accuracy, and how to improve the sensitivity and accuracy of anomaly detection.

[0006] To achieve the above objectives, in a first aspect, the present application provides a method for performing network information security analysis using big data, comprising the following steps: Get network data within a given time window; Preprocess network data; Calculate the access frequency and number of incorrect login attempts of the resource under test using preprocessed network data; Calculate the correlation coefficient between access frequency and number of incorrect login attempts to identify the first security / threat signal; The network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal. Corresponding measures are automatically triggered according to the first safety / threat signal and the second safety / threat signal.

[0007] Furthermore, within a given time window, network data is obtained, including: Determine the time frame for the analysis; Collect network data within a time range from multiple channels including network logs, application logs, user behavior data, and external threat intelligence; The network data includes: access request records, login attempt records, error codes and user activity tracks.

[0008] Furthermore, the network data is preprocessed, including: Remove irrelevant information, duplicate records and noise data from network data; Convert network data collected from multiple channels into a unified format; Identify and automatically annotate IP addresses, timestamps, request paths, status codes, and user IDs within a time range.

[0009] Furthermore, the calculation formula of the access frequency is: Where n is the total number of requests in the time window, is the request path, is the target path, is an indicator function, which is 1 if the path of the i-th request is equal to the target path, otherwise it is 0, Δt is the time window length, and W is the access frequency.

[0010] Furthermore, the calculation formula for the number of incorrect login attempts is: Where m is the total number of login attempts within the time window, is the status code, Is the status code for failed login. is an indicator function indicating whether it is a failed login attempt. If the status code of the jth login attempt indicates a failure, it is 1, otherwise it is 0. is the number of incorrect login attempts.

[0011] Further, calculating the correlation coefficient between the access frequency and the number of incorrect login attempts to identify the first security / threat signal includes: Calculate the correlation coefficient r between access frequency and number of incorrect login attempts: ; Among them, X and Y represent the access frequency and the number of incorrect login attempts, respectively. is the sample size, S X and S Y are their standard deviations respectively; The type of relationship between the calculated access frequency and the number of incorrect login attempts is determined based on the calculated correlation coefficient value r, and a first safety / threat signal is obtained.

[0012] Furthermore, the network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal, including: Construct graph structure data, including taking each user and resource to be tested as a node; taking the access frequency and number of incorrect login attempts of the user and resource to be tested as edges; The GraphSAGE model is used to convert graph structured data into low-dimensional vector representation, capture the characteristics of nodes and edges, and identify the second security / threat signals.

[0013] Furthermore, it also includes: Adjust the sampling strategy of the GraphSage model according to the characteristics of the current node and the network data before and after the time.

[0014] Furthermore, if the first safety / threat signal or the second safety / threat signal is detected, protective measures are automatically triggered, including: Isolate suspicious IP addresses; Enhanced certification requirements; Send an alert notification.

[0015] In a second aspect, the present application further provides a system for performing network information security analysis using big data, and a method for performing network information security analysis using big data based on any of the above items, including: Initialization module: obtain network data within a given time window; Data preprocessing module: preprocess network data; Data calculation module: uses pre-processed network data to calculate the access frequency and number of incorrect login attempts of the resource to be tested; First identification module: calculates the correlation coefficient between the access frequency and the number of incorrect login attempts, and identifies the first security / threat signal; Second identification module: The network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal. Automatic response module: automatically triggers corresponding measures according to the first safety / threat signal and the second safety / threat signal.

[0016] Beneficial effects of the present invention: Compared with the prior art, the present invention provides a method and system for network information security analysis using big data. By introducing big data processing technology, it can process massive network data in real time and efficiently, ensuring the timeliness and integrity of the data. This not only improves the speed of data processing, but also reduces the risk of data loss; combined with the correlation coefficient calculation of access frequency and the number of incorrect login attempts and the graph neural network model, potential security threats can be identified at multiple levels. In particular, the GraphSage model can capture complex user and resource interaction patterns, significantly improve the accuracy and coverage of anomaly detection, and introduce an adaptive sampling strategy to dynamically adjust the sampling probability according to the historical behavior of the node and the current context, so that the system can better adapt to changes in node behavior, further improving the sensitivity and accuracy of anomaly detection; by setting reasonable thresholds and automated response mechanisms, the corresponding protective measures can be triggered immediately when security threats are detected, greatly shortening the response time and reducing potential losses; the present invention has the ability of continuous monitoring and automatic optimization, and gradually optimizes the model parameters through continuous learning and feedback to improve the overall performance of the system. This self-learning mechanism ensures that the system can respond to emerging security threats and maintain long-term effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 It is a functional block diagram of the system of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The purpose of the present invention is to provide a method and system for network information security analysis using big data, which specifically includes the following steps: Data collection: within a given time window, obtain network data, including network logs, application logs, user behavior data, etc. Data preprocessing: clean, standardize and extract features of the original network data to prepare for subsequent analysis. Calculate access frequency and number of incorrect login attempts: based on the preprocessed data, calculate the access frequency and number of incorrect login attempts of the resource to be tested. Correlation coefficient calculation: calculate the correlation coefficient between the access frequency and the number of incorrect login attempts, and identify the first security / threat signal. Graph structuring and GNN model: construct network data into a graph structure, use the GNN model for graph embedding and anomaly detection, and identify the second security / threat signal. Automatically trigger response measures: according to the first and second security / threat signals, automatically trigger corresponding protection measures, such as isolating suspicious IP addresses, enhancing authentication requirements, etc. Provide solutions for how to effectively protect the network from various security threats.

[0020] Example 1, please refer to Figure 1 As shown, the present application provides a method for performing network information security analysis using big data, comprising the following steps: Get network data within a given time window; Preprocess network data; Calculate the access frequency and number of incorrect login attempts of the resource under test using preprocessed network data; Calculate the correlation coefficient between access frequency and number of incorrect login attempts to identify the first security / threat signal; The network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal. Corresponding measures are automatically triggered according to the first safety / threat signal and the second safety / threat signal.

[0021] In the specific implementation process, network data is obtained within a given time window, including: Determine the time frame for the analysis; Collect network data within a time range from multiple channels including network logs, application logs, user behavior data, and external threat intelligence; The network data includes: access request records, login attempt records, error codes and user activity tracks.

[0022] In the specific implementation process, the network data is preprocessed, including: Remove irrelevant information, duplicate records and noise data from network data; Convert network data collected from multiple channels into a unified format; Identify and automatically annotate IP addresses, timestamps, request paths, status codes, and user IDs within a time range.

[0023] Specifically, the introduction of social network analysis and IoT device data has expanded the data source, provided a more comprehensive security view, and can process massive network data in real time and efficiently, ensuring the timeliness and integrity of the data. This not only improves the speed of data processing, but also reduces the risk of data loss.

[0024] Specifically, NLP and deep learning models can be used for intelligent labeling and classification to improve the efficiency and accuracy of data processing.

[0025] In the specific implementation process, the calculation formula of the access frequency is: Where n is the total number of requests in the time window, is the request path, is the target path, is an indicator function, which is 1 if the path of the i-th request is equal to the target path, otherwise it is 0, Δt is the time window length, and W is the access frequency.

[0026] In the specific implementation process, the calculation formula for the number of incorrect login attempts is: Where m is the total number of login attempts within the time window, is the status code, Is the status code for failed login. is an indicator function indicating whether it is a failed login attempt. If the status code of the jth login attempt indicates a failure, it is 1, otherwise it is 0. is the number of incorrect login attempts.

[0027] In the specific implementation process, the correlation coefficient between the access frequency and the number of incorrect login attempts is calculated to identify the first security / threat signal, including: Calculate the correlation coefficient r between access frequency and number of incorrect login attempts: ; Among them, X and Y represent the access frequency and the number of incorrect login attempts, respectively. is the sample size, S X and S Y are their standard deviations respectively; The type of relationship between the calculated access frequency and the number of incorrect login attempts is determined based on the calculated correlation coefficient value r, and a first safety / threat signal is obtained.

[0028] Specifically, by calculating the correlation between the number of incorrect login attempts and the access frequency, we can assess whether there are signs of brute force attacks, and determine the type of relationship between the variables (positive correlation, negative correlation, or no obvious linear relationship) based on the calculated correlation coefficient value (ranging from -1 to +1). A high positive value indicates a strong positive correlation, meaning that as one variable increases, the other variable also tends to increase; a high negative value indicates the opposite. This helps identify which factors may be risk indicators for cyber attacks or security incidents.

[0029] In the specific implementation process, the network data is constructed into a graph structure, with nodes representing users or resources and edges representing interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal, including: Construct graph structure data, including taking each user and resource to be tested as a node; taking the access frequency and number of incorrect login attempts of the user and resource to be tested as edges; The GraphSAGE model is used to convert graph structured data into low-dimensional vector representation, capture the characteristics of nodes and edges, and identify the second security / threat signals.

[0030] Specifically, the detection capability of the system is enhanced by identifying complex attack patterns through graph structured data and GNN models.

[0031] In the specific implementation process, it also includes: Adjust the sampling strategy of the GraphSage model according to the characteristics of the current node and the network data before and after the time.

[0032] Specifically, the adaptive defense strategy is implemented through the reinforcement learning model, which improves the flexibility and responsiveness of the system.

[0033] More specifically, N ( v ) represents a node v The neighbor set of C ( v , t ) represents a node v At the point in time t Network data before and after (e.g., behavior patterns, interaction frequency, etc.), H ( v ) represents a node v Historical behavior data; First, for each neighbor node u ∈ N ( v ) Calculate an importance weight W ( u ; v , t), the weight can be based on the node's historical behavior, the current context, and other factors (such as the node's degree centrality). Suppose there is a function f (⋅) to combine these factors: ; In order to ensure that the sum of the sampling probabilities of all neighbor nodes is 1, the weights calculated above need to be normalized: ; Based on the normalized weights, we can define a sampling distribution Ps ( u ∣ v , t ), indicating a slave node v Select a node from the neighbors u Probability: ; Introducing a dynamic adjustment factor D ( v , t ), which can be based on the node v The final sampling probability is adjusted based on the current abnormal behavior or risk level. This factor can be a value obtained through a machine learning model or other means, reflecting the risk level of the node: ; Finally, again P ′( u ∣ v , t ) is normalized to obtain the final probability distribution for sampling: ; Through the above method, the sampling strategy of the GraphSage model is adjusted according to the characteristics of the current node and the network data before and after the time, so as to more effectively capture the security threat signals in the network. This method is particularly suitable for nodes whose behaviors change greatly over time and have potential security risks.

[0034] In the specific implementation process, if the first safety / threat signal or the second safety / threat signal is detected, the protective measures are automatically triggered, including: Isolate suspicious IP addresses; Enhanced certification requirements; Send an alert notification.

[0035] Specifically, corresponding protection measures are automatically triggered according to different security / threat signals, which enhances the flexibility and response speed of the system.

[0036] Example 2, please refer to Figure 2As shown, the present application also provides a system for performing network information security analysis using big data, and a method for performing network information security analysis using big data based on any of the above items, including: Initialization module: obtain network data within a given time window; Data preprocessing module: preprocess network data; Data calculation module: uses pre-processed network data to calculate the access frequency and number of incorrect login attempts of the resource to be tested; First identification module: calculates the correlation coefficient between the access frequency and the number of incorrect login attempts, and identifies the first security / threat signal; Second identification module: The network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal. Automatic response module: automatically triggers corresponding measures according to the first safety / threat signal and the second safety / threat signal.

[0037] Specifically, the present invention provides a comprehensive and efficient network information security analysis method by comprehensively utilizing big data processing technology and advanced graph neural network models. It not only solves the shortcomings of traditional security protection methods in processing large-scale data and complex attack patterns, but also significantly improves the real-time and accuracy of the system through dynamic adaptive sampling and automated response mechanisms. Ultimately, this solution can effectively protect the network from various security threats and provide more reliable security protection for enterprises and individual users.

[0038] Through the description of the above implementation methods, the embodiments of the present invention can be provided as methods and systems. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0039] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for network information security analysis using big data, characterized in that: The following steps are involved: Get network data within a given time window; Preprocess network data; Calculate the access frequency and number of incorrect login attempts of the resource under test using preprocessed network data; Calculate the correlation coefficient between access frequency and number of incorrect login attempts to identify the first security / threat signal; The network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal. Corresponding measures are automatically triggered according to the first safety / threat signal and the second safety / threat signal.

2. The method for performing network information security analysis using big data according to claim 1, characterized in that: The obtaining of network data within a given time window includes: Determine the time frame for the analysis; Collect network data within a time range from multiple channels including network logs, application logs, user behavior data, and external threat intelligence; The network data includes: access request records, login attempt records, error codes and user activity tracks.

3. The method for performing network information security analysis using big data according to claim 1, characterized in that: The preprocessing of network data includes: Remove irrelevant information, duplicate records and noise data from network data; Convert network data collected from multiple channels into a unified format; Identify and automatically annotate IP addresses, timestamps, request paths, status codes, and user IDs within a time range.

4. The method for performing network information security analysis using big data according to claim 1, characterized in that: The calculation formula of the access frequency is: Where n is the total number of requests in the time window, is the request path, is the target path, is an indicator function, which is 1 if the path of the i-th request is equal to the target path, otherwise it is 0, Δt is the time window length, and W is the access frequency.

5. The method for performing network information security analysis using big data according to claim 1, characterized in that: The calculation formula for the number of incorrect login attempts is: Where m is the total number of login attempts within the time window, is the status code, Is the status code for failed login. is an indicator function indicating whether it is a failed login attempt. If the status code of the jth login attempt indicates a failure, it is 1, otherwise it is 0. is the number of incorrect login attempts.

6. The method for performing network information security analysis using big data according to claim 1, characterized in that: The calculating the correlation coefficient between the access frequency and the number of incorrect login attempts and identifying the first security / threat signal includes: Calculate the correlation coefficient r between access frequency and number of incorrect login attempts: ； Among them, X and Y represent the access frequency and the number of incorrect login attempts, respectively. is the sample size, S X and S Y are their standard deviations respectively; The type of relationship between the calculated access frequency and the number of incorrect login attempts is determined based on the calculated correlation coefficient value r, and a first safety / threat signal is obtained.

7. The method for performing network information security analysis using big data according to claim 1, characterized in that: The network data is constructed into a graph structure, where nodes represent users or resources, and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal, including: Construct graph structure data, including taking each user and resource to be tested as a node; taking the access frequency and number of incorrect login attempts of the user and resource to be tested as edges; The GraphSAGE model is used to convert graph structured data into low-dimensional vector representation, capture the characteristics of nodes and edges, and identify the second security / threat signals.

8. The method for performing network information security analysis using big data according to claim 7, characterized in that: Also includes: Adjust the sampling strategy of the GraphSage model according to the characteristics of the current node and the network data before and after the time.

9. The method for performing network information security analysis using big data according to claim 1, characterized in that: If the first safety / threat signal or the second safety / threat signal is detected, protective measures are automatically triggered, including: Isolate suspicious IP addresses; Enhanced certification requirements; Send an alert notification.

10. A system for performing network information security analysis using big data, based on a method for performing network information security analysis using big data as claimed in any one of claims 1 to 9, characterized in that: include: Initialization module: obtain network data within a given time window; Data preprocessing module: preprocess network data; Data calculation module: uses pre-processed network data to calculate the access frequency and number of incorrect login attempts of the resource to be tested; First identification module: calculates the correlation coefficient between the access frequency and the number of incorrect login attempts, and identifies the first security / threat signal; Second identification module: The network data is constructed into a graph structure, where nodes represent users or resources and edges represent interaction relationships. The GraphSage model is used for graph embedding and anomaly detection to identify the second security / threat signal. Automatic response module: automatically triggers corresponding measures according to the first safety / threat signal and the second safety / threat signal.

Images