Network security log management method and system

By collecting and preprocessing network security logs in real time, using classification models and multi-level behavior analysis, combining individual, group and global behavior analysis, the problem of insufficient data processing and correlation analysis capabilities in the existing technology is solved, and rapid identification is achieved and respond to complex attacks, improving security incident processing efficiency and resource utilization.

CN119011279AActive Publication Date: 2024-11-22SHANDONG BINZHOU NAT AGRI SCI & TECH PARK MANAGEMENT SERVICE CENT (BINZHOU YELLOW RIVER DELTA EFFICIENT ECOLOGICAL IND MODERN TECH RES INST)

Patent Information

Application Number
CN202411250705.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-22
Estimated Expiration
2044-09-06

AI Technical Summary

Technical Problem

The existing network security log management methods are difficult to effectively process massive security log data, making it difficult to detect security incidents in a timely manner. They lack the ability to analyze the correlation of multi-source heterogeneous security log data, making it difficult to identify complex attack behaviors and traceability attack paths.

Method used

Provide a network security log management method, including real-time collection and preprocessing of network security logs, using a classification model based on historical log data for preliminary classification, conducting real-time multi-level behavior analysis, and performing different storage strategies based on classification results and behavior analysis results. Use multi-source logs to associate potential threat events, locate the source of the problem, and generate security incident reports.

Benefits of technology

By combining individual, group and global behavior analysis, false positives are reduced, complex attack behavior identification ability is improved, response time is shortened, security incident processing efficiency is improved, and resource utilization efficiency is improved through differentiated storage strategies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011279A_ABST
    Figure CN119011279A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of network security, in particular to a network security log management method and system. The objective of the invention is to solve the uncertainty problem that multi-dimensional anomaly detection and complex event association analysis cannot be effectively handled. According to the method, the multi-source network security logs are collected and preprocessed, the preprocessed network security logs are preliminarily classified, the classified network security logs are subjected to multi-level behavior analysis to identify the abnormal risk in the classified data, and meanwhile, the abnormal risk in the classified data is identified based on the preliminary classification result and the behavior analysis result of the network security logs. According to the method, the network security logs are respectively stored according to different storage strategies, and finally, the problem source of the potential threat event is positioned based on the stored multi-source network security logs, and the detailed security event is generated, so that the abnormality can be found more quickly, the response time is shortened, and the processing efficiency of the security event is improved. And safety personnel can be helped to identify real threats more quickly and more accurately.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a network security log management method and system. Background Art

[0002] With the rapid development of the Internet and information technology, network security threats are becoming increasingly severe, various attack methods are emerging in an endless stream, and network security incidents occur frequently. Enterprises and network systems generate massive amounts of security log data every day. These logs record information including but not limited to network activities, system time, and security policies, and contain rich security situation awareness data.

[0003] However, existing network security log management methods cannot effectively process the massive security log data generated by enterprises and network systems, resulting in security incidents in the security log data being difficult to detect in a timely manner. At the same time, they lack the ability to perform correlation analysis on multi-source heterogeneous security log data, making it difficult to identify complex attack behaviors and trace attack paths.

[0004] Therefore, there is an urgent need to develop a network security log management method and system to overcome the shortcomings of the prior art. Summary of the invention

[0005] In order to solve the uncertainty problem that the prior art cannot effectively deal with multi-dimensional anomaly detection and complex event correlation analysis, the present invention provides such a network security log management method, including the following steps:

[0006] S1: collecting and preprocessing network security logs from various sources in real time, and preprocessing the network security logs, wherein the network security logs include firewall logs, intrusion detection system logs and access logs;

[0007] S2: using a log classification model built based on the historical network security log data, preliminarily classifying the preprocessed network security log into different categories, including abnormal activities, potential abnormalities, and normal activities;

[0008] S3: Perform real-time multi-level behavior analysis on the classified network security logs to identify anomalies and potential risks in the classified data;

[0009] S4: Based on the classification result and behavior analysis result of the network security log, the network security log is stored separately according to different storage strategies;

[0010] S5: Based on the stored multi-source network security logs, potential threat events are correlated, the source of the problem is located, and a detailed security incident report is generated.

[0011] Furthermore, the preliminary classification of the preprocessed network security logs specifically includes:

[0012] S21: extract key features from the network security log and calculate the feature vector of each log record in the network security log;

[0013] S22: Input the feature vector into a pre-trained log classification model to obtain a probability distribution of the log record belonging to an abnormal activity, a potential abnormality, or a normal activity;

[0014] S23: Determine the final category of the log record by threshold comparison according to the probability distribution;

[0015] S24: Regularly adjust the set threshold according to the latest classification results to adapt to changes in the network environment.

[0016] Further, the determining the final category of the log record by threshold comparison specifically includes:

[0017] S231: Calculate the abnormal activity probability P a and the potential abnormal probability P b The sum of the probabilities P num ;

[0018] S232: If P num If the probability of the abnormal activity and the potential abnormality is greater than the preset threshold value T1, the probability of the abnormal activity and the potential abnormality is further compared, wherein judgment thresholds T2 and T3 are also preset, and T1>T2>T3:

[0019] If P a >P b :When P a >T2, the log record is determined to be an abnormal activity;

[0020] When P a ≤T2, and P a >T3, the log record is determined to be a potential exception;

[0021] When P a ≤T3, the daily record is judged to be a normal activity;

[0022] If P b >P a :When P b >T3, the log record is determined to be a potential exception;

[0023] When P b ≤T3, the log record is determined to be a normal activity;

[0024] S233: If P numIf the value is not greater than the preset threshold value T1, the log record is directly determined to be a normal activity.

[0025] Furthermore, when training the log recognition model, n key features are first extracted from the log records to form a feature vector X = [x1, x2, ..., x n ];

[0026] Then calculate the preliminary risk score R of the log record:

[0027]

[0028] Finally, the category probability distribution P(K) of the log record is calculated according to the preliminary risk score R:

[0029]

[0030] Wherein, P(K) represents the probability that the log record belongs to the abnormal activity, the potential abnormality or the normal activity; j represents the index of the category, a is the abnormal activity category, b is the potential risk category, and c is the normal activity category; β K is a regulating factor used to control the degree of distinction between categories; K represents the category to which the target belongs; and R is a risk score used to represent a comprehensive evaluation of the log record by all relevant features.

[0031] Furthermore, when the classified network security logs are subjected to real-time multi-level behavior analysis, the multi-level behavior analysis includes individual behavior analysis, group behavior analysis, global behavior analysis and correlation analysis:

[0032] In individual behavior analysis, a user behavior baseline model is constructed, and the session data of each user or device is extracted in real time. The deviation between the current session data and the user behavior baseline model is calculated. When there is an individual in the session data that exceeds a preset deviation threshold, the individual is marked as abnormal.

[0033] In the group behavior analysis, the classified network security logs are grouped according to users, devices and applications, the overall activity pattern of each group is analyzed, and a clustering algorithm is used to determine group activities that are inconsistent with normal work processes, and the identified abnormal group behaviors are marked as abnormal;

[0034] In global behavior analysis, a time series analysis method is used to analyze the behavior trends of the collected network security logs to identify prominent behavior change patterns;

[0035] In the association analysis, the results of the individual behavior analysis, the group behavior analysis and the global behavior analysis are comprehensively evaluated to identify potential threat events.

[0036] Furthermore, the individual behavior analysis is used to identify uncommon conversation patterns and abnormal conversation times in real time, the group behavior analysis is periodically used to identify group activities that are inconsistent with normal work processes, and the global behavior analysis is used to continuously monitor behavior changes in network security logs; when an anomaly is identified in the individual behavior analysis or the group behavior analysis, the individual behavior analysis, the group behavior analysis and the global behavior analysis results are correlated and analyzed to establish a correlation relationship between individual behavior, group behavior and global behavior, and based on the correlation relationship, the level and credibility of potential risks are judged.

[0037] Furthermore, the level and credibility of the potential risk determination include:

[0038] Constructing a multi-layer association graph, taking individual behavior, group behavior and global behavior as nodes in the multi-layer association graph, and establishing association edges between individual behavior nodes, group behavior nodes and global behavior nodes;

[0039] Defining association strength, assigning a corresponding association strength value to each of the association edges, and quantifying the association strength between nodes;

[0040] Risk parameter propagation, assigning an initial risk score to the corresponding node based on the results of individual behavior analysis, group behavior analysis, and global behavior analysis; the risk score of the node is updated based on the risk scores of adjacent nodes and the association strength;

[0041] Determine the risk level and credibility, and determine the level of potential risk based on the final risk score and the preset level threshold; and evaluate the credibility based on the amount of node data calculated by the parameter risk score in the multi-layer association graph, the strength of the association edge, and the length of the risk propagation path.

[0042] Furthermore, the different storage strategies include:

[0043] S41: Based on the risk level of the network security log, a corresponding storage tiering level is set, including a fast-access storage medium, a low-cost high-capacity storage medium, and a hybrid storage architecture of the two storage media;

[0044] S42: according to the classification results and behavior analysis results of the network security logs, setting storage priorities, wherein abnormal activity logs have the highest storage priority, potential risk logs have a medium priority, and normal activity logs have the lowest priority;

[0045] S43: Based on the results of risk assessment, dynamically adjust the allocation of storage resources; use fast-access storage media for high-risk logs, use low-cost and high-capacity storage media for low-risk logs, and use a hybrid storage architecture of two storage media for risk logs;

[0046] S44: By monitoring the access frequency of network security logs and the usage of storage media in real time, a multi-objective optimization algorithm is used to dynamically adjust the storage ratio and location of different types of logs between fast-access storage media, low-cost and high-capacity storage media, and hybrid storage architectures;

[0047] S45: Regularly optimize and organize storage resources, clean up redundant or outdated normal activity logs, and compress and archive potential abnormal logs that are accessed infrequently.

[0048] Furthermore, the correlating of potential threat events and locating the source of the problem includes:

[0049] S51: sorting the collected network security logs by timestamps to ensure the time continuity of events in the network security logs;

[0050] S52: Screening and marking the network security logs by key information to identify existing duplicate or related events, wherein the key information includes IP addresses and time types;

[0051] S53: Use machine learning algorithms to analyze the correlation between labeled network security logs, build event correlation graphs, and generate visual event development paths;

[0052] S54: Analyze the associated network security logs and the event development path based on the anomaly detection model trained with historical network security logs to identify abnormal nodes and abnormal paths;

[0053] S55: Automatically generate a security analysis report based on the event association diagram and the anomaly identification result.

[0054] Furthermore, a network security log management system includes:

[0055] A log acquisition module is used to collect and preprocess network security logs from various sources in real time, and preprocess the network security logs, wherein the network security logs include firewall logs, intrusion detection logs and access logs;

[0056] A log classification module, used to preliminarily classify the pre-processed network security logs into different categories;

[0057] Behavior analysis module, which is used to perform multi-level behavior analysis on the network security logs after preliminary classification to identify anomalies and potential risks in the classified data;

[0058] A log storage module, used to store network security logs separately with different storage strategies according to the analysis results and behavior analysis results of the network logs;

[0059] The source analysis module is used to correlate potential threat events based on the stored multi-source network security logs and locate the source of potential threat events.

[0060] Beneficial effects:

[0061] 1. The present invention combines individual behavior analysis, group behavior analysis and global behavior analysis, and effectively reduces false positives by associating the connections among the three, and improves the ability to identify complex attack behaviors. At the same time, through real-time analysis and dynamic threshold adjustment, the present invention can detect anomalies faster, shorten response time, and improve the efficiency of handling security incidents;

[0062] 2. The present invention stores network security logs on different media according to the classification results and behavior analysis results of network security logs, and utilizes differentiated storage strategies such as fast access storage media, low-cost and high-capacity storage, and hybrid storage architecture, effectively improving resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of the method of the present invention;

[0064] Figure 2 It is a system structure diagram of the present invention. DETAILED DESCRIPTION

[0065] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0066] This embodiment discloses a network security log management method, as shown in the attached Figure 1 As shown, the following steps are included:

[0067] S1: Collect and pre-process network security logs from various sources in real time, and pre-process the network security logs, wherein the network security logs include but are not limited to firewall logs, intrusion detection system logs and access logs.

[0068] Specifically, since the network security log data is obtained from too many sources, its format is diverse and the content is complicated. Therefore, after collecting the network security log data, it needs to be preprocessed, including but not limited to data cleaning, data conversion and data normalization, to clean up invalid information, duplicate information and format errors in the network security log, and convert log data in different formats into a unified format.

[0069] S2: Use the log classification model built based on historical network security log data to preliminarily classify the preprocessed network security logs into different categories, including but not limited to abnormal activities, potential anomalies, and normal activities.

[0070] Specifically, the abnormal activity category can be used to classify malicious activities that may exist in network security log data, including but not limited to abnormal login attempts and data leakage; the potential abnormality category can be used to classify security risks that may exist in network security log data, including but not limited to configuration errors and system vulnerabilities; the normal activity category can be used to classify routine operations and time in network security log data, including but not limited to user login and file access.

[0071] The log records in the network security log are accurately divided into corresponding categories, including:

[0072] S21: Extract key features from the network security log and calculate the feature vector of each log record, where the key features include but are not limited to time features, entity features, operation features, location features, and content features.

[0073] S22: Input the feature vector into a pre-trained log classification model to obtain the probability distribution of whether the log record belongs to abnormal activity, potential abnormality or normal activity.

[0074] Specifically, when training the log recognition model, n key features are first extracted from the log records of historical network security logs to form a feature vector X = [x1, x2, ..., x n ];

[0075] Then calculate the preliminary risk score R of the log record:

[0076]

[0077] Finally, the category probability distribution P(K) of the log record is calculated based on the preliminary risk score R:

[0078]

[0079] Where P(K) represents the probability that a log record belongs to abnormal activity, potential abnormality, or normal activity; j represents the category index traversed in the category probability calculation, a is the abnormal activity category, b is the potential abnormality category, and c is the normal activity category; β K is an adjustment factor, by controlling the adjustment factor, the degree of distinction between categories can be controlled; K represents the category to which the target belongs, that is, it can be one of the categories of abnormal activity, potential abnormality or normal activity; R is the risk score, which is a scalar value obtained by weighted linear combination of all features of the log records extracted from S21, providing a corresponding simplified measurement for each log record, representing the comprehensive evaluation of the log record by all relevant features.

[0080] S23: Determine the final category of the log record by threshold comparison according to the probability distribution.

[0081] The specific steps of using threshold comparison to determine the classification of log records include:

[0082] S231: Calculate the probability of abnormal activity P a and potential abnormal probability P b The sum of the probabilities P num ;

[0083] S232: If P num If it is greater than the preset threshold value T1, the probability of abnormal activity and potential abnormality is further compared, where judgment thresholds T2 and T3 are also preset, and T1>T2>T3:

[0084] If P a >P b :When P a >T2, the log record is considered abnormal activity;

[0085] When P a ≤T2, and P a >T3, the log record is judged as a potential exception;

[0086] When P a ≤T3, then daily records are judged as normal activities;

[0087] If P b >P a :When P b >T3, the log record is judged as a potential exception;

[0088] When P b ≤T3, the log record is considered normal activity;

[0089] S233: If P num If it is not greater than the preset threshold T1, the log record is directly determined to be a normal activity.

[0090] Specifically, T1 is the overall risk threshold, which is used to quickly filter out logs that may have risks. It is the highest threshold; T2 is the high risk threshold, which is used to identify high-risk events that need to be understood and concerned, and it can distinguish between high risk and low risk; T3 is the low risk threshold, which is used to capture minor but noteworthy risks, and it can distinguish between risk and normal activities.

[0091] S24: Regularly adjust the set threshold according to the latest classification results to adapt to changes in the network environment.

[0092] S3: Perform real-time multi-level behavioral analysis on classified network security log data to identify anomalies and potential risks in the classified data.

[0093] Specifically, after analyzing and preliminarily classifying the network security log data, it is necessary to use comprehensive contextual information for in-depth analysis to identify complex attack patterns or subtle abnormal behaviors. These complex attack patterns or subtle abnormal behaviors may contain multiple steps or be disguised. Therefore, it is necessary to perform real-time multi-level behavioral analysis on the preliminarily classified network security log data.

[0094] The real-time multi-level behavior analysis of classified network security log data includes individual behavior analysis, group behavior analysis, global behavior analysis and correlation analysis:

[0095] Individual behavior analysis is used to identify uncommon conversation patterns or abnormal conversation times in real time. In individual behavior analysis, a user behavior baseline model is constructed, and the conversation data of each user or device is extracted in real time. The deviation between the current conversation data and the user behavior baseline model is calculated. When there are individuals in the conversation data that exceed the preset deviation threshold, the individuals are marked as abnormal.

[0096] Group behavior analysis is used to regularly identify group activities that are inconsistent with normal work processes. In group behavior analysis, the classified network security log data is grouped according to users, devices, and applications, and the overall activity pattern of each group is analyzed. A clustering algorithm is used to determine group activities that are inconsistent with normal work processes, and the identified abnormal group behaviors are marked as abnormal.

[0097] Global behavior analysis is used to continuously monitor behavior changes in network security logs. In global behavior analysis, time series analysis methods are used to analyze the behavior trends of collected network security log data to identify prominent behavior change patterns.

[0098] Correlation analysis is used to identify abnormal situations in individual behavior analysis or group behavior, and to correlate the results of individual behavior analysis, group behavior analysis, and global behavior analysis to determine the level and credibility of potential risks. In correlation analysis, the correlation between individual behavior, group behavior, and global behavior is established to determine the level and credibility of potential risks, including:

[0099] A multi-layer association graph is constructed, and individual behaviors, group behaviors and global behaviors are used as nodes in the multi-layer association graph; wherein, association edges can be established between individual behavior nodes according to information such as entities and operations; association edges can be established between group behavior nodes according to information such as categories, locations and time; association edges are established between individual behavior nodes and group behavior nodes according to belonging relationships, and association edges are established between global behavior nodes and individual behavior nodes and group behavior nodes according to information such as time and impact range;

[0100] Define the association strength, assign a corresponding association strength value to each association edge, and quantify the association strength between nodes; the calculation of the association strength value takes time correlation, frequency correlation, importance correlation and behavior similarity as considerations;

[0101] Risk parameter propagation, at the initial stage, according to the results of individual behavior analysis, group behavior analysis and global behavior analysis, the corresponding node is assigned an initial risk score to indicate the possibility of security risk in the node; then, the initial risk score is propagated in the multi-layer association graph, that is, the risk score of the node is updated according to the risk score and association strength of the adjacent nodes; the calculation formula of the risk score is:

[0102]

[0103] Among them, D i is the risk score of node i, D i0 is the initial risk score of node i, N(i) is the set of neighboring nodes of node i, α is the attenuation coefficient during the propagation of risk parameters, and w ij is the strength of the association edge between node i and node j, D j is the risk score of node j;

[0104] Determine the risk level and credibility, preset the level threshold, divide the potential risk level into low, medium and high, and judge the level of potential risk based on the final risk score combined with the preset level threshold; and evaluate the risk credibility based on the amount of node data calculated by the parameter risk score in the multi-layer association graph, the strength of the associated edge and the length of the risk propagation path. Among them, the more nodes involved in the calculation, the higher the strength of the associated edge and the shorter the risk propagation path, the higher the risk credibility.

[0105] S4: Based on the classification results and behavior analysis results of the network security logs, the network security logs are stored separately according to different storage strategies.

[0106] Specifically, in order to improve storage efficiency and query speed, different storage strategies need to be adopted to store network security logs separately:

[0107] S41: Based on the risk level of network security logs, set the corresponding storage tiering level, including fast-access storage media, low-cost high-capacity storage media, and a hybrid storage architecture of the two storage media;

[0108] S42: according to the classification results and behavior analysis results of the network security logs, setting storage priorities, wherein abnormal activity logs have the highest storage priority, potential risk logs have a medium priority, and normal activity logs have the lowest priority;

[0109] S43: Based on the results of risk assessment, dynamically adjust the allocation of storage resources; use fast-access storage media for high-risk logs, use low-cost and high-capacity storage media for low-risk logs, and use a hybrid storage architecture of two storage media for risk logs;

[0110] S44: By monitoring the access frequency of network security logs and the usage of storage media in real time, a multi-objective optimization algorithm is used to dynamically adjust the storage ratio and location of different types of logs between fast-access storage media, low-cost and high-capacity storage media, and hybrid storage architectures;

[0111] S45: Regularly optimize and organize storage resources, clean up redundant or outdated normal activity logs, and compress and archive low-frequency access potential abnormal logs; for high-risk network security logs, retain complete log information and convert them into medium-risk data regularly; for medium-risk network security logs, extract key information and convert them into low-risk data after the retention period expires; for low-risk network security logs, securely delete them after the retention period expires;

[0112] S5: Based on the stored multi-source network security logs, potential threat events are correlated, the source of the problem is located, and a detailed security incident report is generated.

[0113] Specifically, potential threat events are correlated and the source of the problem is located, including:

[0114] S51: sorting the collected network security logs in a unified timestamp format by timestamp to ensure the time continuity of events in the network security logs;

[0115] S52: Screening and marking network security logs by key information, including but not limited to IP addresses and time types, to identify duplicate or related events;

[0116] S53: Use machine learning algorithms to analyze the correlation between marked network security logs, build an event correlation graph, and generate a visual event development path; the constructed event correlation graph is dynamically updated as new network security logs are added, reflecting changes in network security situation in real time;

[0117] S54: Based on the anomaly detection model trained on historical network security logs, the associated network security logs and event development paths are analyzed to identify abnormal nodes and abnormal paths, and the network security logs or time related to potential threat events are located according to the abnormal nodes, and the abnormal correlation between the abnormal path events is determined;

[0118] S55: Automatically generate a security analysis report based on the event correlation diagram and anomaly detection results, including but not limited to event description, impact assessment, source analysis and response suggestions.

[0119] In this embodiment, a network security log management system is also disclosed, as shown in the attached Figure 2 As shown, it includes a log acquisition module, a log classification module, a behavior analysis module, a log storage module and a source analysis module.

[0120] The log acquisition module is used to collect and preprocess network security logs from various sources in real time, and preprocess network security logs. Network security logs include firewall logs, intrusion detection logs and access logs.

[0121] The log classification module is used to perform preliminary classification of the preprocessed network security logs into different categories.

[0122] The behavior analysis module is used to perform multi-level behavior analysis on the network security logs after preliminary classification and identify anomalies and potential risks in the classified data.

[0123] The log storage module is used to store network security logs separately according to different storage strategies based on the analysis results of network logs and behavior analysis results.

[0124] The source analysis module is used to correlate potential threat events based on the stored multi-source network security logs and locate the source of potential threat events.

[0125] The above-mentioned embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person skilled in the art, several modifications, improvements and substitutions can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims

1. A network security log management method, It is characterized in that The following steps are included: S1: collecting and preprocessing network security logs from various sources in real time, and preprocessing the network security logs, wherein the network security logs include firewall logs, intrusion detection system logs and access logs; S2: using a log classification model built based on the historical network security log data, preliminarily classifying the preprocessed network security log into different categories, including abnormal activities, potential abnormalities, and normal activities; S3: Perform real-time multi-level behavior analysis on the classified network security logs to identify anomalies and potential risks in the classified data; S4: Based on the classification result and behavior analysis result of the network security log, the network security log is stored separately according to different storage strategies; S5: Based on the stored network security logs from multiple sources, potential threat events are correlated, the source of the problem is located, and a detailed security incident report is generated.

2. A network security log management method according to claim 1, It is characterized in that The preliminary classification of the pre-processed network security log specifically includes: S21: extracting key features from the network security log, and calculating a feature vector for each log record in the network security log; S22: Inputting the feature vector into a pre-trained log classification model to obtain a probability distribution of whether the log record belongs to an abnormal activity, a potential abnormality, or a normal activity; S23: Determine the final category of the log record by threshold comparison according to the probability distribution; S24: Regularly adjust the set threshold according to the latest classification results to adapt to changes in the network environment.

3. A network security log management method according to claim 2, It is characterized in that Determining the final category of the log record by threshold comparison specifically includes: S231: Calculate the abnormal activity probability P a and the potential abnormal probability P b The sum of the probabilities P num ; S232: If P num If the probability of the abnormal activity and the potential abnormality is greater than the preset threshold value T1, the probability of the abnormal activity and the potential abnormality is further compared, wherein judgment thresholds T2 and T3 are also preset, and T1>T2>T3: If P a >P b :When P a >T2, the log record is determined to be an abnormal activity; When P a ≤T2, and P a >T3, the log record is determined to be a potential exception; When P a ≤T3, the daily record is judged to be a normal activity; If P b >P a :When P b >T3, the log record is determined to be a potential exception; When P b ≤T3, the log record is determined to be a normal activity; S233: If P num If the value is not greater than the preset threshold value T1, the log record is directly determined to be a normal activity.

4. A network security log management method according to claim 2, It is characterized in that When training, the log recognition model first extracts n key features from the log records to form a feature vector X=[x 1 ,x 2 ,…,x n ]; Then calculate the preliminary risk score R of the log record: Finally, the category probability distribution P(K) of the log record is calculated according to the preliminary risk score R: Wherein, P(K) represents the probability that the log record belongs to the abnormal activity, the potential abnormality or the normal activity; j represents the index of the category, a is the abnormal activity category, b is the potential risk category, and c is the normal activity category; β K is a regulating factor used to control the degree of distinction between categories; K represents the category to which the target belongs; and R is a risk score used to represent a comprehensive evaluation of the log record by all relevant features.

5. A network security log management method according to claim 1, It is characterized in that When performing real-time multi-level behavior analysis on the classified network security logs, the multi-level behavior analysis includes individual behavior analysis, group behavior analysis, global behavior analysis and correlation analysis: In individual behavior analysis, a user behavior baseline model is constructed, and the session data of each user or device is extracted in real time. The deviation between the current session data and the user behavior baseline model is calculated. When there is an individual in the session data that exceeds a preset deviation threshold, the individual is marked as abnormal. In the group behavior analysis, the classified network security logs are grouped according to users, devices and applications, the overall activity pattern of each group is analyzed, and a clustering algorithm is used to determine group activities that are inconsistent with normal work processes, and the identified abnormal group behaviors are marked as abnormal; In global behavior analysis, a time series analysis method is used to analyze the behavior trends of the collected network security logs to identify prominent behavior change patterns; In the association analysis, the results of the individual behavior analysis, the group behavior analysis and the global behavior analysis are comprehensively evaluated to identify potential threat events.

6. A network security log management method according to claim 5, It is characterized in that The individual behavior analysis is used to identify uncommon conversation patterns and abnormal conversation times in real time, the group behavior analysis is used periodically to identify group activities that are inconsistent with normal work processes, and the global behavior analysis is used to continuously monitor behavior changes in network security logs; when an anomaly is identified in the individual behavior analysis or the group behavior analysis, the individual behavior analysis, the group behavior analysis and the global behavior analysis results are correlated and analyzed to establish a correlation relationship between individual behavior, group behavior and global behavior, and based on the correlation relationship, the level and credibility of potential risks are judged.

7. A network security log management method according to claim 6, It is characterized in that The level and credibility of judging potential risks include: Constructing a multi-layer association graph, taking individual behavior, group behavior and global behavior as nodes in the multi-layer association graph, and establishing association edges between individual behavior nodes, group behavior nodes and global behavior nodes; Defining association strength, assigning a corresponding association strength value to each of the association edges, and quantifying the association strength between nodes; Risk parameter propagation: assigning initial risk scores to corresponding nodes based on the results of individual behavior analysis, group behavior analysis, and global behavior analysis; The risk score of the node is updated according to the risk scores of the adjacent nodes and the association strength; Determine the risk level and credibility, and determine the level of potential risk based on the final risk score and the preset level threshold; The credibility is evaluated based on the amount of node data calculated based on the parameter risk score in the multi-layer association graph, the strength of the association edge and the length of the risk propagation path.

8. A network security log management method according to claim 1, It is characterized in that The different storage strategies include: S41: Based on the risk level of the network security log, a corresponding storage tiering level is set, including a fast-access storage medium, a low-cost high-capacity storage medium, and a hybrid storage architecture of the two storage media; S42: according to the classification results and behavior analysis results of the network security logs, setting storage priorities, wherein abnormal activity logs have the highest storage priority, potential risk logs have a medium priority, and normal activity logs have the lowest priority; S43: Based on the results of risk assessment, dynamically adjust the allocation of storage resources; use fast-access storage media for high-risk logs, use low-cost and high-capacity storage media for low-risk logs, and use a hybrid storage architecture of two storage media for risk logs; S44: By monitoring the access frequency of network security logs and the usage of storage media in real time, a multi-objective optimization algorithm is used to dynamically adjust the storage ratio and location of different types of logs between fast-access storage media, low-cost and high-capacity storage media, and hybrid storage architectures; S45: Regularly optimize and organize storage resources, clean up redundant or outdated normal activity logs, and compress and archive potential abnormal logs that are accessed infrequently.

9. A network security log management method according to claim 1, It is characterized in that The process of correlating potential threat events and locating the source of the problem includes: S51: sorting the collected network security logs by timestamps to ensure the time continuity of events in the network security logs; S52: Screening and marking the network security logs by key information to identify existing duplicate or related events, wherein the key information includes IP addresses and time types; S53: Use machine learning algorithms to analyze the correlation between labeled network security logs, build event correlation graphs, and generate visual event development paths; S54: Analyze the associated network security logs and the event development path based on the anomaly detection model trained with historical network security logs to identify abnormal nodes and abnormal paths; S55: Automatically generate a security analysis report based on the event association diagram and the anomaly identification result.

10. A network security log management system, It is characterized in that Included are: A log acquisition module is used to collect and preprocess network security logs from various sources in real time, and preprocess the network security logs, wherein the network security logs include firewall logs, intrusion detection logs and access logs; A log classification module, used to preliminarily classify the pre-processed network security logs into different categories; Behavior analysis module, which is used to perform multi-level behavior analysis on the network security logs after preliminary classification to identify anomalies and potential risks in the classified data; A log storage module, used to store network security logs separately with different storage strategies according to the analysis results and behavior analysis results of the network logs; The source analysis module is used to correlate potential threat events based on the stored multi-source network security logs and locate the source of potential threat events.

Citation Information

Patent Citations

  • Anomaly analysis method and device for network data

    CN111245793A

  • Efficient network security protection method and system and storage medium

    CN118074951A

  • Data mining system and method based on computer network security

    CN118199988A

  • Network information security protection system

    CN118353702A

  • Power system security event log management method

    CN118468342A

Cited By

  • Industrial network security log collection method and system and medium

    CN119440961A

  • Network security log analysis method based on large language model

    CN119484162A

  • Private cloud security management method based on multi-dimensional feature data analysis

    CN119922014A

  • A Private Cloud Security Management Method Based on Multidimensional Feature Data Analysis

    CN119922014B

  • Security integration method and device for data classification and malicious behavior analysis and medium

    CN120639431A