Multi-source network security log data fusion analysis method and system

Abstract

The invention discloses a multi-source network security log data fusion analysis method and system, and relates to the technical field of network security, and the system comprises an alarm log collection module which is used for collecting original alarm logs of a multi-source device, carrying out the format unification, content mapping and merging of the original alarm logs, generating a standardized log, and storing the standardized log in a database; respectively storing the original alarm log and the standardized log; and the network security threat identification module is used for inputting the standardized log into a pre-trained network security threat identification model and outputting a security threat identification result. According to the method, the standard logs are generated by performing format unification, content mapping and merging on the original alarm logs, so that the security threat identification model can process and analyze the logs more effectively, and thus the speed and accuracy of threat detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and more particularly to a multi-source network security log data fusion analysis method and system. Background Art

[0002] At present, various security devices in the network operate independently and lack unified coordination, resulting in high rates of missed reports and false alarms of network threats, duplication of work in log analysis and event processing, and time-consuming and labor-intensive report production and difficulty in processing. At the same time, security devices manufactured by different manufacturers are heterogeneous, and the log formats they generate vary, the content varies in detail, and the risk assessment criteria also vary, which makes it difficult to obtain useful information. In addition, although the existing technology has made certain progress in multi-source log fusion, in the complex, open, and compatible application environment of basic telecommunications enterprises, the support for log fusion and parsing capabilities of third-party security products and platforms is poor, and it is impossible to efficiently deal with large-scale log files or records. Therefore, how to solve the above problems is an urgent problem that those skilled in the art need to solve. Summary of the invention

[0003] In view of this, the present invention provides a multi-source network security log data fusion analysis method and system, which overcomes the above-mentioned defects.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] A multi-source network security log data fusion analysis system, comprising:

[0006] An alarm log collection module is used to collect original alarm logs of multi-source devices, unify the format, map and merge the contents of the original alarm logs, generate standardized logs, and store the original alarm logs and the standardized logs separately;

[0007] The network security threat identification module is used to input the standardized log into a pre-trained network security threat identification model and output a security threat identification result; the network security threat identification model is constructed based on the CNN-BiLSTM network.

[0008] Optionally, the alarm log collection module includes:

[0009] Multi-analysis engine submodule: used to receive the original alarm logs of multiple source devices and identify the types of the original alarm logs; automatically match the corresponding analysis rules according to the types of the original alarm logs through the built-in analysis rule library, and convert the original alarm logs into alarm logs with a unified format;

[0010] The multi-dimensional mapping submodule is used to obtain standard alarm logs according to the alarm logs with unified format by using the preset mapping table and static-dynamic mapping mechanism;

[0011] The merging submodule is used to filter false alarms and merge similar alarms on the standard alarm logs to generate standardized logs.

[0012] Optionally, the multi-dimensional mapping submodule includes a static mapping unit and a dynamic mapping unit;

[0013] The static mapping unit is used to perform pattern matching on the alarm logs with a unified format according to the static rule base. If the match is successful, the key fields in the alarm logs are identified and extracted, and the key fields are mapped to standard fields based on a preset mapping table; if the match fails, the alarm logs are transmitted to the dynamic mapping unit;

[0014] The dynamic mapping unit is used to input the alarm log into a dynamic parsing model to obtain an estimated field, and to construct a temporary mapping rule based on historical data, and to map the estimated field into a standard field based on the temporary mapping rule.

[0015] Optionally, the merging submodule includes:

[0016] A false alarm filtering unit, used to perform multi-feature analysis on the standard alarm log, and identify and filter the false alarm log;

[0017] A graph database unit, used to obtain the attack behavior features involved in the filtered multiple standard alarm logs according to the pre-built graph database, and merge the standard alarm logs of the same type of threats based on the pre-built attack rule fingerprint library;

[0018] The alarm compression unit is used to generate security events according to the ATTCK matrix and the attack chain model, and compress the merged standard alarm log based on the security events using a graph algorithm.

[0019] Optionally, the network security threat identification module includes:

[0020] A spatial feature extraction submodule, used for performing multi-scale feature extraction on the standardized log to obtain spatial features;

[0021] A time feature extraction submodule, used for performing time series analysis on the spatial features to obtain extracted features;

[0022] The recognition submodule is used to obtain the recognition result according to the extracted features.

[0023] A multi-source network security log data fusion analysis method, the specific steps are:

[0024] Collecting original alarm logs of multiple source devices, unifying the formats, mapping the contents and merging the original alarm logs, generating standardized logs, and storing the original alarm logs and the standardized logs respectively;

[0025] The standardized log is input into a pre-trained network security threat identification model, and a security threat identification result is output; the network security threat identification model is constructed based on a CNN-BiLSTM network.

[0026] Optionally, the steps of generating the standardized log are:

[0027] Receive the original alarm logs of the multi-source devices and identify the types of the original alarm logs; automatically match the corresponding parsing rules according to the types of the original alarm logs through the built-in parsing rule library, and convert the original alarm logs into alarm logs with a unified format;

[0028] Obtain standard alarm logs using a preset mapping table and static-dynamic mapping mechanism based on alarm logs with a unified format;

[0029] False alarms are filtered out and similar alarms are merged on the standard alarm logs to generate standardized logs.

[0030] Optionally, the steps for obtaining the standard alarm log are:

[0031] The alarm log is pattern matched according to the static rule base. If the match is successful, the key fields in the alarm log are identified and extracted, and the key fields are mapped to standard fields based on a preset mapping table. If the match fails, the alarm log is input into a dynamic parsing model to obtain estimated fields, and a temporary mapping rule is constructed based on historical data. Based on the temporary mapping rule, the estimated fields are mapped to standard fields to generate a standard alarm log.

[0032] Optionally, the steps of merging the standard alarm logs are:

[0033] Perform multi-feature analysis on the standard alarm logs, and identify and filter false alarm logs;

[0034] Acquire the attack behavior features involved in the filtered multiple standard alarm logs according to the pre-built graph database, and merge the standard alarm logs of the same type of threats based on the pre-built attack rule fingerprint library;

[0035] Security events are generated according to the ATTCK matrix and the attack chain model, and the merged standard alarm logs are compressed based on the generated security events using a graph algorithm.

[0036] Optionally, the step of obtaining the security threat identification result is:

[0037] Inputting the standardized log into a pre-trained network security threat identification model to extract spatial features and temporal features;

[0038] A recognition result is generated based on the extracted spatial features and the extracted temporal features.

[0039] It can be seen from the above technical solutions that the present invention discloses a multi-source network security log data fusion analysis method and system, which has the following beneficial effects compared with the prior art:

[0040] Improve threat detection efficiency and accuracy: By unifying the format, mapping the content and merging the original alarm logs, standardized logs are generated, so that the security threat identification model can process and analyze these logs more effectively, thereby improving the speed and accuracy of threat detection.

[0041] Reduce false alarm and missed alarm rates: By filtering false alarms and grouping similar alarms, repetitive work is effectively reduced, and the false alarm and missed alarm rates in log analysis are reduced, which not only improves the reliability of the system, but also reduces the workload of security managers.

[0042] Improve security response speed: Due to the use of uncertainty quantification methods that combine spatial and temporal features, network attacks can be quickly and accurately identified in the early stages, winning a valuable time window for taking defensive measures in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A schematic diagram of the system structure provided by the present invention;

[0045] Figure 2 A schematic diagram of the content mapping process provided by the present invention;

[0046] Figure 3 The present invention provides a flow chart of the method. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] On the one hand, an embodiment of the present invention discloses a multi-source network security log data fusion analysis system. Figure 1 As shown, including:

[0049] The alarm log collection module is used to collect the original alarm logs of multi-source devices, unify the format, map and merge the original alarm logs, generate standardized logs, and store the original alarm logs and standardized logs separately;

[0050] The network security threat identification module is used to input standardized logs into a pre-trained network security threat identification model and output security threat identification results; the network security threat identification model is built based on the CNN-BiLSTM network.

[0051] In one embodiment, the alarm log collection module includes:

[0052] Multi-analysis engine submodule: used to receive original alarm logs from multiple source devices and identify the types of original alarm logs; through the built-in analysis rule library, automatically match the corresponding analysis rules according to the type of the original alarm log, and convert the original alarm log into an alarm log with a unified format;

[0053] The multi-dimensional mapping submodule is used to obtain standard alarm logs according to the alarm logs with unified format by using the preset mapping table and static-dynamic mapping mechanism;

[0054] The merging submodule is used to filter out false alarms and merge similar alarms from standard alarm logs to generate standardized logs.

[0055] In one implementation, the multi-dimensional mapping submodule includes a static mapping unit and a dynamic mapping unit;

[0056] The static mapping unit is used to perform pattern matching on the alarm logs with a unified format according to the static rule base. If the match is successful, the key fields in the alarm logs are identified and extracted, and the key fields are mapped to standard fields based on a preset mapping table; if the match fails, the alarm logs are transmitted to the dynamic mapping unit;

[0057] The dynamic mapping unit is used to input the alarm log into the dynamic parsing model to obtain the estimated field, and to build a temporary mapping rule based on the historical data, and to map the estimated field to the standard field based on the temporary mapping rule.

[0058] Furthermore, firstly, based on the syslog protocol, the big data component Logstash collects original alarm logs from various types of network security devices (such as DDOS, NDR, 5G edge security gateway, WAF, EDR, content compliance platform, data security platform, risk control platform, etc.). These logs may come from different manufacturers, different versions of security devices, and different adapted security scenarios. The generated alarm logs are different in format, name, quantity, meaning, etc., which brings great challenges to the realization of efficient and prepared alarm response disposal. Therefore, before using the data, it is necessary to standardize the data, specifically:

[0059] When receiving the original alarm log, the multi-analysis engine submodule first identifies the type of log by checking the log header information or specific tags. Then, using the built-in parsing rule library, it automatically matches and applies the corresponding parsing rules (such as regular expressions, grok parsing, JSON parsing, delimiter parsing, KV mode, CEF extraction, syslog standard parsing or other custom rules) according to the log type to convert the original log into an alarm log with a unified format.

[0060] The types of original alarm logs include network alarms, terminal alarms, and audit alarms. The standard field information contained in each type is as follows:

[0061] Network alarm: alarm time, attack source IP, source IP ownership, destination IP, destination IP ownership, threat type, attack type, alarm level, reporting device type, reporting device IP, region, attack details, etc.

[0062] Terminal alarm: alarm device type, device manufacturer, terminal IP, device IP, MAC, alarm level, client version, operating system, alarm time, reporting time, area, terminal name, terminal type, affiliated business system, threat type, attack type, attack method, etc.

[0063] Audit alarm: alarm time, region, reporting province, equipment manufacturer, equipment source, equipment source IP, equipment location, risk object, object ownership, risk object description, threat type, risk type, risk subcategory, and alarm level.

[0064] Among them, when the alarm log contains information such as the five-tuple, it will be classified as a network alarm; EDR, DLP and other devices that monitor based on terminal behavior will be classified as terminal alarms; content compliance platforms, audit platforms and other devices that mainly monitor compliance-related behaviors will be classified as audit alarms. In specific applications, the appropriate type can be matched according to the device situation.

[0065] Furthermore, in the multi-dimensional mapping sub-module, according to the mapping table preset in the system, the preset static-dynamic mapping mechanism is used to obtain the standard alarm log, specifically: the multi-dimensional mapping sub-module first receives the alarm log in a unified format from the multi-parsing engine sub-module, and first uses the static rule library to try to match the known pattern, identify and extract the key fields in the log, such as timestamp, event type, source address, etc.; if the match fails, the alarm log is transferred to the dynamic parsing model to use natural language processing (NLP) technology and context-aware algorithms to understand the content of unstructured text, and infer possible new fields and their meanings based on similarities with historical cases; for those fields marked as "dynamic", the system will analyze their distribution characteristics based on historical data and build temporary mapping rules.

[0066] Furthermore, after the field mapping is completed, the original field value and the standardized field value are mapped. For example, for the attack type field, the original field value is [0, 1, 2], and the mapping dictionary can be configured to convert it into [weak password, Trojan horse, XSS] to achieve standardization of field values.

[0067] like Figure 2 As shown, this embodiment matches and maps threat types, attack types, and alarm levels through a mapping table. This embodiment matches and maps the original alarm type information with 11 pre-set threat types, hundreds of attack types, and three alarm levels, thereby achieving standardization of threat types, standardization of attack types, and unification of alarm levels for heterogeneous alarms.

[0068] In order to achieve standardized description and response processing flow of security events, this embodiment further standardizes the types of events. After the alarm triggers and generates a security event, the security event type is marked to achieve standardized unification of event types and descriptions. A partial event type table is shown in Table 1.

[0069] Table 1

[0070]

[0071] In one embodiment, the merging submodule includes:

[0072] False alarm filtering unit, used to perform multi-feature analysis on standard alarm logs, identify and filter false alarm logs;

[0073] A graph database unit, used to obtain attack behavior features involved in multiple filtered standard alarm logs according to a pre-built graph database, and merge standard alarm logs of the same type of threats based on a pre-built attack rule fingerprint library;

[0074] The alarm compression unit is used to generate security events according to the ATTCK matrix and the attack chain model, and compress the merged standard alarm logs based on the generated security events using a graph algorithm.

[0075] Furthermore, the false alarm filtering unit includes:

[0076] Preliminary Classification Subunit: Apply machine learning algorithms to train AI models so that they can automatically learn and distinguish the difference between normal and malicious activities;

[0077] Marking subunit: When a newly generated alarm log enters the system, the AI ​​model first performs a preliminary classification and marks the logs that may be false alarms;

[0078] Human review subunit: Security analysts then review logs flagged as potential false positives and decide whether to accept the AI ​​model’s recommendations or adjust model parameters to improve accuracy;

[0079] Feedback loop subunit: A feedback loop mechanism is introduced to allow security analysts to feed back actual processing results to the AI ​​model, thereby continuously optimizing model performance.

[0080] Furthermore, the graph database unit includes:

[0081] A relation storage subunit, used to store alarm logs and their mutual relations in the form of nodes and edges;

[0082] The relationship analysis subunit is used to perform correlation analysis on the alarm logs stored in the graph database based on the graph algorithm to identify possible complex attack patterns;

[0083] The dynamic update sub-unit continuously monitors changes in network behavior patterns, updates and improves the attack rule fingerprint library in real time, and ensures that it can respond to new network attack patterns in a timely manner.

[0084] Furthermore, the false alarm filtering unit can use the combination of security experts' experience judgment and artificial intelligence (AI) models to identify and filter potential false alarms. In this embodiment, the experience judgment of security experts and the learning ability of AI models are effectively combined through semi-supervised machine learning, and automatic and intelligent screening is performed from aspects such as attack results in alarms, historical related event judgment, business traffic rules, threat intelligence, and multi-feature associations in traffic (including horizontal analysis of multi-field collaboration in the same alarm, and vertical correlation analysis of traffic behavior trends in the same traffic sequence). From the massive alarms, the alarms triggered by normal business, missed scan alarms, and unsuccessful alarms are identified, and the merging and screening strategies are optimized, thereby greatly reducing noise, improving the "signal-to-noise ratio", making the real alarms emerge, and improving the accuracy of security event mining, so as to achieve the effect of using the least manpower to judge the most security risks. For example, when the system detects that an IP address frequently triggers alarms, but after analysis, it is found that the IP address belongs to the normal business traffic within the company, the false alarm filtering unit will use the AI ​​model to intelligently identify the alarm log of the IP address and filter out the false alarms. At the same time, the unit will continue to learn and optimize the AI ​​model to improve the accuracy and efficiency of false alarm filtering.

[0085] Furthermore, the graph database unit receives alarm logs from various security devices and extracts alarm associations, such as attack source, attack target, attack time, etc. Then, it uses the pre-built attack rule fingerprint library to merge alarm logs of the same type of threats. For example, when the system detects multiple DDoS attack alarms from the same IP address, the graph database unit will merge these alarm logs into one record and display the associations between them. At the same time, the unit will also analyze the attack behavior characteristics between these alarm logs, such as attack intensity, attack frequency, etc., to further understand the attacker's behavior pattern.

[0086] The alarm compression unit first uses the ATTCK matrix and attack chain model to classify and mark the alarm logs, and then combines the customized security event model to use the graph algorithm to identify and compress multiple alarm logs belonging to the same attack event. The threat types of the system are classified and labeled in the form of the ATTCK matrix to identify different techniques and strategies used by attackers, such as exploiting vulnerabilities, social engineering, malware, etc. Multiple alarms are matched with different techniques and strategies in the ATTCK framework, and multiple alarm logs of the same source but different threat types are summarized into a preliminary security event. At the same time, combined with the six-step attack chain model of reconnaissance, intrusion, permission acquisition, command control, malicious behavior, and trace cleaning, the nature of preliminary security events of different attack progress is analyzed to form a final security event to further compress alarms. For example, primary security events in the reconnaissance stage within a specified time are merged into a final security event, and primary security events in the command control stage are immediately formed into a final security event.

[0087] In one embodiment, the storage strategy is mainly divided into two parts. The first part is the storage of the original alarm log, and the second part is the storage of the standardized log. For the storage of the alarm log, a storage solution combining file storage + TiDB + ES is adopted. In order to meet the traceability and compliance requirements, the original alarm log part will be compressed and stored in the form of files. For the standardized log, a storage architecture combining TiDB and ES is adopted. The cold data with earlier time series, lower access frequency, and infrequent updates is transferred to ES for storage. TiDB is used to store hot data. After the separation of storage and computing, the computing unit and the storage unit are further decoupled, and each unit can achieve independent dynamic expansion and contraction. The storage architecture of this solution can dynamically adjust resource configuration according to actual needs, improving the flexibility and scalability of the system.

[0088] In one embodiment, the encryption technology of rsyslog+gnults is used to achieve the purpose of improving the security of logs and solve the problem of low security of traditional log transmission.

[0089] In one embodiment, the network security threat identification module includes:

[0090] The spatial feature extraction submodule is used to extract multi-scale features from the standardized logs to obtain spatial features;

[0091] The time feature extraction submodule is used to perform time series analysis on spatial features to obtain extracted features;

[0092] The recognition submodule is used to generate recognition results based on the extracted features.

[0093] Furthermore, the spatial feature extraction submodule adopts a spatial feature extraction method based on a multi-scale convolutional neural network. This method uses convolution kernels of different sizes to capture features of different scales, and introduces an attention mechanism so that the model can automatically learn which features are more important for prediction, thereby enhancing the performance and accuracy of the model.

[0094] The time feature extraction submodule uses a bidirectional long short-term memory network with residual connections to process the time series characteristics of the received spatial feature extraction submodule output data, improve the capture accuracy of time-related features, and dynamically adjust the time window size input to the LSTM according to the time interval of the event, so as to more flexibly respond to different types of security events;

[0095] The identification submodule identifies network security threats based on the output of the time feature extraction submodule and outputs the identification results.

[0096] In another embodiment, a policy scheduling submodule is also included, which introduces an uncertainty quantification method to evaluate the probability distribution of the model output and then generate a deployment instruction.

[0097] Furthermore, the spatial feature extraction submodule is used to perform multi-scale feature extraction on standardized logs to obtain spatial features. For example, for DNS logs, domain name length, character ratio, domain name information entropy, number of subdomains, domain name binary ratio, domain name vowel ratio, etc. can be used as input features for feature extraction through the CNN network. At the same time, the subdomains are dictionary encoded as the input of the CNN convolutional neural network to further capture the spatial features of the domain name.

[0098] The time feature extraction submodule is used to perform time series analysis on spatial features to obtain extracted features. For example, for DNS logs, time dimension features such as the number of abnormal second-level domain name requests per second, the number of abnormal second-level domain name requests per hour, the request failure ratio per hour, the number of different subdomain name requests per hour, and the request type ratio of TXT per hour can be counted. Then, these time dimension features are used as the input of the BiLSTM network for time series analysis to capture time-related features. At the same time, the time window size input to the LSTM is dynamically adjusted according to the time interval of the event to more flexibly respond to different types of security events.

[0099] The policy scheduling submodule is used to generate policy scheduling instructions based on the generated recognition results. This submodule introduces uncertainty quantification methods to evaluate the probability distribution of model outputs, provides confidence scores, and generates scheduling instructions. Specifically, it uses Monte Carlo to model uncertainty, uses the Monte Carlo method to sample the model, generates multiple sampling samples, and uses these samples to calculate the statistical characteristics of the recognition results. Through multiple samplings, each time using different model parameters or data sampling, a set of prediction results is generated, thereby providing uncertainty information about the recognition results; based on the recognition results and the uncertainty information of the recognition results, a policy scheduling instruction is generated, and the response strategy pre-stored in the scheduling policy library is dispatched according to the mobilization instruction to ensure network security.

[0100] In one embodiment, the model adopts an incremental learning framework, allowing the model to continuously update weights as new data arrives without retraining the entire model, thereby maintaining the real-time and adaptability of the system.

[0101] In another aspect, an embodiment further discloses a multi-source network security log data fusion analysis method, such as Figure 3 As shown, the specific steps are:

[0102] Collect the original alarm logs of multi-source devices, unify the format, map and merge the original alarm logs, generate standardized logs, and store the original alarm logs and standardized logs separately;

[0103] The standardized logs are input into the pre-trained network security threat identification model to output the security threat identification results; the network security threat identification model is built based on the CNN-BiLSTM network.

[0104] In one embodiment, the steps for generating a standardized log are:

[0105] Receive original alarm logs from multiple source devices and identify the types of original alarm logs; automatically match corresponding parsing rules according to the types of original alarm logs through the built-in parsing rule library to convert original alarm logs into alarm logs with unified format;

[0106] Obtain standard alarm logs using a preset mapping table and static-dynamic mapping mechanism based on alarm logs with a unified format;

[0107] Filter false alarms and merge similar alarms in standard alarm logs to generate standardized logs.

[0108] In one embodiment, the steps for obtaining the standard alarm log are:

[0109] The alarm log is pattern matched according to the static rule base. If the match is successful, the key fields in the alarm log are identified and extracted, and the key fields are mapped to standard fields based on the preset mapping table. If the match fails, the alarm log is input into the dynamic parsing model to obtain the estimated fields, and a temporary mapping rule is built based on historical data. The estimated fields are mapped to standard fields based on the temporary mapping rule to generate a standard alarm log.

[0110] In one embodiment, the steps for merging standard alarm logs are:

[0111] Perform multi-feature analysis on standard alarm logs to identify and filter false alarm logs;

[0112] Obtain attack behavior features involved in multiple standard alarm logs based on the pre-built graph database, and merge standard alarm logs of the same type of threats based on the pre-built attack rule fingerprint library;

[0113] Security events are generated according to the ATTCK matrix and attack chain model, and the merged standard alarm logs are compressed based on the generated security events using graph algorithms.

[0114] In one embodiment, the steps of obtaining the security threat identification result are:

[0115] Input the standardized logs into the pre-trained network security threat identification model to extract spatial and temporal features;

[0116] The recognition results are obtained based on the extracted spatial features and temporal features.

[0117] Furthermore, the network security threat identification model is a model built based on the CNN-BiLSTM network. A training set is constructed to iteratively train the CNN-BiLSTM network to obtain the network security threat identification model. Based on the results output by the model, corresponding response measures are taken, including but not limited to alarming, isolating the affected system or further investigation.

[0118] Assuming that the above system is deployed in an enterprise intranet, if an employee's computer is found to be infected with a ransomware virus, the system will quickly identify this abnormal activity and classify it as a "terminal infection" security scenario. Next, according to the previously set rules, the system will first SSH to the firewall to block the IP address of the infected machine to prevent the virus from spreading; then notify the desktop management system to push a warning message to other employees; and finally contact the access control system to disconnect the infected machine from the network. At the same time, the system will also record all actions taken and their effectiveness in order to further improve its own protection mechanism.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0120] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-source network security log data fusion analysis system, characterized in that: include: An alarm log collection module is used to collect original alarm logs of multi-source devices, and unify the format, map and merge the contents of the original alarm logs to generate standardized logs, and store the original alarm logs and the standardized logs separately; A network security threat identification module, used to input the standardized log into a pre-trained network security threat identification model and output a security threat identification result; The network security threat identification model is built based on the CNN-BiLSTM network; The alarm log collection module includes: Multi-analysis engine submodule: used to receive the original alarm logs of multiple source devices and identify the types of the original alarm logs; automatically match the corresponding analysis rules according to the types of the original alarm logs through the built-in analysis rule library, and convert the original alarm logs into alarm logs with a unified format; The multi-dimensional mapping submodule is used to obtain standard alarm logs according to the alarm logs with unified format by using the preset mapping table and static-dynamic mapping mechanism; The merging submodule is used to filter false alarms and merge similar alarms on the standard alarm logs to generate standardized logs.

2. According to claim 1, a multi-source network security log data fusion analysis system is characterized in that: The multi-dimensional mapping submodule includes a static mapping unit and a dynamic mapping unit; The static mapping unit is used to perform pattern matching on the alarm logs with a unified format according to the static rule base, and if the match is successful, identify and extract the key fields in the alarm logs, and map the key fields to standard fields based on a preset mapping table; If the match fails, transmitting the alarm log to the dynamic mapping unit; The dynamic mapping unit is used to input the alarm log into a dynamic parsing model to obtain an estimated field, and to construct a temporary mapping rule based on historical data, and to map the estimated field into a standard field based on the temporary mapping rule.

3. According to claim 1, a multi-source network security log data fusion analysis system is characterized in that: The merging submodule comprises: A false alarm filtering unit, used to perform multi-feature analysis on the standard alarm log, and identify and filter the false alarm log; A graph database unit, used to obtain the attack behavior features involved in the filtered multiple standard alarm logs according to the pre-built graph database, and merge the standard alarm logs of the same type of threats based on the pre-built attack rule fingerprint library; The alarm compression unit is used to generate security events according to the ATTCK matrix and the attack chain model, and compress the merged standard alarm log based on the security events using a graph algorithm.

4. The multi-source network security log data fusion analysis system according to claim 1 is characterized in that: The network security threat identification module includes: A spatial feature extraction submodule, used for performing multi-scale feature extraction on the standardized log to obtain spatial features; A time feature extraction submodule, used for performing time series analysis on the spatial features to obtain extracted features; The recognition submodule is used to obtain the recognition result according to the extracted features.

5. A multi-source network security log data fusion analysis method, characterized in that: The specific steps are: Collecting original alarm logs of multiple source devices, unifying the formats, mapping the contents and merging the original alarm logs, generating standardized logs, and storing the original alarm logs and the standardized logs respectively; Inputting the standardized log into a pre-trained network security threat identification model and outputting a security threat identification result; the network security threat identification model is constructed based on a CNN-BiLSTM network; The steps for generating the standardized log are as follows: Receive the original alarm logs of the multi-source devices and identify the types of the original alarm logs; automatically match the corresponding parsing rules according to the types of the original alarm logs through the built-in parsing rule library, and convert the original alarm logs into alarm logs with a unified format; Obtain standard alarm logs using a preset mapping table and static-dynamic mapping mechanism based on alarm logs with a unified format; False alarms are filtered out and similar alarms are merged on the standard alarm logs to generate standardized logs.

6. A multi-source network security log data fusion analysis method according to claim 5, characterized in that: The steps for obtaining the standard alarm log are: Performing pattern matching on the alarm log according to the static rule base, if the match is successful, identifying and extracting key fields in the alarm log, and mapping the key fields to standard fields based on a preset mapping table; If the match fails, the alarm log is input into the dynamic parsing model to obtain the estimated field, and a temporary mapping rule is constructed based on the historical data. The estimated field is mapped to a standard field based on the temporary mapping rule to generate a standard alarm log.

7. The multi-source network security log data fusion analysis method according to claim 5 is characterized in that: The steps for merging the standard alarm logs are as follows: Perform multi-feature analysis on the standard alarm logs, and identify and filter false alarm logs; Acquire the attack behavior features involved in the filtered multiple standard alarm logs according to the pre-built graph database, and merge the standard alarm logs of the same type of threats based on the pre-built attack rule fingerprint library; Security events are generated according to the ATTCK matrix and the attack chain model, and the merged standard alarm logs are compressed based on the security events using a graph algorithm.

8. The multi-source network security log data fusion analysis method according to claim 5 is characterized in that: The steps for obtaining the security threat identification result are: Inputting the standardized log into a pre-trained network security threat identification model to extract spatial features and temporal features; A recognition result is obtained according to the extracted spatial features and the temporal features.

Images