A network security threat information early warning method and system based on big data

Abstract

The application discloses the network security early warning technical field, disclose a kind of network security threat information early warning method and system based on big data, comprising: based on big data collection and integration about network security internal data and external data, natural language processing analysis unstructured threat intelligence, threat entity association is built by knowledge graph;Based on big data of network security threat information and deep learning constructs adaptive threat model, uses convolutional neural network CNN to identify abnormal mode in encrypted traffic;Through long short-term memory network LSTM establishes user-device behavior baseline;In combination with graph neural network GNN, the threat diffusion path is predicted, a threat risk quantization model is developed, and the information risk value is calculated by comprehensively considering the attack success rate, asset value, influence range value and response delay;Based on the information risk value calculated, combined with information risk attenuation factor, risk prediction is carried out on network security threat information.

Description

Technical Field

[0001] This invention relates to the field of network security early warning technology, specifically to a method and system for early warning of network security threats based on big data. Background Technology

[0002] Existing technologies rely on limited data sources, focusing only on internal network logs or simple external open-source information, making it difficult to comprehensively understand the cybersecurity threat landscape, break down data silos, and achieve a complete and accurate understanding of cybersecurity threats. Some technologies fail to effectively standardize data from different sources during data integration, leading to inconsistent data formats and increasing the difficulty of subsequent analysis. For large amounts of unstructured threat intelligence, there is a lack of effective parsing methods. Traditional methods use single technologies for threat detection, such as simple rule matching or signature-based detection, which are insufficient for detecting new or complex cybersecurity threats. Existing threat detection models lack adaptability and cannot adjust to the dynamic changes in cybersecurity threats. When assessing cybersecurity threat risks, considering only a few factors is insufficient, failing to comprehensively and accurately assess risks. Existing risk prediction technologies use relatively simple methods, based on simple statistical analysis of historical data, which lacks accuracy and timeliness in predicting future risks, pays little attention to the cross-domain propagation of cybersecurity threats, and does not fully consider the ability of threats to breach network boundaries. Therefore, there is a need to provide a big data-based method and system for early warning of cybersecurity threats. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for early warning of network security threats based on big data. To solve the above-mentioned problems in the prior art, this invention achieves this through the following technical solution:

[0004] Firstly, a method for early warning of cybersecurity threats based on big data includes the following steps:

[0005] Based on big data, internal and external cybersecurity data are collected and preprocessed, natural language processing is used to parse unstructured threat intelligence, and knowledge graphs are used to construct the relationship between threat entities.

[0006] Based on big data cybersecurity threat information and deep learning, an adaptive threat model is built to identify abnormal patterns in encrypted traffic; a threat risk quantification model is developed to comprehensively consider attack success rate, asset value, impact range value and response latency to calculate information risk value.

[0007] Based on the calculated information risk value, and combined with the information risk attenuation factor, risk prediction is made for cybersecurity threat information.

[0008] Secondly, a big data-based cybersecurity threat information early warning system includes the following modules:

[0009] Data collection module: Based on big data, collects and integrates internal and external data related to cybersecurity;

[0010] Data preprocessing module: For structured data, it compares and merges with existing structured information in the knowledge graph; for unstructured data, it performs in-depth analysis based on Natural Language Processing (NLP) technology; it identifies entities related to cybersecurity threats from unstructured text, using a pre-trained Named Entity Recognition (NER) model, and fine-tunes a model based on the pre-trained BERT language model using the Transformer deep learning architecture on cybersecurity datasets to improve recognition accuracy; based on the continuous influx and updates of new internal and external data, it continuously uses NLP technology to analyze new data, adding new threat entities and relationships to the knowledge graph; when new attack methods targeting application vulnerabilities are discovered, it promptly updates the corresponding nodes and edges in the knowledge graph; it uses existing information in the knowledge graph to perform relationship reasoning and discover potential threat associations;

[0011] Threat Model Building Module: Based on big data cybersecurity threat information and deep learning, an adaptive threat model is built, using a convolutional neural network (CNN) to identify abnormal patterns in encrypted traffic; a user-device behavior baseline is established through a long short-term memory (LSTM) network.

[0012] Threat Risk Quantification Module: Combines graph neural network (GNN) to predict threat propagation paths, develops a threat risk quantification model, and comprehensively considers attack success rate, asset value, impact range value and response delay to calculate information risk value;

[0013] Risk prediction module: Based on the calculated information risk value, combined with the information risk attenuation factor, risk prediction is performed on network security threat information.

[0014] The beneficial effects of this invention are:

[0015] 1. Comprehensive data collection methods acquire cybersecurity-related information from multiple perspectives, resulting in a more complete and accurate understanding of cybersecurity threats. Text cleaning removes noise and irrelevant information, improving data quality. NER is used to extract key entities from unstructured text, and relationships between threat entities are extracted based on semantic analysis results. A knowledge graph is then constructed. Different processing methods are applied to structured and unstructured data to ensure timely processing and utilization. Existing information in the knowledge graph is used for relationship reasoning to discover potential threat associations, enhancing the system's ability to understand and analyze cybersecurity threat information.

[0016] 2. An adaptive threat model is constructed based on big data and deep learning. The detection response delay obtained by combining the time interval from threat occurrence to detection and the threat assessment and processing time is used to calculate the information risk value, and the risk level is classified accordingly. Based on the calculated information risk value, a historical information risk value sequence is obtained. The information risk attenuation factor is calculated by combining the historical information risk value sequence with the verification node value. The risk prediction value is obtained by weighting the information risk attenuation factor, the risk information entropy value and the spatiotemporal change gradient. The model is continuously iterated and updated to adapt to the dynamic changes of threats, promptly detect potential security risks, and provide timely early warning information for network security protection. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the steps of a network security threat information early warning method based on big data provided in Embodiment 1 of the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of a network security threat information early warning system based on big data, provided in Embodiment 2 of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] Example 1

[0022] like Figure 1 As shown in the figure, the network security threat information early warning method based on big data provided by this embodiment of the invention specifically includes the following steps:

[0023] Step 1: Collect internal and external data on cybersecurity, preprocess them to obtain structured or unstructured threat intelligence, parse the unstructured threat intelligence through natural language processing, and use knowledge graphs to construct the relationship between threat entities;

[0024] In step one:

[0025] Firstly, the specific process for collecting internal and external data related to network security is as follows:

[0026] Internal network security data is acquired through internal storage, including network system logs, endpoint behavior, vulnerability scans, and asset inventories. Simultaneously, external network security data is collected through big data analytics, including open-source data, threat intelligence platforms, and dark web data.

[0027] By aggregating internal and external data into a unified data storage platform, preparations can be made for subsequent processing. During the data integration process, the data needs to be standardized to ensure that the data formats from different sources are consistent.

[0028] The network system logs generated by firewalls, intrusion detection systems, servers and various applications are collected based on internal storage. The key data recorded in the network system logs include, but are not limited to: network traffic, user login attempts, system operations and error messages. Firewall logs show the source IP, destination IP, port number and whether the connection is allowed.

[0029] Based on internal data analysis, the company's network assets are reviewed, covering both hardware and software assets, including but not limited to: servers, computers, network switches, operating systems, business applications and version information, as well as the departments and responsible persons associated with these assets. The scope of this asset information in assessing the threat impact is clearly defined. and asset value ;

[0030] By aggregating internal and external data into a unified data storage platform, preparations can be made for subsequent processing. During the data integration process, the data needs to be standardized to ensure that the data formats from different sources are consistent.

[0031] Secondly, the specific process of preprocessing to obtain structured or unstructured threat intelligence is as follows:

[0032] When new internal and external data flow into the system, the new internal and external data are transferred to the real-time data cache. The real-time cache performs preliminary classification and filtering of the new data, distinguishing between structured and unstructured data.

[0033] For structured data, if it is an accurate vulnerability scan report from an internal system, it can be directly compared and fused with the existing structured information in the knowledge graph;

[0034] The collected external data contains a large amount of irrelevant information, noise, and formatting errors; text cleaning is performed to remove Hypertext Markup Language (HTML) tags, special characters, and stop words, convert the text to plain text format, and perform case conversion; the cleaned text is then segmented into individual words or tags.

[0035] It should be noted that Hypertext Markup Language (HTML) is a markup language that uses a series of tags to unify the format of documents on the web, connecting scattered data resources into a logical whole. Removing HTML tags improves text readability, facilitates text processing, and reduces error display.

[0036] Using Named Entity Recognition (NER) technology, key entities are extracted from unstructured text, excluding but not limited to: hacker organization names, malicious IP addresses, common vulnerability disclosure CVE numbers, vulnerability names, virus names, Trojan families, operating system names, and application names.

[0037] It should be noted that Named Entity Recognition (NER) technology is used to identify entities with specific meanings in text; Common Vulnerability Disclosure (CVE) numbers give a public name to widely recognized information security vulnerabilities or exposed weaknesses, helping users share data in their respective independent vulnerability databases and vulnerability assessment tools;

[0038] The identified entities are classified and labeled into categories including, but not limited to, malware, vulnerabilities, and assets, in preparation for building nodes in the knowledge graph;

[0039] By utilizing dependency parsing and semantic role labeling techniques, we can understand the semantic relationships between words in a sentence and perform word segmentation and sentence segmentation on the text.

[0040] The role of dependency parsing is to reveal the syntactic structure by analyzing the dependency relationships between components within a linguistic unit. Semantic role labeling is a shallow semantic analysis technique that aims to analyze the relationship between predicates and arguments in a sentence.

[0041] Based on the analysis results, the relationships between threatening entities are extracted to form edges in the knowledge graph;

[0042] Thirdly, the specific process of parsing unstructured threat intelligence through natural language processing is as follows:

[0043] For unstructured data, in-depth analysis is performed based on Natural Language Processing (NLP) techniques;

[0044] It should be noted that Natural Language Processing (NLP) technology refers to enabling computers to understand, process, and generate human natural language, thereby achieving effective communication and interaction between humans and computers using natural language, including but not limited to: knowledge from multiple disciplines such as linguistics, computer science, and mathematics.

[0045] Specifically, unstructured data is collected from multiple sources, and regular expressions are used to remove noise and irrelevant information from the data;

[0046] Convert all text to a uniform case format, usually lowercase, to avoid semantic understanding errors caused by different case values;

[0047] The NLTK library for natural language processing is used for stemming and lemmatization to reduce words to their basic forms and reduce vocabulary diversity.

[0048] This method identifies entities related to cybersecurity threats from unstructured text, uses a pre-trained Named Entity Recognition (NER) model, and fine-tunes the pre-trained language model BERT based on the Transformer deep learning architecture on a cybersecurity dataset to improve recognition accuracy.

[0049] It should be noted that the Named Entity Recognition (NER) model is used to identify entities with specific meanings in text and extract meaningful nouns or phrases from the text, providing a foundation for building knowledge graphs. The deep learning architecture Transformer is used to process long sequence data and capture dependencies in the data through a self-attention mechanism, thereby improving the model's generative ability. The pre-trained language model BERT uses a bidirectional Transformer architecture to capture deep dependencies between words and solve the problem of polysemy.

[0050] Fourthly, specifically, the process of constructing the relationship between threat entities using knowledge graphs is as follows:

[0051] Threat entities identified and extracted using Natural Language Processing (NLP) technology are used as nodes in a knowledge graph, and relationships between entities are used as edges. Attributes are added to each node and edge. For example, the attributes of malware nodes may include release time and propagation path; the attributes of infection relationship edges include, but are not limited to, infection time and infection scope.

[0052] Knowledge graph storage is based on graph databases, which can handle the relationship query of nodes and edges, and quickly retrieve and analyze the associations between threat entities.

[0053] Based on the continuous inflow and updates of new internal and external data, we continuously use Natural Language Processing (NLP) technology to analyze new data, add new threat entities and relationships to the knowledge graph, and promptly update the corresponding nodes and edges in the knowledge graph when new attack methods targeting application vulnerabilities are discovered.

[0054] By leveraging existing information in knowledge graphs, we can perform relational reasoning to uncover potential threat connections.

[0055] For example, it is known that vulnerability A exists in software B, which is provided by vendor C. By reasoning, it is deduced that vulnerability A may affect the potential relationships of vendor C's customers, and these potential relationships are added to the knowledge graph to further improve the association network of threat intelligence.

[0056] The technical solution of this invention is as follows: A comprehensive data collection method obtains network security-related information from multiple perspectives, breaking down data silos and providing a more complete and accurate understanding of network security threats. During data integration, data from different sources is standardized to ensure consistent data formats. Multiple natural language processing technologies are comprehensively used to remove noise and irrelevant information through text cleaning, improving data quality. Key entities are extracted from unstructured text using NER, and relationships between threat entities are extracted based on semantic analysis results. A knowledge graph is constructed to intuitively display the associations between threat entities. When new internal and external data flow into the system, the data is transferred to a real-time data cache in real time and undergoes preliminary classification and filtering. Different processing methods are adopted for structured and unstructured data to ensure timely processing and utilization of data. Existing information in the knowledge graph is used for relationship reasoning to discover potential threat associations, thereby improving the system's ability to understand and analyze network security threat information.

[0057] Example 2

[0058] like Figure 1 As shown in the figure, the network security threat information early warning method based on big data provided by this embodiment of the invention specifically includes the following steps:

[0059] Step 2: Based on big data network security threat information and deep learning, construct an adaptive threat model, use convolutional neural networks (CNN) to identify abnormal patterns in encrypted traffic; establish a user-device behavior baseline through long short-term memory networks (LSTM); combine graph neural networks (GNN) to predict threat propagation paths, develop a threat risk quantification model, and comprehensively consider attack success rate, asset value, impact range, and response latency to calculate information risk value.

[0060] It should be noted that Convolutional Neural Networks (CNNs) are a type of feedforward neural network that includes convolutional computation and has a deep structure; Long Short-Term Memory Networks (LSTMs) are a type of recurrent neural network used to address the long-term dependency problem present in general recurrent neural networks; and Graph Neural Networks (GNNs) are used to learn graph-structured data using neural networks, extracting and discovering features and patterns in graph-structured data.

[0061] In step two;

[0062] Firstly, specifically, the process of obtaining the success rate of cybersecurity threat information attacks is as follows:

[0063] The Common Vulnerability Scoring System (CVSS) obtains a basic vulnerability score for cybersecurity threat information. It assesses the attacker's capabilities by evaluating their historical attack records, tools, and techniques, and assesses the defense capabilities of the target system's various security measures. Finally, it calculates the attack success rate by weighting the basic vulnerability score, the attacker's capabilities, and the defense capabilities. The weighting factors for the vulnerability baseline score, attacker capability, and attack defense capability can be set to 0.51, 0.32, and 0.17, respectively. The weighting factor values ​​can be adjusted according to the actual situation.

[0064] It should be noted that the Vulnerability Scoring System (CVSS) is an open framework for communicating the characteristics and severity of software vulnerabilities, quantifying vulnerability risk levels, and standardizing communication language.

[0065] Secondly, specifically, the process for obtaining the information risk value of cybersecurity threat information is as follows:

[0066] The system collects the time interval from the occurrence of a cybersecurity threat to its detection by the detection system. After a threat is detected, it obtains the time spent by the system in assessing the threat, determining its type, severity, and scope of impact. The processing times are then combined to obtain the detection response latency. ;

[0067] Based on the impact range of the collected cybersecurity threat information and the asset value of the target, the attack success rate and detection response latency are calculated comprehensively using the formula. The information risk value of cybersecurity threat information is calculated. ,in, This indicates the success rate of attacks targeting cybersecurity threats. , This represents the value of detection assets used to detect cybersecurity threats. , This indicates the scope of impact of cybersecurity threat information. This indicates the delay in responding to cybersecurity threat information. This represents the cross-domain propagation coefficient of cybersecurity threat information. This represents the preset response impact index;

[0068] It should be noted that the cross-domain propagation coefficient represents the cross-boundary capability of cybersecurity threat information to break through network boundaries. The cross-domain propagation coefficient is obtained by weighting protocol penetration, supply chain contamination, and artificial intelligence adaptability. The weight factors for protocol penetration, supply chain contamination, and artificial intelligence adaptability can be 0.415, 0.345, and 0.24, respectively. The weight factor values ​​can be adjusted according to the actual situation.

[0069] Based on the obtained information risk values, they are sorted from largest to smallest. The information risk values ​​are compared with risk thresholds, and the risk levels of network security threat information are divided into three levels: high risk, medium risk, and low risk, according to the size of the information risk value. The specific risk threshold boundaries are adjusted and determined according to the enterprise or organization's risk tolerance, the importance of the business, and the complexity of the network system.

[0070] Step 3: Based on the calculated information risk value, perform risk prediction on network security threat information through calculation and analysis;

[0071] In step three:

[0072] Firstly, specifically, the process for obtaining risk prediction values ​​for cybersecurity threat information is as follows:

[0073] Based on the calculated information risk value, obtain the historical information risk value sequence within a unit period. Set the unit time granularity, obtain the number of verified tactical nodes in the historical attack chain, and compare it with the preset unit node number to obtain the verified node value. ,in, This represents the number of intervals in the historical information risk value sequence;

[0074] It should be noted that the unit cycle is a collection cycle preset by those skilled in the art and can be adjusted according to actual conditions;

[0075] The historical information risk value sequence and the verification node value are obtained through the formula. Calculate the information risk attenuation factor ,in, This represents the risk value of historical information. This represents the preset risk attenuation coefficient. This represents the interval number of the historical information risk value sequence, and e represents the natural constant.

[0076] Based on information risk attenuation factor By using an exponential decay mechanism, historical risk data is assigned different weights according to their time proximity. Since recent risks have a greater impact, this reflects the natural decay characteristics of threats over time. By observing the changes in the risk sequence after decay, the long-term evolution pattern of threats can be identified.

[0077] The defense priority is dynamically adjusted based on the trend of information risk value and information risk decay factor.

[0078] For example, if the cybersecurity threat information is at a high risk level and the information risk attenuation factor is less than a preset risk attenuation threshold... If the information risk value is low, the investment in defense resources will be reduced, and the network defense will gradually shift to a lower level based on the information risk value, while routine monitoring will be conducted; if the network security threat information is of a low risk level and the information risk attenuation factor is greater than or equal to the preset risk attenuation threshold, the network security threat information will be reduced. If so, increase investment in defense resources, gradually shift to more advanced network defenses based on information risk values, and conduct key monitoring;

[0079] It should be noted that, based on historical risk values, the system assigns a higher weight to recent risk values. Controlling the rate of memory decay, To suppress noise interference from complex attack chains, the preset parameters in the formula are set by those skilled in the art based on actual conditions and historical experience, and can be adjusted according to actual conditions;

[0080] Based on the knowledge graph, the probability distribution of each feature dimension is calculated, the entropy value of each dimension is calculated by applying the Shannon entropy formula, and the risk information entropy value is calculated by combining the asset value weight and the propagation coefficient weight. Based on the real-time acquisition of the dark web forum transaction popularity change rate by the quantum crawler, the spatiotemporal change gradient of dark web threat activities is analyzed.

[0081] It should be noted that quantum web crawlers utilize the parallel processing capabilities of quantum computing to optimize the efficiency of traditional web crawlers.

[0082] The risk prediction value is obtained by weighting the information risk attenuation factor, risk information entropy value and spatiotemporal change gradient. The weight factors of information risk attenuation factor, risk information entropy value and spatiotemporal change gradient can be 0.45, 0.3 and 0.25 respectively. The weight factor values ​​can be adjusted according to the actual situation.

[0083] It is important to note that the information risk attenuation factor plays a crucial role in cybersecurity threat information risk analysis: The information risk attenuation factor weights historical information risk values ​​by time interval, making the impact of recent risk data on the current assessment significantly higher than that of long-term data, reflecting the time-sensitive nature of the risk; by combining verification node values ​​to adjust the credibility weight of historical data, the information risk attenuation factor enhances the reference value of actually verified attack chain information in risk prediction; the information risk attenuation factor integrates the attenuated historical risk sequence to analyze the long-term attenuation trend and short-term fluctuations of risk values, providing a dynamic basis for predicting future risk evolution and assisting in the formulation of forward-looking defense strategies. Through a preset risk attenuation coefficient, the attenuation rate of historical data can be flexibly controlled, enabling the model to adapt to the risk change patterns under different business scenarios.

[0084] The weights of the information risk attenuation factor, the risk information entropy value, and the spatiotemporal change gradient are set by those skilled in the art based on actual conditions and historical experience, and can be adjusted according to actual conditions;

[0085] The risk prediction value is calculated iteratively based on the risk prediction value calculation formula to obtain the predicted risk value in each subsequent unit time granularity. As new information risk value data is generated, the prediction is continuously updated to adapt to the dynamic changes of network security threats.

[0086] The technical solution of this invention is as follows: Integrating internal and external network security data, utilizing natural language processing technology to analyze unstructured threat intelligence, constructing a knowledge graph, updating and inferring potential threat associations in real time based on new internal and external data inflows, building an adaptive threat model based on big data and deep learning, using CNN to identify abnormal patterns in encrypted traffic, LSTM to establish a user-device behavior baseline, and GNN to predict threat propagation paths; calculating the attack success rate by comprehensively considering a general vulnerability scoring system, attacker capabilities, and the defense capabilities of protective measures; combining the time interval from threat occurrence to detection and the threat assessment and processing time to obtain the detection response delay; and considering the impact range, asset value, attack success rate, detection response delay, and... The cross-domain propagation coefficient is used to calculate the information risk value, and risk levels are classified accordingly. Based on the calculated information risk value, a historical information risk value sequence is obtained. The verification node value is obtained by the ratio of the number of verified tactical nodes in the historical attack chain to the preset number of unit nodes. The information risk attenuation factor is calculated by combining the historical information risk value sequence and the verification node value. The risk information entropy value is obtained by reasoning based on the knowledge graph. The spatiotemporal change gradient of dark web threat activities is analyzed by using quantum crawling to obtain the change rate of dark web forum transaction popularity. The risk prediction value is calculated by weighting the information risk attenuation factor, risk information entropy value and spatiotemporal change gradient, and iteratively updated to adapt to the dynamic changes of threats, timely detection of potential security risks, and timely early warning information for network security protection.

[0087] Example 3

[0088] like Figure 1 As shown in the figure, the network security threat information early warning system based on big data provided in this embodiment of the invention specifically includes the following modules:

[0089] Data collection module: Based on big data, collects and integrates internal and external data related to cybersecurity;

[0090] Data preprocessing module: For structured data, it compares and merges with existing structured information in the knowledge graph; for unstructured data, it performs in-depth analysis based on Natural Language Processing (NLP) technology; it identifies entities related to cybersecurity threats from unstructured text, using a pre-trained Named Entity Recognition (NER) model, and fine-tunes a model based on the pre-trained BERT language model using the Transformer deep learning architecture on cybersecurity datasets to improve recognition accuracy; based on the continuous influx and updates of new internal and external data, it continuously uses NLP technology to analyze new data, adding new threat entities and relationships to the knowledge graph; when new attack methods targeting application vulnerabilities are discovered, it promptly updates the corresponding nodes and edges in the knowledge graph; it uses existing information in the knowledge graph to perform relationship reasoning and discover potential threat associations;

[0091] Threat Model Building Module: Based on big data cybersecurity threat information and deep learning, an adaptive threat model is built, using a convolutional neural network (CNN) to identify abnormal patterns in encrypted traffic; a user-device behavior baseline is established through a long short-term memory (LSTM) network.

[0092] Threat Risk Quantification Module: Combines graph neural network (GNN) to predict threat propagation paths, develops a threat risk quantification model, and comprehensively considers attack success rate, asset value, impact range value and response delay to calculate information risk value;

[0093] Risk prediction module: Based on the calculated information risk value, combined with the information risk attenuation factor, risk prediction is performed on network security threat information.

[0094] The above provides a detailed description of one embodiment of the present invention, but the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. The above formulas are all dimensionless numerical calculations, and the formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art based on actual conditions and historical experience, and can be adjusted according to actual conditions. All equivalent changes and improvements made in accordance with the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A method for early warning of network security threat information based on big data, characterized in that, The method comprises the following steps: Collect internal data and external data of network security based on big data, and perform preprocessing, natural language processing to analyze unstructured threat intelligence, and build threat entity correlation relationship through knowledge graph; Build an adaptive threat model based on network security threat information and deep learning of big data, identify abnormal patterns in encrypted traffic, develop a threat risk quantification model, and calculate information risk value by considering attack success rate, asset value, influence range value and response delay; Based on the calculated information risk value, the information risk decay factor is combined to calculate and analyze the risk prediction of network security threat information; The process of obtaining the information risk value is: Based on the influence range value of the network security threat information and the asset value of the threat object, the attack success rate and the detection response delay calculated comprehensively, the information risk value is calculated by the formula . , , , , , , , , wherein, the attack success rate of the network security threat information is , the detection asset value of the detection network security threat information is , the influence range value of the network security threat information is , the response delay of the network security threat information is , the cross-domain propagation coefficient of the network security threat information is , and the preset response influence index is . The process of obtaining the risk prediction value is: The obtained historical information risk value sequence and the verification node value pass through formula The information risk attenuation factor is calculated , wherein The historical information risk value is The preset risk attenuation coefficient is The interval number of the historical information risk value sequence is The verification node value is e, which represents a natural constant, and the defense priority is dynamically adjusted according to the trend of the information risk value and the information risk attenuation factor; Wherein, based on the calculated information risk value, the historical information risk value sequence in a unit period is obtained , set the unit time granularity, obtain the number of verified tactical nodes in the historical attack chain and perform ratio processing with the preset unit node number to obtain the verified node value ; Based on the knowledge graph, the probability distribution of each feature dimension is calculated, the Shannon entropy formula is used to calculate the entropy value of each dimension, the risk information entropy value is calculated by combining the asset value weight and the cross-domain propagation coefficient weight, and the dark web forum transaction heat change rate is obtained in real time based on quantum crawling, and the spatio-temporal change gradient of dark web threat activities is analyzed; Based on the obtained information risk decay factor, risk information entropy value and spatio-temporal change gradient, the risk prediction value is obtained by weighted calculation, based on the new information risk value data, the prediction is updated constantly, and the dynamic change of network security threat is adapted. 2.The big data-based network security threat information early warning method according to claim 1, characterized in that, The analysis process of the external data is: Based on the collected external data, the cleaned text is segmented into single words or tokens; Key entities are extracted from unstructured text by using named entity recognition technology; The identified entities are classified and labeled, and the entities are classified to prepare for building nodes in the knowledge graph; The semantic relationship between words in a sentence is understood by using dependency syntax analysis and semantic role labeling technology. 3.The big data-based network security threat information early warning method according to claim 1, characterized in that, The process of obtaining the threat entity correlation relationship is: Based on the semantic analysis result, the relationship between threat entities is extracted to form the edge of the knowledge graph; The threat entities recognized and extracted through natural language processing technology are used as the nodes of the knowledge graph, and the relationship between the entities is used as the edge of the knowledge graph; The knowledge graph is stored based on a graph database, the graph database processes the relationship query of nodes and edges, quickly retrieves and analyzes the correlation between threat entities, uses existing information in the knowledge graph for relationship reasoning, and discovers potential threat correlations. 4.The big data-based network security threat information early warning method according to claim 1, characterized in that, The analysis process of the internal data and the external data is: Based on the new internal data and external data flowing into the system, structured data and unstructured data are distinguished; For structured data, comparison and fusion are performed with existing structured information in the knowledge graph; For unstructured data, in-depth analysis is performed based on natural language processing technology, and fine-tuning is performed on the data set in the network security field.

5. The big data-based network security threat information early warning method according to claim 1, characterized in that, The process of obtaining the attack success rate is: The Common Vulnerability Scoring System obtains the vulnerability base score of network security threat information, and the attack success rate is calculated by weighting the vulnerability base score, attacker ability and attack resistance ability.

6. The big data-based network security threat information early warning method according to claim 1, characterized in that, The process of obtaining the detection response delay is: The time interval from the occurrence of a network security threat to the discovery of the threat by a detection system is collected, and after the threat is detected, the time taken by the system to evaluate the threat, determine the threat type, severity and impact range value is obtained, the processing time is integrated to obtain the detection response delay.

7. The big data-based network security threat information early warning method according to claim 1, characterized in that, The verification node value acquisition process is: Based on the calculated information risk value, the historical information risk value sequence in the unit period is obtained, the unit time granularity is set, the number of verified tactical nodes in the historical attack chain is obtained and the preset unit node number is processed by ratio to obtain the verification node value.

8. A big data-based network security threat information early warning system, characterized in that, Comprise: Data collection module: based on big data collection and integration of internal and external data related to network security; Data preprocessing module: based on new internal and external data, fine-tune the data set in the field of network security; Threat model construction module: based on big data network security threat information and deep learning to build adaptive threat model; Threat risk quantification module: combine graph neural network to predict threat diffusion path, develop threat risk quantification model, comprehensively consider attack success rate, asset value, impact range value and response delay to calculate information risk value; Risk prediction module: based on the calculated information risk value, combine information risk decay factor to calculate and analyze the risk prediction of network security threat information; The information risk value acquisition process is: Based on the influence range value of the network security threat information and the asset value of the threat object, the attack success rate and the detection response delay calculated comprehensively, the information risk value is calculated by the formula The information risk value is calculated , The attack success rate of the network security threat information is The detection asset value of the network security threat information is , The influence range value of the network security threat information is The response delay of the network security threat information is The cross-domain propagation coefficient of the network security threat information is The preset response influence index is represented The risk prediction value acquisition process is: The historical information risk value sequence and the verification node value are calculated through the formula The information risk attenuation factor is calculated , wherein is the historical information risk value, is a preset risk attenuation coefficient, is the interval number of the historical information risk value sequence, is the verification node value, and e represents a natural constant; according to the trend of the information risk value and the information risk attenuation factor, the defense priority is dynamically adjusted; Wherein, based on the calculated information risk value, the historical information risk value sequence in a unit period is obtained , set the unit time granularity, obtain the number of verified tactical nodes in the historical attack chain and perform ratio processing with the preset unit node number to obtain the verified node value ; Based on the knowledge graph, the probability distribution of each feature dimension is calculated, the Shannon entropy formula is applied to calculate the entropy value of each dimension, the risk information entropy value is calculated by combining the asset value weight and the cross-domain propagation coefficient weight, based on the dark web forum transaction heat change rate obtained by the quantum crawler in real time, the spatio-temporal change gradient of the dark web threat activity is analyzed; Based on the obtained information risk decay factor, risk information entropy value and spatio-temporal change gradient, the risk prediction value is calculated by weighting, based on the new information risk value data, the prediction is updated constantly, and the dynamic change of network security threat is adapted.

Images