An intelligent network information security early warning method and system based on big data

Abstract

The application relates to the technical field of network security, and discloses an intelligent network information security early warning method and system based on big data, which comprises the following steps: collecting multi-source heterogeneous data of monitoring network equipment in real time, performing multi-dimensional feature extraction on the data to generate a structured feature vector set; constructing a multi-modal threat detection model to generate a threat detection result; fusing the threat detection result with external threat intelligence, constructing a threat intelligence knowledge base, and calculating a comprehensive risk value of network threats; dynamically setting an early warning threshold, adaptively determining the threat detection result, generating graded early warning information, and visually displaying the graded early warning information through a multi-dimensional situation view. The application converts original data into threat signals by collecting multi-source heterogeneous data, forms a complete attack chain by fusing isolated threat points, and realizes unified management and correlation analysis of security data.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to an intelligent network information security early warning method and system based on big data. Background Technology

[0002] The current security architecture is characterized by various types of security data, such as network traffic data, terminal logs, application logs, and threat intelligence, being scattered across different systems, forming serious information silos. Security teams need to switch between multiple consoles and manually correlate and analyze data from different sources, which is inefficient and prone to missing critical information.

[0003] Traditional security systems generate thousands of alerts daily, leaving security teams chronically fatigued and critical threats often buried in the noise. Furthermore, current security detection systems typically employ single detection technologies, resulting in significant blind spots in defenses. Attackers can easily gain access simply by circumventing specific detection mechanisms. Simultaneously, traditional systems struggle to handle multi-stage, multi-vector composite attacks and cannot identify the attacker's complete intent. Security teams lack a clear understanding of the overall situation, making it difficult to predict attack trends and assess potential risks. They also lack timely and accurate security status information, leading to inefficient resource allocation and unclear protection priorities. Summary of the Invention

[0004] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides an intelligent network information security early warning method based on big data to solve the above problems.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an intelligent network information security early warning method based on big data, comprising: Real-time acquisition and monitoring of multi-source heterogeneous data from network devices; quality protection processing of the acquired raw data; generation of raw security datasets in a unified format. The original security dataset is preprocessed, multi-dimensional features are extracted, and the extracted features are standardized, normalized, and augmented to generate a set of structured feature vectors. Based on the set of feature vectors, a multimodal threat detection model is constructed. The multimodal threat detection model includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module, and generates threat detection results. The threat detection results are integrated with external threat intelligence to construct a threat intelligence knowledge base. The threat detection results are then enhanced and scored based on the threat intelligence knowledge base to calculate the comprehensive risk value of network threats. Based on the comprehensive risk value, the early warning threshold is dynamically set, the threat detection results are adaptively judged, the graded early warning information is generated, and the graded early warning information is visualized through a multi-dimensional situation view.

[0006] As a preferred embodiment of the intelligent network information security early warning method based on big data described in this invention, generating a unified format of the original security dataset includes: Data acquisition devices deployed at network boundaries, terminal devices, cloud platforms, and security devices are used to collect multi-source heterogeneous data in real time. The multi-source heterogeneous data includes network traffic data, terminal behavior data, security device logs, and cloud environment data. The raw data collected is subjected to quality inspection, including data integrity review, data timeliness review, and data consistency review. Data that does not meet the quality requirements is marked and re-collected or supplemented. The collected data undergoes privacy protection processing, including data anonymization, encryption of fields that retain relevance, and range-based processing of numerical values.

[0007] As a preferred embodiment of the intelligent network information security early warning method based on big data described in this invention, the generation of a structured feature vector set includes: Based on the original security dataset, a distribution analysis was performed on each data field, and an adaptive data cleaning algorithm was used to identify and process abnormal data. Multi-dimensional feature extraction of data is achieved through time series feature extraction, graph structure feature construction, text semantic feature extraction, and behavioral pattern feature extraction. The extracted features are standardized and normalized, and synthetic minority class oversampling is used to augment the attack samples. The data stream is divided into fixed-size time windows, and data processing is performed independently within each window, ultimately generating a set of structured feature vectors.

[0008] As a preferred embodiment of the intelligent network information security early warning method based on big data described in this invention, the multi-dimensional feature extraction includes: Sliding window statistical features are extracted from time series data. Data statistics are calculated within the time window, and autocorrelation coefficients and spectral features are calculated to extract time series features. By treating network entities as graph nodes and relationships between entities as graph edges, a dynamic knowledge graph is constructed. A centrality index is calculated for each node, and a weight is calculated for each edge, in order to construct and extract graph structure features. For the text description content in the security log, a deep learning model is used to extract semantic features. The extracted features include at least one of the following: threat type, attack method, and scope of impact, in order to extract text semantic features. For user and entity behavior sequences, behavioral pattern features are extracted, Markov transition probability matrices of the behavior sequences are calculated, frequent behavioral patterns are extracted, and behavioral anomaly is calculated to extract time series features.

[0009] As a preferred embodiment of the intelligent network information security early warning method based on big data described in this invention, the construction of a multimodal threat detection model includes: A time series anomaly detection module is constructed, which uses a long short-term memory network combined with an attention mechanism to analyze time series data. It identifies time series anomalies by calculating reconstruction errors and dynamically determines key time steps using attention weights. A graph structure anomaly detection module is constructed, which uses graph convolutional networks to model network topology and entity relationships, aggregates neighbor node information, calculates the local and global structural anomalies of nodes, and identifies abnormal subgraphs and cooperative attack behaviors. A multimodal fusion detection module is constructed, which uses a multi-input neural network to process time-series data, graph data and text data respectively. The outputs of each module are dynamically weighted and combined through an attention fusion layer. A multi-task learning strategy is adopted to simultaneously optimize anomaly detection, threat classification and attack phase identification tasks. Based on unsupervised and semi-supervised learning mechanisms, anomalies are determined by reconstruction error, the discriminative feature is enhanced by contrastive loss function, and samples are selected for labeling based on uncertainty score and representativeness score. Adversarial robustness enhancement strategies include adversarial training to generate adversarial examples to improve the model's anti-interference ability, input verification and sanitization mechanisms to filter malicious inputs, and model ensemble defense to improve detection stability through heterogeneous model voting fusion. By integrating context-aware detection strategies, both temporal and spatial context information are detected, and detection sensitivity and threat weight are dynamically adjusted.

[0010] As a preferred embodiment of the intelligent network information security early warning method based on big data described in this invention, the fusion of the threat detection results with external threat intelligence includes: Collect threat intelligence from multiple sources, build an intelligence quality assessment system, and dynamically score intelligence sources; Standardize and disambiguate threat intelligence, define threat intelligence data models, identify malware variants, analyze the duration, frequency, and geographical distribution characteristics of attack activities through spatiotemporal correlation diagrams, and build a threat intelligence knowledge base. Based on the aforementioned threat intelligence knowledge base, an intelligence enhancement and detection optimization strategy is set up. A multi-factor threat scoring algorithm is used to combine the basic detection score, intelligence enhancement coefficient, and environmental adjustment coefficient to obtain a comprehensive risk value of network threats.

[0011] As a preferred embodiment of the intelligent network information security early warning method based on big data described in this invention, the generation of hierarchical early warning information includes: Define a base threshold, and dynamically adjust it in conjunction with threat environment adjustment coefficient, business importance adjustment coefficient, and intelligence credibility adjustment coefficient to determine the dynamic threshold for early warning; By combining the comprehensive risk value of the network threat with the dynamic warning threshold, the warning levels are divided, and graded warning information is generated. A multi-dimensional situation view is then visualized, which includes a global threat situation heatmap, a dynamic network topology map, or a time-series threat trend map.

[0012] Secondly, the present invention provides an intelligent network information security early warning system based on big data, comprising: The data acquisition module is used to collect multi-source heterogeneous data from monitoring network devices in real time, perform quality protection processing on the collected raw data, and generate a raw security dataset in a unified format. The feature extraction module is used to preprocess the original security dataset, extract multi-dimensional features from the data, and perform standardization, normalization and data augmentation on the extracted features to generate a set of structured feature vectors. The detection module is used to construct a multimodal threat detection model based on the feature vector set. The multimodal threat detection model includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module, and generates threat detection results. The evaluation module is used to integrate the threat detection results with external threat intelligence, construct a threat intelligence knowledge base, enhance the scoring of the threat detection results based on the threat intelligence knowledge base, and calculate the comprehensive risk value of network threats. The early warning module is used to dynamically set early warning thresholds based on the comprehensive risk value, perform adaptive early warning judgment on threat detection results, generate graded early warning information, and visualize the graded early warning information through a multi-dimensional situation view.

[0013] Thirdly, the present invention provides a computer device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the intelligent network information security early warning method based on big data.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the aforementioned intelligent network information security early warning method based on big data.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention collects multi-source heterogeneous data, intelligently preprocesses to eliminate data noise and extract key features, transforms raw data into threat signals through multimodal detection, forms a complete attack chain from isolated threat points through intelligence fusion, and adopts adaptive early warning to transform threat signals into dynamic early warning information, ultimately visualizing the complex network security situation and realizing unified management and correlation analysis of security data. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the overall process of an intelligent network information security early warning method based on big data according to an embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0019] Reference Figure 1 As one embodiment of the present invention, a method for intelligent network information security early warning based on big data is provided, comprising: S101 collects and monitors multi-source heterogeneous data from network devices in real time, performs quality protection processing on the collected raw data, and generates a raw security dataset in a unified format. S102, perform data preprocessing on the original security dataset, extract multi-dimensional features from the data, and perform standardization, normalization and data augmentation on the extracted features to generate a set of structured feature vectors. S103, based on the feature vector set, construct a multimodal threat detection model. The multimodal threat detection model includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module, and generates threat detection results. S104 integrates threat detection results with external threat intelligence to build a threat intelligence knowledge base, enhances the scoring of threat detection results based on the threat intelligence knowledge base, and calculates the comprehensive risk value of network threats; S105 dynamically sets early warning thresholds based on comprehensive risk values, adaptively determines early warnings based on threat detection results, generates tiered early warning information, and visualizes the tiered early warning information through a multi-dimensional situational view.

[0020] Preferably, generating a raw security dataset in a uniform format includes: Data acquisition devices deployed at network boundaries, terminal devices, cloud platforms, and security devices are used to collect multi-source heterogeneous data in real time. This multi-source heterogeneous data includes network traffic data, terminal behavior data, security device logs, and cloud environment data. The raw data collected is subjected to quality inspection, including data integrity review, data timeliness review, and data consistency review. Data that does not meet the quality requirements is marked and re-collected or supplemented. The collected data undergoes privacy protection processing, including data anonymization, encryption of fields that retain relevance, and range-based processing of numerical values.

[0021] Specifically, network traffic data is collected by deploying network traffic mirroring ports on network switches and perimeter firewalls. Raw data packets are obtained through network data packets. Non-critical traffic is sampled probabilistically during peak hours, while critical business traffic is collected in full. After acquiring data packets, key features such as packet header information, load characteristics, session duration, and data transmission volume are extracted to generate network flow records.

[0022] It collects terminal behavior data and monitors key behaviors of servers and terminal devices in real time, such as process creation, file operations, registry modifications, and network connections. When abnormal behaviors are detected, such as large-scale file encryption or abnormal process tree structures, it automatically increases the granularity of data collection and records detailed behavior logs. It also records data on security-related events such as user login behavior, permission changes, and access to sensitive files.

[0023] Collect security device logs, including log data from security devices such as firewalls, intrusion detection systems, antivirus software, and web application firewalls. Convert log data from different vendors and in different formats into a unified data structure, including information such as time, event type, source address, destination address, and severity level.

[0024] Collect data from the cloud platform, including virtual machine monitoring data, API call logs, and storage access records, and establish a mapping relationship between containers and the host machine to ensure that security events can be accurately associated with the corresponding physical resources.

[0025] Furthermore, during data collection, data quality checks are performed. This includes integrity checks by verifying the completeness of data packets and the absence of missing log fields, timeliness checks by marking data delay times, and data consistency checks by verifying the consistency of records of the same event in different data sources. Raw data that does not meet the quality requirements is marked, and the marked data is then re-collected or supplemented.

[0026] Privacy protection is implemented during data collection by desensitizing user identity information and sensitive business data, using deterministic encryption technology to encrypt fields that need to retain relevance, and employing generalization technology to range precise values, such as IP addresses and geographical locations, to ensure both data availability and privacy protection in subsequent early warning analysis.

[0027] Preferably, the generated structured feature vector set includes: Based on the original security dataset, a distribution analysis is performed on each data field, and an adaptive data cleaning algorithm is used to identify and process abnormal data. Multi-dimensional feature extraction of data is achieved through time series feature extraction, graph structure feature construction, text semantic feature extraction, and behavioral pattern feature extraction. The extracted features are standardized and normalized, and synthetic minority class oversampling is used to augment the attack samples. The data stream is divided into fixed-size time windows, and data processing is performed independently within each window, ultimately generating a set of structured feature vectors.

[0028] Specifically, distribution analysis is performed on each data field, calculating statistical characteristics such as mean, standard deviation, and quantiles. For numerical data, the normalized value of each data point is calculated, which is the difference between the data point and the mean divided by the standard deviation. When the absolute value of the normalized value exceeds a dynamic threshold, it is judged as an outlier. The dynamic threshold is automatically adjusted based on the skewness and kurtosis of the data distribution, combined with expert experience. For categorical data, the frequency of each category is counted. Categories with a frequency below a set threshold and no obvious correlation with other categories are marked as outliers. The threshold can be set to 0.1%. During cleaning, the original data records of outlier data are retained, and a cleaning log is generated.

[0029] Preferred multi-dimensional feature extraction includes: Sliding window statistical features are extracted from time series data. Data statistics are calculated within the time window, and autocorrelation coefficients and spectral features are calculated to extract time series features. By treating network entities as graph nodes and relationships between entities as graph edges, a dynamic knowledge graph is constructed. A centrality index is calculated for each node, and a weight is calculated for each edge, in order to construct and extract graph structure features. For the text description content in the security log, a deep learning model is used to extract semantic features. The extracted features include at least one of the following: threat type, attack method, and scope of impact, in order to extract text semantic features. For user and entity behavior sequences, behavioral pattern features are extracted, Markov transition probability matrices of the behavior sequences are calculated, frequent behavioral patterns are extracted, and behavioral anomaly is calculated to extract time series features.

[0030] In this embodiment, when extracting time series features, sliding window statistical features are extracted from time series data such as network traffic and system calls. Within the time window, statistics such as mean, variance, maximum, minimum, and rate of change can be calculated. The window size is dynamically adjusted according to the data characteristics, ranging from 30 to 60 minutes. Simultaneously, autocorrelation coefficients and spectral features are calculated. The autocorrelation coefficient is used to assess the correlation of data at different time points, and the spectral features are analyzed for periodicity using Fast Fourier Transform (FFT). The time series data is preprocessed, and FFT is performed to obtain frequency components. Based on these frequency components, the power spectral density is calculated. ,in, Let N be the k-th frequency component, and N be the sequence length. Then, the dominant frequency feature is extracted, where dominant frequency = ( ), that is, the frequency component with the highest power, to calculate the spectral entropy.

[0031] Furthermore, joint time-frequency features are extracted using wavelet transform, and wavelet basis functions are selected. Given scale parameter a and translation parameter b, calculate the continuous wavelet transform: ; in, These are wavelet coefficients. For scale parameters, For translation parameters, It is a time series signal. For the complex conjugate of wavelet basis functions, The time variable is used, and then wavelet energy features are extracted. Calculated as Calculate wavelet entropy ,in, Let be the wavelet entropy at scale a.

[0032] Furthermore, deep learning feature extraction is performed, constructing a two-layer Long Short-Term Memory (LSTM) temporal encoder. The first LSTM layer has an input dimension equal to the original feature dimension and a hidden layer dimension of 128, using the tanh activation function. The second LSTM layer has an input dimension of 128 and a hidden layer dimension of 64, using the hyperbolic tangent tanh activation function. The output layer uses a fully connected layer to map the 64-dimensional hidden state to a 32-dimensional feature vector. The model controls the information flow through a gating mechanism: an input gate, a forget gate, and an output gate. The forget gate determines the proportion of historical information retained, the input gate controls the addition of new information, and the output gate determines the output of the current state.

[0033] In this embodiment, a graph structure feature is constructed, treating entities such as network hosts, users, and processes as graph nodes, and relationships such as network connections, file access, and process creation as graph edges, thus building a dynamic knowledge graph. Centrality metrics, such as degree centrality, betweenness centrality, and proximity centrality, are calculated for each node to reflect its importance in the network. Weights are calculated for each edge, based on factors such as connection frequency and data transmission volume, to reflect the tightness of relationships. The graph structure is updated periodically to capture the evolutionary trend of the network topology.

[0034] Specifically, define node types, including host nodes, user nodes, process nodes, file nodes, and network connection nodes. Define edge types, including access relationship edges (user-host), execution relationship edges (process-file), communication relationship edges (host-host), and dependency relationship edges (service-service). Calculate edge weights: communication relationship edge weight = data transmission volume × communication frequency × communication duration; access relationship edge weight = access frequency × access depth × access duration.

[0035] Further, node features are initialized: host node feature vector = [CPU utilization, memory utilization, network traffic, number of open ports, number of running processes, operating system type encoding], user node feature vector = [login frequency, operation permission level, behavior pattern entropy, activity time period distribution, access resource type distribution], and process node feature vector = [CPU utilization, memory utilization, file operation frequency, number of network connections, parent process ID encoding]. Then, graph neural network feature extraction is performed using a three-layer graph convolutional network. The first layer aggregates first-order neighbor information, and the new feature of node u = ReLU(Σ[normalized weights (u,v)×(weight matrix×original feature of node v+bias vector)]), where v∈neighboring nodes, and the normalized weights (u,v) are the reciprocal of the square root of the degree of node u multiplied by the degree of node v. The second layer aggregates second-order neighbor information, using the output of the first layer as input, expanding the aggregation range to second-order neighbors, and introducing an attention mechanism: attention coefficient (u,v) = softmax(LeakyReLU(attention vector)). T×[weight matrix × node u feature ||weight matrix × node v feature])); The third layer graph convolution generates the final node representation, integrates global graph structure information, and uses graph pooling operation to extract graph-level features, graph-level features = (1 / total number of nodes) × Σ(node ​​features × node centrality weight).

[0036] Furthermore, extract the statistical features of the graph structure and calculate the betweenness centrality of the nodes: ; in, The total number of shortest paths from node s to node t. Let be the number of shortest paths passing through node v, and s,t be all node pairs; then calculate the clustering coefficients. : ; in, This represents the actual number of connections between node v's neighbors. The number of neighbors of node v; calculate community structure characteristics, use the Leuven algorithm to detect communities, and determine community modularity. The calculation is as follows: ; Where m is the total number of communities. Let be the number of edges within community c, and 2m be the total number of edges in the graph.

[0037] In this embodiment, the extraction of text semantic features involves using a deep learning model to extract semantic features from the textual descriptions in the security logs. First, a pre-trained language model is used to convert the text into a high-dimensional vector representation, such as the Transformer-Based Bidirectional Encoder Representation Language Model (BERT). Then, principal component analysis is used to compress the vector to a fixed dimension. A domain dictionary is constructed to enhance the model's understanding of security-related vocabulary. The extracted features include semantic information such as threat type, attack methods, and scope of impact.

[0038] Specifically, a cybersecurity terminology dictionary is established, mapping ransomware to ransomware and exploit to vulnerability exploitation. Named entities are identified, and a Bidirectional Long Short-Term Memory Network-Conditional Random Field (BiLSTM-CRF) model is used to identify threat subjects, attack targets, vulnerability numbers, and malware names. The BiLSTM layer processes forward text sequences through forward LSTM and backward LSTM through backward text sequences; the CRF layer is used to model label transfer constraints to ensure the rationality of entity boundaries; then semantic disambiguation is performed, and polysemous words are disambiguated in context. For example, the command-line interface shell is mapped to reverse shell in the reverse command-line interface and to shell script in shell script.

[0039] An improved BERT model architecture is adopted, with an input layer containing 768-dimensional word embeddings, 768-dimensional positional embeddings, and 768-dimensional paragraph embeddings. The encoding layer is a 12-layer Transformer encoder, each layer including a multi-head self-attention mechanism (12 heads, 64 dimensions per head), a feedforward neural network, a hidden layer dimension of 3072, Gaussian Error Linearity (GELU) activation function, layer normalization, and residual connections. The output layer uses [CLS]-labeled output vectors as the semantic representation of the entire text. The model is trained on a training set, including cybersecurity technical documents, vulnerability descriptions, threat reports, etc., using masked language modeling. 15% of the words are randomly masked, with 80% replaced with [MASK], 10% replaced with random words, and 10% left unchanged. Then, the model predicts the next sentence, determining whether two sentences are consecutive. The loss function is 0.5 × mask prediction loss + 0.5 × sentence relationship prediction loss. Fine-tuning can be performed on a security event description dataset, adding a threat type classification task.

[0040] Furthermore, semantic similarity and semantic relevance are calculated using cosine similarity, which is obtained by α×cosine similarity + β×Jacartes similarity + γ×edit distance normalized value, where α=0.6 is the semantic weight, β=0.3 is the lexical weight, and γ=0.1 is the structural weight.

[0041] In this embodiment, the extraction of behavioral pattern features involves extracting behavioral pattern features from the behavioral sequences of users and entities. This includes calculating the Markov transition probability matrix of the behavioral sequence (the probability of transitioning from one behavioral state to another), extracting frequent behavioral patterns (using sequence mining algorithms to identify frequently occurring behavioral combinations), and calculating behavioral anomaly levels based on the degree of deviation from normal behavioral patterns. These features can effectively capture abnormal behavioral patterns, such as privilege abuse and data theft.

[0042] Specifically, the process first constructs a sequence of behaviors and defines behavioral events, which can include login events, file access events, network connection events, permission change events, and abnormal operation events. Sequence codes are generated, with each behavioral event encoded as [behavior type, target object, operation result, timestamp, context information]. A time-series window is then created using a sliding window with a window size of 24 hours and a step size of 1 hour. An improved PrefixSpan sequence pattern mining algorithm is employed to calculate the support of individual behavioral items. The support of a behavioral item is calculated by dividing the number of sequences containing that behavioral item by the total number of sequences. A projection database is constructed, retaining only behavioral items with support greater than a minimum support threshold of 0.05. Frequent sequences are identified, and the sequence support is calculated by dividing the number of users containing that sequence by the total number of users. Finally, anomaly scores are calculated, which are 1 - (frequency of the sequence among normal users / frequency of the sequence among abnormal users).

[0043] Furthermore, based on the behavior sequence, statistical behavior characteristics are analyzed, including behavior frequency characteristics, behavior diversity characteristics, behavior temporal characteristics, and behavior path characteristics. Among these, the behavior frequency characteristic is the number of behaviors per unit time, calculated by dividing the total number of behaviors by the time window length; the behavior diversity characteristic is calculated using behavior entropy, expressed as... ,in, Let T be the entropy of behavior, and T be the total number of behavior types. Let C be the number of occurrences of behavior type i, and C be the total number of behaviors; the behavior time sequence feature is the mean of behavior intervals; the behavior path feature is obtained through Markov transition probabilities.

[0044] Furthermore, feature extraction is performed after deep behavior encoding to construct a hierarchical behavior encoder. The bottom encoder uses a gated recurrent unit (GRU) network to encode a single session behavior, with a hidden layer dimension of 128. The network is updated by updating gates, resetting gates, candidate hidden states, and the current hidden state. The high-level encoder uses an attention mechanism to aggregate multiple session representations, with session weights as follows: ; in, Let q be the attention weight for session i, and q be the query vector. Let M be the feature representation of session i, M be the total number of sessions, and the final behavioral feature be... The anomaly detection head uses a Siamese network structure to calculate behavioral similarity, i.e., the similarity of normal behavior sample pairs = cos(normal behavior feature 1, normal behavior feature 2), and the similarity of abnormal behavior sample pairs = cos(normal behavior feature, abnormal behavior feature). The loss function is max(0, abnormal similarity - normal similarity + boundary value).

[0045] Furthermore, the extracted features are standardized to eliminate dimensional differences between different features. Numerical features are scaled to the [0,1] interval using min-max normalization; categorical features are converted to binary vectors using one-hot encoding. For features with extremely skewed distributions, a logarithmic or Box-Cox transformation is first performed to approximate a normal distribution before standardization. The statistical distribution information of the original features is preserved during the standardization process.

[0046] Simultaneously, data augmentation and balancing are performed. To address the scarcity of attack samples in security data, Synthetic Minority Oversampling Technique (SMOTE) is employed to generate new attack samples. In the feature space, for each minority class sample, a neighbor is randomly selected from its K nearest neighbors (typically K=5), and a new sample point is generated on the line segment connecting these two samples. Normal samples are undersampled by randomly deleting some normal samples to achieve a reasonable positive-to-negative sample ratio, which can be between 1:1 and 1:5. During data augmentation, the spatiotemporal correlation of samples is maintained to avoid generating unrealistic samples.

[0047] Simultaneously, real-time stream processing is optimized by dividing the data stream into fixed-size time windows, such as 1 second, 10 seconds, and 1 minute, and processing the data independently within each window. For features requiring long-term statistical analysis, such as user historical behavior statistics, state variables are maintained, and only the relevant states are updated when new data is generated, avoiding duplicate calculations. When the processing speed cannot keep up with the data generation speed, the window size is automatically adjusted or the processing precision is reduced to ensure the system's real-time performance.

[0048] It should be noted that through this step, the raw data is transformed into a high-quality set of feature vectors. Each feature vector contains hundreds of security-related features, which can comprehensively characterize the state and behavior of network entities and provide accurate input for subsequent threat detection.

[0049] Preferably, constructing a multimodal threat detection model includes: A time series anomaly detection module is constructed, which uses a long short-term memory network combined with an attention mechanism to analyze time series data. It identifies time series anomalies by calculating reconstruction errors and dynamically determines key time steps using attention weights. A graph structure anomaly detection module is constructed, which uses graph convolutional networks to model network topology and entity relationships, aggregates neighbor node information, calculates the local and global structural anomalies of nodes, and identifies abnormal subgraphs and cooperative attack behaviors. A multimodal fusion detection module is constructed, which uses a multi-input neural network to process time-series data, graph data and text data respectively. The outputs of each module are dynamically weighted and combined through an attention fusion layer. A multi-task learning strategy is adopted to simultaneously optimize anomaly detection, threat classification and attack phase identification tasks. Based on unsupervised and semi-supervised learning mechanisms, anomalies are determined by reconstruction error, the discriminative feature is enhanced by contrastive loss function, and samples are selected for labeling based on uncertainty score and representativeness score. Adversarial robustness enhancement strategies include adversarial training to generate adversarial examples to improve the model's anti-interference ability, input verification and sanitization mechanisms to filter malicious inputs, and model ensemble defense to improve detection stability through heterogeneous model voting fusion. By integrating context-aware detection strategies, both temporal and spatial context information are detected, and detection sensitivity and threat weight are dynamically adjusted.

[0050] Specifically, the multimodal threat detection model includes a basic detection layer, a feature fusion layer, and a decision output layer. The basic detection layer includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module. The feature fusion layer uses attention-weighted fusion to calculate the confidence weights of the outputs of each sub-model, and dynamically adjusts the contribution of each modality through a gating mechanism to generate a unified threat detection feature vector. The decision output layer contains a threat classifier and an anomaly scorer. The threat classifier outputs threat type labels, including normal, suspicious, and malicious, while the anomaly scorer outputs anomaly probability values ​​in the range of 0-1.

[0051] The temporal anomaly detection module employs a Long Short-Term Memory (LSTM) network model to analyze time-series data of network traffic and system behavior. The LSTM network includes memory units and three gating mechanisms: an input gate, a forget gate, and an output gate, effectively capturing long-term dependencies. The input layer receives pre-processed 32-dimensional temporal features, the hidden layer contains 128 LSTM units, and the output layer outputs a 16-dimensional temporal anomaly feature vector and a temporal anomaly probability score. During training, reconstruction error is used as the loss function; reconstruction error is small for normal data and large for anomalous data. An attention mechanism is introduced to focus the model on the time steps that have the greatest impact on the detection results, improving detection accuracy.

[0052] The graph anomaly detection module uses a Graph Convolutional Network (GCN) to analyze network topology and entity relationships. Employing a graph autoencoder architecture, it detects anomalies by minimizing graph reconstruction error. The GCN updates the representation of the current node by aggregating information from neighboring nodes. With 2-3 layers, each GCN layer contains a weight matrix and a non-linear activation function (ReLU activation function is acceptable). Node features are updated through a message-passing mechanism. The graph attention layer calculates node importance, with 4 attention heads and 8 dimensions per head. The graph pooling layer uses global average pooling to generate a graph-level representation. The output layer is a fully connected layer with 32 input dimensions and 16 output dimensions, using the ReLU activation function. Finally, the graph anomaly score is calculated as follows: ; in, To plot outlier scores, For graph-level representation, The mean value is represented by the normal graph level. This is the weighting coefficient, with a value of 0.3. To reconstruct the adjacency matrix, It is the Frobenius norm.

[0053] The multimodal fusion detection module processes temporal, graph, and text data separately using a multi-input neural network. The temporal branch uses LSTM, the graph branch uses GCN, and the text branch uses a Transformer encoder. The input dimension is 768, the hidden layer dimension is 512, and the number of heads is 8. LayerNorm and residual connections are employed. The outputs of the three branches are weighted and combined using an attention fusion layer, which includes weights for the influence of text on behavior and the influence of behavior on text. These weights are dynamically adjusted based on the importance of each modality to the current detection task. The fused features are then input into a fully connected layer. The model employs a contrastive learning mechanism to maximize the similarity of normal samples and minimize the similarity of abnormal samples. During training, the contrastive loss is minimized, and finally, a fused anomaly score is calculated. ,in, As a feature of fusion, The output is the final fused abnormal feature vector and threat probability, which is the mean of the normal sample features.

[0054] It should be noted that this invention employs unsupervised and semi-supervised learning mechanisms. Through anomaly detection using an autoencoder, a deep autoencoder model is constructed, comprising an encoder and a decoder. The encoder compresses input features into a low-dimensional latent space (32-64 dimensions), while the decoder reconstructs the original input. The model is trained on normal data to accurately reconstruct normal patterns; reconstruction errors on abnormal data significantly increase. A dynamic threshold is set based on expert experience; when the reconstruction error exceeds the threshold, it is considered an anomaly. Contrastive learning is used to enhance the model's ability to distinguish between normal and abnormal patterns. Positive and negative sample pairs are constructed, with positive pairs representing samples of the same category and negative pairs representing samples of different categories. A contrastive loss function is used to narrow the distance between positive sample pairs and widen the distance between negative sample pairs. Positive sample pairs can represent normal behavior of the same user at different times, while negative sample pairs can represent normal behavior and aggressive behavior. Through contrastive learning, the model can learn more discriminative feature representations. The model can also actively learn sample selection, intelligently select the most valuable samples for manual annotation, calculate the uncertainty score of each unlabeled sample, such as the entropy value of the predicted probability, and the representativeness score, i.e. the similarity with the labeled samples. It combines the two scores to select the samples that most need to be labeled, and prioritizes the labeling of samples with high uncertainty and high representativeness, so as to obtain the greatest model performance improvement with the minimum manual annotation cost.

[0055] During adversarial training, adversarial examples are generated to enhance the model's robustness. Fast Gradient Sign Method (FGSM) is used to generate adversarial examples, calculating the gradient of the loss function with respect to the input. Small perturbations are added along the gradient sign direction, and the original and adversarial examples are mixed for training, enabling the model to resist minor input perturbations. Simultaneously, defensive distillation is employed, using soft labels to train a second model (the model's prediction probability of the original samples) to improve model smoothness. Before inputting data into the model, rigorous validation and cleansing are performed. Numerical features are checked for reasonable ranges, text features are filtered for special characters and abnormal patterns, and graph structure features are used to verify the rationality of nodes and edges. When malicious manipulation of certain features is detected, the weights of these features are automatically reduced or the relevant detection modules are temporarily disabled. The model can simultaneously perform ensemble detection on multiple heterogeneous models. Each model uses different architectures, training data, or parameter settings. Detection results are voted on or weighted and fused. An alarm is triggered only when a majority of models determine a threat. The detection differences between models are periodically evaluated, and overly similar models are replaced to maintain the diversity of the ensemble. Furthermore, when model performance abnormally degrades, it is automatically isolated and retrained.

[0056] Furthermore, after each module completes the initial anomaly scoring and before the weighted fusion of features, context-aware detection is performed. Environmental context information such as organizational structure, business processes, and asset value is integrated into the detection process. A security level is assigned to each asset, which can be determined based on its business importance and data sensitivity through expert experience. Permission levels are assigned to each user. When detecting threats, alarm thresholds are adjusted according to the environmental context: alarm thresholds are lowered for abnormal behavior of high-security-level assets, and alarm thresholds are raised for privileged operations by low-privilege users. Temporal context is analyzed, considering the impact of time factors on threat detection, identifying periodic patterns in normal business operations, such as working hours and batch processing times. Abnormal behavior detected outside of working hours is assigned a higher threat weight, and the temporal anomaly degree of the behavior is calculated, i.e., the degree of deviation between the current behavior and historical behavior during the same period. Spatial context correlation is performed to analyze the spatial distribution characteristics of threats, i.e., correlation analysis is conducted on attack behaviors from the same geographical region, the same IP segment, and the same organizational unit to identify coordinated attacks. Abnormal access across regions and departments is monitored closely, especially when the access path violates the principle of least privilege. A spatial threat heat map is constructed to visualize the threat density in different regions and allocate defense resources accordingly.

[0057] It should be noted that this step, through collaborative analysis of multimodal data, enables a comprehensive analysis of network activity from multiple dimensions and modalities, identifies known and unknown threats, provides high-precision threat detection capabilities, improves the accuracy and coverage of threat detection, and ultimately outputs detection results, including anomaly scores, threat types, confidence levels, and detection time.

[0058] Preferably, fusing threat detection results with external threat intelligence includes: Collect threat intelligence from multiple sources, build an intelligence quality assessment system, and dynamically score intelligence sources; Standardize and disambiguate threat intelligence, define threat intelligence data models, identify malware variants, analyze the duration, frequency, and geographical distribution characteristics of attack activities through spatiotemporal correlation diagrams, and build a threat intelligence knowledge base. Based on a threat intelligence knowledge base, an intelligence enhancement and detection optimization strategy is established. A multi-factor threat scoring algorithm is used to synthesize the basic detection score, intelligence enhancement coefficient, and environment adjustment coefficient to obtain a comprehensive risk value for network threats. Specifically, the intelligence enhancement coefficient is dynamically adjusted based on the severity and credibility of the matched intelligence, while the environment adjustment coefficient is calculated in real-time according to the importance of the target asset and the business status.

[0059] Specifically, multi-source threat intelligence is collected by acquiring high-quality threat intelligence from multiple external intelligence platforms. Intelligence types can include malicious IP addresses, malicious domains, file hashes, attack tool characteristics, and attack organization profiles. Factors such as the quality and relevance of intelligence sources are empirically set, and dynamic weights are assigned to intelligence from each platform to ensure that high-quality data is prioritized or collected in sufficient quantities to extract key information such as attacker organization names, malware names, and affected products. Cross-validation of multiple independent intelligence sources is performed. Optionally, evidence weights and state consistency scores can be calculated and compared with corresponding thresholds to filter false intelligence. Evidence weights can be calculated using the weight coefficients of intelligence sources and expert-defined confidence and reliability scores. State consistency scores are calculated as the ratio of the threat status of an intelligence source to the total number of intelligence sources participating in the verification. Optionally, dynamic expert scoring can be performed on intelligence of different threat types such as IP addresses, domain names, and files. Based on the verification results, intelligence sources are graded, i.e., a quality score is calculated by weighting the evidence weights and state consistency scores, and intelligence sources with quality scores exceeding a set threshold are added to the threat intelligence knowledge base.

[0060] Furthermore, the threat intelligence from the previous step is converted into a unified STIX 2.1 standard format, and multi-dimensional associations are established through a knowledge graph. This involves defining a standard data model, achieving heterogeneous format conversion, constructing a threat intelligence knowledge graph, and integrating content, structure, and temporal features for association analysis. Node similarity is calculated to establish multi-dimensional relationships. Specifically, a unified threat intelligence data model is defined, including entities such as threat subjects (attackers, attack tools, attack targets, attack methods, time windows, and scope of impact). Intelligence data from different sources and in different formats are mapped to this unified model, establishing standardized field mapping relationships. For example, terms like malware, viruses, and Trojans from different intelligence sources are uniformly mapped to malicious program entities. For attacker organization names, string similarity algorithms, such as Jaccard similarity and edit distance, are used, along with contextual semantic analysis, to determine if they point to the same organization. For malware variants, clustering is performed based on multi-dimensional features such as code similarity, behavioral characteristics, and network communication patterns to identify different variants of the same malware family, establishing unique entity identifiers to ensure that records of the same entity in different intelligence sources can be correctly associated. A threat intelligence knowledge graph is constructed, and node similarity is calculated by weighting content similarity, structural similarity, and temporal similarity. A spatiotemporal association graph is constructed through graph convolutional networks and attention mechanisms to associate threat events in time and space dimensions. The time dimension is used to analyze the duration, frequency, and cycle of attack activities to identify persistent threat (APT) activities. The spatial dimension is used to analyze the geographical distribution of attack sources and the network location distribution of target assets to identify distributed attacks.

[0061] Furthermore, intelligence-enhanced detection optimization is implemented, transforming threat intelligence into enhanced detection features. IP reputation features are constructed for known malicious IP addresses, including historical malicious activity frequency, recent activity time, and attack type distribution. Behavioral fingerprints are extracted from malware families, including file operation patterns, network communication characteristics, and persistence mechanisms. TTPs (tactical, technical, and process) features are constructed for attack organizations, including commonly used attack methods, toolchains, and target selection preferences. These enhanced features are combined with internal detection features to improve detection accuracy. A multi-factor threat scoring algorithm is established, integrating internal detection results and external intelligence information. Threat score = basic detection score × intelligence enhancement coefficient × environment adjustment coefficient. The basic detection score is the output probability of the multi-modal threat detection model. The intelligence enhancement coefficient is based on the severity and credibility of the matched threat intelligence; for example, the coefficient is set to 2.0 for matching advanced malware and 1.2 for matching ordinary malware. The environment adjustment coefficient is based on the importance of the target asset and the current business status; for example, it is set to 1.5 during core business operations. This step achieves refined priority ranking of threats through multi-factor scoring.

[0062] It should be noted that this step can transform scattered, multi-source threat intelligence into unified and actionable security knowledge, significantly improving the scope and accuracy of threat detection and providing strong support for real-time early warning.

[0063] Preferably, the generation of tiered early warning information includes: Define a base threshold, and dynamically adjust it in conjunction with threat environment adjustment coefficient, business importance adjustment coefficient, and intelligence credibility adjustment coefficient to determine the dynamic threshold for early warning; By combining the comprehensive risk value of network threats with the dynamic warning threshold, warning levels are classified, and graded warning information is generated. Multi-dimensional situational view visualization is also performed, including a global threat situation heatmap, a dynamic network topology map, or a time-series threat trend map.

[0064] Specifically, a preset baseline warning threshold, determined based on historical threat data analysis, is set to 0.6. A threat warning is triggered when the overall risk value exceeds this threshold. The threshold can be dynamically adjusted based on the threat environment, business importance, and intelligence credibility. The threat environment is determined by continuously monitoring the frequency of threat alerts over the past 24 hours. When a high-incidence period is detected (i.e., more than 50 high-risk alerts within 24 hours), the threat environment adjustment coefficient is automatically set to 0.9. Under normal threat conditions, this coefficient remains at 1.0. Business importance is adjusted according to the security importance of different businesses. For example, for core business systems such as core databases and critical business applications, the adjustment coefficient is set to 0.8; for important business systems such as office systems... For general systems and email systems, the adjustment coefficient is set to 0.9 to appropriately increase sensitivity. For ordinary business systems, such as test and development environments, the adjustment coefficient is 1.0. The intelligence credibility adjustment coefficient is set according to the degree of verification of threat intelligence. When threat intelligence is confirmed by three or more independent trusted sources, the intelligence credibility adjustment coefficient is set to 0.85 to lower the warning threshold and ensure that high-credibility threats are not missed. When intelligence is confirmed by two trusted sources, the adjustment coefficient is set to 0.95. When intelligence is confirmed by only one source, the adjustment coefficient remains at 1.0 to maintain the baseline threshold and avoid false alarms caused by low-credibility intelligence. The final threshold calculation multiplies the base threshold by the above three coefficients to obtain the final dynamic warning threshold.

[0065] Furthermore, based on the calculated dynamic threshold and the comprehensive risk value of the threat, the early warning information is divided into three clear levels. A high-risk warning is triggered when the comprehensive risk value of the threat simultaneously meets two conditions: a risk value greater than or equal to 0.8 and a risk value exceeding the dynamic warning threshold. A medium-risk warning is triggered when the comprehensive risk value of the threat exceeds the dynamic warning threshold but does not reach 0.8. A low-risk warning is triggered when the comprehensive risk value of the threat does not exceed the dynamic warning threshold, but the difference is within 0.2. No warning is generated when the comprehensive risk value is more than 0.2 below the dynamic threshold. Finally, the warning level and corresponding initial notification are sent to the relevant personnel. For threat behaviors with the same attack source IP, the same target system, and the same attack type, only one warning message is generated within a 10-minute time window. Optionally, when the same attack source is detected to have performed continuous attack steps such as scanning, vulnerability exploitation, and data theft within a short period, these scattered alarms are automatically aggregated into a complete attack chain alarm.

[0066] Furthermore, multi-dimensional situational awareness can include global threat situational heatmaps, dynamic network topology maps, or time-series threat trend maps. Global threat situational heatmaps utilize Geographic Information System (GIS) technology to visualize threat distribution on a map, with each geographic location represented by a heat point. The size of the point represents the number of threats, and the color intensity represents the severity of the threat. Dynamic network topology maps visualize network topology and attack paths. Nodes represent network devices, servers, and terminals, edges represent network connections, node size reflects asset importance, and color reflects security status. When an attack is detected, the attack path is automatically highlighted, with red arrows indicating the attack direction, and the attack stages are displayed, including reconnaissance, intrusion, lateral movement, and data theft. Time-series threat trend maps use multi-indicator time-series charts to display the changing trends of key security indicators, including threat quantity trend lines, threat type distribution stacking diagrams, response time change curves, and system health indices.

[0067] It should be noted that this invention collects multi-source heterogeneous data, intelligently preprocesses to eliminate data noise and extract key features, transforms raw data into threat signals through multimodal detection, forms complete attack chains from isolated threat points through intelligence fusion, and uses adaptive early warning to transform threat signals into dynamic early warning information. Ultimately, it visualizes the complex network security situation, achieving unified management and correlation analysis of security data. Through the synergistic effect of multi-source heterogeneous data fusion and collection and multimodal threat detection models, it constructs a three-dimensional threat cognition capability, achieving comprehensive coverage of network threats. The multimodal analysis capability ensures cross-validation of threats from multiple dimensions such as network traffic, terminal behavior, log data, and threat intelligence, and combines external threat intelligence for cross-validation, forming an understanding of attacker intent, attack methods, and attack targets, significantly improving the accuracy of threat detection.

[0068] The above is an illustrative scheme of an intelligent network information security early warning method based on big data according to this embodiment. It should be noted that the technical solution of this intelligent network information security early warning system based on big data and the technical solution of the aforementioned intelligent network information security early warning method based on big data belong to the same concept. Details not described in detail in the technical solution of the intelligent network information security early warning system based on big data in this embodiment can be found in the description of the technical solution of the aforementioned intelligent network information security early warning method based on big data.

[0069] This embodiment provides an intelligent network information security early warning system based on big data, including: The data acquisition module is used to collect multi-source heterogeneous data from monitoring network devices in real time, perform quality protection processing on the collected raw data, and generate a raw security dataset in a unified format. The feature extraction module is used to preprocess the original security dataset, extract multi-dimensional features from the data, and perform standardization, normalization and data augmentation on the extracted features to generate a set of structured feature vectors. The detection module is used to construct a multimodal threat detection model based on a set of feature vectors. The multimodal threat detection model includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module, and generates threat detection results. The assessment module is used to integrate threat detection results with external threat intelligence, build a threat intelligence knowledge base, enhance the scoring of threat detection results based on the threat intelligence knowledge base, and calculate the comprehensive risk value of network threats. The early warning module is used to dynamically set early warning thresholds based on comprehensive risk values, adaptively determine early warning results based on threat detection results, generate graded early warning information, and visualize the graded early warning information through a multi-dimensional situation view.

[0070] This embodiment also provides a computer device suitable for intelligent network information security early warning based on big data, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement a big data-based intelligent network information security early warning method as proposed in the above embodiments.

[0071] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a big data-based intelligent network information security early warning method as proposed in the above embodiments.

[0072] The storage medium proposed in this embodiment belongs to the same inventive concept as the method for implementing intelligent network information security early warning based on big data proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0073] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0074] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for intelligent network information security early warning based on big data, characterized in that, include: Real-time acquisition and monitoring of multi-source heterogeneous data from network devices; quality protection processing of the acquired raw data; generation of raw security datasets in a unified format. The original security dataset is preprocessed, multi-dimensional features are extracted, and the extracted features are standardized, normalized, and augmented to generate a set of structured feature vectors. Based on the set of feature vectors, a multimodal threat detection model is constructed. The multimodal threat detection model includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module, and generates threat detection results. The threat detection results are integrated with external threat intelligence to construct a threat intelligence knowledge base. The threat detection results are then enhanced and scored based on the threat intelligence knowledge base to calculate the comprehensive risk value of network threats. Based on the comprehensive risk value, the early warning threshold is dynamically set, the threat detection results are adaptively judged, the graded early warning information is generated, and the graded early warning information is visualized through a multi-dimensional situation view.

2. The intelligent network information security early warning method based on big data as described in claim 1, characterized in that, Generating a raw security dataset in a uniform format includes: Data acquisition devices deployed at network boundaries, terminal devices, cloud platforms, and security devices are used to collect multi-source heterogeneous data in real time. The multi-source heterogeneous data includes network traffic data, terminal behavior data, security device logs, and cloud environment data. The raw data collected is subjected to quality inspection, including data integrity review, data timeliness review, and data consistency review. Data that does not meet the quality requirements is marked and re-collected or supplemented. The collected data undergoes privacy protection processing, including data anonymization, encryption of fields that retain relevance, and range-based processing of numerical values.

3. The intelligent network information security early warning method based on big data as described in claim 1, characterized in that, Generating a structured set of feature vectors includes: Based on the original security dataset, a distribution analysis was performed on each data field, and an adaptive data cleaning algorithm was used to identify and process abnormal data. Multi-dimensional feature extraction of data is achieved through time series feature extraction, graph structure feature construction, text semantic feature extraction, and behavioral pattern feature extraction. The extracted features are standardized and normalized, and synthetic minority class oversampling is used to augment the attack samples. The data stream is divided into fixed-size time windows, and data processing is performed independently within each window, ultimately generating a set of structured feature vectors.

4. The intelligent network information security early warning method based on big data as described in claim 3, characterized in that, The multi-dimensional feature extraction includes: Sliding window statistical features are extracted from time series data. Data statistics are calculated within the time window, and autocorrelation coefficients and spectral features are calculated to extract time series features. By treating network entities as graph nodes and relationships between entities as graph edges, a dynamic knowledge graph is constructed. A centrality index is calculated for each node, and a weight is calculated for each edge, in order to construct and extract graph structure features. For the text description content in the security log, a deep learning model is used to extract semantic features. The extracted features include at least one of the following: threat type, attack method, and scope of impact, in order to extract text semantic features. For user and entity behavior sequences, behavioral pattern features are extracted, Markov transition probability matrices of the behavior sequences are calculated, frequent behavioral patterns are extracted, and behavioral anomaly is calculated to extract time series features.

5. The intelligent network information security early warning method based on big data as described in claim 1, characterized in that, Building a multimodal threat detection model includes: A time series anomaly detection module is constructed, which uses a long short-term memory network combined with an attention mechanism to analyze time series data. It identifies time series anomalies by calculating reconstruction errors and dynamically determines key time steps using attention weights. A graph structure anomaly detection module is constructed, which uses graph convolutional networks to model network topology and entity relationships, aggregates neighbor node information, calculates the local and global structural anomalies of nodes, and identifies abnormal subgraphs and cooperative attack behaviors. A multimodal fusion detection module is constructed, which uses a multi-input neural network to process time-series data, graph data and text data respectively. The outputs of each module are dynamically weighted and combined through an attention fusion layer. A multi-task learning strategy is adopted to simultaneously optimize anomaly detection, threat classification and attack phase identification tasks. Based on unsupervised and semi-supervised learning mechanisms, anomalies are determined by reconstruction error, the discriminative feature is enhanced by contrastive loss function, and samples are selected for labeling based on uncertainty score and representativeness score. Adversarial robustness enhancement strategies include adversarial training to generate adversarial examples to improve the model's anti-interference ability, input verification and sanitization mechanisms to filter malicious inputs, and model ensemble defense to improve detection stability through heterogeneous model voting fusion. By integrating context-aware detection strategies, both temporal and spatial context information are detected, and detection sensitivity and threat weight are dynamically adjusted.

6. The intelligent network information security early warning method based on big data as described in claim 5, characterized in that, The fusion of the threat detection results with external threat intelligence includes: Collect threat intelligence from multiple sources, build an intelligence quality assessment system, and dynamically score intelligence sources; Standardize and disambiguate threat intelligence, define threat intelligence data models, identify malware variants, analyze the duration, frequency, and geographical distribution characteristics of attack activities through spatiotemporal correlation diagrams, and build a threat intelligence knowledge base. Based on the aforementioned threat intelligence knowledge base, an intelligence enhancement and detection optimization strategy is set up. A multi-factor threat scoring algorithm is used to combine the basic detection score, intelligence enhancement coefficient, and environmental adjustment coefficient to obtain a comprehensive risk value of network threats.

7. The intelligent network information security early warning method based on big data as described in claim 6, characterized in that, The generation of tiered early warning information includes: Define a base threshold, and dynamically adjust it in conjunction with threat environment adjustment coefficient, business importance adjustment coefficient, and intelligence credibility adjustment coefficient to determine the dynamic threshold for early warning; By combining the comprehensive risk value of the network threat with the dynamic warning threshold, the warning levels are divided, and graded warning information is generated. A multi-dimensional situation view is then visualized, which includes a global threat situation heatmap, a dynamic network topology map, or a time-series threat trend map.

8. A big data-based intelligent network information security early warning system, employing the big data-based intelligent network information security early warning method as described in any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to collect multi-source heterogeneous data from monitoring network devices in real time, perform quality protection processing on the collected raw data, and generate a raw security dataset in a unified format. The feature extraction module is used to preprocess the original security dataset, extract multi-dimensional features from the data, and perform standardization, normalization and data augmentation on the extracted features to generate a set of structured feature vectors. The detection module is used to construct a multimodal threat detection model based on the feature vector set. The multimodal threat detection model includes a temporal anomaly detection module, a graph structure anomaly detection module, and a multimodal fusion detection module, and generates threat detection results. The evaluation module is used to integrate the threat detection results with external threat intelligence, construct a threat intelligence knowledge base, enhance the scoring of the threat detection results based on the threat intelligence knowledge base, and calculate the comprehensive risk value of network threats. The early warning module is used to dynamically set early warning thresholds based on the comprehensive risk value, perform adaptive early warning judgment on threat detection results, generate graded early warning information, and visualize the graded early warning information through a multi-dimensional situation view.

9. A computer device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the intelligent network information security early warning method based on big data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, which, when executed by a processor, implement the steps of the intelligent network information security early warning method based on big data as described in any one of claims 1 to 7.

Images