Cybersecurity state characterization method, cybersecurity intelligent decision method, apparatus

Abstract

The application discloses a network security state representation method and a network security intelligent decision method and device. A topology graph is constructed based on network traffic data and environment context data in a target network security scene, and a network traffic branch feature vector is obtained by extracting features of key subgraphs from the topology graph and splicing and aggregating the features. A user behavior branch feature vector is obtained from user behavior data. The two vectors are fused to obtain a fusion feature vector of the target network security scene. Network traffic data, environment context data, user behavior data and system logs in the target network security scene are obtained. A double-branch cross-fusion network of a network security intelligent decision model obtains the fusion feature vector of the target network security scene based on the network security state representation method. An RLHF-based model extracts a context vector from the system logs, splices the fusion feature vector and the context vector to obtain an enhanced feature vector, and an intelligent decision result is output by a reinforcement learning decision module.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method for representing network security status, a method for intelligent network security decision-making, and an apparatus. Background Technology

[0002] With the rapid development of the internet and information technology, cybersecurity threats are becoming increasingly complex and varied. Traditional security measures such as firewalls, intrusion detection systems (IDS), and antivirus software are no longer sufficient to cope with new types of attacks. In recent years, artificial intelligence and machine learning technologies have been widely used in the field of cybersecurity. By analyzing massive amounts of data, they can automatically identify potential security threats and make cybersecurity decisions.

[0003] A prerequisite for cybersecurity decision-making is an accurate understanding of the current cybersecurity status of the environment. This is essential to ensuring the security and reliability of cybersecurity decisions. Currently, existing technologies often suffer from insufficient accuracy in representing the cybersecurity status, which in turn affects the accuracy, security, and reliability of cybersecurity decisions. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method for representing network security status, a method for intelligent network security decision-making, and an apparatus, with the aim of achieving accurate representation of network security status, thereby improving the accuracy, security, and reliability of network security decisions.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] The first aspect of this application provides a method for characterizing network security status, the method comprising:

[0007] A topology graph is constructed based on network traffic data and environmental context data in the target network security scenario. Key subgraphs are extracted from the topology graph, and subgraph features are extracted from the key subgraphs and then spliced ​​and aggregated to obtain network traffic branch feature vectors.

[0008] User behavior branch feature vectors are obtained from user behavior data in the target network security scenario using BiLSTM;

[0009] By fusing the network traffic branch feature vector and the user behavior branch feature vector, a fused feature vector of the target network security scenario is obtained; the fused feature vector is used to characterize the network security status of the target network security scenario.

[0010] In an optional implementation, fusing the network traffic branch feature vector and the user behavior branch feature vector to obtain the fused feature vector of the target network security scenario includes:

[0011] The network traffic branch feature vector and the user behavior branch feature vector are normalized respectively to obtain the normalized network traffic branch feature vector and the normalized user behavior branch feature vector.

[0012] The normalized network traffic branch feature vector is processed using a first gating network unit to obtain a first processing result vector; the normalized user behavior branch feature vector is processed using a second gating network unit to obtain a second processing result vector; the first gating network unit includes a first weight matrix and a first bias parameter corresponding to the network traffic branch feature vector; the second gating network unit includes a second weight matrix and a second bias parameter corresponding to the user behavior branch feature vector.

[0013] An activation function is used to generate a first gating weight coefficient vector based on the first processing result vector; an activation function is used to generate a second gating weight coefficient vector based on the second processing result vector.

[0014] The first gating weight coefficient vector is multiplied element-wise with the normalized network traffic branch feature vector to obtain the first product vector; the second gating weight coefficient vector is multiplied element-wise with the normalized user behavior branch feature vector to obtain the second product vector.

[0015] The first product vector is added to the second product vector to obtain the fused feature vector of the target network security scenario.

[0016] In an optional implementation, the construction of the topology map based on network traffic data and environmental context data in the target network security scenario includes:

[0017] A network traffic graph is constructed based on the network traffic data, with IP addresses and service ports as nodes and traffic sessions as edges; an environment context graph is constructed based on the environment context data, with devices as nodes and network topology and load balancing as edges.

[0018] Based on the mapping relationship between IP and devices, an association is established between the network traffic graph and the environment context graph, and a unified topology graph is formed based on the association and the network traffic graph and the environment context graph.

[0019] In an optional implementation, extracting key subgraphs from the topology graph includes:

[0020] Based on high-frequency traffic rules, a first type of key sub-graph is extracted from the topology graph where the session frequency of the source IP is higher than a preset frequency or the port traffic is higher than a preset traffic.

[0021] Based on the high-load device rules, a second type of key subgraph is extracted from the topology graph, where the CPU utilization rate continuously exceeds a preset utilization rate for a preset duration or the memory usage exceeds a safety baseline.

[0022] In an optional implementation, the step of extracting subgraph features from the key subgraph and then concatenating and aggregating them to obtain a network traffic branch feature vector includes:

[0023] A multi-layer graph neural network is used to extract subgraph features layer by layer from each of the key subgraphs;

[0024] The extracted subgraph features are concatenated and aggregated using global average pooling to form an aggregated graph feature vector.

[0025] The feature dimension of the aggregated graph feature vector is compressed by a fully connected layer, and then processed by a nonlinear activation function to finally obtain the network traffic branch feature vector.

[0026] In an optional implementation, the user behavior data includes user behavior logs, which in turn include user time-series features; these features include the user's login time, operation frequency, and permission changes; obtaining the user behavior branch feature vector from the user behavior data in the target network security scenario using BiLSTM includes:

[0027] The user's temporal features are processed in chronological order by a forward LSTM and in reverse chronological order by a backward LSTM to obtain a first hidden state sequence generated by the forward LSTM and a second hidden state sequence generated by the backward LSTM.

[0028] The first hidden state sequence and the second hidden state sequence are concatenated at each time step to form a bidirectional hidden state sequence. Then, a global average pooling operation is performed on the bidirectional hidden state sequence to generate the user behavior branch feature vector.

[0029] A second aspect of this application provides a cybersecurity intelligent decision-making method, the method comprising:

[0030] Network traffic data, environmental context data, user behavior data, and system logs from the target network security scenario are acquired and used as input to the network security intelligent decision-making model; the network security intelligent decision-making model includes a dual-branch cross-fusion network and a model based on human feedback reinforcement learning (RLHF).

[0031] The network security status-based representation method of the dual-branch cross-fusion network is used to obtain the fusion feature vector of the target network security scenario; the fusion feature vector is used to represent the network security status of the target network security scenario; the network security status representation method is the method described in any implementation of the first aspect.

[0032] The context vector is extracted from the system log using a feature extractor in the RLHF-based model;

[0033] The fused feature vector and the context vector are concatenated through the feature concatenation layer of the RLHF-based model to obtain the enhanced feature vector;

[0034] The enhanced feature vector is used as the input to the reinforcement learning decision module in the RLHF-based model. The reinforcement learning decision module analyzes the enhanced feature vector and outputs an intelligent decision result for the target network security scenario.

[0035] In an optional implementation, the network security intelligent decision-making model is formed by training an initial decision-making model based on multimodal historical data; the multimodal historical data includes network traffic historical data, environmental context historical data, user behavior historical data, system historical logs, and manual feedback data in the same network security scenario;

[0036] The initial decision model includes: an initial two-branch cross-fusion network and an initial reinforcement learning model; the initial reinforcement learning model includes: a pre-trained model, a splicing layer, and a supervised fine-tuning model.

[0037] The process of training the initial decision model to obtain the cybersecurity intelligent decision model includes:

[0038] The initial dual-branch cross-fusion model obtains the historical fusion feature vector based on the historical network traffic data, the historical environmental context data, and the historical user behavior data;

[0039] The historical context vector is extracted by the pre-trained model based on the system's historical logs;

[0040] The concatenation layer in the initial reinforcement learning model concatenates the historical fusion feature vector and the historical context vector into a historical augmented feature vector.

[0041] The supervised fine-tuning model predicts network security risk categories based on the historical enhanced feature vectors, and adjusts the parameters of the supervised fine-tuning model based on the predicted network security risk categories and the actual category labels;

[0042] By employing the RLHF reinforcement learning method and combining it with the aforementioned human feedback data, further reinforcement learning is carried out on the supervised fine-tuning model after parameter adjustment to obtain the aforementioned intelligent decision-making model for cybersecurity; the human feedback data includes one or more of the following: feedback data from cybersecurity experts, feedback data from cybersecurity teams, and external threat intelligence.

[0043] In an optional implementation, the initial dual-branch cross-fusion model includes: a network traffic branch network, a user behavior branch network, and a dynamic fusion network;

[0044] The process of obtaining the historical fusion feature vector by the initial dual-branch cross-fusion model based on the historical network traffic data, the historical environmental context data, and the historical user behavior data includes:

[0045] A historical topology map is constructed from the network traffic branch network based on the historical network traffic data and the historical environmental context data. Historical key subgraphs are extracted from the historical topology map, and features of historical subgraphs are extracted from the historical key subgraphs and then spliced ​​and aggregated to obtain a historical network traffic branch feature vector.

[0046] The user behavior branch network obtains historical user behavior branch feature vectors from the historical user behavior data through BiLSTM;

[0047] The historical fusion feature vector is obtained by fusing the historical network traffic branch feature vector and the historical user behavior branch feature vector by the dynamic fusion network.

[0048] In an optional implementation, the pre-trained model structure includes a 6-layer Transformer encoder; the extraction of the historical context vector from the system historical logs by the pre-trained model includes:

[0049] The system's historical logs are cleaned and standardized.

[0050] The pre-trained model is trained using a self-supervised task based on masked language modeling (MLM). The semantic encoding capability of the pre-trained model is trained through a dynamic masking mechanism, and the training parameters of the pre-trained model are optimized. During the training process of the pre-trained model, the semantic understanding capability of the pre-trained model is preserved by minimizing the MLM loss.

[0051] The pre-trained model generates a historical context vector containing log information, which is used as a feature extractor in the network security intelligent decision-making model.

[0052] In an optional implementation, the structure of the supervised fine-tuning model includes a 4-layer Transformer encoder and a 2-layer Multilayer Perceptron (MLP) classifier; the step of the supervised fine-tuning model predicting cybersecurity risk categories based on the historical enhanced feature vectors, and adjusting the parameters of the supervised fine-tuning model based on the predicted cybersecurity risk categories and the actual category labels, includes:

[0053] The 4-layer Transformer encoder learns cross-modal feature interactions based on the historical enhanced feature vectors.

[0054] The probability distribution of three network security risk categories is predicted using a two-layer MLP; the three network security risk categories include: normal state, low-risk warning, and high-risk blocking.

[0055] If the probability distribution of the three cybersecurity risk categories predicted by the supervised fine-tuning model does not match the actual category labels, the parameters of the 4-layer Transformer encoder and the 2-layer MLP are jointly updated and adjusted during the supervised fine-tuning phase until the early stopping condition is met.

[0056] In an optional implementation, the training of the supervised fine-tuning phase of the supervised fine-tuning model is performed offline; the method of using RLHF reinforcement learning and combining it with the human feedback data to further strengthen learning on the basis of the parameter-adjusted supervised fine-tuning model to obtain the network security intelligent decision-making model includes:

[0057] After the supervised fine-tuning model with adjusted parameters is deployed offline, the RLHF reinforcement learning method is used in conjunction with the human feedback data to start online learning of the supervised fine-tuning model with adjusted parameters.

[0058] During online learning, the parameters of the 4-layer Transformer encoder of the supervised fine-tuning model with adjusted parameters are frozen, and only the parameters of the 2-layer MLP are adjustable; the policy network predicts the network security risk category based on the historical enhanced feature vectors and outputs the decision result; based on the human feedback data, a reward value is calculated using a reward function; a value network is used to generate a state value assessment result based on the decision result output by the policy network and the reward value; the policy network is adjusted in conjunction with the state value assessment result and the reward value; the human feedback data includes correction labels for the human feedback decision results;

[0059] The optimization objective of the policy network is to maximize the reward obtained by the decision sequence, and the optimization objective of the value network is to minimize the mean square error between TD-Target and the current state value estimate, wherein TD-Target is the sum of the immediate reward and the discounted next state value estimate;

[0060] The policy network and the value network share the 4-layer Transformer encoder. The policy network has a 2-layer MLP policy head, and the value network has a 1-layer MLP value head.

[0061] A third aspect of this application provides a network security status characterization device, the device comprising:

[0062] The first branch feature extraction module is used to construct a topology graph based on network traffic data and environmental context data in the target network security scenario, extract key subgraphs from the topology graph, extract subgraph features from the key subgraphs, and then splice and aggregate them to obtain a network traffic branch feature vector.

[0063] The second branch feature extraction module is used to obtain user behavior branch feature vectors from user behavior data in the target network security scenario using BiLSTM.

[0064] The fusion module is used to fuse the network traffic branch feature vector and the user behavior branch feature vector to obtain a fused feature vector of the target network security scenario; the fused feature vector is used to characterize the network security status of the target network security scenario.

[0065] A fourth aspect of this application provides a cybersecurity intelligent decision-making device, the device comprising:

[0066] The input data acquisition module is used to acquire network traffic data, environmental context data, user behavior data, and system logs in the target network security scenario as input to the network security intelligent decision-making model; the network security intelligent decision-making model includes a dual-branch cross-fusion network and a model based on human feedback reinforcement learning (RLHF).

[0067] The dual-branch cross-fusion module is used to obtain a fusion feature vector of the target network security scenario through the network security status representation method of the dual-branch cross-fusion network; the fusion feature vector is used to represent the network security status of the target network security scenario; the network security status representation method is the method described in any implementation of the first aspect;

[0068] The log context feature extraction module is used to extract the context vector from the system log through the feature extractor in the RLHF-based model;

[0069] The concatenation module is used to concatenate the fused feature vector and the context vector through the feature concatenation layer of the RLHF-based model to obtain an enhanced feature vector;

[0070] The intelligent decision-making module is used to take the enhanced feature vector as input to the reinforcement learning decision-making module in the RLHF-based model, and use the reinforcement learning decision-making module to analyze the enhanced feature vector and output an intelligent decision result for the target network security scenario.

[0071] A fifth aspect of this application provides a computer device, the device comprising: a processor and a memory communicatively connected to each other;

[0072] The memory stores computer programs;

[0073] The processor is configured to run the computer program to implement the network security status characterization method as described in any implementation of the first aspect, or to implement the network security intelligent decision-making method as described in any implementation of the second aspect.

[0074] Compared with the prior art, this application has the following beneficial effects:

[0075] The network security status representation method provided in this application extracts feature vectors from two different schemes in the target network security scenario and fuses them to achieve a high-coverage and high-accuracy representation of the network security status. The method includes: constructing a topology graph based on network traffic data and environmental context data in the target network security scenario; extracting and aggregating features from key subgraphs to obtain network traffic branch feature vectors; obtaining user behavior branch feature vectors from user behavior data in the target network security scenario using BiLSTM; and fusing the two types of vectors to obtain a fused feature vector for the target network security scenario, which represents the network security status of the target network security scenario. Because this method fuses network traffic branch feature vectors and user behavior branch feature vectors, it fully considers network traffic, environmental context, and user behavior in representing the network security status. Furthermore, the extraction methods of the two feature vectors are matched with the temporal characteristics of the constructed topology graph and the user behavior data, thereby achieving effective extraction of key features. The fused feature vector obtained based on this fusion can more accurately represent the network security status in the target network security scenario compared to existing technologies. This improves the accuracy, security, and reliability of network security decisions.

[0076] The network security intelligent decision-making method provided in this application first acquires data from multiple modalities in the target network security scenario, including network traffic data, environmental context data, user behavior data, and system logs. This data serves as input to the network security intelligent decision-making model. Then, by employing a dual-branch cross-fusion network of the network security intelligent decision-making model based on the network security status representation method proposed in this application, a fused feature vector of the target network security scenario is obtained. Next, based on the Human Feedback-Based Reinforcement Learning (RLHF) model within the network security intelligent decision-making model, the feature extractor in the RLHF model extracts the context vector from the system logs. The feature concatenation layer in the RLHF model then concatenates and fuses the feature vector and the context vector to obtain an enhanced feature vector. Finally, this enhanced feature vector is used as input to the reinforcement learning decision-making module in the RLHF-based model. The reinforcement learning decision-making module analyzes this enhanced feature vector machine and outputs the intelligent decision result. Because the network security decision-making in this application is based on the fusion feature vector of the target network security scenario and combined with the RLHF-based model for analysis and decision-making, it can continuously optimize and learn by combining human feedback (such as data from network security experts or network security teams, or external threat intelligence), thereby improving the accuracy and adaptability of network security decision-making and avoiding the problem of high false alarm rate caused by simplified and rigid decision-making mechanisms, which seriously affects the efficiency of security operations. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 A flowchart illustrating a method for characterizing network security status as provided in an embodiment of this application;

[0079] Figure 2 An implementation architecture diagram of a network security status characterization method provided in this application embodiment;

[0080] Figure 3 A flowchart illustrating a network security intelligent decision-making method provided in this application embodiment;

[0081] Figure 4 An implementation architecture diagram of the network security intelligent decision-making method provided in the embodiments of this application;

[0082] Figure 5 A flowchart illustrating how to train an initial decision-making model to obtain a cybersecurity intelligent decision-making model;

[0083] Figure 6 This is a schematic diagram illustrating how RLHF reinforcement learning, combined with human feedback data, is used to further reinforce a supervised fine-tuning model with adjusted parameters, as provided in an embodiment of this application.

[0084] Figure 7 A schematic diagram of the system architecture used in the network security intelligent decision-making method provided in the embodiments of this application;

[0085] Figure 8 A schematic diagram of the structure of a network security status characterization device provided in an embodiment of this application;

[0086] Figure 9 This is a schematic diagram of the structure of a network security intelligent decision-making device provided in an embodiment of this application. Detailed Implementation

[0087] As described earlier, an accurate grasp of the cybersecurity status determines the quality of cybersecurity decisions. Specifically, inaccurate representation of the cybersecurity status will negatively impact the accuracy, security, and reliability of cybersecurity decisions. Analysis reveals that current technologies for representing the cybersecurity status rely on relatively limited data sources, resulting in incomplete data analysis related to cybersecurity. Furthermore, the data analysis methods and feature extraction techniques used for cybersecurity have not been adapted or adjusted accordingly. These factors contribute to inaccurate representation of the cybersecurity status, thereby affecting cybersecurity decisions.

[0088] Through research, the inventors have proposed a method for representing network security status, a method for intelligent network security decision-making, and a device. The aim is to extract and fuse features from multimodal data using a dual-branch cross-fusion network architecture, ultimately achieving an accurate representation of network security status, thereby improving the quality of network security decision-making and overcoming existing technical problems.

[0089] To facilitate understanding of the technical solution of this application, several technical terms involved in this application will be introduced and explained first.

[0090] RLHF (Reinforcement Learning with Human Feedback) is a learning method that combines reinforcement learning with human feedback. In RLHF, human feedback is used to guide or correct the model's learning process. This feedback can be scoring, ranking, or direct policy suggestions, helping to improve learning efficiency and decision quality, especially in complex or high-risk decision-making environments.

[0091] GAT (Graph Attention Networks): GAT (Graph Attention Networks) is a type of graph neural network (GNN) that uses attention mechanisms to process graph-structured data. By dynamically assigning different weights to each node in the graph, GAT is able to capture the complex relationships and dependencies between nodes.

[0092] Subgraph sampling: Subgraph sampling is the process of extracting subgraphs from a large graph, with the aim of reducing computational complexity and improving algorithm efficiency. By selecting key nodes and edges in the graph, subgraphs can preserve the main features and structure of the original graph, while reducing the difficulty of data processing and the required computational resources.

[0093] Multimodal fusion: Multimodal fusion originally refers to combining data from different sources or types (such as text, images, sound, etc.) to form a unified and richer data representation. In this application, it refers to the technology for unified representation of heterogeneous data such as network traffic, behavior logs, and system logs.

[0094] Gating Mechanism: A gating mechanism is a technique for controlling the flow of information, determining which information should be passed to the next stage of the system. In machine learning, gating mechanisms can be used to dynamically adjust the importance of different features or data streams, thereby optimizing the performance and adaptability of the model.

[0095] Bidirectional Long Short-Term Memory (BiLSTM) networks: BiLSTM is a special type of recurrent neural network (RNN) that contains two LSTM layers, one processing the forward sequence and the other processing the reverse sequence. This structure allows the model to consider both past and future information of the sequence simultaneously.

[0096] Pre-trained lightweight DistilBERT: DistilBERT is a pre-trained BERT model that reduces the size and computational requirements of the original BERT architecture while maintaining BERT's performance as much as possible.

[0097] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0098] See Figure 1 and Figure 2 ,in Figure 1 A flowchart illustrating a method for characterizing network security status provided in this application embodiment. Figure 2 This is an implementation architecture diagram for methods of representing network security status. (Example:) Figure 1 The methods for representing network security status as shown include:

[0099] S101. Construct a topology graph based on network traffic data and environmental context data in the target network security scenario, extract key subgraphs from the topology graph, extract subgraph features from the key subgraphs, and then splice and aggregate them to obtain network traffic branch feature vectors.

[0100] In this embodiment of the application, to achieve an accurate representation of the network security status in the target network security scenario, it is first necessary to acquire multimodal data in the scenario, including network traffic data, environmental context data, and user behavior data. Feature extraction is performed on the network traffic data and environmental context data in step S101; feature extraction is performed on the user behavior data in step S102.

[0101] like Figure 2 In the implementation architecture shown, the area indicated by the dashed line can be understood as a dual-branch cross-fusion network. This dual-branch cross-fusion network includes a network traffic branch network, a user behavior branch network, and a fusion network for performing vector fusion. This step S101 can be specifically defined by… Figure 2 The network traffic branch network shown in the figure constructs a topology graph based on network traffic data and environmental context data, extracts key subgraphs from it, and splices and aggregates the subgraph features in the key subgraphs. Finally, the network traffic branch network outputs a network traffic branch feature vector.

[0102] The following is an exemplary introduction and explanation of several key processes involved in step S101.

[0103] The first step is the construction of the graph structure. In this embodiment, constructing a topology graph based on network traffic data and environmental context data in the target network security scenario can include:

[0104] A network traffic graph is constructed based on network traffic data, with IP addresses and service ports as nodes and traffic sessions as edges. An environment context graph is constructed based on environment context data, with devices as nodes and network topology and load balancing as edges. Based on the mapping relationship between IP addresses and devices, an association is established between the network traffic graph and the environment context graph. Based on this association, a unified topology graph is formed using the network traffic graph and the environment context graph.

[0105] Table 1 illustrates the construction concept of the topology graph described above. Referring to Table 1, the first column from left to right shows the basic structure of the graph, such as nodes and edges; the second column shows the specific meaning of nodes and edges in the network traffic graph; the third column shows the specific meaning of nodes and edges in the context graph; and the fourth column shows the relationship between the network traffic graph and the context graph.

[0106] Table 1

[0107]

[0108] As described above, in this embodiment of the application, two graphs are constructed to mine the characteristics of network traffic branches: a network traffic graph and an environment context graph. IP addresses and service ports are mapped to nodes in the network traffic graph, and traffic sessions are mapped to edges between nodes in the network traffic graph. Similarly, devices such as servers and switches are mapped to nodes in the environment context graph, and network topology and load balancing are mapped to edges between nodes in the environment context graph. Then, by combining the relationships between nodes in these two graphs, they are joined into a unified topology graph (also a heterogeneous graph). Specifically, this can be achieved by using the attention mechanism of GAT to dynamically model the interaction relationships between nodes.

[0109] Next, subgraph sampling is performed. In this embodiment, key subgraphs can be extracted based on heuristic rules (including but not limited to rules for high-frequency traffic, high-load devices, etc.). In optional implementations, extracting key subgraphs from the topology graph may include:

[0110] Based on high-frequency traffic rules, the first type of key subgraph is extracted from the topology graph where the session frequency of the source IP is higher than the preset frequency or the port traffic is higher than the preset traffic. Based on high-load device rules, the second type of key subgraph is extracted from the topology graph where the CPU utilization rate continuously exceeds the preset utilization rate for a preset duration or the memory usage exceeds the security baseline.

[0111] Heuristic rules are employed to extract high-value subgraph structures from large-scale data topology graphs through key subgraph sampling, thereby improving the efficiency of high-value feature extraction. High-frequency traffic rules extract key subgraphs where the source IP has an excessively high session frequency (e.g., >1000 times per unit time) or a sudden surge in port traffic (e.g., far exceeding the daily average). These key subgraphs can be used to detect DDoS attacks, port scanning, brute-force attacks, and other attacks. As an example, high-load device rules extract key subgraphs where CPU utilization is >90% for 5 minutes or memory usage exceeds the security baseline. These subgraphs can be used to effectively detect memory trojan attacks, resource exhaustion attacks, and other attacks. Multiple key subgraphs can be extracted through subgraph sampling. Multiple sampling ensures that the model can capture different parts of the graph, thus providing a more comprehensive characterization of the GAT's graph structure information.

[0112] The following describes an example implementation process for extracting subgraph features from key subgraphs and then concatenating and aggregating them to obtain network traffic branch feature vectors. This process may include:

[0113] A multi-layer graph neural network (GAT) is used to extract subgraph features layer by layer from each key subgraph. The multi-layer GAT employs a multi-head attention mechanism to iteratively update the node features in the graph, capturing complex interactions within the subgraph, such as interactions between device states and network traffic. Furthermore, residual connection layers can be used to avoid gradient vanishing, thus fully preserving the original feature information.

[0114] Applying multi-layer Gaussian Interaction (GAT) to the sampled key subgraphs allows for the extraction of relationships between multiple subgraphs. The output of each layer in the multi-layer GAT serves as the input to the next layer. Through layer-by-layer propagation and updating, the model can extract more complex relationships, making it suitable for feature extraction between multiple subgraphs. After progressively extracting features from each key subgraph using multi-layer GAT, global average pooling can be used to concatenate and aggregate the features from multiple subgraphs, forming an aggregated graph feature vector. This aggregated graph feature vector, as a single vector, can represent the entire subgraph. Finally, a fully connected layer compresses the feature dimension of the aggregated graph feature vector, and a non-linear activation function (e.g., ReLU) is applied to process it, ultimately yielding the network traffic branch feature vector.

[0115] F net =ReLU(W fc ⋅H agg +b fc ); Formula (1)

[0116] In formula (1) F net W represents the feature vector of network traffic branches. fc Let b represent the weight matrix of the fully connected layer. fc H represents the bias vector of the fully connected layer. agg This represents the aggregated graph feature vector.

[0117] Compared to time-series models like LSTM, the Network Traffic Branch Network (BRN) achieves structured modeling and local focus through GAT (Graph Attention) and subgraph sampling, addressing the core pain points of network security detection, such as difficulty in capturing complex relationships and low computational efficiency on a large scale. It demonstrates significant advantages in scenarios like DDoS detection, APT defense, and ransomware blocking. Essentially, the BRN deeply integrates the "relational features" (node ​​interactions) and "local features" (key subgraphs) of network traffic, preserving the semantic information of the graph structure while achieving efficient implementation through dynamic attention and sampling strategies. This provides a more suitable technical solution for real-time security decision-making in complex network environments.

[0118] S102. Obtain user behavior branch feature vectors from user behavior data in the target network security scenario using BiLSTM.

[0119] In optional implementations, user behavior data includes user behavior logs, which contain user temporal characteristics. For example, user temporal characteristics include user login time, operation frequency, and permission changes. Anomalies in these behaviors may indicate network security risks, necessitating analysis and mining to extract features that contribute to accurate network security status representation.

[0120] In this embodiment, user temporal features can be processed sequentially using a forward LSTM and in reverse chronological order using a backward LSTM to obtain a first hidden state sequence generated by the forward LSTM and a second hidden state sequence generated by the backward LSTM. Then, the first and second hidden state sequences are concatenated at each time step to form a bidirectional hidden state sequence. Finally, global average pooling is performed on the bidirectional hidden state sequence to generate a user behavior branch feature vector. In this embodiment, the application of BiLSTM can capture long-distance temporal dependencies and effectively extract key features from user temporal features. These features not only contain direct information about user behavior but also implicitly contain complex relationships and patterns between behaviors. This forms one of the materials that can be used to accurately characterize network security status, namely the user behavior branch feature vector, which can be represented as F. beh .

[0121] S103. Merge the network traffic branch feature vector and the user behavior branch feature vector to obtain the fused feature vector of the target network security scenario. The fused feature vector is used to characterize the network security status of the target network security scenario.

[0122] In this embodiment, the network traffic branch feature vector and the user behavior branch feature vector can be directly fused, or a more flexible and dynamic method can be used to fuse them. The following is an illustrative example of the latter implementation method.

[0123] In optional implementations, network traffic branch feature vectors and user behavior branch feature vectors are fused to obtain a fused feature vector for the target network security scenario, including:

[0124] The network traffic branch feature vector is processed using a first gating network unit to obtain a first processed result vector; the user behavior branch feature vector is processed using a second gating network unit to obtain a second processed result vector. The first gating network unit includes a first weight matrix and a first bias parameter corresponding to the network traffic branch feature vector; the second gating network unit includes a second weight matrix and a second bias parameter corresponding to the user behavior branch feature vector.

[0125] Next, an activation function is used to generate a first gating weight coefficient vector based on the first processing result vector; then, an activation function is used to generate a second gating weight coefficient vector based on the second processing result vector. These two gating weight coefficient vectors are used to dynamically regulate the fusion of network traffic branch feature vectors and user behavior branch feature vectors.

[0126] Specifically, the first gating weight coefficient vector is multiplied element-wise with the network traffic branch feature vector to obtain the first product vector; the second gating weight coefficient vector is multiplied element-wise with the user behavior branch feature vector to obtain the second product vector. Finally, the first product vector and the second product vector are added to obtain the fused feature vector of the target network security scenario. The above process can be detailed in the formula below.

[0127] F fusion = σ(W g ⋅F net + b g )⊙F net +σ(W b ⋅F beh +b b )⊙F beh ; Formula (2)

[0128] In formula (2), F net and F beh These represent the network traffic branch feature vector and the user behavior branch feature vector, respectively. g and b g These represent the first weight matrix and the first bias parameter, respectively, used to adjust the importance of the network traffic branch feature vectors. b and b b represents the second weight matrix and the second bias parameter, respectively, used to adjust the importance of the feature vectors in the user behavior branch. ⊙ represents the Hadamard product, used to multiply the weighted feature vector element-wise with the original feature vector, used to adjust the feature scale or perform feature selection. σ() is the activation function; for example, Sigmoid or ReLU can be used. Activation functions are used to introduce non-linearity, helping the model learn more complex feature interactions.

[0129] It should be noted that if there is a difference in the dimensions between the network traffic branch feature vector and the user behavior branch feature vector, then in this application, the network traffic branch feature vector F can also be... net and user behavior branch feature vector F beh After normalization using layer normalization to eliminate dimensional differences, the models are then fused. This improves the fairness of the fusion process and the model convergence efficiency. Based on the above normalization process, formula (2) can also be evolved into:

[0130] F fusion = σ(W g ⋅F' net + b g )⊙F' net +σ(W b ⋅F' beh +b b )⊙F' beh ; Formula (3)

[0131] In formula (3), F' net and F' beh These represent the normalized network traffic branch feature vector and the normalized user behavior branch feature vector, respectively.

[0132] Combining the implementation of formula (3), in the optional implementation, the network traffic branch feature vector and the user behavior branch feature vector are fused to obtain the fused feature vector of the target network security scenario. This includes: processing the normalized network traffic branch feature vector using a first gating network unit to obtain a first processing result vector; processing the normalized user behavior branch feature vector using a second gating network unit to obtain a second processing result vector. Next, an activation function is used to generate a first gating weight coefficient vector based on the first processing result vector; an activation function is used to generate a second gating weight coefficient vector based on the second processing result vector. The first gating weight coefficient vector is multiplied element-wise with the normalized network traffic branch feature vector to obtain a first product vector; the second gating weight coefficient vector is multiplied element-wise with the normalized user behavior branch feature vector to obtain a second product vector. Finally, the first product vector and the second product vector are added together to obtain the fused feature vector of the target network security scenario. It should be noted that in the embodiments of this application, the first weight matrix, the first bias parameter, the second weight matrix, and the second bias parameter can all be obtained through backpropagation optimization based on the accuracy of the network's classification results. Optimization during the learning process can better reflect the contribution of different features to the final task.

[0133] Combining the formula (3) introduced above and the fusion process of the feature vectors obtained from the two branches, it can be seen that in the dual-branch cross-fusion strategy introduced in this application embodiment, the gating mechanism is the key to achieving dynamic fusion. First, the normalized network traffic branch feature vector F' net and the normalized user behavior branch feature vector F' beh Each feature vector is passed through an independent gated network unit consisting of a weight matrix and bias parameters. Then, an activation function is used to process the features into two gated weight coefficient vectors (each containing generated gated weight coefficients between 0 and 1), dynamically reflecting the importance of each branch's features. Next, Hadamard product is used to select and scale the feature vectors. Finally, the weighted feature vectors are summed to form a fused feature vector F. fusion This mechanism dynamically adjusts the cross-features of the two branches through learnable parameters, and then optimizes the weight matrix and bias parameters within the gated network unit through backpropagation, ensuring that the fusion strategy is compatible with downstream tasks.

[0134] The network security status representation method provided in this application extracts feature vectors from two different schemes in the target network security scenario and fuses them to achieve a high-coverage and high-accuracy representation of the network security status. The method includes: constructing a topology graph based on network traffic data and environmental context data in the target network security scenario; extracting and aggregating features from key subgraphs to obtain network traffic branch feature vectors; obtaining user behavior branch feature vectors from user behavior data in the target network security scenario using BiLSTM; and fusing the two types of vectors to obtain a fused feature vector for the target network security scenario, which represents the network security status of the target network security scenario. Because this method fuses network traffic branch feature vectors and user behavior branch feature vectors, it fully considers network traffic, environmental context, and user behavior in representing the network security status. Furthermore, the extraction methods of the two feature vectors are matched with the temporal characteristics of the constructed topology graph and the user behavior data, thereby achieving effective extraction of key features. The fused feature vector obtained based on this fusion can more accurately represent the network security status in the target network security scenario compared to existing technologies. This improves the accuracy, security, and reliability of network security decisions.

[0135] The preceding embodiments primarily introduced the inventor's newly proposed method for representing network security status. Next, the implementation of intelligent network security decision-making will be described in detail. The intelligent network security decision-making method proposed in this application also utilizes the fused feature vector formed by the network security status representation method of the preceding embodiments.

[0136] See Figure 3The figure is a flowchart of a network security intelligent decision-making method provided in an embodiment of this application. Figure 4 This is a diagram illustrating an implementation architecture of the network security intelligent decision-making method provided in this application. (Combined with...) Figure 4 As can be seen, the intelligent decision-making model for network security includes a dual-branch cross-fusion network and a model based on RLHF. In this application, in order to utilize... Figure 4 The network security intelligent decision-making model shown requires at least multimodal data, including network traffic data, environmental context data, user behavior data, system logs, and human feedback data. Network traffic data, environmental context data, and user behavior data need to be provided to the dual-branch cross-fusion network of the network security intelligent decision-making model to extract features and form a fused feature vector. This process can refer to the network security status representation method described in the previous embodiment. System logs and human feedback data are provided to the RLHF-based model to extract contextual features from the system logs and optimize the decision-making accuracy of the RLHF-based model, improving its ability to identify new threats and its adaptability to complex threat environments. Figure 3 As shown, intelligent decision-making methods for cybersecurity include:

[0137] S301. Obtain network traffic data, environmental context data, user behavior data, and system logs from the target network security scenario as input to the network security intelligent decision-making model.

[0138] like Figure 4 As shown, the network traffic data and environmental context data obtained in the target network security scenario are input into the network traffic branch network of the dual-branch cross-fusion network to obtain the network traffic branch feature vector; the user behavior data obtained in the target network security scenario are input into the user behavior branch network of the dual-branch cross-fusion network to obtain the user behavior branch feature vector; the system logs obtained in the target network security scenario are input into the RLHF-based model, which includes a feature extractor that can be used to extract the context vector from the system logs, as detailed in step S303 below.

[0139] S302. Obtain the fusion feature vector of the target network security scenario through a network security status representation method based on a dual-branch cross-fusion network.

[0140] This step primarily utilizes the network security status representation method described in the previous embodiments. A fused feature vector is obtained based on the dual-branch cross-fusion network, and this fused feature vector is used to characterize the network security status of the target network security scenario. Since this process has been described and explained in detail in the previous embodiments, it will not be repeated here.

[0141] S303. Extract the context vector from the system log using the feature extractor in the RLHF-based model.

[0142] In this embodiment, the feature extractor in the RLHF-based model is a pre-trained model, and therefore can be understood as a pre-trained model. During pre-training, self-supervised learning can be used to extract general semantic features from massive system logs, thereby solving the problem of scarce labeled data in the security field. In specific implementation, tasks such as Masked Language Modeling (MLM) similar to BERT can be used, but are not limited to, to enable the model to learn the contextual association patterns of log events.

[0143] As an example, considering factors such as model performance, computational resources, model deployment, and real-time response speed, the pre-trained lightweight DistilBERT model can be selected as a feature extractor. DistilBERT learns from BERT through knowledge distillation, retaining 97% of BERT's language understanding capabilities while reducing parameters by 40% and improving inference speed by 60%. The DistilBERT model architecture is similar to BERT, with its core architecture being a 6-layer Transformer encoder, only half the size of the original BERT model. Through the self-supervised learning of the DistilBERT model, not only can the problem of scarce labeled data in the security domain be solved, but the model's ability to understand text content can also be improved by performing contextual modeling on unstructured text in system logs, such as error descriptions and operation records.

[0144] In the specific implementation of the pre-training phase, it is necessary to obtain system historical logs (the number of system historical logs depends on training requirements). The pre-trained model extracts historical context vectors from the system historical logs, including: Step 1, cleaning and standardizing the system historical logs. Step 2, training the pre-trained model based on the designed Masked Language Modeling (MLM) self-supervised task, training the semantic encoding capability of the pre-trained model through a dynamic masking mechanism, and optimizing the training parameters of the pre-trained model. In the example, the batch size in step 2 is 128, and the learning rate is 2e-5. It should be noted that the batch size and learning rate should be set relatively small to adapt to real-time requirements. During the training process of the pre-trained model, the semantic understanding capability of the pre-trained model can be preserved by minimizing the MLM loss. Step 3, generating historical context vectors containing log information through the pre-trained model, which is used as a feature extractor in the network security intelligent decision-making model.

[0145] In this embodiment, the network structure of the pre-trained model does not participate in the subsequent fine-tuning and reinforcement learning online learning process. That is, the feature extractor is used after pre-training is completed and does not undergo further optimization or adjustment along with other network structures.

[0146] After the feature extractor is pre-trained, it can be used to extract context vectors from system logs in the target cybersecurity scenario. In the example, the feature extractor generates a 256-dimensional context vector containing log information, providing features for downstream anomaly detection classification tasks.

[0147] S304. The fused feature vector and context vector are concatenated through the feature concatenation layer of the RLHF-based model to obtain the enhanced feature vector.

[0148] In this embodiment, the RLHF-based model includes a feature concatenation layer. This layer concatenates the fused feature vector output by the dual-branch cross-fusion network with the context vector obtained in step S303, thereby achieving the integration and enhancement of multimodal features. The concatenated vector is called the enhanced feature vector.

[0149] As an example, the 256-dimensional context vector generated by the feature extractor is combined with the 256-dimensional fused feature vector F output by the two-branch cross-fusion network. fusion The concatenation results in a 512-dimensional enhanced feature vector F. enhanced This concatenation operation fully utilizes the information from text semantic understanding and structured features, solving the problem of insufficient expressive power of single-modal features and providing a more comprehensive representation for subsequent classification tasks.

[0150] S305. The enhanced feature vector is used as the input to the reinforcement learning decision module in the RLHF-based model. The reinforcement learning decision module analyzes the enhanced feature vector and outputs intelligent decision results for the target network security scenario.

[0151] like Figure 4 As shown, the RLHF-based model also includes a reinforcement learning decision module. In this application, the enhanced feature vector obtained in step S304 is used as the input to this reinforcement learning decision module. The reinforcement learning decision module uses this enhanced feature vector as a basis to make intelligent decisions about the target cybersecurity scenario. The final decision is the output of the intelligent decision result.

[0152] In one possible implementation, the reinforcement learning decision module is designed and trained to output probability distributions for three cybersecurity risk categories: normal state, low-risk warning, and high-risk blocking. The cybersecurity risk category with the highest probability distribution is taken as the intelligent decision result. For example, if the probability of the normal state, low-risk warning, and high-risk blocking output by the reinforcement learning decision module is the highest, then the low-risk warning is taken as the intelligent decision result.

[0153] It should be noted that, in this embodiment, the multimodal data may also include human feedback data, which includes feedback data from cybersecurity experts and teams, as well as external threat intelligence. The concept of "external" refers to information relative to that held by internal cybersecurity experts and teams. Furthermore, cybersecurity experts and teams can provide correction labels for the decision results from human feedback during the learning phase of the reinforcement learning decision module. For example, if the predicted classification reflected in the intelligent decision result does not match the actual classification, the cybersecurity expert or team can provide the correct classification result, i.e., a correction label for the decision result, to improve the accuracy of the intelligent decision. Human feedback data in the multimodal data plays a role in providing new knowledge, enabling the model to continuously acquire the latest threat information, which is then fed back to the offline-trained reinforcement learning decision module. This drives the reinforcement learning decision module to perform online learning and policy optimization, improving its ability to identify new threats and continuously optimizing the overall model performance.

[0154] The network security intelligent decision-making method provided in this application first acquires data from multiple modalities in the target network security scenario, including network traffic data, environmental context data, user behavior data, and system logs. This data serves as input to the network security intelligent decision-making model. Then, by employing a dual-branch cross-fusion network of the network security intelligent decision-making model based on the network security status representation method proposed in this application, a fused feature vector of the target network security scenario is obtained. Next, based on the Human Feedback-Based Reinforcement Learning (RLHF) model within the network security intelligent decision-making model, the feature extractor in the RLHF model extracts the context vector from the system logs. The feature concatenation layer in the RLHF model then concatenates and fuses the feature vector and the context vector to obtain an enhanced feature vector. Finally, this enhanced feature vector is used as input to the reinforcement learning decision-making module in the RLHF-based model. The reinforcement learning decision-making module analyzes this enhanced feature vector machine and outputs the intelligent decision result. Because the network security decision-making in this application is based on the fusion feature vector of the target network security scenario and combined with the RLHF-based model for analysis and decision-making, it can continuously optimize and learn by combining human feedback (such as data from network security experts or network security teams, or external threat intelligence), thereby improving the accuracy and adaptability of network security decision-making and avoiding the problem of high false alarm rate caused by simplified and rigid decision-making mechanisms, which seriously affects the efficiency of security operations.

[0155] In the optional implementation, the cybersecurity intelligent decision-making model is formed by training the initial decision-making model based on multimodal historical data. The term "initial decision-making model" can be used to represent the state of the cybersecurity intelligent decision-making model during the training phase. Its basic structure is similar to that of the cybersecurity intelligent decision-making model. To distinguish it from the cybersecurity intelligent decision-making model, the two main components of the initial decision-making model are referred to as: the initial two-branch cross-fusion network and the initial reinforcement learning model. The initial two-branch cross-fusion network, after training, corresponds to the two-branch cross-fusion network of the cybersecurity intelligent decision-making model; the initial reinforcement learning model, after training, corresponds to the RLHF-based model in the cybersecurity intelligent decision-making model.

[0156] Multimodal historical data includes historical network traffic data, historical environmental context data, historical user behavior data, historical system logs, and manual feedback data within the same cybersecurity scenario.

[0157] In optional implementations, the initial two-branch cross-fusion model includes: a network traffic branch network, a user behavior branch network, and a dynamic fusion network. The initial reinforcement learning model includes: a pre-trained model, a splicing layer, and a supervised fine-tuning model.

[0158] This application employs a three-stage training paradigm of "pre-training-fine-tuning-reinforcement learning" for the initial reinforcement learning model, integrating self-supervised learning, supervised learning, and human expert knowledge to construct a cybersecurity intelligent decision-making model adapted to dynamic network threats. It also builds a closed-loop system of "monitoring-decision-feedback-optimization," overcoming the challenge of static models' inability to adapt to new types of attacks.

[0159] See Figure 5 The diagram illustrates the process of training an initial decision-making model to obtain a cybersecurity intelligent decision-making model. Figure 5 As shown, the process of training the initial decision model to obtain the cybersecurity intelligent decision model includes:

[0160] S501. The initial dual-branch cross-fusion model obtains the historical fusion feature vector based on historical network traffic data, historical environmental context data, and historical user behavior data.

[0161] Specifically, in this step, the network traffic branch network constructs a historical topology map based on historical network traffic data and historical environmental context data, extracts historical key subgraphs from the historical topology map, extracts features from the historical key subgraphs, and then concatenates and aggregates them to obtain historical network traffic branch feature vectors. The user behavior branch network obtains historical user behavior branch feature vectors from historical user behavior data using BiLSTM. The dynamic fusion network fuses the historical network traffic branch feature vectors and historical user behavior branch feature vectors to obtain historical fused feature vectors. This process is similar to the technical implementation of the network security status representation method described in the previous embodiments. Therefore, this step can refer to the description in the previous embodiments, including the method of extracting feature vectors and the method of fusing feature vectors, which will not be repeated here.

[0162] S502. The historical context vector is extracted from the system's historical logs by the pre-trained model.

[0163] The historical context vector is extracted from the system's historical logs by a pre-trained model, including: Step 1, cleaning and standardizing the system's historical logs. Step 2, training the pre-trained model based on the designed Masked Language Modeling (MLM) self-supervised task, training the semantic encoding capability of the pre-trained model through a dynamic masking mechanism, and optimizing the training parameters of the pre-trained model. During the training process, the semantic understanding capability of the pre-trained model can be preserved by minimizing the MLM loss. Step 3, generating a historical context vector containing log information through the pre-trained model, which is used as a feature extractor in the network security intelligent decision-making model.

[0164] S503: The concatenation layer in the initial reinforcement learning model concatenates the historical fusion feature vector and the historical context vector into a historical enhanced feature vector.

[0165] Assuming both the historical fusion feature vector and the historical context vector are 256-dimensional vectors, the resulting historical enhancement feature vector after concatenation is a 512-dimensional vector. This concatenation step fully utilizes the information from textual semantic understanding and structured features, solving the problem of insufficient representational power of single-modal features and providing a more comprehensive representation for subsequent classification tasks.

[0166] S504. The supervised fine-tuning model predicts cybersecurity risk categories based on historical enhanced feature vectors, and adjusts the parameters of the supervised fine-tuning model based on the predicted cybersecurity risk categories and the actual category labels.

[0167] In this embodiment, the supervised fine-tuning model is a downstream model within the initial reinforcement learning model, located downstream of the pre-trained model. Its basic function is to predict network security risk categories based on upstream transmitted data and adjust the parameters of the supervised fine-tuning model based on the predicted categories and actual category labels. As previously mentioned, the initial reinforcement learning model adopts a three-stage training paradigm of "pre-training-fine-tuning-reinforcement learning," integrating self-supervised learning, supervised learning, and human expert knowledge. Step S504 describes the "fine-tuning stage" of this three-stage process. For the supervised fine-tuning model, its input is the historical enhancement feature vector obtained in step S503.

[0168] The supervised fine-tuning model, also known as the SFT model, consists of a 4-layer Transformer encoder and a 2-layer Multilayer Perceptron (MLP) classifier. Based on historical enhanced feature vectors, the supervised fine-tuning model predicts cybersecurity risk categories. The parameters of the supervised fine-tuning model are then adjusted based on the predicted cybersecurity risk categories and the actual category labels, including:

[0169] Cross-modal feature interactions are learned based on historical enhanced feature vectors through a 4-layer Transformer encoder. In the classifier, two fully connected layers combine the ReLU activation function and Dropout mechanism to prevent overfitting; the output layer of the classifier can specifically use Softmax to generate probability distributions for three cybersecurity risk categories: normal state, low-risk warning, and high-risk blocking.

[0170] If the probability distribution of the three cybersecurity risk categories predicted by the supervised fine-tuning model does not match the actual category labels, the parameters of the 4-layer Transformer encoder and the 2-layer MLP will be jointly updated and adjusted during the supervised fine-tuning phase until the early stopping condition is met.

[0171] In practical applications, the AdamW optimizer and early stopping mechanism can be used to adapt to small sample data, ensuring the model's generalization ability. Cross-entropy loss function can be used to optimize classification performance; in this application, Focal Loss can be chosen to address the class imbalance problem. Dynamically allocating weights between text and multimodal features through feature concatenation and attention mechanisms further enhances the model's sensitivity to key features. During the supervised fine-tuning phase, the parameters of both the 4-layer Transformer encoder and the 2-layer MLP network structure are adjusted. The pre-trained model, located upstream of the supervised fine-tuning model, does not participate in the adjustments during the supervised fine-tuning phase after its own training is complete.

[0172] In the optional implementation, training for the supervised fine-tuning phase of the model is performed offline. After completing the supervised fine-tuning phase, the parameter-adjusted supervised fine-tuned model can be deployed. The supervised fine-tuned model provides the initial strategy for the reinforcement learning phase, avoiding the need for reinforcement learning to explore from scratch and improving learning efficiency.

[0173] S505. Using the RLHF reinforcement learning method and combining it with human feedback data, further reinforcement learning is carried out on the basis of the supervised fine-tuning model after parameter adjustment to obtain a cybersecurity intelligent decision-making model.

[0174] Human feedback data includes one or more of the following: feedback data from cybersecurity experts, feedback data from cybersecurity teams, and external threat intelligence.

[0175] Figure 6 This illustration shows a method for further reinforcement learning of a supervised fine-tuning model with adjusted parameters using RLHF reinforcement learning combined with human feedback data, as provided in this embodiment of the application. The initial reinforcement learning model employs a three-stage training paradigm of "pre-training-fine-tuning-reinforcement learning," with this step focusing on the "reinforcement learning" stage. After offline deployment of the supervised fine-tuning model with adjusted parameters, online learning of the supervised fine-tuning model is initiated using RLHF reinforcement learning combined with human feedback data.

[0176] During online learning, the parameters of the 4-layer Transformer encoder in the supervised fine-tuning model are frozen, and only the parameters of the 2-layer MLP are adjustable. This stage can also be understood as initializing the supervised fine-tuning model after offline deployment.

[0177] The reinforcement learning phase involves three important components: (1) the reward function; (2) the policy network (Actor); and (3) the value network (Critic). The policy network and the value network can share a 4-layer Transformer encoder. By sharing the underlying Transformer structure, parameter computation and computational overhead can be reduced.

[0178] Furthermore, the policy network has a 2-layer MLP policy head; the value network has a 1-layer MLP value head, used for direct value estimation. The optimization objective of the policy network is to maximize the reward obtained from the decision sequence, while the optimization objective of the value network is to minimize the mean squared error between the TD-Target and the current state value estimate, where TD-Target is the sum of the immediate reward and the discounted next state value estimate.

[0179] See Figure 6 As shown, in this step, the policy network predicts the network security risk category based on historical enhanced feature vectors and outputs the decision results. Figure 6In this context, the enhanced features refer to the historical enhanced feature vectors obtained through step S503 described above. Figure 6 The initialization of the supervised fine-tuning model in this context refers to the offline deployment of a supervised fine-tuning model with frozen parameters, where only the parameters of the 2-layer Transformer encoder are adjustable. Based on human feedback data, a reward function is used to calculate the reward value, thus transforming the human feedback into reward information to guide policy updates during reinforcement learning stages and optimize the model's prediction results. The reward function optimizes the policy network using the PPO algorithm; see [link to relevant documentation]. Figure 6 The PPO optimization signal is used in the policy network, which provides a three-class decision result: normal, low risk, or high risk, and an automated response is adapted accordingly. The response for normal is to allow passage, for low risk is to limit flow, and for high risk is to block. Human feedback data includes correction labels for the manually fed-in decision results, which can then be used to calculate the reward value in the reward function.

[0180] The value network directly fits the TD-Target, optimizing by minimizing the mean squared error between the current state value estimate and the TD-Target. Based on the decision results and reward values ​​output by the policy network, a state value evaluation result is generated and fed back to the policy function. During the reinforcement learning phase, the policy network is adjusted in conjunction with the state value evaluation results and reward values. This improves the stability and accuracy of training. This architecture forms a closed loop of a dynamic defense mechanism of "monitoring-decision-feedback-optimization," making it more adaptable to dynamic and complex network attacks.

[0181] During the reinforcement learning phase, the value network learns to accurately evaluate the value of a state, while the policy network adjusts its policy within a safe range to maximize returns. To improve training efficiency and address the issue of imbalanced samples, high-risk decision samples can be prioritized and given appropriate higher weights.

[0182] The reward function design can employ various reward mechanisms: for example, a base reward of +1 point is given when the system correctly identifies a threat, and an additional +0.5 point is awarded for a rapid response within 1 second; a -1 point is deducted for a false alarm, and a -2 point penalty is imposed for a missed alarm. Security experts can fine-tune key decisions by ±0.5 points based on actual circumstances. The reward function effectively balances the three core indicators of detection accuracy, response speed, and operational reliability. It should be noted that the reward function mentioned in this application can be configured by cybersecurity experts according to business scenarios or designed as needed; this patent only lists some simple rules.

[0183] Figure 7This is a schematic diagram of the system architecture used in the network security intelligent decision-making method provided in the embodiments of this application. It includes a multimodal data input layer, a dual-branch cross-fusion feature layer, a reinforcement learning model layer based on human feedback, and an intelligent decision output layer.

[0184] like Figure 7 As shown in the embodiments of this application, a network security intelligent decision-making system based on multimodal feature fusion and reinforcement learning is proposed. The system first integrates multi-source data such as user behavior, network traffic, and system logs through a multimodal data input layer. After preprocessing, this data is input into a dual-branch cross-fusion feature layer. The dual-branch cross-fusion feature layer includes a network traffic branch and a behavior log branch (which can also be a user behavior branch). The network traffic branch uses GAT+ subgraph sampling to extract topological features, while the behavior log branch uses BiLSTM to model temporal features. These features are then fused using a dynamic gating mechanism to generate a fused feature vector. In the reinforcement learning model layer based on human feedback, the system adopts a three-stage training paradigm of "pre-training-fine-tuning-reinforcement learning," continuously optimizing the decision-making strategy through RLHF. Finally, the intelligent decision output layer implements real-time three-classification responses (normal / low-risk / high-risk) and automated rate limiting / blocking operations.

[0185] In contrast, existing technologies have the following shortcomings that need to be overcome:

[0186] Disadvantage 1: Insufficient multimodal data fusion capability.

[0187] Typically, single-type data processing, such as analyzing only network traffic, is not considered, and multi-source fusion is not addressed. This single-modal analysis fails to capture cross-data source correlation characteristics, leading to detection blind spots. When faced with complex attacks requiring comprehensive multi-dimensional data such as network traffic, behavior logs, and system audits, the system struggles to fully assess the threat landscape, resulting in increased false negatives and high false negatives, severely impacting security effectiveness.

[0188] Disadvantage 2: Limited dynamic adaptability.

[0189] Typically, these are static models trained offline and then deployed in a fixed location. When faced with zero-day attacks or new threats such as APTs, the system cannot autonomously adjust its detection strategies and must rely on manual retraining of the model. This lag makes it difficult for the system to adapt to the evolution of attack methods in a timely manner, putting it at a disadvantage in offensive and defensive confrontations and increasing the window of opportunity for network security protection.

[0190] Disadvantage 3: The decision-making mechanism is too rigid.

[0191] Response strategies are typically generated using a pre-defined static rule base. In complex real-world network environments, this rigid decision-making lacks the flexibility to adapt to specific scenarios, leading to numerous false alarms or over-response. Particularly when handling boundary cases, the system cannot make context-aware, flexible judgments like human experts, reducing security operation efficiency.

[0192] Disadvantage 4: Insufficient utilization of expert knowledge.

[0193] The system failed to effectively integrate the experience and knowledge of cybersecurity experts into its decision-making cycle. Relying solely on algorithms for automatic analysis and judgment, it was prone to making incorrect decisions when encountering anomalies not covered by training data. The lack of a mechanism for continuously incorporating expert feedback prevented the system from accumulating and utilizing domain knowledge during operation, thus limiting the continuous evolution of its detection capabilities.

[0194] The technical solutions described in this application have the following outstanding advantages:

[0195] Advantages of Dynamic Weighted Fusion of Multimodal Data: This application's embodiments, by fusing multimodal features, can more comprehensively reflect the network security status, reducing false negatives and false negatives. Compared to existing single-data-source solutions, this adaptive fusion mechanism significantly improves the accuracy of network security status representation. The innovative dual-branch cross-fusion architecture combines GAT network topology analysis with BiLSTM behavioral sequence modeling, achieving dynamic feature fusion through a gating mechanism, fully integrating various multimodal data related to network security. This feature extraction model and fusion strategy, designed specifically for the characteristics of different data, offers better representational features compared to existing solutions.

[0196] Advantages of network traffic branching based on GAT and subgraph sampling: In network security detection scenarios, network traffic is essentially highly structured heterogeneous data, containing multiple types of nodes such as IP, port, device, and traffic volume, as well as edge attributes such as connection relationships and topological degree. However, traditional time series models such as LSTM or Large Language Models (LLM) face two major bottlenecks in complex network attack detection due to insufficient structural adaptability: poor structural adaptability and low computational efficiency.

[0197] The integration of online learning capabilities and expert feedback mechanisms: The introduction of the RLHF mechanism constructs a closed-loop system of "monitoring-decision-feedback-optimization," incorporating self-learning and reinforcement learning mechanisms. This allows the model to automatically adapt to new types of attacks, improving the flexibility and accuracy of decision-making. Security experts directly participate in model optimization through correcting labels and providing threat intelligence feedback, enabling the system to continuously learn. This innovative approach to introducing the RLHF mechanism into the cybersecurity field constructs a dynamic defense system driven by expert knowledge. By transforming real-time corrective feedback from security experts into optimization signals for the PPO algorithm, security decision boundaries are optimized under expert supervision. Compared to traditional static models, this significantly improves defense adaptability in complex threat environments.

[0198] In summary, the embodiments of this application implement an "end-to-end dynamic defense closed-loop" system architecture. Through an innovative dual-branch cross-fusion architecture and a Human Feedback Reinforcement Learning (RLHF) mechanism, dynamic threat detection and intelligent response are achieved. Furthermore, a Graph Attention Network (GAT) is used to process network topology features, combined with bidirectional LSTM modeling of user behavior temporal features, and a gating mechanism enables adaptive fusion of multimodal data. This significantly improves the accuracy of threat detection and response efficiency in complex network environments, and solves the problems of single data sets and rigid models in traditional technologies.

[0199] It should be noted that the system architecture and model architecture provided in this application can be adjusted according to actual conditions. For example, the network traffic branch based on GAT and subgraph sampling can be replaced by the GraphSAGE model with lower computational complexity. The pre-trained model based on a large amount of log information can also be replaced by other general large language models instead of the lightweight DistilBERT model.

[0200] Based on the methods described in the foregoing embodiments, this application also provides a network security status characterization device and a network security intelligent decision-making device. Figure 8 A schematic diagram of the structure of a network security status characterization device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a network security intelligent decision-making device provided in an embodiment of this application.

[0201] like Figure 8 The illustrated device for characterizing network security status includes:

[0202] The first branch feature extraction module 81 is used to construct a topology graph based on network traffic data and environmental context data in the target network security scenario, extract key subgraphs from the topology graph, extract subgraph features from the key subgraphs, and then splice and aggregate them to obtain network traffic branch feature vectors.

[0203] The second branch feature extraction module 82 is used to obtain user behavior branch feature vectors from user behavior data in the target network security scenario through BiLSTM.

[0204] The fusion module 83 is used to fuse network traffic branch feature vectors and user behavior branch feature vectors to obtain a fused feature vector of the target network security scenario; the fused feature vector is used to characterize the network security status of the target network security scenario.

[0205] In the optional implementation, fusion module 83 is specifically used for:

[0206] The network traffic branch feature vector and the user behavior branch feature vector are normalized respectively to obtain the normalized network traffic branch feature vector and the normalized user behavior branch feature vector.

[0207] The normalized network traffic branch feature vector is processed using a first gating network unit to obtain a first processing result vector; the normalized user behavior branch feature vector is processed using a second gating network unit to obtain a second processing result vector; the first gating network unit includes a first weight matrix and a first bias parameter corresponding to the network traffic branch feature vector; the second gating network unit includes a second weight matrix and a second bias parameter corresponding to the user behavior branch feature vector.

[0208] An activation function is used to generate a first gating weight coefficient vector based on the first processing result vector; an activation function is used to generate a second gating weight coefficient vector based on the second processing result vector.

[0209] The first gating weight coefficient vector is multiplied element-wise with the normalized network traffic branch feature vector to obtain the first product vector; the second gating weight coefficient vector is multiplied element-wise with the normalized user behavior branch feature vector to obtain the second product vector.

[0210] The first product vector is added to the second product vector to obtain the fused feature vector of the target network security scenario.

[0211] In the optional implementation, the first branch feature extraction module 81 is specifically used for:

[0212] A network traffic graph is constructed based on the network traffic data, with IP addresses and service ports as nodes and traffic sessions as edges; an environment context graph is constructed based on the environment context data, with devices as nodes and network topology and load balancing as edges.

[0213] Based on the mapping relationship between IP and devices, an association is established between the network traffic graph and the environment context graph, and a unified topology graph is formed based on the association and the network traffic graph and the environment context graph.

[0214] In the optional implementation, the first branch feature extraction module 81 is specifically used for:

[0215] Based on high-frequency traffic rules, a first type of key sub-graph is extracted from the topology graph where the session frequency of the source IP is higher than a preset frequency or the port traffic is higher than a preset traffic.

[0216] Based on the high-load device rules, a second type of key subgraph is extracted from the topology graph, where the CPU utilization rate continuously exceeds a preset utilization rate for a preset duration or the memory usage exceeds a safety baseline.

[0217] In the optional implementation, the first branch feature extraction module 81 is specifically used for:

[0218] A multi-layer graph neural network is used to extract subgraph features layer by layer from each of the key subgraphs;

[0219] The extracted subgraph features are concatenated and aggregated using global average pooling to form an aggregated graph feature vector.

[0220] The feature dimension of the aggregated graph feature vector is compressed by a fully connected layer, and then processed by a nonlinear activation function to finally obtain the network traffic branch feature vector.

[0221] In an optional implementation, the user behavior data includes user behavior logs, which in turn include user time-series features; these features include the user's login time, operation frequency, and permission changes; the second branch feature extraction module 82 is specifically used for:

[0222] The user's temporal features are processed in chronological order by a forward LSTM and in reverse chronological order by a backward LSTM to obtain a first hidden state sequence generated by the forward LSTM and a second hidden state sequence generated by the backward LSTM.

[0223] The first hidden state sequence and the second hidden state sequence are concatenated at each time step to form a bidirectional hidden state sequence. Then, a global average pooling operation is performed on the bidirectional hidden state sequence to generate the user behavior branch feature vector.

[0224] like Figure 9 The illustrated network security intelligent decision-making device includes:

[0225] The input data acquisition module 91 is used to acquire network traffic data, environmental context data, user behavior data and system logs in the target network security scenario as input to the network security intelligent decision model; the network security intelligent decision model includes a dual-branch cross-fusion network and a model based on human feedback reinforcement learning (RLHF);

[0226] The dual-branch cross-fusion module 92 is used to obtain a fused feature vector of the target network security scenario through a network security status representation method based on the dual-branch cross-fusion network; the fused feature vector is used to represent the network security status of the target network security scenario; the network security status representation method is the method described in any implementation of the first aspect;

[0227] Log context feature extraction module 93 is used to extract context vectors from system logs through a feature extractor in an RLHF-based model;

[0228] The concatenation module 94 is used to concatenate the fused feature vector and the context vector through the feature concatenation layer of the RLHF-based model to obtain the enhanced feature vector;

[0229] The intelligent decision module 95 is used to take the enhanced feature vector as input to the reinforcement learning decision module in the RLHF-based model, and use the reinforcement learning decision module to analyze the enhanced feature vector and output intelligent decision results for the target network security scenario.

[0230] In an optional implementation, the network security intelligent decision-making model is formed by training an initial decision-making model based on multimodal historical data; the multimodal historical data includes network traffic historical data, environmental context historical data, user behavior historical data, system historical logs, and manual feedback data in the same network security scenario;

[0231] The initial decision model includes: an initial two-branch cross-fusion network and an initial reinforcement learning model; the initial reinforcement learning model includes: a pre-trained model, a splicing layer, and a supervised fine-tuning model; the network security intelligent decision-making device further includes a training module 96, used to train the initial decision model to obtain the network security intelligent decision-making model. The training module 96 is specifically used for:

[0232] The initial dual-branch cross-fusion model obtains the historical fusion feature vector based on the historical network traffic data, the historical environmental context data, and the historical user behavior data;

[0233] The historical context vector is extracted by the pre-trained model based on the system's historical logs;

[0234] The concatenation layer in the initial reinforcement learning model concatenates the historical fusion feature vector and the historical context vector into a historical augmented feature vector.

[0235] The supervised fine-tuning model predicts network security risk categories based on the historical enhanced feature vectors, and adjusts the parameters of the supervised fine-tuning model based on the predicted network security risk categories and the actual category labels;

[0236] By employing the RLHF reinforcement learning method and combining it with the aforementioned human feedback data, further reinforcement learning is carried out on the supervised fine-tuning model after parameter adjustment to obtain the aforementioned intelligent decision-making model for cybersecurity; the human feedback data includes one or more of the following: feedback data from cybersecurity experts, feedback data from cybersecurity teams, and external threat intelligence.

[0237] In an optional implementation, the initial dual-branch cross-fusion model includes: a network traffic branch network, a user behavior branch network, and a dynamic fusion network;

[0238] Training module 96 is specifically used for:

[0239] A historical topology map is constructed from the network traffic branch network based on the historical network traffic data and the historical environmental context data. Historical key subgraphs are extracted from the historical topology map, and features of historical subgraphs are extracted from the historical key subgraphs and then spliced ​​and aggregated to obtain a historical network traffic branch feature vector.

[0240] The user behavior branch network obtains historical user behavior branch feature vectors from the historical user behavior data through BiLSTM;

[0241] The historical fusion feature vector is obtained by fusing the historical network traffic branch feature vector and the historical user behavior branch feature vector by the dynamic fusion network.

[0242] In an optional implementation, the pre-trained model includes a 6-layer Transformer encoder; the training module 96 is specifically used for:

[0243] The system's historical logs are cleaned and standardized.

[0244] The pre-trained model is trained using a self-supervised task based on masked language modeling (MLM). The semantic encoding capability of the pre-trained model is trained through a dynamic masking mechanism, and the training parameters of the pre-trained model are optimized. During the training process of the pre-trained model, the semantic understanding capability of the pre-trained model is preserved by minimizing the MLM loss.

[0245] The pre-trained model generates a historical context vector containing log information, which is used as a feature extractor in the network security intelligent decision-making model.

[0246] In an optional implementation, the structure of the supervised fine-tuning model includes a 4-layer Transformer encoder and a 2-layer Multilayer Perceptron (MLP) classifier; the training module 96 is specifically used to: learn cross-modal feature interactions based on the historical enhanced feature vectors through the 4-layer Transformer encoder.

[0247] The probability distribution of three network security risk categories is predicted using a two-layer MLP; the three network security risk categories include: normal state, low-risk warning, and high-risk blocking.

[0248] If the probability distribution of the three cybersecurity risk categories predicted by the supervised fine-tuning model does not match the actual category labels, the parameters of the 4-layer Transformer encoder and the 2-layer MLP are jointly updated and adjusted during the supervised fine-tuning phase until the early stopping condition is met.

[0249] In an optional implementation, the training of the supervised fine-tuning phase of the supervised fine-tuning model is performed offline; the training module 96 is specifically used for:

[0250] After the supervised fine-tuning model with adjusted parameters is deployed offline, the RLHF reinforcement learning method is used in conjunction with the human feedback data to start online learning of the supervised fine-tuning model with adjusted parameters.

[0251] During online learning, the parameters of the 4-layer Transformer encoder of the supervised fine-tuning model with adjusted parameters are frozen, and only the parameters of the 2-layer MLP are adjustable; the policy network predicts the network security risk category based on the historical enhanced feature vectors and outputs the decision result; based on the human feedback data, a reward value is calculated using a reward function; a value network is used to generate a state value assessment result based on the decision result output by the policy network and the reward value; the policy network is adjusted in conjunction with the state value assessment result and the reward value; the human feedback data includes correction labels for the human feedback decision results;

[0252] The optimization objective of the policy network is to maximize the reward obtained by the decision sequence, and the optimization objective of the value network is to minimize the mean square error between TD-Target and the current state value estimate, wherein TD-Target is the sum of the immediate reward and the discounted next state value estimate;

[0253] The policy network and the value network share the 4-layer Transformer encoder. The policy network has a 2-layer MLP policy head, and the value network has a 1-layer MLP value head.

[0254] In addition, based on the aforementioned method, this application also provides a computer device, which includes a processor and a memory that are communicatively connected to each other;

[0255] The memory stores computer programs;

[0256] A processor is used to run computer programs to implement the network security status characterization method as described in the foregoing embodiments, or to implement the network security intelligent decision-making method as described in the foregoing embodiments.

[0257] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and equipment embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0258] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A network security intelligent decision-making method, characterized in that, include: Network traffic data, environmental context data, user behavior data, and system logs from the target network security scenario are acquired and used as input to the network security intelligent decision-making model; the network security intelligent decision-making model includes a dual-branch cross-fusion network and a model based on human feedback reinforcement learning (RLHF). The fused feature vector of the target network security scenario is obtained through the network security status representation method based on the dual-branch cross-fusion network; The fused feature vector is used to characterize the network security status of the target network security scenario; The context vector is extracted from the system log using a feature extractor in the RLHF-based model; The fused feature vector and the context vector are concatenated through the feature concatenation layer of the RLHF-based model to obtain the enhanced feature vector; The enhanced feature vector is used as the input to the reinforcement learning decision module in the RLHF-based model. The reinforcement learning decision module analyzes the enhanced feature vector and outputs an intelligent decision result for the target network security scenario. The method for representing the network security status includes: A topology graph is constructed based on network traffic data and environmental context data in the target network security scenario. Key subgraphs are extracted from the topology graph, and subgraph features are extracted from the key subgraphs and then spliced ​​and aggregated to obtain network traffic branch feature vectors. User behavior branch feature vectors are obtained from user behavior data in the target network security scenario using BiLSTM; By fusing the network traffic branch feature vector and the user behavior branch feature vector, a fused feature vector of the target network security scenario is obtained; the fused feature vector is used to characterize the network security status of the target network security scenario.

2. The method according to claim 1, characterized in that, The network security intelligent decision-making model is formed by training an initial decision-making model based on multimodal historical data; the multimodal historical data includes network traffic history data, environmental context history data, user behavior history data, system historical logs, and manual feedback data in the same network security scenario; The initial decision model includes: an initial two-branch cross-fusion network and an initial reinforcement learning model; the initial reinforcement learning model includes: a pre-trained model, a splicing layer, and a supervised fine-tuning model. The process of training the initial decision model to obtain the cybersecurity intelligent decision model includes: The initial dual-branch cross-fusion model obtains a historical fusion feature vector based on the historical network traffic data, the historical environmental context data, and the historical user behavior data; The historical context vector is extracted by the pre-trained model based on the system's historical logs; The concatenation layer in the initial reinforcement learning model concatenates the historical fusion feature vector and the historical context vector into a historical augmented feature vector. The supervised fine-tuning model predicts network security risk categories based on the historical enhanced feature vectors, and adjusts the parameters of the supervised fine-tuning model based on the predicted network security risk categories and the actual category labels; By employing the RLHF reinforcement learning method and combining it with the aforementioned human feedback data, further reinforcement learning is carried out on the supervised fine-tuning model after parameter adjustment to obtain the aforementioned intelligent decision-making model for cybersecurity; the human feedback data includes one or more of the following: feedback data from cybersecurity experts, feedback data from cybersecurity teams, and external threat intelligence.

3. The method according to claim 2, characterized in that, The initial dual-branch cross-fusion model includes: a network traffic branch network, a user behavior branch network, and a dynamic fusion network; The process of obtaining the historical fusion feature vector by the initial dual-branch cross-fusion model based on the historical network traffic data, the historical environmental context data, and the historical user behavior data includes: A historical topology map is constructed from the network traffic branch network based on the historical network traffic data and the historical environmental context data. Historical key subgraphs are extracted from the historical topology map, and features of historical subgraphs are extracted from the historical key subgraphs and then spliced ​​and aggregated to obtain a historical network traffic branch feature vector. The user behavior branch network obtains historical user behavior branch feature vectors from the historical user behavior data through BiLSTM; The historical fusion feature vector is obtained by fusing the historical network traffic branch feature vector and the historical user behavior branch feature vector by the dynamic fusion network.

4. The method according to claim 2, characterized in that, The pre-trained model structure includes a 6-layer Transformer encoder; the historical context vector extracted by the pre-trained model based on the system's historical logs includes: The system's historical logs are cleaned and standardized. The pre-trained model is trained using a self-supervised task based on masked language modeling (MLM). The semantic encoding capability of the pre-trained model is trained through a dynamic masking mechanism, and the training parameters of the pre-trained model are optimized. During the training process of the pre-trained model, the semantic understanding capability of the pre-trained model is preserved by minimizing the MLM loss. The pre-trained model generates a historical context vector containing log information, which is used as a feature extractor in the network security intelligent decision-making model.

5. The method according to claim 2, characterized in that, The supervised fine-tuning model comprises a 4-layer Transformer encoder and a 2-layer Multilayer Perceptron (MLP) classifier. The process involves the supervised fine-tuning model predicting cybersecurity risk categories based on the historical enhanced feature vectors, and adjusting the parameters of the supervised fine-tuning model based on the predicted cybersecurity risk categories and the actual category labels. This includes: The 4-layer Transformer encoder learns cross-modal feature interactions based on the historical enhanced feature vectors. The probability distribution of three network security risk categories is predicted using a two-layer MLP; the three network security risk categories include: normal state, low-risk warning, and high-risk blocking. If the probability distribution of the three cybersecurity risk categories predicted by the supervised fine-tuning model does not match the actual category labels, the parameters of the 4-layer Transformer encoder and the 2-layer MLP are jointly updated and adjusted during the supervised fine-tuning phase until the early stopping condition is met.

6. The method according to claim 5, characterized in that, The training of the supervised fine-tuning phase of the supervised fine-tuning model is performed offline; the method of using RLHF reinforcement learning and combining it with the human feedback data to further strengthen learning on the basis of the parameter-adjusted supervised fine-tuning model, to obtain the network security intelligent decision-making model, includes: After the supervised fine-tuning model with adjusted parameters is deployed offline, the RLHF reinforcement learning method is used in conjunction with the human feedback data to start online learning of the supervised fine-tuning model with adjusted parameters. During online learning, the parameters of the 4-layer Transformer encoder of the supervised fine-tuning model with adjusted parameters are frozen, and only the parameters of the 2-layer MLP are adjustable; the policy network predicts the network security risk category based on the historical enhanced feature vectors and outputs the decision result; based on the human feedback data, a reward value is calculated using a reward function; a value network is used to generate a state value assessment result based on the decision result output by the policy network and the reward value; the policy network is adjusted in conjunction with the state value assessment result and the reward value; the human feedback data includes correction labels for the human feedback decision results; The optimization objective of the policy network is to maximize the reward obtained by the decision sequence, and the optimization objective of the value network is to minimize the mean square error between TD-Target and the current state value estimate, wherein TD-Target is the sum of the immediate reward and the discounted next state value estimate; The policy network and the value network share the 4-layer Transformer encoder. The policy network has a 2-layer MLP policy head, and the value network has a 1-layer MLP value head.

7. The method according to claim 1, characterized in that, The process of fusing the network traffic branch feature vector and the user behavior branch feature vector to obtain the fused feature vector of the target network security scenario includes: The network traffic branch feature vector and the user behavior branch feature vector are normalized respectively to obtain the normalized network traffic branch feature vector and the normalized user behavior branch feature vector. The normalized network traffic branch feature vector is processed using a first gating network unit to obtain a first processing result vector; the normalized user behavior branch feature vector is processed using a second gating network unit to obtain a second processing result vector; the first gating network unit includes a first weight matrix and a first bias parameter corresponding to the network traffic branch feature vector; the second gating network unit includes a second weight matrix and a second bias parameter corresponding to the user behavior branch feature vector. An activation function is used to generate a first gating weight coefficient vector based on the first processing result vector; an activation function is used to generate a second gating weight coefficient vector based on the second processing result vector. The first gating weight coefficient vector is multiplied element-wise with the normalized network traffic branch feature vector to obtain the first product vector; the second gating weight coefficient vector is multiplied element-wise with the normalized user behavior branch feature vector to obtain the second product vector. The first product vector is added to the second product vector to obtain the fused feature vector of the target network security scenario.

8. The method according to claim 1, characterized in that, The construction of the topology map based on network traffic data and environmental context data in the target network security scenario includes: A network traffic graph is constructed based on the network traffic data, with IP addresses and service ports as nodes and traffic sessions as edges; an environment context graph is constructed based on the environment context data, with devices as nodes and network topology and load balancing as edges. Based on the mapping relationship between IP and devices, an association is established between the network traffic graph and the environment context graph, and a unified topology graph is formed based on the association and the network traffic graph and the environment context graph.

9. The method according to claim 1 or 8, characterized in that, The extraction of key subgraphs from the topology graph includes: Based on high-frequency traffic rules, a first type of key sub-graph is extracted from the topology graph where the session frequency of the source IP is higher than a preset frequency or the port traffic is higher than a preset traffic. Based on the high-load device rules, a second type of key subgraph is extracted from the topology graph, where the CPU utilization rate continuously exceeds a preset utilization rate for a preset duration or the memory usage exceeds a safety baseline.

10. The method according to claim 1 or 8, characterized in that, The step of extracting subgraph features from the key subgraph and then concatenating and aggregating them to obtain a network traffic branch feature vector includes: A multi-layer graph neural network is used to extract subgraph features layer by layer from each of the key subgraphs; The extracted subgraph features are concatenated and aggregated using global average pooling to form an aggregated graph feature vector. The feature dimension of the aggregated graph feature vector is compressed by a fully connected layer, and then processed by a nonlinear activation function to finally obtain the network traffic branch feature vector.

11. The method according to claim 1, characterized in that, The user behavior data includes user behavior logs, which include user time-series features; the user time-series features include user login time, operation frequency, and permission changes; obtaining user behavior branch feature vectors from the user behavior data in the target network security scenario using BiLSTM includes: The user's temporal features are processed in chronological order by a forward LSTM and in reverse chronological order by a backward LSTM to obtain a first hidden state sequence generated by the forward LSTM and a second hidden state sequence generated by the backward LSTM. The first hidden state sequence and the second hidden state sequence are concatenated at each time step to form a bidirectional hidden state sequence. Then, a global average pooling operation is performed on the bidirectional hidden state sequence to generate the user behavior branch feature vector.

12. A network security intelligent decision-making device, characterized in that, include: The input data acquisition module is used to acquire network traffic data, environmental context data, user behavior data, and system logs in the target network security scenario as input to the network security intelligent decision-making model; the network security intelligent decision-making model includes a dual-branch cross-fusion network and a model based on human feedback reinforcement learning (RLHF). The dual-branch cross-fusion module is used to obtain the fused feature vector of the target network security scenario through the network security status representation method of the dual-branch cross-fusion network; The fused feature vector is used to characterize the network security status of the target network security scenario; The log context feature extraction module is used to extract the context vector from the system log through the feature extractor in the RLHF-based model; The concatenation module is used to concatenate the fused feature vector and the context vector through the feature concatenation layer of the RLHF-based model to obtain an enhanced feature vector; The intelligent decision-making module is used to take the enhanced feature vector as input to the reinforcement learning decision-making module in the RLHF-based model, and use the reinforcement learning decision-making module to analyze the enhanced feature vector and output intelligent decision-making results for the target network security scenario. The method for representing the network security status includes: A topology graph is constructed based on network traffic data and environmental context data in the target network security scenario. Key subgraphs are extracted from the topology graph, and subgraph features are extracted from the key subgraphs and then spliced ​​and aggregated to obtain network traffic branch feature vectors. User behavior branch feature vectors are obtained from user behavior data in the target network security scenario using BiLSTM; By fusing the network traffic branch feature vector and the user behavior branch feature vector, a fused feature vector of the target network security scenario is obtained; the fused feature vector is used to characterize the network security status of the target network security scenario.

13. A computer device, characterized in that, include: Interconnected processors and memory; The memory stores computer programs; The processor is configured to run the computer program to implement the network security intelligent decision-making method as described in any one of claims 1-11.

Images