Honeynet deployment method and system based on graph convolutional neural network and reinforcement learning network

Abstract

This invention discloses a honeypot deployment method and system based on graph convolutional neural networks (GCNNs) and reinforcement learning networks. The method constructs a cascaded GCNN and actor-critic network. The GCNN includes a feature extraction module and an adjacency feature module, while the actor-critic network includes parallel actor and critic networks. The network model is trained using a dataset. The network topology is input into the GCNN to obtain node feature matrices and standardized adjacency matrices, which are then processed by graph convolution and input into the actor-critic network to obtain probability distributions and estimates. The network topology to be deployed is input into the trained model. Nodes with probabilities greater than a preset threshold are deployed as high-interaction honeypots, while the remaining nodes are deployed as low-interaction honeypots. This invention extracts network topology features through GCNNs and combines them with reinforcement learning to achieve adaptive deployment of honeypot types, improving the flexibility and adaptability of honeypot deployment.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to a honeynet deployment method and system based on graph convolutional neural networks and reinforcement learning networks. Background Technology

[0002] With the rapid development of the internet, cyberattacks have become increasingly complex and diverse. Traditional security measures, such as firewalls and intrusion detection systems, while providing some protection, are proving insufficient in the face of increasingly sophisticated attack methods. Firewalls primarily protect network boundaries by filtering and blocking illegitimate access requests, while intrusion detection systems detect potential attacks by monitoring network traffic and system activity. However, these passive defenses often fall short when facing advanced persistent threats, zero-day attacks, and other sophisticated attack techniques.

[0003] To improve the effectiveness of honeypots in luring attackers, for example, patent CN117938473A, entitled "A Deception Defense Method Based on Reinforcement Learning for High Attack-Defense Interaction in Multi-Honeypot Scenarios," describes a method that utilizes a traffic discrimination module to collect external traffic data in real-time and filter malicious traffic. It then uses an attack prediction module to extract and classify the attack characteristics of malicious traffic, forming the observation state of the honeypot defense agent and predicting defense response strategies in an improved PPO-P model. Based on the defense response strategy, it selects the honeypots to react and the corresponding response actions required by each honeypot. A reward function is used to reward and punish the honeypots, continuously feeding the reward values ​​back into the improved PPO-P model. After iterative prediction, the optimal defense response strategy is obtained. While this invention has the advantage of efficiently identifying and responding to malicious attacks, its drawback is that it cannot change the network topology and interaction type of the honeypots based on the attacker's attack information. Once identified and marked by the attacker, it loses its ability to lure attackers, lacking flexibility and adaptability. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a honeypot deployment method and system based on graph convolutional neural networks and reinforcement learning networks, thereby improving the flexibility and adaptability of honeypot deployment.

[0005] This invention is achieved through the following technical solution: Firstly, this application provides a honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks, comprising the following steps: Step 1: Construct a network topology dataset, which includes multiple network topologies, each containing multiple network nodes; Step 2: Construct a cascaded graph convolutional neural network and an actor-critic network; The graph convolutional neural network includes a feature extraction module and an adjacency feature module. The feature extraction module is used to extract multidimensional features of network nodes, and the adjacency feature module is used to extract the adjacency matrix of the network topology and perform standardization processing. The actor-critic network includes a parallel actor network and a critic network. The actor network is used to output the action probability distribution, and the critic network is used to output the state estimate. Step 3: Train the constructed network model using the dataset. Input the network topology into the graph convolutional neural network to obtain the node feature matrix and the standardized adjacency matrix. After graph convolution operation, input the matrix into the actor-critic network to obtain the probability distribution and estimate. Update the network parameters through reinforcement learning algorithm to obtain the trained network model. Step 4: Input the network topology to be deployed into the trained network model. Determine the honeypot interaction type of each network node based on the probability distribution output by the actor network. Deploy nodes with a probability greater than a preset threshold as high-interaction honeypots and deploy the remaining nodes as low-interaction honeypots.

[0006] Preferably, the construction of the network topology dataset in step 1 includes at least one of the following methods: Randomly initialize and generate multiple network topologies; Collect topology data in real network environments; A graph generation model is used to generate network topologies with specific statistical properties. Evolutionary algorithms are used to generate diverse network topologies through iterative optimization.

[0007] Preferably, the feature extraction module in step 2 includes an IP feature vector module, a degree feature vector module, a centrality feature vector module, and a topology feature vector module arranged in parallel, as well as a feature fusion module connected to the output of the four vector modules; The IP feature vector module is used to convert the IP address of a network node into a binary representation to extract the IP feature vector; The degree feature vector module is used to calculate the in-degree and out-degree of network nodes to extract degree feature vectors; The centrality feature vector module is used to calculate the degree centrality, proximity centrality, betweenness centrality, and eigenvector centrality of network nodes to extract centrality feature vectors; The topology feature vector module is used to calculate the local clustering coefficients and neighbor degree distributions of network nodes to extract topology feature vectors.

[0008] Preferably, the graph convolutional neural network in step 2 further includes multiple convolutional layers sequentially cascaded with the outputs of the feature extraction module and the adjacent feature module; The multi-layer convolutional layer is used to perform graph convolution operations on the node feature matrix and the normalized adjacency matrix, and after aggregating the neighbor node information, outputs an output feature matrix containing topological features.

[0009] Preferably, the node feature matrix is ​​calculated as follows: The IP address of each network node is converted into a binary representation to obtain the IP feature vector; Calculate the degree feature of each node according to the network type: calculate the node degree for undirected networks, and calculate the in-degree and out-degree for directed networks respectively. Combine the degree values ​​to form a degree feature vector. Calculate the centrality features of each node, including degree centrality, proximity centrality, betweenness centrality, and eigenvector centrality, and combine the four centrality measures to form a central eigenvector; Calculate the local clustering coefficient and neighbor degree distribution of each node, and combine the calculation results to form a topological feature vector; The IP feature vector, degree feature vector, centrality feature vector, and topology feature vector are fused to obtain a comprehensive feature representation of each node, and then a node feature matrix is ​​constructed.

[0010] Preferably, the degree centrality Proximity centrality Betweenness centrality eigenvector centrality Local clustering coefficient And neighbor degree distribution The calculation formula is:

[0011]

[0012]

[0013]

[0014]

[0015]

[0016] in, It is the adjacency characteristic matrix eigenvalues, It is a node The degree, It is a node and nodes The shortest path distance between them It is a node and nodes The number of shortest paths between them Through nodes The number of shortest paths, It is a node The actual number of edges between neighbors. It is a node The set of neighboring nodes.

[0017] Preferably, the output feature matrix obtained after graph convolution in step 4 is calculated in the following way: The node feature matrix and the normalized adjacency matrix are input into a multi-layer graph convolutional network. The neighbor node information is aggregated through layer-by-layer convolution operations to obtain the final output feature matrix. The output feature matrix integrates the network node’s own features and the topological structure information of multi-hop neighbors.

[0018] Preferably, the graph convolution operation is at least one of the following: Basic graph convolution, Chebyshev graph convolution, spectral graph convolution, attention graph convolution, hybrid graph convolution, or gated graph convolution.

[0019] Secondly, this application provides a honeynet deployment system based on graph convolutional neural networks and reinforcement learning networks, including: The dataset construction module is used to construct a network topology dataset, which includes multiple network topologies, each containing multiple network nodes. The network model building module is used to build cascaded graph convolutional neural networks and actor-critic networks; The graph convolutional neural network includes a feature extraction module and an adjacency feature module. The feature extraction module is used to extract multidimensional features of network nodes, and the adjacency feature module is used to extract the adjacency matrix of the network topology and perform standardization processing. The actor-critic network includes a parallel actor network and a critic network. The actor network is used to output the action probability distribution, and the critic network is used to output the state estimate. The model training module is used to train the constructed network model using the dataset. The network topology is input into the graph convolutional neural network to obtain the node feature matrix and the standardized adjacency matrix. After graph convolution operation, it is input into the actor-critic network to obtain the probability distribution and the estimated value. The network parameters are updated through the reinforcement learning algorithm to obtain the trained network model. The honeypot deployment module is used to input the network topology to be deployed into the trained network model, determine the honeypot interaction type of each network node based on the probability distribution of the actor network output, deploy nodes with a probability greater than a preset threshold as high-interaction honeypots, and deploy the remaining nodes as low-interaction honeypots.

[0020] Thirdly, this application provides a computer device, including: a processor and a computer-readable storage medium; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks as described above.

[0021] Compared with the prior art, the present invention has the following beneficial technical effects: This application proposes a honeynet deployment method based on graph convolutional neural networks (GCNNs) and reinforcement learning networks. First, a dataset containing multiple network topologies is constructed as the training foundation. Then, a cascaded GCNN and actor-critic network are built. The GCNN extracts multi-dimensional features of network nodes and the adjacency matrix of the network topology through a feature extraction module and an adjacency feature module, respectively, and performs standardization processing. The actor-critic network outputs action probability distributions and state estimates through parallel actor and critic networks, respectively. Next, the network model is trained using the dataset. The network topology is input into the GCNN to obtain node feature matrices and standardized adjacency matrices. After graph convolution operations, these are input into the actor-critic network to obtain probability distributions and estimates. The network parameters are updated using a reinforcement learning algorithm to obtain the trained model. Finally, the network topology to be deployed is input into the trained model. Based on the probability distribution output by the actor network, the honeypot interaction type of each node is determined. Nodes with probabilities greater than a threshold are deployed as high-interaction honeypots, and the rest are deployed as low-interaction honeypots. This scheme fully exploits the structural features of network topology and the attribute features of nodes through graph convolutional neural networks, solving the problem that traditional methods cannot dynamically adjust honeypot types according to the topology. It introduces an actor-critic reinforcement learning framework to achieve adaptive optimization of deployment strategies, improving the flexibility and targeting of honeypot deployment. Node classification is achieved by comparing probability distribution with thresholds, effectively controlling resource consumption while ensuring the trapping effect. The trained model can be applied to new network topology environments and has good generalization ability.

[0022] This application also proposes a honeynet deployment system based on graph convolutional neural networks and reinforcement learning networks, an electronic device, and a computer storage medium, which possess all the advantages of the aforementioned honeynet deployment methods based on graph convolutional neural networks and reinforcement learning networks. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to the present invention; Figure 2 This is the network model of the present invention. A structural diagram. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0027] A honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks includes the following steps: Step 1: Randomly initialize multiple network topologies. Each network topology contains multiple network nodes, and each network node is assigned multiple ports. A portion of the network topologies are used to form the training set, and the rest are used to form the test set. Step 2: Construct a network model, including a graph convolutional neural network and an actor-critic network connected in sequence; wherein, the graph convolutional neural network is used to extract node features and topological information, and the actor-critic network is used for policy optimization and value evaluation. This network model employs a cascaded architecture of a graph convolutional neural network (GCNN) and an actor-critic network. The GCNN extracts node IP features, degree features, centrality features, and topological features in parallel through a feature extraction module. Simultaneously, it extracts the adjacency matrix of the network topology through an adjacency feature module and performs standardization, deeply fusing node attribute information with topological structure information. The actor-critic network operates in parallel with an actor network and a critic network. The actor network outputs action probability distributions through a linear combination layer and a softmax distribution layer, while the critic network outputs state estimates through a linear combination layer, forming a complete policy optimization and value assessment framework. This network model fully leverages the advantages of GCNNs in processing graph-structured data, effectively capturing the topological dependencies and structural features between network nodes. The multi-dimensional design of the feature extraction module ensures the comprehensiveness of node information, characterizing node attributes from multiple perspectives such as IP address, degree centrality, proximity centrality, and betweenness centrality. The parallel architecture of the actor-critic network enables collaborative optimization of the policy network and the value network. The actor network focuses on learning the optimal deployment policy, while the critic network provides accurate state value assessments. These two networks mutually reinforce each other, improving the efficiency and stability of policy learning.

[0028] Step 3: Initialize the number of iterations, set the maximum number of iterations, and initialize the parameters of the graph convolutional neural network, the actor network parameters, and the critic network parameters; Step 4: Perform forward propagation on the network model, input the training set into the network model, extract the node feature matrix and standardized adjacency matrix through the graph convolutional neural network, obtain the output feature matrix after multiple convolutions, and then calculate the probability distribution and estimated value of the training network topology through the actor-critic network. The network topology training set is input into the constructed network model. The feature extraction module of the graph convolutional neural network extracts the node feature matrix, and the adjacency feature module extracts the adjacency matrix and performs standardization. The node feature matrix and the standardized adjacency matrix are input into multiple convolutional layers for graph convolution operation. After aggregating the neighbor node information, the output feature matrix is ​​obtained. The output feature matrix is ​​input into the actor-critic network. The actor network calculates the probability distribution of the training network topology, and the critic network calculates the estimated value of the training network topology. This realizes the end-to-end mapping from the original network topology data to decision probability and value assessment.

[0029] Step 5: Update the parameters of the graph convolutional neural network, actor network, and critic network based on the forward propagation results. Determine whether the maximum number of iterations has been reached. If so, the trained network model is obtained; otherwise, return to step 4 to continue iterating. Reinforcement learning algorithms are used to update the parameters of the graph convolutional neural network, actor network, and critic network, increasing the selection probability of high-value actions and improving the accuracy of value assessment. The algorithm checks if the current iteration count has reached the preset maximum; if so, a trained network model is obtained; otherwise, it returns to step 4 to continue iterative training. Through repeated iterative optimization, the network model gradually learns the optimal honeypot deployment strategy.

[0030] Step 6: Input the test set into the trained network model and perform forward propagation to obtain the probability distribution of the test network topology. Update the nodes with a probability distribution greater than a preset threshold as high-interaction honeypot nodes, and update the remaining nodes as low-interaction honeypot nodes.

[0031] See Figure 1 and 2 A honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks includes the following steps: Step 1: Construct network topology training and test sets: Randomly initialize multiple network topologies, each containing multiple network nodes, and each network node is assigned multiple ports. Select some network topologies to form the training set, and the rest to form the test set. Specifically, random initialization Each network topology contains [number] network topologies. There are 1 network node, and each network node is randomly assigned. Ports, and among them The network topology structures are used to form a network topology training set, and the remaining ones are used to form a network topology training set. A network topology test set is composed of several network topologies; in, , , , ; In some embodiments, topology data from real-world network environments, such as enterprise networks, data center networks, or campus network topologies, are collected as the basis for training and testing sets, making the model more closely aligned with real-world application scenarios.

[0032] In some embodiments, Erd is used The s–Rényi random graph model, the Watts–Strogatz small-world network model, or the Barabási–Albert scale-free network model generate network topologies with specific statistical properties.

[0033] In some embodiments, evolutionary computation methods such as genetic algorithms or particle swarm optimization are used to generate diverse network topologies through iterative optimization.

[0034] The network topology training set is used to train the network model, while the network topology test set is used to evaluate the performance of the trained network model and deploy honeypot nodes. By randomly initializing diverse network topologies, the model can learn the node characteristics and deployment strategies under different network structures, thereby enhancing the model's generalization ability.

[0035] Step 2: Construct the network model This includes sequentially connected graph convolutional neural networks and actor-critic networks; Among them, graph convolutional neural networks are used to extract node features and topological information, while actor-critic networks are used for policy optimization and value evaluation. See Figure 2 The graph convolutional neural network is used to extract node features and topological information. The graph convolutional neural network includes a feature extraction module and an adjacent feature module arranged in parallel, as well as three convolutional layers that are sequentially cascaded with the outputs of the feature extraction module and the adjacent feature module.

[0036] The feature extraction module is used to extract multidimensional features of network nodes, including parallel IP feature vector modules, degree feature vector modules, centrality feature vector modules and topology feature vector modules, as well as a feature fusion module connected to the output of these four vector modules. The adjacency feature module is used to extract the adjacency relationships of the network topology, including a cascaded adjacency matrix extraction module and a normalization processing module.

[0037] Actor-critic networks are used for policy optimization and value evaluation. An actor-critic network consists of parallel actor networks and critic networks. The actor network is used to output the action probability distribution. The actor network consists of sequentially cascaded linear combination layers and probability layers. Activation layer, linear combination layer and Distribution layer; The critic network is used to output state estimates, including the first linear combination layer in a sequential cascade, and the estimate. Activation layer and second linear combination layer.

[0038] In some embodiments, the graph convolutional neural network may employ a graph attention network, a GraphSAGE network, a simplified graph convolutional network, a hyperbolic graph convolutional network, or an adaptive graph convolutional network.

[0039] In some embodiments, the actor-critic network can adopt the following alternative architecture: Advantage actor-critic network: Introduces an advantage function to replace the original estimate, reducing the variance of the policy gradient.

[0040] Asynchronous advantages of actor-critic network: It uses multiple parallel actor-learners to interact with the environment asynchronously, which improves training efficiency and stability.

[0041] Soft actor-critic network: Introduces an entropy regularization term into the objective function to balance exploration and exploitation, thereby improving the randomness and robustness of the policy.

[0042] Distributed proximal policy optimization network: It adopts a distributed architecture to collect experience and updates the policy through a pruned proxy objective function.

[0043] Dual-delay deep deterministic policy gradient network: Employs a dual-critic network to reduce overestimation of the value function and introduces objective policy smoothing regularization.

[0044] The network model constructed in this application fully extracts the structural and attribute features of network nodes through graph convolutional neural networks, and then uses an actor-critic network to achieve policy optimization and value evaluation, forming an end-to-end honeypot deployment decision framework. Graph convolutional neural networks can effectively process graph-structured network topology data and capture the dependencies between nodes; the actor-critic network combines the advantages of policy gradient methods and value function methods, and can achieve stable and efficient learning in continuous action space.

[0045] Step 3: Initialize network model parameters: Initialize the number of iterations, set the maximum number of iterations, and initialize the graph convolutional neural network parameters, actor network parameters, and critic network parameters; Specifically, the initial iteration count is: The maximum number of iterations is , , No. The next iteration of the network model based on graph convolutional neural networks and actor-critic. The parameters of the graph convolutional neural network, actor network, and critic network are respectively: , and and order ; In some embodiments, the maximum number of iterations can be set using the following strategy: Adaptive early stopping mechanism: If the validation set loss does not decrease for N consecutive iterations, training will be terminated early to prevent overfitting.

[0046] Dynamic adjustment based on convergence speed: The maximum number of iterations is dynamically adjusted according to the rate of loss reduction during training.

[0047] Multi-stage training strategy: Set different maximum number of iterations and learning rates in stages, first coarse tuning and then fine tuning.

[0048] Based on computing resources: The maximum number of iterations can be flexibly set according to available computing resources and time budget.

[0049] In some embodiments, the learning rate can be configured with the following scheduling strategy: Step descent strategy: After a certain number of iterations, the learning rate is multiplied by a decay factor.

[0050] Cosine annealing strategy: The learning rate is periodically adjusted according to the cosine function to help escape local optima.

[0051] Adaptive learning rate strategy: Use adaptive optimizers such as Adam and RMSprop to automatically adjust the learning rate.

[0052] Warm-up strategy: Use a small learning rate at the beginning of training and gradually increase it to the preset value to stabilize the training process.

[0053] Step 4: Analyze the network model. Forward propagation: The training set is input into the network model, and the node feature matrix and normalized adjacency matrix are extracted through the graph convolutional neural network. After multiple convolutions, the output feature matrix is ​​obtained. Then, the probability distribution and estimated value of the training network topology are calculated through the actor-critic network. Specifically, the network topology training set is used as the network model. The input is propagated forward to obtain Probability distribution of training network topology and The estimated value of the training network topology : S4.1 Extract the node feature matrix and the standardized adjacency matrix; The network topology training set is used as the network model. The input is used to extract IP feature vectors, degree feature vectors, centrality feature vectors, and topological feature vectors based on the node information of each network topology. These are then fused to obtain a feature matrix of 70 nodes. The method is as follows: Convert the IP address of each node into a binary representation to extract the IP feature vector; Calculate the in-degree and out-degree of each network node, and combine the in-degree and out-degree values ​​to form the degree feature vector of the node; Calculate the degree centrality of each node. Proximity centrality Betweenness centrality and eigenvector centrality To extract the centrality feature vector; Calculate the local clustering coefficient for each node. And neighbor degree distribution To extract topological feature vectors; By fusing the above-mentioned feature vectors, a node feature matrix is ​​obtained. ; Degree centrality Proximity centrality Betweenness centrality eigenvector centrality Local clustering coefficient And neighbor degree distribution The calculation formula is:

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] in, It is the adjacency characteristic matrix eigenvalues, It is a node The degree, It is a node and nodes The shortest path distance between them It is a node and nodes The number of shortest paths between them Through nodes The number of shortest paths, It is a node The actual number of edges between neighbors. It is a node The set of neighboring nodes.

[0060] 2) The adjacency feature module extracts 70 adjacency matrices based on the network topology. 70 standardized adjacency matrices were obtained through normalization. :

[0061]

[0062] in, It is the identity matrix; It is a degree matrix.

[0063] In some embodiments, the standardization of the adjacency matrix can also be achieved through random walk standardization, symmetric standardization, or power-law standardization.

[0064] By extracting multidimensional features and standardizing the adjacency matrix, the original network topology data is transformed into an input format suitable for graph convolutional neural network processing, fully preserving the network's structural information and node attribute information.

[0065] S4.2, The feature matrix of 70 nodes and 70 standardized adjacency matrices Perform three convolutional operations to obtain 70 output feature matrices. :

[0066]

[0067] in, yes Activation function; The weight parameters for the first, second, and third convolutional layers are as follows: , and ; In some embodiments, the convolution operation is a Chebyshev graph convolution, a spectral graph convolution, an attention graph convolution, or a hybrid graph convolution.

[0068] Through multi-layer graph convolution operations, each node can aggregate information from its multi-hop neighbors, thereby learning richer topological features and inter-node dependencies.

[0069] S4.3, the actor network and the critic network respectively calculate each output feature matrix. In state probability distribution under Valuation :

[0070]

[0071] in, Indicates the current status of the training equipment; Including weight parameters and ; Including weight parameters and .

[0072] The probability of each node being selected as a high-interaction honeypot is output through the actor network, and the value of the current network state is evaluated through the critic network, providing a basis for subsequent policy updates.

[0073] Step 5: Iteratively train the network model: Use reinforcement learning algorithm to update the parameters of the graph convolutional neural network, actor network, and critic network based on the forward propagation results. Determine whether the maximum number of iterations has been reached. If so, the trained network model is obtained; otherwise, return to step 4 to continue iterating. In this embodiment, the graph convolutional neural network parameters, actor network parameters, and critic network parameters are respectively determined based on the results of step 4. , and The network model for this iteration is updated to obtain the current iteration model. and judge If true, then the trained network model is obtained. Otherwise, let , And proceed to step 4; By iteratively optimizing network parameters, the model gradually learns the optimal honeypot deployment strategy. During parameter updates, the actor network adjusts its strategy based on the valuation provided by the critic network, while the critic network updates its valuation based on actual returns. The two networks mutually promote and optimize each other.

[0074] Step 6: Input the test set into the trained network model and perform forward propagation to obtain the probability distribution of the test network topology. Determine the honeypot interaction type based on the probability distribution, that is, update the nodes with a probability distribution greater than a preset threshold as high-interaction honeypot nodes, and update the remaining nodes as low-interaction honeypot nodes.

[0075] The network topology test set is used as the trained network model. The input is propagated forward to obtain Probability distribution of test network topology and valuation and probability distribution With a preset threshold satisfy The node at the specified time is updated to a high-interaction honeypot node, and the remaining nodes are updated to low-interaction honeypot nodes.

[0076] In some embodiments, honeypot interaction types may include the following hierarchies: High-interaction honeypots: Simulate real operating systems and application services, providing a complete interactive environment and recording detailed attack behaviors, but they consume high resources and pose significant risks.

[0077] Interactive honeypots: Simulate some service functions, provide limited interactive capabilities, with moderate resource consumption and controllable risks.

[0078] Low-interaction honeypots: only simulate service ports and basic responses, with low resource consumption and low risk, suitable for large-scale deployment.

[0079] Dynamic interactive honeypot: The interaction level is dynamically adjusted according to the attack behavior. It is low interaction in the early stage of the attack and upgrades to high interaction as the attack progresses.

[0080] In this embodiment, the network topology test set is used as the trained network model. The input is propagated forward to obtain Probability distribution of test network topology and valuation and probability distribution With a preset threshold satisfy The node at the specified time is updated to a high-interaction honeypot node, and the remaining nodes are updated to low-interaction honeypot nodes.

[0081] Through the above steps, this embodiment achieves dynamic deployment of honeypots based on graph convolutional neural networks and reinforcement learning networks. It can adaptively configure nodes as honeypots with different interaction types according to network topology characteristics, improving the flexibility and adaptability of honeypot deployment. Compared with existing technologies, this invention can dynamically change the network topology and interaction type of the honeypots based on attacker information. Even after being identified by an attacker, it can still be reconfigured to continue trapping, significantly enhancing the persistent trapping capability of the honeypot.

[0082] Example 2 Based on the above-mentioned honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks, this application provides a honeynet deployment system based on graph convolutional neural networks and reinforcement learning networks, including: The dataset construction module is used to construct a network topology dataset, which includes multiple network topologies, each containing multiple network nodes. The network model building module is used to build cascaded graph convolutional neural networks and actor-critic networks; The graph convolutional neural network includes a feature extraction module and an adjacency feature module. The feature extraction module is used to extract multidimensional features of network nodes, and the adjacency feature module is used to extract the adjacency matrix of the network topology and perform standardization processing. The actor-critic network includes a parallel actor network and a critic network. The actor network is used to output the action probability distribution, and the critic network is used to output the state estimate. The model training module is used to train the constructed network model using the dataset. The network topology is input into the graph convolutional neural network to obtain the node feature matrix and the standardized adjacency matrix. After graph convolution operation, it is input into the actor-critic network to obtain the probability distribution and the estimated value. The network parameters are updated through the reinforcement learning algorithm to obtain the trained network model. The honeypot deployment module is used to input the network topology to be deployed into the trained network model, determine the honeypot interaction type of each network node based on the probability distribution of the actor network output, deploy nodes with a probability greater than a preset threshold as high-interaction honeypots, and deploy the remaining nodes as low-interaction honeypots.

[0083] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0084] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0085] An electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks as described in any of the above embodiments.

[0086] Another electronic device provided in this application embodiment may further include: an input port connected to a processor for transmitting multimodal data collected by an external acquisition device to the processor; a display unit connected to the processor for displaying the processor's processing results to the outside world; and a communication module connected to the processor for enabling communication between the electronic device and the outside world. The display unit may be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module includes, but is not limited to, Mobile High Definition Link (HML), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), and wireless connection (including Wi-Fi, Bluetooth, Bluetooth Low Energy, and IEEE 802.11s-based communication technology).

[0087] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks as described in any of the above embodiments.

[0088] For descriptions of relevant parts of the honeynet deployment system, electronic device, and computer-readable storage medium based on graph convolutional neural networks and reinforcement learning networks provided in this application, please refer to the detailed descriptions of the corresponding parts in the honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks provided in this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0089] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks, characterized in that, Includes the following steps: Step 1: Construct a network topology dataset, which includes multiple network topologies, each containing multiple network nodes; Step 2: Construct a cascaded graph convolutional neural network and an actor-critic network; The graph convolutional neural network includes a feature extraction module and an adjacency feature module. The feature extraction module is used to extract multidimensional features of network nodes, and the adjacency feature module is used to extract the adjacency matrix of the network topology and perform standardization processing. The actor-critic network includes a parallel actor network and a critic network. The actor network is used to output the action probability distribution, and the critic network is used to output the state estimate. Step 3: Train the constructed network model using the dataset. Input the network topology into the graph convolutional neural network to obtain the node feature matrix and the standardized adjacency matrix. After graph convolution operation, input the matrix into the actor-critic network to obtain the probability distribution and estimate. Update the network parameters through reinforcement learning algorithm to obtain the trained network model. Step 4: Input the network topology to be deployed into the trained network model. Determine the honeypot interaction type of each network node based on the probability distribution output by the actor network. Deploy nodes with a probability greater than a preset threshold as high-interaction honeypots and deploy the remaining nodes as low-interaction honeypots.

2. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 1, characterized in that, The network topology dataset constructed in step 1 includes at least one of the following methods: Randomly initialize and generate multiple network topologies; Collect topology data in real network environments; A graph generation model is used to generate network topologies with specific statistical properties. Evolutionary algorithms are used to generate diverse network topologies through iterative optimization.

3. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 1, characterized in that, The feature extraction module in step 2 includes an IP feature vector module, a degree feature vector module, a centrality feature vector module, and a topology feature vector module arranged in parallel, as well as a feature fusion module connected to the output of the four vector modules. The IP feature vector module is used to convert the IP address of a network node into a binary representation to extract the IP feature vector; The degree feature vector module is used to calculate the in-degree and out-degree of network nodes to extract degree feature vectors; The centrality feature vector module is used to calculate the degree centrality, proximity centrality, betweenness centrality, and eigenvector centrality of network nodes to extract centrality feature vectors; The topology feature vector module is used to calculate the local clustering coefficients and neighbor degree distributions of network nodes to extract topology feature vectors.

4. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 1, characterized in that, The convolutional neural network in step 2 also includes multiple convolutional layers that are sequentially cascaded with the outputs of the feature extraction module and the adjacent feature module; The multi-layer convolutional layer is used to perform graph convolution operations on the node feature matrix and the normalized adjacency matrix, and after aggregating the neighbor node information, outputs an output feature matrix containing topological features.

5. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 1, characterized in that, The node feature matrix is ​​calculated as follows: The IP address of each network node is converted into a binary representation to obtain the IP feature vector; Calculate the degree feature of each node according to the network type: calculate the node degree for undirected networks, and calculate the in-degree and out-degree for directed networks respectively. Combine the degree values ​​to form a degree feature vector. Calculate the centrality features of each node, including degree centrality, proximity centrality, betweenness centrality, and eigenvector centrality, and combine the four centrality measures to form a central eigenvector; Calculate the local clustering coefficient and neighbor degree distribution of each node, and combine the calculation results to form a topological feature vector; The IP feature vector, degree feature vector, centrality feature vector, and topology feature vector are fused to obtain a comprehensive feature representation of each node, and then a node feature matrix is ​​constructed.

6. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 5, characterized in that, Degree centrality Proximity centrality Betweenness centrality eigenvector centrality Local clustering coefficient And neighbor degree distribution The calculation formula is: in, It is the adjacency characteristic matrix eigenvalues, It is a node The degree, It is a node and nodes The shortest path distance between them It is a node and nodes The number of shortest paths between them Through nodes The number of shortest paths, It is a node The actual number of edges between neighbors. It is a node The set of neighboring nodes.

7. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 1, characterized in that, The output feature matrix obtained after graph convolution in step 4 is calculated in the following way: The node feature matrix and the normalized adjacency matrix are input into a multi-layer graph convolutional network. The neighbor node information is aggregated through layer-by-layer convolution operations to obtain the final output feature matrix. The output feature matrix integrates the network node’s own features and the topological structure information of multi-hop neighbors.

8. The honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks according to claim 7, characterized in that, The graph convolution operation is at least one of the following: Basic graph convolution, Chebyshev graph convolution, spectral graph convolution, attention graph convolution, hybrid graph convolution, or gated graph convolution.

9. A honeynet deployment system based on graph convolutional neural networks and reinforcement learning networks, characterized in that, include: The dataset construction module is used to construct a network topology dataset, which includes multiple network topologies, each containing multiple network nodes. The network model building module is used to build cascaded graph convolutional neural networks and actor-critic networks; The graph convolutional neural network includes a feature extraction module and an adjacency feature module. The feature extraction module is used to extract multidimensional features of network nodes, and the adjacency feature module is used to extract the adjacency matrix of the network topology and perform standardization processing. The actor-critic network includes a parallel actor network and a critic network. The actor network is used to output the action probability distribution, and the critic network is used to output the state estimate. The model training module is used to train the constructed network model using the dataset. The network topology is input into the graph convolutional neural network to obtain the node feature matrix and the standardized adjacency matrix. After graph convolution operation, it is input into the actor-critic network to obtain the probability distribution and the estimated value. The network parameters are updated through the reinforcement learning algorithm to obtain the trained network model. The honeypot deployment module is used to input the network topology to be deployed into the trained network model, determine the honeypot interaction type of each network node based on the probability distribution of the actor network output, deploy nodes with a probability greater than a preset threshold as high-interaction honeypots, and deploy the remaining nodes as low-interaction honeypots.

10. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the honeynet deployment method based on graph convolutional neural networks and reinforcement learning networks as described in any one of claims 1 to 8.

Images