A Dynamic Key Generation and Distribution Method Based on Improved GAN-RL

Abstract

This invention relates to the fields of information security and communication networks, and discloses a dynamic key generation and distribution method based on an improved GAN-RL. This method combines Generative Adversarial Networks (GANs) and Reinforcement Learning (RL) to improve the security, real-time performance, and adaptability of the key generation and distribution process. The method includes the following steps: S1, communication environment feature acquisition and initialization; S2, key generation based on an improved GAN-RL; S3, inputting communication environment features into a reinforcement learning model; S4, real-time adjustment and optimization of the theoretical strategy generated by the reinforcement learning model; S5, key distribution execution and feedback optimization. This dynamic key generation and distribution method based on improved GAN-RL exhibits strong dynamic adaptability, improves the efficiency and security of key distribution, effectively reduces network congestion, increases the distribution success rate, and meets the specific needs of different devices and nodes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information security and communication networks; in particular, a dynamic key generation and distribution method based on improved GAN-RL. BACKGROUND

[0002] In the field of information security, key generation and distribution is of great significance. Traditional key management methods rely heavily on static key generation and manual distribution. Although this method can ensure security to a certain extent, it has great limitations when facing large-scale and dynamic network environments. Especially in the fields of distributed networks and Internet of Things, traditional methods often cannot balance real-time performance, adaptability, and security. Therefore, how to achieve efficient and secure key generation and distribution in a dynamic environment has become a hot research topic.

[0003] However, traditional key distribution methods still have many limitations and defects. For example, static key generation and distribution cannot cope with rapidly changing network states and are vulnerable to attacks, leading to key leakage or tampering. Manual distribution increases the complexity of operations and security risks. In addition, delays, packet loss, and node anomalies during key distribution further limit the application effect of traditional methods.

[0004] In recent years, as cutting-edge technologies, generative adversarial networks (GAN) and reinforcement learning (RL) have gradually been applied to the field of key management. Generative adversarial networks can adaptively generate more random and required keys through adversarial learning between the generator and the discriminator. Reinforcement learning dynamically optimizes the distribution strategy by obtaining feedback from the environment, thereby making the best decisions in a changing network environment. As a self-learning method, it compensates for the shortcomings of traditional methods in dynamic environments through experience accumulation and strategy optimization.

[0005] However, the existing generative adversarial networks and reinforcement learning in the application of key generation and distribution still have many problems. For example, how to combine these two technologies to form a complete key generation and distribution system? How to use real-time data in the network to optimize the generation rules of the keys? How to adjust the generator and discriminator parameters in the generative adversarial network according to the feedback to improve the security and efficiency of key distribution? These problems need to be solved.

[0006] Traditional key distribution methods often fail to respond in a timely manner and adapt flexibly in the face of dynamic network environments. For example, once the traditional static key is generated, it cannot be optimized and adjusted according to network state or environmental changes. Therefore, researchers have proposed an algorithm based on the combination of generative adversarial networks (GAN) and reinforcement learning (RL), which improves the security and intelligence level of the key generation and distribution process through deep learning technology and adaptive strategies. However, in practical applications, how to effectively combine these two algorithms and form a systematic key generation and distribution method still needs further research and exploration. SUMMARY

[0007] The technical problem to be solved by the present application is to provide an improved GAN-RL-based dynamic key generation and distribution method that combines generative adversarial networks (GAN) and reinforcement learning (RL) to improve the security, real-time performance, and adaptability of the key generation and distribution process.

[0008] The technical solution adopted by the present application to solve the technical problem is: an improved GAN-RL-based dynamic key generation and distribution method, comprising the following steps:

[0009] S1, communication environment feature collection and initialization; the communication environment feature collection includes collecting the topology of the communication network, collecting network node information, collecting network state information, and collecting device attribute information;

[0010] S2, key generation based on improved generative adversarial network;

[0011] S3, inputting the communication environment features into the reinforcement learning model;

[0012] S4, real-time adjustment and optimization of the theoretical strategy generated by the reinforcement learning model;

[0013] S5, key distribution execution and feedback optimization.

[0014] Specifically, collecting the topology of the communication network includes the following steps:

[0015] S111, deploying sensor devices at each node of the communication network, and collecting the connection information between nodes in real time through these devices to obtain the connection state of the nodes ; the represents the connection between node i and node j, where:

[0016]

[0017] By collecting the connection between each node one by one, the following connection matrix is generated :

[0018]

[0019] Each row or column in the matrix represents the connection relationship between a node and other nodes, with 1 indicating a connection and 0 indicating no connection.

[0020] S112. Use a web crawling algorithm to traverse the global connectivity of the network and identify the set of direct neighboring nodes for each node. ,in:

[0021] ;

[0022] Based on the connection matrix C, the adjacent nodes of each node can be obtained:

[0023] The set of adjacent nodes of node A is ;

[0024] The set of adjacent nodes of node B is ;

[0025] The set of adjacent nodes of node C is ;

[0026] The set of adjacent nodes of node D is ;

[0027] S113. While collecting connection information, record the communication link characteristics between nodes, including: link type: confirm whether the link is unidirectional or bidirectional; link bandwidth. Communication bandwidth between data acquisition nodes:

[0028]

[0029]

[0030]

[0031]

[0032] Based on this data, bandwidth matrix B is formed:

[0033]

[0034] S114. Combining the connection matrix C and bandwidth information B, construct the topology of the communication network. ;

[0035] Among them, the node set Represents all nodes in the network:

[0036] ={A,B,C,D};

[0037] edge set Indicates the communication link between nodes:

[0038] The bandwidth of edge (A,B) is 100 Mbps;

[0039] The bandwidth of edge (B,C) is 50 Mbps;

[0040] The bandwidth of edge (C,D) is 150 Mbps;

[0041] The bandwidth of edge (A,D) is 200 Mbps;

[0042] S115. Based on the collected data, provide complete basic data for subsequent key generation and distribution.

[0043] Specifically, collecting network node information includes the following steps:

[0044] S121. Enable the node information collection function in the network monitoring equipment, and obtain the basic attribute information of each node in the network, including the number and type of nodes, through the SNMP protocol; assuming that there are N nodes in the network, use a set Represents a set of nodes;

[0045] S122, for each node Collect its specific attribute information, including: node type, node function, connection bandwidth, and communication method;

[0046] (1) Node type:

[0047] The device type identifier determines whether a node is a router, switch, server, or terminal device.

[0048] Use a label Represents a node Type:

[0049] ;

[0050] (2) Node functions:

[0051] Determine the network functions performed by the nodes;

[0052] Use sets Represents a node Functions:

[0053] ;

[0054] Among them, Relay refers to a node that has the function of providing data forwarding / relay to other nodes in the network and undertakes the relay of communication traffic; Compute refers to a node that has computing or processing capabilities and can perform data analysis, task calculation and other processing operations; Storage refers to a node that has data storage capabilities and can be used to cache, back up or distribute data in the network.

[0055] (3) Connection bandwidth:

[0056] From node Bandwidth to other nodes Information, and records the total connection bandwidth of the nodes:

[0057]

[0058] in Represents a node and The bandwidth between.

[0059] (4) Communication method:

[0060] The communication method of the acquisition node is identified by a flag. express:

[0061] ;

[0062] S123. Integrate the above data into a node attribute table. Except for the first row, each row in the node attribute table represents the specific attributes of the node, including type, function, connection bandwidth and communication method.

[0063] Specifically, the collected network status information includes network latency, packet loss rate, bandwidth utilization, and network congestion.

[0064] The network latency refers to the communication network latency between nodes in the acquisition network; that is, the time required for data to be sent from the source node to the target node, which is measured by calculating the round-trip time.

[0065] For each pair of nodes and ,Delay It can be calculated using the following formula:

[0066] ;

[0067] in From node To the node Round-trip time. Delay value. It will be collected and recorded for subsequent analysis;

[0068] The packet loss rate represents the proportion of data packets lost in a network, and is typically measured by network monitoring tools by periodically sending probe packets. In network monitoring systems, the packet loss rate... This refers to the node To the node The packet loss rate is calculated using the following formula:

[0069]

[0070] The bandwidth utilization rate refers to the actual usage of network links, reflecting whether the communication links between nodes are close to saturation; bandwidth utilization rate The calculation formula is:

[0071] ;

[0072] in It is a link ( , Actual bandwidth usage on ) This is the total bandwidth of the link;

[0073] Network congestion is a comprehensive indicator calculated by monitoring information such as latency, packet loss, and bandwidth utilization among multiple nodes, used to assess the overall network load. In network monitoring systems, congestion... The weighted average is expressed using the following formula:

[0074]

[0075] in, , , These are the weights for latency, packet loss rate, and bandwidth utilization, respectively. , , These are the latency, packet loss rate, and bandwidth utilization of the i-th link, respectively, and N is the number of links in the network.

[0076] Specifically, collecting device attribute information includes the following steps:

[0077] S131. Collect hardware attribute information of each device in the network in real time through dedicated monitoring tools or operating system interfaces, including the device's computing power, memory size, and storage capacity; the device's computing power is mainly evaluated through CPU performance and number of cores, and the CPU frequency and total memory capacity can be obtained through the interface provided by the operating system.

[0078] S132. Collect the battery status and power consumption information of the device. Battery information, such as remaining power and charging rate, can be obtained through battery monitoring tools provided by the operating system. (Remaining power...) The calculation formula is:

[0079] ;

[0080] Battery consumption rate It can be estimated using the following formula:

[0081] ;

[0082] in, The change in device power is represented by Δt, which is the time interval.

[0083] S133. Monitor the power consumption of the device in real time using dedicated energy monitoring tools or hardware modules, including the power consumption of components such as CPU, memory, storage, and network; and the total energy consumption of the device. This can be obtained by calculating the sum of the power consumption of each component:

[0084] ;

[0085] in, Let be the power consumption of the i-th component, and NN be the number of components monitored in the device.

[0086] Specifically, step S1, communication environment feature acquisition and initialization, includes feature data standardization processing:

[0087] The feature data standardization process includes the following steps:

[0088] S141. Preprocess the various types of data obtained in step S1, including the topology of the communication network, network node information, network status information, and device attribute information.

[0089] S142. Standardization of Topology Data: Topology is usually represented by a connection matrix C, where each element... This indicates whether node i and node j are directly connected.

[0090] Transform the connection matrix C into a flattened one-dimensional vector. This flattened connection matrix can be implemented in the following way:

[0091] ;

[0092] Here, Flatten represents the transformation of a two-dimensional connection matrix into a one-dimensional vector; this standardized connection information can be input into generative adversarial networks (GANs) and reinforcement learning models;

[0093] Standardization of node information: Node information includes the number of nodes, node type, and node function; the number of nodes N is directly used as the input feature; the node type is converted into a one-hot encoding vector; the node function is standardized using one-hot encoding or numerical encoding.

[0094] ;

[0095] in, For the type or function label of the node;

[0096] Standardization of network state data: Network state data typically has different dimensions and numerical ranges; Z-score standardization is used to process this data.

[0097] ;

[0098] Where X represents the original data, μ represents the mean of the data, and σ represents the standard deviation of the data. This standardized network state data will be within a uniform scale, which is beneficial for subsequent model training.

[0099] Standardization of device attribute data: Device attribute data includes computing power, memory size, remaining power, and energy consumption; it is processed through Min-Max standardization or Z-score standardization; Min-Max standardization maps the data to the [0, 1] interval.

[0100] ;

[0101] in, and These are the minimum and maximum values ​​of the feature, respectively.

[0102] Furthermore, the key generation based on the improved generative adversarial network in step S2 includes the following steps:

[0103] S21. Input random noise and communication environment characteristics into the generator of the generative adversarial network;

[0104] S211. The generator in a Generative Adversarial Network (GAN) requires an initial input, which is random noise z. This noise provides randomness to the generated key, ensuring that each key generation is unique. To achieve sufficient randomness, typically...

[0105] A noise vector z of length N is obtained by sampling from the standard normal distribution: this random noise z is used as an initial input to the generator in the generative adversarial network;

[0106]

[0107] Where N is the dimension of the noise vector, and each element... They are all independently drawn from the standard normal distribution;

[0108] S212. The network's communication environment characteristic data is provided to the generator as input; the network's communication environment characteristic data includes the network's topology, node information, network status, and device attributes; the network's communication environment characteristic data is represented as a vector e of size M, where each... It is a specific feature of the network:

[0109]

[0110] The length M of the communication environment feature e depends on the network information collected in step S1;

[0111] The input noise vector z and the network communication environment feature data vector e are fused to obtain a new input vector x:

[0112]

[0113] The vector x consists of two parts: random noise and the communication environment characteristics of the network, with a length of N+M;

[0114] S213. Input this fused input vector x into the generator of the generative adversarial network to generate the key k. The generator process is represented as follows:

[0115]

[0116] in, The mapping function represents the generator. The generator's output key k is used as a parameter and will be passed to the discriminator as a candidate key for quality evaluation.

[0117] S22. The generator outputs candidate keys and passes them to the discriminator;

[0118] S221. Pass the candidate key k to the discriminator. The discriminator is used to evaluate the quality of the key and determine whether the key is random enough and resistant to prediction.

[0119] The discriminator determines the quality of a key by evaluating its entropy, distribution characteristics, and anti-predictability; entropy is an important indicator of key randomness and uncertainty, calculated using the following formula:

[0120] ;

[0121] in, Let be the probability distribution of the i-th bit of key k, and n be the length of the key;

[0122] The discriminator evaluates the key based on its distribution characteristics; the discriminator uses the chi-square test or the Kullback-Leibler divergence statistic method to measure the uniformity of the key based on its distribution characteristics.

[0123] The evaluation process of the discriminator is represented as follows:

[0124] ;

[0125] Among them, Evaluate( ) is the evaluation function of the discriminator, which outputs a scalar value representing the quality of the key;

[0126] If the discriminator deems the generated key to be of high quality, indicating that it has sufficient randomness and distribution characteristics, the discriminator will output a high evaluation value, indicating that the key meets the requirements;

[0127] If the discriminator deems the key quality poor, the generator will receive feedback from the discriminator and adjust the key generation algorithm based on the discriminator's feedback until the generated key meets the requirements.

[0128] S23. The discriminator feeds back the evaluation results to the generator to optimize the generator's network parameters.

[0129] The discriminator's evaluation results are fed back to the generator as feedback signals to update the generator's network parameters; the generator and discriminator are optimized together during training, using an adversarial training approach.

[0130] (1) The feedback mechanism and optimization process are as follows:

[0131] The evaluation results are fed back as an error signal: the output of the discriminator. It will be fed back to the generator as an error signal to guide the generator's optimization direction;

[0132] (2) Parameter update of the generator: The generator updates the network parameters through backpropagation and gradient descent; after each key generation, the generator calculates the loss function. , representing the generator's loss; this loss function is calculated based on the discriminator's evaluation results, as shown below:

[0133]

[0134] Where E represents the expected value, and k is the candidate key generated by the generator. and These are the parameters for the generator and the discriminator, respectively. It is the discriminator's evaluation output of the key k;

[0135] The generator optimizes its parameters by minimizing the loss function, thereby improving the quality of the key; this process can be achieved using gradient descent.

[0136]

[0137] Where η is the learning rate. It is the gradient of the generator loss function with respect to its parameters;

[0138] (3) Optimization of target feedback adjustment: In adversarial training, the generator and the discriminator are in a game state; the generator continuously adjusts its parameters to improve the quality of its generated keys, making it increasingly difficult for the discriminator to judge the authenticity of the keys; as training progresses, the quality of the keys generated by the generator will gradually improve, eventually reaching the standard that meets the encryption requirements of communication networks.

[0139] S24. Once the generated key meets the set quality standard, it is stored in the key pool and awaits distribution.

[0140] The optimized generator will generate keys that meet the quality standards; the discriminator will perform a final evaluation of the generated keys. If the key's entropy, distribution characteristics, and anti-predictability meet the predetermined security standards, the key is considered qualified; at this time, the generated key k will be stored in the key pool, waiting for subsequent distribution operations.

[0141] The generated key pool can be represented as follows:

[0142]

[0143] Where K is the key pool, containing multiple generated keys, each key They all possess a certain degree of randomness and security, and can meet subsequent distribution needs.

[0144] Furthermore, in step S3, the reinforcement learning model inputting communication environment features includes the following steps:

[0145] S31. Construct the state space and action space of the reinforcement learning model.

[0146] Construct the state space (S) and action space (A) of the reinforcement learning model;

[0147] (1) Construction of the state space

[0148] The state space refers to the environmental information that the reinforcement learning model can perceive at each step of the decision; the state space is composed of multiple key communication environment features, specifically including network topology information, node information, network state information, and device attribute information.

[0149] The network topology information refers to the connection relationships between nodes obtained in step S1. The network topology can be represented as an adjacency matrix C:

[0150] ;

[0151] in, This reflects whether node i and node j in the network are directly connected;

[0152] The node information represents the attributes of each node, including its type, computing power, and function, forming the node's feature vector. :

[0153] ;

[0154] The network status information includes the real-time status of the network, namely bandwidth utilization, latency, and packet loss rate; assuming This represents the bandwidth of network node i. Represents latency and packet loss rate. Then it is:

[0155] ;

[0156] The device attribute information includes the node's energy consumption information and remaining power; the device status can be represented as:

[0157] ;

[0158] After standardization, network topology information, node information, network status information, and device attribute information form states in state space S. Each state is a combination of multiple feature vectors, as follows:

[0159] ;

[0160] (2) Construction of action space

[0161] The motion space mainly consists of the following:

[0162] Path selection: Assume there are multiple key distribution paths The choice of each path can be represented as a discrete action. :

[0163] ;

[0164] Node selection: The node selection action can be represented as:

[0165]

[0166] in It is a node selected from the current network state that is suitable for receiving the key;

[0167] S32, Training the reinforcement learning model

[0168] The training process includes everything from initializing the model to adjusting the policy through continuous feedback to obtain optimal decisions in a dynamic network environment; it includes the following steps:

[0169] S321. Initialization of the training environment

[0170] First, the construction of the training environment is based on the state space S and action space A constructed in step S31, and needs to include the following key elements:

[0171] Network state simulation: Based on actual collected data or hypothetical datasets, simulate various states in a communication network; simulate changes in various network conditions through these data so that reinforcement learning models can adapt to different network states;

[0172] Node and device status simulation: Simulate the working status and energy consumption of different nodes, as well as the connection relationships between nodes;

[0173] Reward function design: Define the reward function This is used to measure the effect of each strategy after its execution;

[0174] A higher reward is given if the key is successfully distributed and the network load is below a set threshold; a lower reward is given if the key distribution fails or the network latency is high. The reward function is as follows:

[0175]

[0176] in, A positive reward is given when the key is successfully distributed; a negative reward is given when the network latency is high; and a negative reward is given when the device consumes too much energy.

[0177] S322, Training process of reinforcement learning model

[0178] The training process is based on reinforcement learning methods; the reinforcement learning method employs Q-learning, Deep Q-Network (DQN), or policy gradient methods; the training process includes the following steps:

[0179] Sampling State and Selecting Action: At each time step t, the model starts from the current state. Choose one action And apply it to a simulated environment to obtain rewards. and the next state :

[0180] ;

[0181] in, This indicates that the policy π is based on the state. The action to choose.

[0182] Update Q-value or policy: Update the Q-value function or policy network based on the selected action and environmental feedback; for Q-learning, the update formula is:

[0183] ;

[0184] Where α is the learning rate and γ is the discount factor. It's a reward. It is the maximum Q value in the next state;

[0185] For the policy gradient method, updating the parameters θ of the policy network is achieved by maximizing the expected reward:

[0186] ;

[0187] in, It is the gradient of the policy, representing the expected reward for choosing an action in a given state;

[0188] Training iteration: By continuously performing multiple training iterations, the model makes optimal key distribution decisions for different network states in a simulated environment;

[0189] S323, Experience Replay and Exploration Mechanisms

[0190] Experience replay and ε-greedy policy mechanisms were introduced into the reinforcement learning model;

[0191] Experience replay: In reinforcement learning, experience replay refers to storing the model's historical states, actions, rewards, and next state quadruplets in an experience pool; during each training session, a batch of historical experiences is randomly selected from the pool for training; the formula is as follows:

[0192] ;

[0193] Where D is the experience pool, ( A tuple consists of state, action, reward, and next state; by randomly sampling data from the experience pool, the model can learn from different experiences, avoiding overfitting.

[0194] ε-greedy strategy: This strategy is used to balance exploration and exploitation; during training, the model will exploit the system with a certain probability ( Select random actions to discover potential better strategies; while in the remaining probability In this case, the model will select the optimal action at the moment; as training progresses, As the value gradually decreases, the model relies more on the learned optimal policy; the formula is:

[0195] ;

[0196] This approach allows the model to explore new distribution strategies as well as select the optimal solution from known strategies, thus achieving better convergence.

[0197] Furthermore, the real-time adjustment and optimization of the theoretical strategy generated by the reinforcement learning model in step S4 includes the following steps:

[0198] S41. Key Distribution Strategy Optimization Based on Reinforcement Learning

[0199] First, the reinforcement learning model receives real-time state information from the communication network; this information refers to network performance metrics including bandwidth, latency, and packet loss rate. The model uses the following formula to evaluate the network health status:

[0200] ;

[0201] in, It is the network latency of path p; It is the bandwidth of path p; It is the packet loss rate of path p; , , These are weighting coefficients, representing the impact of latency, bandwidth, and packet loss rate on path quality;

[0202] The model evaluates the quality of each path by calculating network health metrics, and finally selects the path with the highest health score for key distribution.

[0203] The model evaluates key suitability based on key characteristics:

[0204]

[0205] Where Type represents the score of the key type; Length is the key length; and Strength is the encryption strength of the key. , , These are weighting coefficients, reflecting the impact of each key characteristic on adaptability;

[0206] After obtaining the health status of the network paths and the suitability of the keys, the reinforcement learning model needs to select the optimal key distribution path and distribution method based on this information. The model calculates the comprehensive score of each path according to the following formula and selects the path with the highest score:

[0207] ;

[0208] Where P is the set of all possible paths;

[0209] The model selects an appropriate distribution method based on the number of target nodes. For a small number of target nodes, the model might choose unicast distribution; while for a large number of target nodes, the model will choose multicast or groupcast. This decision...

[0210] The distribution method of the model is determined by the following rules:

[0211]

[0212] in, It refers to the number of target nodes; Unicast is suitable for a small number of target nodes; Multicast is suitable for large-scale distribution; Broadcast is suitable for large-scale distribution.

[0213] Each node is assigned a priority using the following formula:

[0214] ;

[0215] in, It is the priority of node n; the priority of a critical node is 1, which means that the node should have priority to obtain the key;

[0216] S42. Reinforcement learning iteratively optimizes the distribution strategy through exploration and experimentation in the action space.

[0217] Through exploration and feedback mechanisms, the model can continuously adjust and improve its distribution strategy to adapt to the dynamically changing network environment and task requirements.

[0218] (1) Exploration and feedback of action space

[0219] In reinforcement learning, the action space is the set of all actions that the model can choose; in key distribution strategies, the action space includes different path selections, distribution methods, and node priority adjustments; the model adjusts its strategy by experimenting in a simulated environment, selecting different actions, and observing environmental feedback.

[0220] For each time step t, the model starts from the current state. Choose an action And apply it to the environment; the action space representation of reinforcement learning is:

[0221] ;

[0222] Where p∈P represents the set of available network paths; m represents the distribution method, which can be unicast, multicast, or groupcast; n∈{0,1} represents the priority of node n, where 0 is a normal node and 1 is a critical node or a high-risk node;

[0223] In each training iteration, the model adjusts according to the current state. Choose an action And perform this action in a simulation environment.

[0224] (2) Reward feedback and strategy update

[0225] Whenever the model selects an action and applies it to the network, the system returns a feedback signal, i.e., a reward; the reward function is designed in the following form:

[0226] ;

[0227] Among them, Success_Reward is the reward value after successful key distribution; Delay is the network latency, the greater the latency, the higher the penalty; Energy is the energy consumption during the distribution process, the higher the energy consumption, the greater the penalty; Risk is the security risk of the network or node, the higher the risk, the greater the penalty. , , These are weighting coefficients, which adjust the impact of various factors on the reward function;

[0228] Through the reward function, the model can obtain feedback information about the current action and adjust its policy based on this feedback; the goal of reinforcement learning is to maximize the long-term accumulated reward, that is, to optimize the key distribution policy.

[0229] (3) Action selection strategy and ε-greedy strategy

[0230] In reinforcement learning, an ε-greedy strategy is employed to balance exploration and exploitation; this strategy gradually reduces randomness during training, allowing the model to transition from extensive exploration in the early stages to local optimization in the later stages; the model has... The probability of choosing a random action, i.e., exploring, is determined by the probability of the action. The probability of choosing the optimal action under the current strategy is used, as detailed below:

[0231] ;

[0232] in, It is the probability of exploration, which gradually decays during training to ensure that the model can explore different strategies extensively in the early stage and make decisions using the best strategies learned in the later stage.

[0233] (4) Strategy optimization and convergence

[0234] After each exploration and feedback iteration, the reinforcement learning model updates its policy based on the new state and reward information; for Q-learning, the update formula is as follows:

[0235] ;

[0236] in, Indicates the state Next action The Q value; α is the learning rate, which controls the step size of each update; γ is the discount factor, which measures the weight of future rewards; The next state The maximum Q value;

[0237] For policy-based algorithms, policy updates are performed using the following formula:

[0238] ;

[0239] in, These are the parameters of the current strategy; It is the policy gradient, representing the gradient at state 100°. The expected reward for the next action selection;

[0240] S43. Generate the final key distribution scheme and pass it to the distribution execution module.

[0241] After repeated training and optimization, the final key distribution strategy will be gradually improved based on step S42, and the final key distribution scheme will be generated. The scheme includes: the selected distribution path, the distribution method and nodes, and the timing and frequency of key distribution.

[0242] Selected distribution path: Optimal path selection;

[0243] Distribution method and nodes: Determine whether to use unicast, multicast, or multicast, and prioritize key nodes for distribution;

[0244] Key distribution timing and frequency: Determine the timing and frequency of key distribution based on network status and node requirements;

[0245] The final distribution scheme is passed to the distribution execution module to perform the key distribution task.

[0246] Furthermore, the key distribution execution and feedback optimization in step S5 includes the following steps:

[0247] S51. Distribute the generated key to the target node according to the distribution strategy output by the reinforcement learning model, and monitor the node response and distribution result in real time during the distribution process;

[0248] First, the optimized distribution strategy of the reinforcement learning model is obtained from step S4; this strategy selects the optimal target node and path based on network state, node requirements, and key characteristics; specifically, it includes:

[0249] Target node selection: The reinforcement learning model determines which nodes should receive the key first by considering factors such as node load and communication requirements; the node set is N={ , ,..., The model will select the set of nodes Ntarget that best meets the requirements. N;

[0250] Distribution path selection: Select the optimal transmission path based on the location of the target node and the network topology; path selection is usually determined by factors such as latency d, bandwidth b, and stability s; the goal of path selection is to minimize latency and maximize bandwidth to ensure key transmission efficiency.

[0251] The target node selection function is:

[0252] ;

[0253] in, It is node load. It's a communication requirement. It's about node stability. , , It refers to the weights of each factor;

[0254] Then, based on the selected target node and path, the key transmission begins; at this point, the generated key is transmitted to the target node through the selected path; during this process, the following transmission methods can be selected:

[0255] Unicast: Transmits the key directly to a single target node;

[0256] Multicast: Transmits a key to multiple nodes simultaneously;

[0257] Multicast: Transmitting a key to a group of nodes;

[0258] Next, during the key transmission process, the system monitors the following key indicators in real time to ensure the smooth progress of the distribution process:

[0259] Node response: Monitor whether the target node has successfully received the key; if the node does not respond or communication fails, the system will mark it as abnormal and prepare for the next step of abnormal handling;

[0260] Distribution delay: Real-time tracking of key transmission delay to ensure it does not exceed a set threshold; the delay is calculated using the following formula:

[0261] ;

[0262] in, It is the time when the target node receives the key. It is the time when the key starts being sent. It is the target number of nodes;

[0263] Distribution success rate: Calculate the proportion of nodes that successfully receive the key; if the proportion of nodes that successfully receive the key is lower than a predetermined threshold, the system will trigger an alarm and re-evaluate the distribution strategy.

[0264]

[0265] in, This is the number of nodes that successfully received the key. This is the total number of target nodes;

[0266] Distribution failure rate: The proportion of nodes that fail to distribute keys also needs to be monitored in a timely manner to avoid a high failure rate during key distribution; its calculation method is as follows:

[0267]

[0268] in, This is the number of nodes that failed to distribute the data. This is the total number of target nodes;

[0269] Finally, during the monitoring process, the system will record relevant data in real time regarding instances of excessive latency, unresponsive nodes, or distribution failures, and feed these anomalies back to the reinforcement learning model for subsequent optimization; specific feedback includes:

[0270] Delay information: Real-time feedback on distribution delays and optimization of the distribution path based on the current network status;

[0271] Node response failure information: This indicates that the target node failed to respond, and it may be necessary to select a different path or node.

[0272] Distribution failure information: Reports nodes where distribution failed, for reference in subsequent strategy adjustments;

[0273] The system optimizes the distribution path and method through this real-time feedback, making the next round of distribution more efficient and stable;

[0274] S52. If an error occurs during the distribution process, the error information is recorded and fed back to the reinforcement learning module. The reinforcement learning model updates its policy based on the feedback information.

[0275] The main steps are as follows:

[0276] S521. Anomaly Detection and Logging: During the key distribution process, the system continuously monitors and detects several key indicators, including:

[0277] High latency: If the latency of a distribution path exceeds a set threshold, the system immediately marks it as abnormal; the latency calculation formula is:

[0278] ;

[0279] if Greater than the set maximum delay threshold If so, it is considered that a delay anomaly has occurred;

[0280] Node failure: If the target node fails to respond to the key reception request within the specified time, or if the connection with the target node fails, the system records the node failure information; the node failure rate can be calculated using the following formula:

[0281] ;

[0282] if Exceeding the set threshold If so, it is marked as a node failure or abnormality;

[0283] Path interruption: If the path is lost or interrupted, preventing the key from being transmitted to the target node, the system records the path interruption event and triggers an alarm;

[0284] S522, Anomaly Feedback to Reinforcement Learning Module: Once an anomaly is detected, the system will feed back the anomaly information to the reinforcement learning module in step S4; the specific feedback information includes:

[0285] Delay anomaly information: The system sends data with recorded delays exceeding the threshold to the reinforcement learning model, and the model adjusts its distribution path selection strategy based on this information;

[0286] Node failure information: If some nodes fail during key distribution, the system will report the failure information of these nodes and adjust the node selection and path optimization strategies through reinforcement learning models;

[0287] Path interruption information: When a path is interrupted, the system will record the path and the state at the time of the interruption, and feed this information back to the reinforcement learning model. The reinforcement learning model will then re-evaluate the path selection and backup strategy.

[0288] S523. Reinforcement learning models adjust their policies based on feedback: Upon receiving abnormal feedback, the reinforcement learning model updates its policy based on the current network environment and the feedback data. The update typically includes the following aspects:

[0289] Path optimization: If some paths have poor latency or reliability, the model will optimize path selection to reduce dependence on these paths;

[0290] Node selection optimization: The reinforcement learning model will reselect more reliable nodes based on node failure data;

[0291] Distribution strategy: If certain distribution patterns result in high latency or failure rates, the model may adjust the distribution method to adapt to changes in the network environment;

[0292] The specific strategy update formula is as follows:

[0293] For the Q-learning method, the policy update formula is:

[0294] ;

[0295] Where α is the learning rate and γ is the discount factor. It's an instant reward. It is the value of the current state-action pair;

[0296] For the policy gradient method, the formula for updating the policy parameters is:

[0297] ;

[0298] in, These are the parameters of the current strategy. It is the gradient of the policy;

[0299] S523, Loop Optimization of Feedback Mechanism:

[0300] Throughout the process, the feedback of abnormal information and the updating of policies form a closed loop; after each round of key distribution, the system continuously optimizes the policy based on the execution results to ensure that the next round of distribution can adapt to changes in network conditions.

[0301] S53. After the key distribution is completed, if the communication environment characteristics or distribution efficiency do not meet the expected value, the input conditions of the generative adversarial network are optimized through the feedback mechanism to further improve the adaptability of key generation and distribution.

[0302] The characteristics of the communication environment and the distribution efficiency during the key generation and distribution process are evaluated, and the input conditions of the generative adversarial network (GAN) are optimized through a feedback mechanism to further improve the adaptability of key generation and distribution.

[0303] Includes the following:

[0304] (1) Evaluate the effectiveness of key distribution and communication environment characteristics: Evaluate key indicators and communication environment characteristics during the distribution process; including:

[0305] Distribution latency: The total time taken from key generation to receipt by the target node; if the latency is too high, it may affect the network's real-time requirements.

[0306] Distribution success rate: Evaluate whether the key has successfully reached all target nodes; if there are any nodes where distribution failed, the system should mark them as failures and conduct further analysis;

[0307] Node response: Check the response speed and success rate of the target node;

[0308] Path stability: Evaluate the stability and reliability of the key distribution path. If the path is found to be unstable or interrupted, optimize the path selection.

[0309] The above evaluation indicators are quantified using the following formula:

[0310] Distribution delay:

[0311] ;

[0312] in, It is the time when the key was received. It is the time when the key is sent. It is the target number of nodes;

[0313] Distribution success rate:

[0314] ;

[0315] in, This is the number of nodes that successfully received the key. This is the total number of target nodes;

[0316] (2) Optimize the input conditions of GAN through feedback mechanism: Based on the above evaluation results, the system will pass the feedback information to the generative adversarial network (GAN) to optimize its input conditions; the goal of the generative adversarial network is to generate a key that meets the requirements, and its input conditions include the following aspects: the characteristics of the network's communication environment, node state requirements and path selection optimization.

[0317] Specific optimization methods include:

[0318] Optimize key generation rules: Optimize the generation rules of GAN by analyzing historical data;

[0319] Adjusting network input characteristics: If certain communication environment characteristics negatively affect key distribution, then the input characteristics of the GAN can be adjusted.

[0320] Improve adaptability: By continuously optimizing the input conditions of GAN, the adaptability of the key generation process is improved, ensuring that keys can be distributed efficiently and securely under different network environments and node requirements;

[0321] The optimization formula for the feedback is as follows:

[0322]

[0323] in, It is the optimized GAN input. This is the current input condition. It is the amount of adjustment to the input conditions based on feedback information, and α is the learning rate.

[0324] Beneficial effects of the present invention

[0325] The dynamic key generation and distribution method based on improved GAN-RL described in this invention has the following advantages:

[0326] 1. Strong Dynamic Adaptability: The dynamic key generation and distribution method based on improved GAN-RL described in this invention, through key generation based on Generative Adversarial Networks (GANs) and strategy optimization using reinforcement learning (RL), can automatically adjust key generation rules and distribution strategies in real-time changing network environments. Compared with traditional static key management methods, this invention can more flexibly adapt to changes in network conditions, such as bandwidth fluctuations, node failures, and path delays, ensuring the security and efficiency of key distribution.

[0327] 2. Improved Efficiency and Security of Key Distribution: The dynamic key generation and distribution method based on improved GAN-RL described in this invention uses reinforcement learning optimization strategies to continuously iterate and optimize decisions during the key distribution process. This not only improves the speed of key distribution but also effectively avoids potential security risks such as path failure and node leakage. Furthermore, generative adversarial networks introduce noise and environmental features, increasing the randomness and complexity of the generated keys and reducing the risk of key cracking.

[0328] 3. Real-time Feedback and Closed-Loop Optimization: The dynamic key generation and distribution method based on improved GAN-RL described in this invention utilizes a reinforcement learning experience replay mechanism and a real-time feedback mechanism to adjust and optimize based on the actual results during the distribution process (such as latency, packet loss rate, node response, etc.). After each distribution task, the system collects anomaly information and feeds it back to the generative adversarial network and reinforcement learning model to ensure that the next key distribution is more efficient and adaptable to the current network environment.

[0329] 4. Flexible Key Generation and Distribution Methods: The dynamic key generation and distribution method based on improved GAN-RL described in this invention supports multiple key distribution methods, including unicast, multicast, and groupcast. By intelligently selecting the most suitable distribution path and method, network congestion can be effectively reduced, distribution success rate can be improved, and the specific needs of different devices and nodes can be met. Attached Figure Description

[0330] Figure 1 This is a flowchart of the dynamic key generation and distribution method based on the improved GAN-RL in an embodiment of the present invention;

[0331] Figure 2 This is a topology diagram of the dynamic key generation and distribution method based on the improved GAN-RL in this embodiment of the invention;

[0332] Figure 3 This is a flowchart of the key generation process for the dynamic key generation and distribution method based on the improved GAN-RL in the verification experiment of this invention. Detailed Implementation

[0333] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0334] like Figure 1 As shown, the dynamic key generation and distribution method based on improved GAN-RL described in this invention includes the following steps:

[0335] S1. Acquisition and initialization of communication environment characteristics;

[0336] In this embodiment, S1 includes the following sub-steps:

[0337] S11. Collect the topology of the communication network;

[0338] The current communication network topology is collected by network sensors or monitoring equipment, including the number of network nodes, the connection relationships between nodes, and the topology type (such as tree, grid, star, etc.), and this information is recorded for analysis of the overall network structure and data flow.

[0339] like Figure 2 The diagram shown is a topology diagram provided by the dynamic key generation and distribution method based on the improved GAN-RL described in this application.

[0340] First, sensor devices are deployed at various nodes of the communication network (such as routers, switches, servers, etc.). These devices collect connection information between nodes in real time to obtain the connection status of the nodes. .here This represents the connection between node i and node j, where:

[0341]

[0342] By collecting the connection information between each node one by one, the following connection matrix is ​​generated. :

[0343]

[0344] Each row or column in the matrix represents the connection relationship between a node and other nodes, with 1 indicating a connection and 0 indicating no connection.

[0345] Next, a network crawling algorithm (such as Depth-First Search (DFS) or Breadth-First Search (BFS) is used to traverse the global connectivity of the network and identify the set of direct neighbors for each node. ,in:

[0346]

[0347] Based on the connection matrix C, the adjacent nodes of each node can be obtained:

[0348] The set of adjacent nodes of node A is .

[0349] The set of adjacent nodes of node B is .

[0350] The set of adjacent nodes of node C is .

[0351] The set of adjacent nodes of node D is .

[0352] Then, while collecting connection information, the characteristics of the communication links between nodes are recorded, including:

[0353] Link type: Confirm whether the link is unidirectional (Simplex) or bidirectional (Full Duplex).

[0354] Link bandwidth Communication bandwidth between data acquisition nodes:

[0355]

[0356]

[0357]

[0358]

[0359] Based on this data, bandwidth matrix B is formed:

[0360]

[0361] Next, combining the connection matrix C and the bandwidth information B, the topology of the communication network is constructed.

[0362] Among them, the node set Represents all nodes in the network:

[0363] ={A,B,C,D}

[0364] edge set Indicates the communication link between nodes:

[0365] The bandwidth of edge (A,B) is 100 Mbps.

[0366] The bandwidth of edge (B,C) is 50 Mbps.

[0367] The bandwidth of edge (C,D) is 150 Mbps.

[0368] The bandwidth of edge (A,D) is 200 Mbps.

[0369] Finally, based on the collected data, a topology map is drawn, such as... Figure 2 As shown in the figure, the network structure and link characteristics are clearly illustrated, providing complete basic data for subsequent optimization of key generation and distribution strategies.

[0370] S12. Collect network node information;

[0371] Collect specific attribute information for each node in the network, including the number of nodes, node type (such as router, server, terminal device), its function (such as relay, computing or storage), and the connection bandwidth and communication method between nodes.

[0372] First, enable the node information collection function in the network monitoring equipment to obtain basic attribute information of each node in the network, including the number and type of nodes, via the SNMP protocol. Assuming the network contains N nodes, use a set... Represents a set of nodes.

[0373] Next, for each node Collect its specific attribute information, including:

[0374] (1) Node type:

[0375] The device type identifier (such as OID information or device category) determines whether a node is a router, switch, server, or terminal device.

[0376] Use a label Represents a node Type:

[0377]

[0378] (2) Node functions:

[0379] Determine the network function that the node performs (such as relay, computing, or storage).

[0380] Use sets Represents a node Functions:

[0381]

[0382] (3) Connection bandwidth:

[0383] From node Bandwidth to other nodes Information (collected from S11) and the total connection bandwidth of the nodes are recorded:

[0384]

[0385] in Represents a node and The bandwidth between.

[0386] (4) Communication method:

[0387] The communication method of the acquisition nodes (unicast, multicast, or groupcast) is identified by the identifier. express:

[0388]

[0389] Finally, the above data is integrated into a node attribute table A. Each row in the table represents the specific attributes of a node, including type, function, connection bandwidth, and communication method.

[0390] Node Type Function Total connection bandwidth Communication mode <![CDATA[v2]]> Router Router {Relay, Storage} 300 Mbps Unicast <![CDATA[v2]]> Server {Compute} 250 Mbps Multicast <![CDATA[v3]]> Terminal device {Storage} 150 Mbps Unicast <![CDATA[v4]]> Router {Relay, Storage} 350 Mbps Multicast

[0391] S13. Collect network status information;

[0392] Continuous monitoring of communication networks is performed using network monitoring tools (such as SNMP, NetFlow, Wireshark, etc.), collecting key metrics related to network performance in real time. These metrics reflect the real-time status of the network and can reveal potential bottlenecks and instabilities in network operation.

[0393] The specific network status information collected includes the following:

[0394] (1) Network latency:

[0395] This measure collects the communication latency (also known as delay) between nodes in the network. Network latency refers to the time required for data to travel from the source node to the destination node, and is usually measured by round-trip time.

[0396] For each pair of nodes and ,Delay It can be calculated using the following formula:

[0397]

[0398] in From node To the node Round-trip time. Delay value. It will be collected and recorded for subsequent analysis.

[0399] (2) Packet loss rate:

[0400] Packet loss rate represents the proportion of data packets lost in a network, and is typically measured by network monitoring tools by periodically sending probe packets. In network monitoring systems, packet loss rate... This refers to the node To the node The packet loss rate is calculated using the following formula:

[0401]

[0402] This metric reflects the stability of the network and the reliability of data transmission. A high packet loss rate may indicate a fault or congestion in the network.

[0403] (3) Bandwidth utilization:

[0404] Bandwidth utilization refers to the actual usage of network links, reflecting whether the communication links between nodes are approaching saturation. The calculation formula is:

[0405]

[0406] in It is a link ( , Actual bandwidth usage on ) This is the total bandwidth of the link. Excessive bandwidth utilization usually indicates that the link is nearing saturation, which may lead to performance degradation.

[0407] (4) Network congestion:

[0408] Network congestion is a comprehensive metric calculated by monitoring information such as latency, packet loss, and bandwidth utilization among multiple nodes. It is used to assess the overall load of a network. In network monitoring systems, congestion is... The weighted average can be expressed by the following formula:

[0409]

[0410] in, , , These are the weights for latency, packet loss rate, and bandwidth utilization, respectively. , , These are the latency, packet loss rate, and bandwidth utilization of the i-th link, respectively, and N is the number of links in the network.

[0411] S14. Collect device attribute information;

[0412] First, hardware attribute information of each device in the network is collected in real time through dedicated monitoring tools or operating system interfaces, including the device's computing power, memory size, and storage capacity. The device's computing power is mainly evaluated through CPU performance and the number of cores. The CPU frequency (unit: GHz) and total memory capacity (unit: GB) can be obtained through interfaces provided by the operating system (such as procfs or Windows WMI).

[0413] Secondly, collecting device battery status and power consumption information is crucial, especially for mobile or low-power devices. Real-time monitoring of remaining battery power and charging status is essential. Battery information can be obtained through operating system-provided battery monitoring tools (such as Windows' BatteryStatus API or Linux's / sys / class / power_supply directory), showing remaining power (in %) and charging rate. Remaining power... The calculation formula is:

[0414]

[0415] Battery consumption rate It can be estimated using the following formula:

[0416]

[0417] in, The data represents the change in device battery power, with Δt representing the time interval (in minutes). This data allows for real-time monitoring of the device's battery status, providing a basis for adjusting key generation and distribution strategies when battery power is low.

[0418] Finally, using specialized energy monitoring tools or hardware modules, the device's power consumption can be monitored in real time, including the power consumption of components such as CPU, memory, storage, and network. The total energy consumption of the device... This can be obtained by calculating the sum of the power consumption of each component:

[0419]

[0420] in, Let N represent the power consumption of the i-th component (e.g., CPU, memory, hard drive), and N be the number of components monitored in the device. This data allows for a comprehensive assessment of the device's energy efficiency and power consumption, providing strong support for optimizing key generation and distribution strategies under resource constraints.

[0421] S15. Feature data standardization processing;

[0422] The various types of data collected in steps S11 to S14 are preprocessed. Since these data types are diverse (e.g., topology is graph data, while network status and device attributes are numerical data), they need to be standardized to ensure effective integration in subsequent models.

[0423] (1) Standardization of topology data: Topology is usually represented by a connection matrix C, where each element This reflects whether nodes i and j are directly connected. To input the topological data into the model, the connection matrix C can be transformed into a flattened one-dimensional vector. In this way, the connection information between each node is uniformly represented by numerical values. This flattened connection matrix can be implemented as follows:

[0424]

[0425] Here, Flatten represents the transformation of a two-dimensional connection matrix into a one-dimensional vector. This standardized connection information can be input into generative adversarial networks (GANs) and reinforcement learning models.

[0426] (2) Standardization of node information: Node information includes the number of nodes, node type, and node function. The number of nodes N can be directly used as an input feature. For node type (such as router, server, etc.), it can be converted into a one-hot encoding vector. For node function (such as computation, storage, forwarding, etc.), one-hot encoding or numerical encoding is also used for standardization:

[0427]

[0428] in, This is a label for the type or function of a node.

[0429] (3) Standardization of network state data: Network state data (such as latency, packet loss rate, bandwidth utilization, etc.) usually have different dimensions and numerical ranges. To ensure that these data can play a role in the training process, they need to be normalized so that all network state data are within the same numerical range. Z-score standardization (zero mean, unit variance) can be used to process these data.

[0430]

[0431] Where X represents the original data, μ represents the mean of the data, and σ represents the standard deviation of the data. This standardized network state data will be within a uniform scale, which is beneficial for subsequent model training.

[0432] (4) Standardization of Equipment Attribute Data: Equipment attribute data includes computing power, memory size, remaining power, and energy consumption. This data is usually numerical and can be processed using Min-Max standardization or Z-score standardization. Min-Max standardization maps the data to the [0, 1] interval.

[0433]

[0434] in, and These are the minimum and maximum values ​​of the feature, respectively. This standardization method ensures that the performance data of all devices are within the same scale, facilitating subsequent analysis.

[0435] (5) Data merging and construction of multidimensional input features: After the above standardization process is completed, all feature data (topology, node information, network status, device attributes) will be merged into a unified multidimensional feature vector. This feature vector will serve as input to subsequent Generative Adversarial Networks (GANs) and reinforcement learning models. The dimensions of the merged multidimensional input feature vector can be represented as:

[0436]

[0437] in For standardized topology data, For node information, For standardized network status data, This refers to standardized device attribute data.

[0438] Through the above steps, all data has been standardized and converted into a unified format, ensuring efficient and stable training in subsequent model training, thereby improving the performance of generative adversarial networks (GANs) and reinforcement learning models.

[0439] S2. Key generation based on improved generative adversarial networks;

[0440] like Figure 3 The diagram shown is a flowchart of the key generation process for the dynamic key generation and distribution method based on improved GAN-RL described in this application.

[0441] S21. Input random noise and communication environment features into the generator of the generative adversarial network (GAN);

[0442] First, the generator in a Generative Adversarial Network (GAN) requires an initial input, which is random noise z. This noise provides randomness to the generated key, ensuring that each key generation is unique. To obtain sufficient randomness, it is typically sampled from a standard normal distribution, resulting in a noise vector z of length N:

[0443]

[0444] Where N is the dimension of the noise vector, and each element They were all independently drawn from the standard normal distribution.

[0445] Next, to ensure that the generated key is not only random but also adaptable to the specific needs of the current communication network, network communication environment feature data is also provided as input to the generator. These communication environment features include the network topology, node information, network status (such as bandwidth and latency), and device attributes (such as computing power and energy consumption), which help the generator better understand the network environment. These communication environment features are represented as a vector e of size M, where each... It refers to a specific characteristic of the network, such as bandwidth or latency.

[0446]

[0447] Therefore, the length M of the communication environment feature e depends on the network information collected in step S1. Specifically, M represents the number of network communication environment features collected in steps S11 to S14. These features mainly reflect the network topology, bandwidth, latency, and other key information.

[0448] In order for the generator to generate a key that meets the requirements based on the characteristics of the communication environment and random noise, the two types of input data are fused to obtain a new input vector x:

[0449]

[0450] This vector x consists of two parts: random noise and communication environment characteristics, with a length of N+M. In this way, the generator can both utilize noise to ensure the unpredictability of the key and adjust the generated key according to the communication environment characteristics, thereby making it more suitable for the current communication network requirements.

[0451] Next, the fused input vector x is fed into the generator of a Generative Adversarial Network (GAN). The generator uses a deep neural network to process the input, performing a series of nonlinear transformations to finally generate a key k. This key not only maintains sufficient randomness but also matches the bandwidth, latency, and other characteristics of the current network. The generator process can be represented as follows:

[0452]

[0453] Among them, G( ) represents the mapping function of the generator. The parameters of the generator are adjusted through training. The generator's output key k is then passed as a candidate key to the discriminator for quality evaluation.

[0454] S22. The generator outputs candidate keys and passes them to the discriminator;

[0455] In step S21, the generator has received the fused input vector x and generated a candidate key k based on this input. The generator's goal is to produce a key with high randomness and adaptability to the current communication network environment through nonlinear transformations and multi-layer neural network computations. However, this generated key may not fully meet the key quality requirements. Therefore, the generator passes the generated candidate key to the discriminator for further evaluation.

[0456] First, the candidate key k is passed to the discriminator D( The discriminator's task is to evaluate the quality of the key, determining whether it is sufficiently random and resistant to prediction. The discriminator assesses metrics such as the key's entropy, distribution characteristics, and resistance to prediction to determine its quality. Entropy is an important indicator of key randomness and uncertainty, typically calculated using the following formula:

[0457]

[0458] in Let be the probability distribution of the i-th bit of key k, and n be the length of the key. The higher the entropy value, the stronger the randomness and unpredictability of the key, and thus the stronger its resistance to attacks.

[0459] In addition, the discriminator also evaluates the key based on other characteristics, such as its distribution. For example, the bit distribution of the key should be relatively uniform to avoid any regularity, which is crucial for preventing traditional cryptanalysis (such as frequency analysis). For the generated key, the discriminator may use statistical methods, such as chi-square test and Kullback-Leibler divergence, based on its distribution characteristics to measure the uniformity of the key.

[0460] The evaluation process of the discriminator can be represented as:

[0461]

[0462] Among them, Evaluate( ) is the evaluation function of the discriminator, which outputs a scalar value representing the quality of the key.

[0463] The discriminator evaluates the quality of the generated key and outputs a score. .

[0464] The score is calculated based on the following four indicators:

[0465]

[0466] in, The entropy randomness of the key. For uniformity of distribution, This represents the autocorrelation suppression degree. To predict resistance. The weights of each indicator are as follows: .

[0467] if If the key is of high quality, it is considered to be stored in the key pool.

[0468] if Then the feedback is sent to the generator for fine-tuning; if If the key is not found, discard it and the generator will regenerate it.

[0469] If the discriminator deems the generated key of high quality, indicating sufficient randomness and distribution characteristics, it will output a high evaluation value, signifying that the key meets the requirements. However, if the discriminator deems the key of poor quality, the generator will receive feedback from the discriminator and then adjust its algorithm to improve the generation process.

[0470] S23. The discriminator feeds back the evaluation results to the generator to optimize the generator's parameters;

[0471] In step S22, the generator generates candidate keys based on random noise and communication environment characteristics, and then performs quality evaluation by a discriminator. The discriminator's evaluation result determines whether the key meets the requirements based on multiple dimensions such as key entropy, distribution characteristics, and anti-predictability. If the discriminator deems the generated key unqualified, the generator needs to adjust its internal parameters to optimize the generation process and generate higher-quality keys.

[0472] To achieve this goal, the discriminator's evaluation results are fed back to the generator as feedback signals to update the generator's network parameters. Specifically, the generator and discriminator optimize together during training using an adversarial training approach. This approach is based on gradient descent, where the generator aims to maximize the discriminator's error, while the discriminator aims to maximize its ability to distinguish between real and generated keys. Through this game-like interaction, they continuously adjust their parameters until they reach a balance.

[0473] (1) The feedback mechanism and optimization process are as follows:

[0474] The evaluation results are fed back as an error signal: discriminator The output evaluation value reflects the quality of the key. If the discriminator deems the key quality poor, its output value will be low. Conversely, a high evaluation value indicates good key quality. The discriminator's output results. This error signal is fed back to the generator to guide its optimization. The generator's optimization goal is to maximize the discriminator's score, making the generated key increasingly difficult for the discriminator to distinguish as a "fake key."

[0475] (2) Generator parameter update: The generator updates the network parameters using backpropagation and gradient descent. After each key generation, the generator calculates the loss function. , representing the generator's loss. This loss function is calculated based on the discriminator's evaluation results, as shown below:

[0476]

[0477] Where E represents the expected value, and k is the candidate key generated by the generator. and These are the parameters for the generator and the discriminator, respectively. It is the discriminator's evaluation output of the key k.

[0478] The generator optimizes its parameters by minimizing the loss function, thereby improving key quality. This process can be achieved using gradient descent.

[0479]

[0480] Where η is the learning rate. It is the gradient of the generator loss function with respect to its parameters.

[0481] (3) Optimization of target feedback adjustment: In adversarial training, the generator and discriminator are in a game-like state. The generator continuously adjusts its parameters to improve the quality of its generated keys, making it increasingly difficult for the discriminator to determine the authenticity of the keys. As training progresses, the quality of the keys generated by the generator will gradually improve, eventually reaching a standard that can meet the encryption requirements of communication networks.

[0482] S24. Once the generated key meets the set quality standard, it is stored in the key pool and awaits distribution.

[0483] The optimized generator will produce keys that meet quality standards. The discriminator will then perform a final evaluation of the generated keys. If the key's entropy, distribution characteristics, and anti-predictability meet predetermined security standards, the key will be considered qualified. At this point, the generated key k will be stored in the key pool, awaiting subsequent distribution operations.

[0484] The generated key pool can be represented as follows:

[0485]

[0486] Where K is the key pool, containing multiple generated keys, each key They all possess a certain degree of randomness and security, which can meet subsequent distribution needs.

[0487] S3, Reinforcement Learning Model Based on Communication Environment Features;

[0488] S31. Construct the state space and action space of the reinforcement learning model.

[0489] In the dynamic key generation and distribution method based on improved GAN-RL described in this application, the core of the reinforcement learning model design lies in how to dynamically generate the optimal key distribution strategy based on the feature data of the communication environment. To achieve this goal, it is first necessary to construct the state space (S) and action space (A) of the reinforcement learning model. These two spaces will directly determine how the model perceives the network environment and how it selects the key distribution strategy.

[0490] (1) Construction of the state space

[0491] The state space refers to the environmental information that a reinforcement learning model can perceive at each decision step. In the dynamic key generation and distribution method based on improved GAN-RL described in this application, the state space consists of several key communication environment features, specifically including the following types of data:

[0492] Network topology information: Based on the connection relationships between nodes obtained in step S11, the network topology can be represented as an adjacency matrix C:

[0493] here This reflects whether nodes i and j in the network are directly connected. Topological information helps the model understand the connection relationships between network nodes.

[0494] Node information: The attributes of each node, such as node type (router, terminal device, etc.), computing power (e.g., CPU, memory), and functions (e.g., computing, storage), constitute the node's feature vector. :

[0495]

[0496] Network status information: This includes real-time network status, such as bandwidth utilization, latency, and packet loss rate. These dynamic characteristics reflect the current performance of the network. Assumptions This represents the bandwidth of network node i. Represents latency and packet loss rate. Then it is:

[0497]

[0498] Device attribute information: This includes node power consumption information, remaining battery power, etc. This information is particularly important for key distribution tasks, especially for battery-powered devices. Device status can be represented as:

[0499]

[0500] After standardization, this information forms the states in the state space S. Each state is a combination of multiple feature vectors, for example:

[0501]

[0502] In this way, the state space contains multi-dimensional communication environment information, enabling reinforcement learning models to perceive the current network environment more comprehensively.

[0503] (2) Construction of action space

[0504] The action space defines all possible actions that a reinforcement learning model can choose. In the key distribution problem, the action space mainly consists of the following:

[0505] Path selection: Given a network topology, the model needs to select the optimal key distribution path. This can be unicast, multicast, or broadcast. Assume there are multiple paths. The choice of each path can be represented as a discrete action. :

[0506]

[0507] Node selection: Determining which target nodes need to receive the key. This selection depends on the nodes' computing power, load status, etc. The node selection action can be represented as:

[0508]

[0509] in It is a node selected from the current network state that is suitable for receiving the key.

[0510] Priority assignment: For multiple candidate nodes, the model may need to assign different priorities to them based on factors such as node security and load.

[0511] The action space combines path selection, node selection, and priority allocation. The choice of each action determines how the key is distributed to the target node.

[0512] S32, Training the reinforcement learning model

[0513] The purpose of training a reinforcement learning model is to enable it to learn the optimal key distribution strategy through repeated training on simulations and historical data. The training process includes initializing the model and adjusting the strategy through continuous feedback to achieve optimal decisions in dynamic network environments. The following are detailed steps:

[0514] (1) Initialization of the training environment

[0515] First, to train a reinforcement learning model, a simulation environment must be set up to mimic a real communication network environment. The construction of the training environment depends on the state space S and action space A constructed in step S31, and needs to include the following key elements:

[0516] Network state simulation: Based on actual collected data or hypothetical datasets, various states in a communication network are simulated, such as bandwidth, latency, and packet loss rate. This data is used to simulate changes in various network conditions, enabling the reinforcement learning model to adapt to different network states.

[0517] Node and device status simulation: Simulate the working status and energy consumption of different nodes (such as routers, switches, and terminal devices), as well as the connection relationships between nodes. These device states will affect the key distribution strategy and effectiveness, so they must be considered during training.

[0518] Reward function design: Define the reward function This is used to measure the effectiveness of each policy after execution. The reward function is designed to guide the model towards the optimal goal. For example, a higher reward is given if the key is successfully distributed and the network load is below a set threshold; a lower reward is given if key distribution fails or network latency is high. A possible reward function form is as follows:

[0519]

[0520] in, A positive reward is given when the key is successfully distributed. A negative reward is given when network latency is high. A negative reward is given when energy consumption is too high.

[0521] (2) Training process of reinforcement learning model

[0522] The training process is based on reinforcement learning algorithms (such as Q-learning, Deep Q-Network (DQN), or policy gradient methods). The core objective is to continuously learn the optimal key distribution policy through interaction with the environment. The training process includes the following steps:

[0523] Sampling State and Selecting Action: At each time step t, the model starts from the current state. Choose one action And apply it to a simulated environment to obtain rewards. and the next state :

[0524]

[0525] in, This indicates that the policy π is based on the state. The action to choose.

[0526] Update the Q-value or policy: Based on the selected action and environmental feedback, update the Q-value function (for Q-learning) or the policy network (for policy gradient methods). For Q-learning, the update formula is:

[0527]

[0528] Where α is the learning rate and γ is the discount factor. It's a reward. It is the maximum Q value in the next state.

[0529] For the policy gradient method, updating the parameters θ of the policy network is achieved by maximizing the expected reward:

[0530]

[0531] in, It is the gradient of the policy, representing the expected reward for choosing an action in a given state.

[0532] Training iterations: Through repeated training iterations, the model will make optimal key distribution decisions in a simulated environment under different network conditions (such as network latency, bandwidth, node load, etc.). The training goal is to enable the model to optimize key distribution performance in complex network environments and avoid overfitting or inefficient decisions.

[0533] (3) Experience playback and exploration mechanism

[0534] To improve the training performance of the model, the dynamic key generation and distribution method based on improved GAN-RL described in this application introduces experience replay and ε-greedy policy mechanisms. These mechanisms enable the reinforcement learning model to explore the policy space more effectively and improve learning efficiency.

[0535] Experience replay: In reinforcement learning, experience replay refers to storing the model's historical states, actions, rewards, and next state (a four-tuple) in an experience pool. During each training session, a batch of historical experiences is randomly selected from this pool for training. This breaks down the correlation between data points and improves learning efficiency. The formula is as follows:

[0536]

[0537] Where D is the experience pool, ( A tuple consists of state, action, reward, and next state. By randomly sampling data from an experience pool, the model can learn from diverse experiences, avoiding overfitting.

[0538] ε-greedy strategy: This strategy balances exploration and exploitation. During training, the model will explore with a certain probability ( ) Select a random action (exploration) to discover potential better strategies; while in the remaining probability ( Under these conditions, the model will select the current optimal action (utilization). As training progresses, As the value gradually decreases, the model relies more on the learned optimal policy. The formula is:

[0539]

[0540] This approach allows the model to explore new distribution strategies as well as select the optimal solution from known strategies, thus achieving better convergence.

[0541] S4. Real-time adjustment and optimization of the theoretical strategies for generating reinforcement learning models;

[0542] S41. Key Distribution Strategy Optimization Based on Reinforcement Learning

[0543] In S3, the reinforcement learning model has already obtained a key distribution strategy that can adapt to various network environments through extensive training and feedback mechanisms. However, in S41, the model needs to adjust and optimize these strategies based on real-time network conditions and key characteristics to adapt them to the specific requirements of the current task.

[0544] First, the reinforcement learning model receives real-time state information from the communication network. This information includes network performance metrics such as bandwidth, latency, and packet loss rate, which directly affect the efficiency and security of key distribution. To quantify the quality of each path, the model uses the following formula to evaluate the network health status:

[0545]

[0546] in, It is the network latency of path p. It is the bandwidth of path p. It is the packet loss rate of path p. , , These are weighting coefficients that represent the impact of latency, bandwidth, and packet loss rate on path quality.

[0547] The model evaluates the quality of each path by calculating network health metrics, and ultimately selects the path with the highest health score for key distribution.

[0548] Besides network conditions, the characteristics of the key itself also affect the distribution strategy. The key type (e.g., symmetric key, asymmetric key), length, and encryption strength determine the security and efficiency of key distribution. Therefore, the model also needs to evaluate key suitability based on the key's characteristics.

[0549]

[0550] Here, Type represents the key type rating (e.g., 1 for symmetric key, 2 for asymmetric key). Length is the key length; longer keys generally mean stronger security. Strength is the encryption strength of the key, usually measured based on the security of the algorithm. , , It is a weighting coefficient that reflects the impact of each key characteristic on adaptability.

[0551] Using this formula, the model can determine the suitability of the key in the current network environment, and thus decide whether to select the key for distribution.

[0552] After obtaining the health status of the network paths and the suitability of the keys, the reinforcement learning model needs to select the optimal key distribution path and method based on this information. The model's goal is to select a path with minimum latency, optimal bandwidth, and minimum packet loss rate, and to choose an appropriate distribution method (unicast, multicast, or groupcast) according to the task requirements. To ensure efficient and secure distribution, the model calculates a comprehensive score for each path using the following formula and selects the path with the highest score:

[0553]

[0554] Here, P is the set of all possible paths. The goal is to minimize the path latency, packet loss rate, and bandwidth bottleneck.

[0555] Next, the model will select an appropriate distribution method based on the number of target nodes. If the number of target nodes is small, the model may choose unicast distribution; while if the number of target nodes is large, the model will choose multicast or groupcast. This decision is made according to the following rules:

[0556]

[0557] in, This refers to the number of target nodes. Unicast is suitable for a small number of target nodes and provides high security; while multicast and broadcast are suitable for large-scale distribution, which can improve efficiency and reduce network load.

[0558] In addition, the model needs to assign a priority to each node, especially for critical or high-risk nodes, which should be prioritized for key distribution. The priority allocation is based on the following formula:

[0559]

[0560] in, This represents the priority of node n. A critical node has a priority of 1, meaning that this node should have priority in obtaining the key.

[0561] S42. Reinforcement learning iteratively optimizes the distribution strategy through exploration and experimentation in the action space.

[0562] In step S41, the reinforcement learning model formulates an initial key distribution strategy by evaluating factors such as the health of network paths and key suitability. Next, step S42 aims to further optimize this strategy through action space exploration and experimentation in reinforcement learning. The core of this step is to use exploration and feedback mechanisms to allow the model to continuously adjust and improve its distribution strategy to adapt to dynamically changing network environments and task requirements.

[0563] (1) Exploration and feedback of action space

[0564] In reinforcement learning, the action space is the set of all actions the model can choose. In key distribution strategies, the action space includes different path selections, distribution methods (such as unicast, multicast, and groupcast), and node priority adjustments. The model adjusts its strategy by experimenting in a simulated environment, selecting different actions, and observing environmental feedback.

[0565] For each time step t, the model starts from the current state. Choose an action This is then applied to the environment. In key distribution tasks, the action space includes not only path selection and distribution methods, but also priority adjustments for different nodes. Specifically, the action space of reinforcement learning can be represented as:

[0566]

[0567] Where p∈P represents the set of available network paths. m represents the distribution method, which can be unicast, multicast, or multicast. n∈{0,1} represents the priority of node n, where 0 represents a normal node and 1 represents a critical or high-risk node.

[0568] In each training iteration, the model adjusts according to the current state. Choose an action And perform this action in a simulated environment.

[0569] (2) Reward feedback and strategy update

[0570] Each time the model selects an action and applies it to the network, the system returns a feedback signal (reward). The design of the reward function is crucial, as it reflects the effectiveness and quality of the action. In key distribution tasks, the reward function considers multiple factors, such as the success or failure of key distribution, network latency, bandwidth utilization, and energy consumption. The reward function can be designed in the following form:

[0571]

[0572] Among them, Success_Reward is the reward value after successful key distribution; Delay is the network latency, the greater the latency, the higher the penalty; Energy is the energy consumption during the distribution process, the higher the energy consumption, the greater the penalty; Risk is the security risk of the network or node, the higher the risk, the greater the penalty. , , These are weighting coefficients, which adjust the impact of various factors on the reward function.

[0573] Through a reward function, the model can obtain feedback information about the current action and adjust its policy based on this feedback. The goal of reinforcement learning is to maximize the long-term cumulative reward, i.e., to optimize the key distribution policy.

[0574] (3) Action selection strategy and ε-greedy strategy

[0575] To strike a balance between exploration and exploitation, a commonly used strategy in reinforcement learning is the ε-greedy strategy. This strategy gradually reduces randomness during training, allowing the model to transition from extensive exploration in the early stages to local optimization in later stages. Specifically, the model has... The probability of choosing a random action (exploration) is 1 - ε, while the probability of choosing the optimal action under the current policy (exploitation) is 1 - ε.

[0576]

[0577] in, It represents the probability of exploration, which gradually decays during training to ensure that the model can explore a wide range of different strategies in the early stages and make decisions using the best strategies it has learned in the later stages.

[0578] By striking a balance between exploration and utilization, the model can not only avoid getting stuck in local optima during training, but also gradually converge to the global optimum.

[0579] (4) Strategy optimization and convergence

[0580] After each exploration and feedback iteration, the reinforcement learning model updates its policy based on the new state and reward information. For Q-learning, the update formula is as follows:

[0581]

[0582] in, Indicates the state Next action The Q value; α is the learning rate, which controls the step size of each update; γ is the discount factor, which measures the weight of future rewards; The next state The maximum Q value.

[0583] For policy-based algorithms (such as policy gradient methods), policy updates are performed using the following formula:

[0584]

[0585] in, These are the parameters of the current strategy; It is the policy gradient, representing the gradient at state 100°. The expected reward for the next action selection.

[0586] Through this iterative update process, reinforcement learning models can continuously adjust their strategies, optimize key distribution paths and methods, and maximize the success rate and network performance during the distribution process.

[0587] S43. Generate the final key distribution scheme and pass it to the distribution execution module.

[0588] After repeated training and optimization, the final key distribution strategy will be gradually improved based on S42, resulting in the final key distribution scheme. This scheme will include:

[0589] Selected distribution path: The optimal path selection ensures that key distribution can be carried out efficiently and reliably.

[0590] Distribution method and nodes: Determine whether to use unicast, multicast or multi-cast, and prioritize key nodes for distribution.

[0591] Key distribution timing and frequency: Determine the timing and frequency of key distribution based on network conditions, node requirements, etc., in order to minimize security risks.

[0592] The final distribution scheme will be passed to the distribution execution module so that the actual key distribution task can be performed.

[0593] S5, Key Distribution Execution and Feedback Optimization.

[0594] S51. Distribute the generated key to the target node according to the distribution strategy output by the reinforcement learning model, and monitor the node response and distribution result in real time during the distribution process;

[0595] In the dynamic key generation and distribution method based on improved GAN-RL described in this application, the key task of S51 is to successfully transmit the generated key to the target node according to the optimal distribution strategy output by the reinforcement learning model in stage S4, and to monitor the node response, distribution success rate, latency, and other indicators in real time during the distribution process. The core objective of the entire process is to ensure that the key can be transmitted efficiently and stably, and to adjust the strategy in a timely manner to deal with potential problems and minimize anomalies that occur during transmission.

[0596] First, the optimized distribution strategy of the reinforcement learning model is obtained from stage S4. This strategy selects the optimal target node and path based on network state, node requirements, and key characteristics. Specifically, it includes:

[0597] Target node selection: The reinforcement learning model determines which nodes should receive the key first by considering factors such as node load and communication requirements. Assume the node set is N = { , ,..., The model will select the set of nodes Ntarget that best meets the requirements. N.

[0598] Distribution path selection: Based on the location of the target node and the network topology, the optimal transmission path is selected. Path selection is typically determined based on factors such as latency (d), bandwidth (b), and stability (s). The goal of path selection is to minimize latency and maximize bandwidth, ensuring efficient key transmission.

[0599] Assume the target node selection function is:

[0600]

[0601] in, It is node load. It's a communication requirement. It's about node stability. , , It is the weight of each factor.

[0602] Then, based on the selected target node and path, the key transmission begins. At this point, the generated key is transmitted to the target node via the selected path. During this process, the following transmission methods can be selected:

[0603] Unicast: Transmits the key directly to a single target node.

[0604] Multicast: Transmits the key to multiple nodes simultaneously.

[0605] Multicast: Transmitting a key to a group of nodes.

[0606] Key transmission can be secured using encryption protocols such as TLS, ensuring the confidentiality of the key during transmission.

[0607] Next, during the key transmission process, the system will monitor the following key indicators in real time to ensure the smooth progress of the distribution process:

[0608] Node Response: Monitors whether the target node has successfully received the key. If the node does not respond or communication fails, the system will mark it as an anomaly and prepare for the next step of anomaly handling.

[0609] Distribution delay: Real-time tracking of key transmission delay to ensure it does not exceed a set threshold. The delay can be calculated using the following formula:

[0610]

[0611] in, It is the time when the target node receives the key. It is the time when the key starts being sent. This is the target number of nodes.

[0612] Distribution success rate: Calculates the percentage of nodes that successfully receive the key. If the percentage of nodes that successfully receive the key falls below a predetermined threshold, the system will trigger an alarm and reassess the distribution strategy.

[0613]

[0614] in, This is the number of nodes that successfully received the key. This represents the total number of target nodes.

[0615] Distribution failure rate: The proportion of nodes that fail to distribute keys. This also needs to be monitored in a timely manner to avoid a high failure rate during key distribution. It is calculated as follows:

[0616]

[0617] in, This is the number of nodes that failed to distribute the data. This represents the total number of target nodes.

[0618] Finally, during the monitoring process, if any issues such as excessive latency, unresponsive nodes, or distribution failures are detected, the system will record the relevant data in real time and feed these anomalies back to the reinforcement learning model for subsequent optimization. Specific feedback includes:

[0619] Delay information: Provides real-time feedback on distribution delays and optimizes the distribution path based on the current network status.

[0620] Node response failure information: This indicates that the target node has failed to respond, and you may need to select a different path or node.

[0621] Distribution failure information: Reports nodes where distribution failed, for reference in subsequent strategy adjustments.

[0622] The system optimizes the distribution path and method through this real-time feedback, making the next round of distribution more efficient and stable.

[0623] S52. If an anomaly occurs during the distribution process (such as high latency or node failure), the anomaly information is recorded and fed back to the reinforcement learning module. The reinforcement learning model updates its policy based on the feedback information.

[0624] In step S52, a key task is to ensure that any potential anomalies, such as node failures, path interruptions, and high latency, can be identified and handled in real time during key distribution. These anomalies not only affect the success rate of key distribution but may also threaten network security and key integrity. Therefore, it is essential to quickly capture and record these anomalies and transmit the feedback information back to the reinforcement learning module, enabling subsequent distribution strategies to be adjusted and optimized based on the latest network conditions.

[0625] The main steps are as follows:

[0626] (1) Anomaly Detection and Recording: During the key distribution process, the system continuously monitors and detects several key indicators. The main monitoring items include:

[0627] High latency: If the latency of a distribution path exceeds a set threshold, the system immediately marks it as abnormal. The latency calculation formula is:

[0628]

[0629] if Greater than the set maximum delay threshold If so, it is considered that a delay anomaly has occurred.

[0630] Node Failure: If the target node fails to respond to the key reception request within the specified time, or if the connection with the target node fails, the system will record node failure information. The node failure rate can be calculated using the following formula:

[0631]

[0632] if Exceeding the set threshold If so, it is marked as a node failure or anomaly.

[0633] Path interruption: If the path is lost or interrupted, preventing the key from being transmitted to the target node, the system will record the path interruption event and trigger an alarm.

[0634] (2) Anomaly Feedback to the Reinforcement Learning Module: Once an anomaly is detected, the system feeds back the anomaly information (such as latency, node failure, path interruption, etc.) to S4 (the reinforcement learning module). Specific feedback information includes:

[0635] Delay anomaly information: The system sends data with recorded delays exceeding the threshold to the reinforcement learning model, and the model adjusts its distribution path selection strategy based on this information.

[0636] Node failure information: If some nodes fail during key distribution, the system will report the failure information of these nodes and adjust the node selection and path optimization strategies through reinforcement learning models.

[0637] Path interruption information: When a path is interrupted, the system will record the path and the state at the time of the interruption, and feed this information back to the reinforcement learning model. The reinforcement learning model will then re-evaluate the path selection and backup strategy.

[0638] (3) Reinforcement learning models adjust their policies based on feedback: Upon receiving abnormal feedback, the reinforcement learning model updates its policy based on the current network environment and the feedback data. The update methods typically include the following aspects:

[0639] Path optimization: If some paths have poor latency or reliability, the model will optimize path selection to reduce reliance on these paths.

[0640] Node selection optimization: The reinforcement learning model will reselect more reliable nodes based on node failure data, thereby improving the distribution success rate.

[0641] Dynamically adjust distribution strategy: If certain distribution modes (such as multicast, groupcast, etc.) lead to high latency or failure rate, the model may adjust the distribution method to adapt to changes in the network environment.

[0642] The specific strategy update formula is as follows:

[0643] For the Q-learning method, the policy update formula is:

[0644]

[0645] Where α is the learning rate and γ is the discount factor. It's an instant reward. It is the value of the current state-action pair.

[0646] For the policy gradient method, the formula for updating the policy parameters is:

[0647]

[0648] in, These are the parameters of the current strategy. It is the gradient of the policy.

[0649] (4) Circular optimization of feedback mechanism: Throughout the process, the feedback of abnormal information and the policy update form a closed loop. After each round of key distribution, the system will continuously optimize the policy based on the execution results (including success rate, latency, node response, etc.) to ensure that the next round of distribution can adapt to changes in network status and ultimately achieve the goal of efficient and stable key transmission.

[0650] S53. After key distribution is completed, if the communication environment characteristics or distribution efficiency do not meet the expected values, the input conditions of the generative adversarial network are optimized through a feedback mechanism to further improve the adaptability of key generation and distribution.

[0651] The core objective of step S53 is to evaluate the communication environment characteristics and distribution efficiency during the key generation and distribution process after the key distribution is completed, and to optimize the input conditions of the generative adversarial network (GAN) through a feedback mechanism, thereby further improving the adaptability of key generation and distribution.

[0652] This process mainly includes two aspects:

[0653] (1) Evaluate the effectiveness of key distribution and communication environment characteristics: Evaluate key indicators (such as latency, success rate, packet loss rate, node response, etc.) and communication environment characteristics (such as network status, bandwidth, path conditions, etc.) during the distribution process. This mainly includes:

[0654] Distribution latency: The total time taken from key generation to receipt by the target node. Excessive latency can impact the network's real-time performance requirements.

[0655] Distribution success rate: Evaluate whether the key successfully reached all target nodes. If any node failed to distribute the key, the system must mark it as a failure and conduct further analysis.

[0656] Node response: Check the response speed and success rate of the target node. If the node fails or the response time is too long, it may affect the quality of key distribution.

[0657] Path stability: Evaluate the stability and reliability of the key distribution path. If the path is found to be unstable or interrupted, the path selection needs to be optimized.

[0658] These evaluation indicators can be quantified using the following formula:

[0659] Distribution delay:

[0660]

[0661] in, It is the time when the key was received. It is the time when the key is sent. This is the target number of nodes.

[0662] Distribution success rate:

[0663]

[0664] in, This is the number of nodes that successfully received the key. This represents the total number of target nodes.

[0665] (2) Optimizing the input conditions of GAN through feedback mechanism: Based on the above evaluation results, the system passes feedback information to the Generative Adversarial Network (GAN) to optimize its input conditions. The goal of the Generative Adversarial Network is to generate a key that meets the requirements, and its input conditions can include the following aspects:

[0666] The characteristics of the network's communication environment, such as bandwidth, latency, and packet loss rate, require GANs to generate more adaptable keys based on these characteristics.

[0667] Node status requirements: Each node may have different requirements, such as key length, encryption method, etc. GAN needs to generate personalized keys according to these requirements.

[0668] Path selection optimization: GAN also needs to generate suitable keys based on the stability and reliability of the network path to ensure that the keys can be transmitted efficiently and securely.

[0669] Specific optimization methods include:

[0670] Optimize key generation rules: By analyzing historical data, optimize the generation rules of GAN to generate keys that are more suitable for the current network environment and node requirements.

[0671] Adjusting network input characteristics: If certain communication environment characteristics have a negative impact on key distribution (such as high latency or high packet loss rate), the input characteristics of GAN can be adjusted so that the generated key can adapt to different network conditions.

[0672] Improve adaptability: By continuously optimizing the input conditions of GAN, the adaptability of the key generation process is improved, ensuring that keys can be distributed efficiently and securely under different network environments and node requirements.

[0673] The optimization formula for the feedback is as follows:

[0674]

[0675] in, It is the optimized GAN input. This is the current input condition. It is the amount of adjustment to the input conditions based on feedback information, and α is the learning rate.

Claims

1. A dynamic key generation and distribution method based on improved GAN-RL, characterized in that, The method comprises the following steps: S1, communication environment feature collection and initialization; the communication environment feature collection comprises collecting the topology structure of the communication network, collecting network node information, collecting network state information, and collecting equipment attribute information; S2, key generation based on an improved generative adversarial network; S21, inputting random noise and communication environment features into the generator of the generative adversarial network; S22, the generator outputs a candidate key and delivers it to the discriminator; S23, the discriminator feeds back the evaluation result to the generator to optimize the network parameters of the generator; The evaluation result of the discriminator is fed back to the generator as a feedback signal for updating the network parameters of the generator; the generator and the discriminator are optimized together in the training process in an adversarial training manner; S24, when the generated key reaches the set quality standard, it is stored in a key pool and waits for distribution; S3, communication environment feature input into a reinforcement learning model; S31, constructing a state space and an action space of the reinforcement learning model; S32, training the reinforcement learning model; The training process comprises adjusting the strategy through continuous feedback from the initial model to obtain the optimal decision in the dynamic network environment; S321, initialization of the training environment; first, the construction of the training environment is based on the state space and the action space constructed in step S31; S322, training process of the reinforcement learning model; The training process is based on a reinforcement learning method; the reinforcement learning method adopts Q learning, deep Q network DQN, or a policy gradient method; S323, experience replay and exploration mechanism; the experience replay and epsilon-greedy strategy mechanism are introduced in the reinforcement learning model; S4, real-time adjustment and optimization of the theoretical strategy generated by the reinforcement learning model; S41, key distribution strategy optimization based on reinforcement learning; First, the reinforcement learning model receives real-time state information from the communication network; the information refers to network performance indicators including the bandwidth, delay, and packet loss rate of the network; the model evaluates the network health condition by using the following formula: ； wherein, is the network delay of path p; is the bandwidth of path p; is the packet loss rate of path p; , , are weight coefficients representing the influence of delay, bandwidth and packet loss rate on the quality of the path. The model evaluates the pros and cons of each path by calculating the network health indicators of each path, and finally selects the path with the highest health degree for key distribution; S42, the reinforcement learning iteratively optimizes the distribution strategy through exploration and experiment of the action space; Through the exploration and feedback mechanism, the model continuously adjusts and improves its distribution strategy to adapt to the dynamically changing network environment and task requirements; S43, generating a final key distribution scheme and delivering it to a distribution execution module; After repeated training and optimization, the final key distribution strategy will be gradually improved based on step S42, and a final key distribution scheme will be generated; the scheme includes the selected distribution path, the distribution mode and nodes, the distribution time and frequency of the key; S5, key distribution execution and feedback optimization; S51, according to the distribution strategy output by the reinforcement learning model, the generated key is distributed to the target node, and the node response and distribution result are monitored in real time during the distribution process; S52, if an abnormality occurs during the distribution process, the abnormal information is recorded and fed back to the reinforcement learning module, and the reinforcement learning model updates its strategy according to the feedback information; S53, after the key distribution is completed, if the environmental characteristics or the distribution efficiency does not reach the expected value, the input condition of the generative adversarial network is optimized through a feedback mechanism, and the adaptability of the key generation and distribution is further improved.

2. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 1, wherein: The topology of the communication network is collected, including the following steps: S111, deploying sensor devices at each node of the communication network, collecting connection information between nodes in real time through these devices to obtain the connection state of the nodes ; the represents the connection between node i and node j, wherein: ； By collecting the connection between each node one by one, the following connection matrix is generated : ； Each row or list in the matrix represents the connection relationship of a node with other nodes, 1 represents connection, and 0 represents no connection; S112, traverse the global connection of the network by using a network crawling algorithm, and identify a direct adjacent node set of each node wherein: ； According to the connection matrix C, the adjacent nodes of each node can be obtained: The set of adjacent nodes of node A is ; The set of neighboring nodes of a node B is denoted by ; The set of neighboring nodes of node C is ; The set of adjacent nodes of node D is ; S113, while collecting the connection information, record the communication link characteristics between the nodes, including: link type: confirm whether the link is one-way or two-way; link bandwidth size : collect the communication bandwidth between the nodes: ； ； ； ； According to these data, a bandwidth matrix B is formed: ； S114, combine the connection matrix C and the bandwidth information B to construct a topology graph of the communication network ; where the set of nodes denotes all nodes in the network: = {A, B, C, D}; edge set representing a communication link between nodes: The bandwidth of edge (A, B) is 100 Mbps; The bandwidth of edge (B, C) is 50 Mbps; The bandwidth of edge (C, D) is 150 Mbps; The bandwidth of edge (A, D) is 200 Mbps; S115, according to the collected data, complete basic data is provided for subsequent key generation and distribution.

3. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 1, wherein: Collecting network node information includes the following steps: S121, enabling node information collection function in the network monitoring device, obtaining the basic attribute information of each node in the network through the SNMP protocol, including the number and type of nodes; assuming that the network contains N nodes, the set represents the node set; S122, for each node Collecting its specific attribute information, including: node type, node function, connection bandwidth and communication mode; (1) Node type: Determine whether the node is a router, switch, server or terminal device through the device type identifier; With a label Indicating the type of node : ； (2) Node function: Determine the network function assumed by the node; with the set representing the function of the node : ； Among them, Relay means that the node has the function of providing data forwarding / relay for other nodes in the network, and undertakes the transfer of communication traffic; Compute means that the node has computing or processing capability, and can perform data analysis, task calculation and other processing operations; Storage: means that the node has data storage function, which can be used for caching, backup or distribution of data in the network; (3) Connection bandwidth: From node Bandwidth to other nodes Information, and record the total connection bandwidth of the node: ； wherein represents a node and bandwidth between (4) Communication mode: Communication mode of collection node and identification Indicates: ； S123, integrate the above data into a node attribute table, and each row in the node attribute table except the first row represents the specific attributes of the node, including type, function, connection bandwidth and communication mode.

4. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 1, wherein: The network state information collected includes network delay, packet loss rate, bandwidth utilization and network congestion degree: The network delay refers to the communication network delay between nodes in the collected network; that is, the time required for data to be sent from the source node to the target node, which is measured by calculating the round trip time; For each pair of nodes and , the delay can be calculated by the following equation: ； wherein is the round-trip time from node to node ; the delay values will be collected and logged for subsequent analysis; The packet loss rate represents the proportion of data packet loss in the network, which is usually measured by a network monitoring tool by periodically sending probe packets; in a network monitoring system, the packet loss rate refers to the data packet loss rate from a node to a node , and the calculation formula is: ； The bandwidth utilization refers to the actual usage of the network link, which reflects whether the communication link between nodes has approached saturation; bandwidth utilization The calculation formula is: ； wherein is the actual bandwidth usage on the link , ), is the total bandwidth of the link; The network congestion degree is a comprehensive index calculated by monitoring the delay, packet loss and bandwidth utilization between multiple nodes, and is used to evaluate the overall load of the network. In the network monitoring system, the congestion degree is expressed by the following weighted average formula: ； wherein, , , are the weights of delay, packet loss rate and bandwidth utilization, respectively, , , are the delay, packet loss rate and bandwidth utilization of the i-th link, respectively, and N is the number of links in the network.

5. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 1, wherein: Collecting device attribute information includes the following steps: S131, real-time collection of hardware attribute information of each device in the network through a special monitoring tool or operating system interface, including the computing power, memory size and storage capacity of the device; the computing power of the device is mainly evaluated through CPU performance and core number, and CPU frequency and total memory capacity can be obtained through the interface provided by the operating system; S132, collect the battery state and energy consumption information of the equipment, the battery information can be obtained by the remaining power and charging rate through the battery monitoring tool provided by the operating system; the remaining power The calculation formula is: ； Battery consumption rate This can be estimated by the following equation: ； wherein is the change in device power, and At is the time interval. S133, through the special energy consumption monitoring tool or hardware module, real-time monitoring of the power consumption of the device, including CPU, memory, storage, network and other components of the power consumption; the total energy consumption of the device The sum of the power consumption of each component can be calculated: ； wherein, P is the power consumption of the ith component, and N is the number of components monitored in the device.

6. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 1, wherein: The communication environment feature collection and initialization in step S1 includes feature data standardization processing: The feature data standardization processing includes the following steps: S141, pre-process various data obtained by collecting the topology of the communication network, collecting network node information, collecting network state information, and collecting device attribute information in step S1; S142, Standardization of topology data: Topology is usually represented by a connection matrix C, where each element reflects whether node i and node j are directly connected; transforming the connection matrix C into a flattened one-dimensional vector The flattened connection matrix can be achieved by ； Among them, Flatten means converting a two-dimensional connection matrix into a one-dimensional vector; the standardized connection information can be input into the generative adversarial network (GAN) and reinforcement learning model; Standardization of node information: node information includes the number of nodes, node types, and node functions; the number of nodes N is directly taken as an input feature; for node types, it is converted into a One-Hot Encoding vector; for node functions, it is standardized using One-Hot Encoding or numerical encoding: ； wherein, is a type or function tag for the node; Standardization of network state data: network state data usually has different dimensions and numerical ranges; Z-score standardization is used to process these data: ； Where X is the original data, μ is the mean of the data, and σ is the standard deviation of the data; after standardization, the network state data will be in the same scale range, which is conducive to the training of subsequent models; Standardization of device attribute data: device attribute data includes computing power, memory size, remaining power, and energy consumption; it is processed by Min-Max standardization or Z-score standardization; Min-Max standardization maps the data to the [0,1] interval: ； wherein, and are the minimum and maximum values of the feature, respectively.

7. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 1 wherein: The key generation based on the improved generative adversarial network in step S2 includes the following steps: S21, input random noise and communication environment features to the generator of the generative adversarial network; S211, the generator in the generative adversarial network needs an initial input, which is random noise z; Sample from the standard normal distribution to get a noise vector z of length N: take this random noise z as an initial input of the generator in the generative adversarial network; ； where N is the dimension of the noise vector, each element is independently drawn from a standard normal distribution; S212, providing the communication environment feature data of the network as input to the generator; the communication environment feature data of the network includes topology of the communication network, node information, network state, and device attributes; representing the communication environment feature data of the network as a vector e with size M, where each is a certain feature in the network: ； Where the length M of the communication environment feature e depends on the network information collected in step S1; M represents the number of network communication environment features collected in steps S1 to S1; Fuse the input data noise vector z and the network communication environment feature data vector e to get a new input vector x: ； Vector x is composed of random noise and network communication environment features, with a length of N+M; S213, input the fused input vector x into the generator of the generative adversarial network to generate the key k, and the process of the generator is represented as: ； wherein, denotes a mapping function of the generator, is a parameter of the generator, the key k output by the generator will be passed as a candidate key to the discriminator for quality evaluation; S22, the generator outputs a candidate key and passes it to the discriminator; S221, pass the candidate key k to a discriminator The role of the discriminator is to evaluate the quality of the key, determining whether it is sufficiently random and resistant to prediction. The discriminator evaluates the key's entropy value, distribution characteristics, and anti-prediction indicators to determine its quality; entropy is an important indicator of key randomness and uncertainty, calculated by the following formula: ； wherein, is the probability distribution of the i-th bit of the key k, n is the length of the key; The discriminator evaluates according to the distribution characteristics of the key; the discriminator uses chi-square test or Kullback-Leibler divergence statistical method based on its distribution characteristics to measure the uniformity of the key; The evaluation process of the discriminator is represented as: ； where Evaluate( ) is the discriminator's evaluation function, which outputs a scalar value representing the quality of the key; The discriminator evaluates the quality of the generated key and outputs a score ; The score is calculated by the following four indicators: ； wherein, is the entropy randomness of the key, is the uniform distribution degree, is the autocorrelation suppression degree, is the predictable resistance degree; the weights of each index are ; If , the key quality is considered high and the key is stored in the key pool; If , the generator is fed back for fine-tuning; if , the key is discarded and the generator is regenerated. If the discriminator considers that the generated key has high quality, it means that it has sufficient randomness and distribution characteristics, and the discriminator will output a high evaluation value, indicating that the key meets the requirements; If the discriminator considers that the quality of the key is poor, the generator will receive feedback from the discriminator, and the generator will adjust the key generation method according to the feedback of the discriminator; until the quality of the generated key meets the requirements; S23, the discriminator feeds back the evaluation result to the generator to optimize the network parameters of the generator; The evaluation result of the discriminator is transmitted back to the generator as a feedback signal for updating the network parameters of the generator; the generator and the discriminator are optimized together in the training process, and an adversarial training method is adopted; (1) The feedback mechanism and optimization process are as follows: The evaluation result is fed back as an error signal: the output result of the discriminator Will be fed back to the generator as an error signal to guide the optimization direction of the generator; (2) Parameter update of the generator: the generator updates the network parameters by the back propagation algorithm and the gradient descent method; after generating the key each time, the generator calculates the loss function , representing the loss of the generator; the calculation of the loss function is based on the evaluation result of the discriminator, as follows: ； wherein E denotes an expected value, k is a candidate key generated by the generator, and are parameters of the generator and the discriminator, respectively, is an evaluation output of the discriminator on the key k; The generator optimizes its parameters by minimizing the loss function, thereby improving the quality of the key; this process can be achieved by gradient descent method: ； where η is the learning rate, is the gradient of the generator loss function with respect to its parameters; (3) Feedback adjustment of optimization target: in the adversarial training, the generator and the discriminator are in a state of game; the generator continuously adjusts the parameters to improve the quality of the generated key, so that the discriminator becomes more and more difficult to judge the authenticity of the key; with the progress of training, the quality of the key generated by the generator will gradually improve, and finally meet the standard of communication network encryption demand; S24, when the generated key reaches the set quality standard, it is stored in the key pool and waits for distribution; The optimized generator will generate a key that meets the quality standard; the discriminator will finally evaluate the generated key, and if the key's entropy, distribution characteristics and anti-prediction indicators meet the predetermined security standards, the key is considered qualified; at this time, the generated key k will be stored in the key pool, waiting for subsequent distribution operation; The generated key pool can be represented as follows: ； Wherein, K is a key pool, containing a plurality of generated keys, each key has certain randomness and security, and can meet subsequent distribution requirements.

8. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 7, wherein: In step S3, the communication environment feature input reinforcement learning model includes the following steps: S31, constructing the state space and action space of the reinforcement learning model; The state space S and the action space A of the reinforcement learning model are constructed; (1) Construction of state space; The state space refers to the environmental information that the reinforcement learning model can perceive at each decision-making step; the state space is composed of multiple key communication environment features, including network topology information, node information, network state information, and device attribute information; The network topology information refers to the connection relationship between nodes obtained in step S1, and the topology structure of the network is represented as an adjacency matrix C: ； wherein, reflects whether node i and node j are directly connected in the network; The node information represents the attributes of each node, the type of node, computing power, function, and feature vector constituting the node : ； The network state information includes real-time state of the network, i.e. bandwidth utilization, delay, packet loss rate; it is assumed that represents bandwidth of network node i, represents delay, packet loss rate is: ； The device attribute information includes node energy consumption information and remaining power; the device state can be represented as: ； The network topology information, node information, network state information and device attribute information, after being standardized, form states in a state space S ; each state is a combination of a plurality of feature vectors, as follows: ； (2) Construction of action space; The action space mainly consists of the following contents: Path selection: Assume there are multiple key distribution paths The selection of each path can be represented as a discrete action : ； Node selection: the node selection action can be represented as: ； wherein is a node selected from the current network state to be suitable for receiving the key; S32, training the reinforcement learning model; The training process includes from initializing the model to adjusting the strategy through continuous feedback, so as to obtain the optimal decision in the dynamic network environment; including the following steps: S321, initialization of training environment; Firstly, the construction of training environment is based on the state space S and the action space A constructed in step S31, and needs to include the following key elements: Network state simulation: based on the actually collected data or the assumed data set, simulate various states in the communication network; through these data simulation, various changes of network conditions are simulated, so that the reinforcement learning model can adapt to different network states; Node and device state simulation: simulate the working state and energy consumption of different nodes, as well as the connection relationship between nodes; Reward function design: define a reward function to measure the effect of each policy after execution; If the key is successfully distributed and the network load is lower than the set threshold, a high reward is given; if the key distribution fails or the network delay is high, a low reward is given; the reward function is as follows: ； wherein, Penalty for Delay indicates that a negative reward is given when the network delay is high; and Penalty for Energy Consumption indicates that a negative reward is given when the device energy consumption is high. S322, the training process of the reinforcement learning model; The training process is based on a reinforcement learning method; the reinforcement learning method uses Q-learning, deep Q-network (DQN), or a policy gradient method; the training process includes the following steps: Sampling state and selecting action: At each time step t, the model selects an action from the current state and applies it to the simulated environment, obtaining a reward and next state : ； wherein, represents an action selected based on the policy π in state s; Update the Q value or policy: update the Q value function or policy network according to the selected action and environmental feedback; for Q-learning, the update formula is: ； where a is the learning rate, g is the discount factor, is the reward, is the maximum Q-value in the next state; For the policy gradient method, the parameter θ of the policy network is updated by maximizing the expected reward: ； wherein, is the gradient of the policy, representing the expected reward of choosing action a in state s; Training iteration: through continuous iteration training, the model makes the optimal key distribution decision for different network states in the simulation environment; S323, experience replay and exploration mechanism; Experience replay and ε-greedy strategy mechanism are introduced in the reinforcement learning model; Experience replay: in reinforcement learning, experience replay refers to storing the four-tuple of model's historical state, action, reward, and next state into an experience pool; each time of training, a batch of historical experience is randomly drawn from the pool for training; the formula is as follows: ； where D is the experience pool, is a four-tuple of state, action, reward, and next state; by randomly sampling data in the experience pool, the model can learn from different experiences and avoid overfitting; ε-greedy strategy: This strategy is used to balance exploration and exploitation; during training, the model will exploit the system with a certain probability ( Select random actions to discover potential better strategies; while in the remaining probability In this case, the model will select the optimal action at the moment; as training progresses, As the value gradually decreases, the model relies more on the learned optimal policy; the formula is: ； This method enables the model to explore new distribution strategies and select the optimal solution among known strategies, achieving better convergence.

9. The improved GAN-RL based dynamic key generation and distribution method as claimed in claim 8, wherein: The real-time adjustment and optimization of the theoretical strategy generated by the reinforcement learning model in step S4 includes the following steps: S41, reinforcement learning-based key distribution strategy optimization; First, the reinforcement learning model receives real-time state information from the communication network; these information refers to network performance indicators including bandwidth, delay, and packet loss rate; the model evaluates the network health status using the following formula: ； wherein, is the network delay of path p; is the bandwidth of path p; is the packet loss rate of path p; , , is a weight coefficient representing the influence of delay, bandwidth and packet loss rate on the quality of the path. The model evaluates the pros and cons of each path by calculating the network health indicators of each path, and finally selects the path with the highest health degree for key distribution; The model evaluates the adaptability of the key based on its characteristics: ； Wherein, Type represents the score of key type; Length is the key length; Strength is the encryption strength of the key; , , is a weight coefficient, reflecting the influence of each key characteristic on the adaptability; After obtaining the health status of the network path and the adaptability of the key, the reinforcement learning model needs to select the optimal key distribution path and distribution method based on this information; the model calculates the comprehensive score of each path according to the following formula and selects the path with the highest score: ； Where P is the set of all available paths; The model selects the appropriate distribution method according to the number of target nodes; for the case of few target nodes, the model may choose unicast distribution; while for the case of many target nodes, the model will choose multicast or groupcast mode; this decision The distribution method of the model is determined by the following rules: ； wherein, is the number of target nodes; Unicast refers to single-casting; Multicast refers to multi-casting; Broadcast refers to group-casting; Multicast and group-casting are suitable for large-scale distribution; Each node is assigned a priority by the following formula: ； wherein, is the priority of node n; the priority of a critical node is 1, indicating that the node should be given the key first; S42, reinforcement learning iteratively optimizes the distribution strategy through exploration and experimentation of the action space; Through exploration and feedback mechanism, the model continuously adjusts and improves its distribution strategy to adapt to the dynamic changes of network environment and task requirements; (1) Exploration and feedback of action space; In reinforcement learning, the action space is the set of all actions that the model can choose; in the key distribution strategy, the action space includes different path selection, distribution method, and node priority adjustment; the model selects different actions and observes environmental feedback through experimentation in the simulation environment, thereby adjusting its strategy; For each time step t, the model takes the current state selects an action and applies it to the environment; the action space for reinforcement learning is represented as: ； Where p e P represents the set of optional network paths; m represents the distribution mode, which can be unicast, multicast or groupcast; n e {0, 1} represents the priority of node n, where 0 is a normal node and 1 is a critical node or a high-risk node; The model, at each training iteration, updates the value function V(s) based on the current state s selecting an action and executing this action in the simulated environment; (2) Reward feedback and policy update; Whenever the model selects an action and applies it to the network, the system returns a feedback signal, i.e., a reward; the reward function is designed in the following form: ； wherein Success_Reward is the reward value after the success of key distribution; Delay is the network delay, the greater the delay, the higher the penalty; Energy is the energy consumption in the distribution process, the higher the energy consumption, the greater the penalty; Risk is the security risk of the network or node, the higher the risk, the greater the penalty; , , is the weight coefficient, which adjusts the influence of each factor on the reward function; Through the reward function, the model can obtain feedback information about the current action and adjust its policy based on this feedback; the goal of reinforcement learning is to maximize the long-term cumulative reward, i.e., to optimize the key distribution strategy; (3) Action selection strategy and ε-greedy strategy; The ε-greedy policy is adopted in reinforcement learning to balance exploration and exploitation; the policy gradually reduces randomness during the training process, enabling the model to transition from extensive exploration in the early stage to local optimization in the later stage; the model has a probability of selecting a random action, i.e., exploration, and a probability of selecting the optimal action under the current policy, i.e., exploitation; the specifics are as follows: ； wherein, is the probability of exploration, which is gradually decayed during the training process to ensure that the model can extensively explore different strategies in the early stage and make decisions using the best strategy learned in the later stage; (4) Policy optimization and convergence; After each exploration and feedback, the reinforcement learning model will update its policy based on the new state and reward information; for Q-learning, the update formula is as follows: ； wherein, represents the Q-value of performing an action under state ; a is the learning rate, controlling the step size of each update; g is the discount factor, measuring the weight of future rewards; is the maximum Q-value under the next state . For policy-based algorithms, policy update is done through the following formula: ； wherein is a parameter of the current policy; is a policy gradient, representing the expected reward of selecting an action in state . S43, generate the final key distribution scheme and pass it to the distribution execution module; After repeated training and optimization, the final key distribution strategy will be gradually improved based on step S42, and the final key distribution scheme will be generated; this scheme includes: selected distribution path, distribution mode and node, key distribution time and frequency; Selected distribution path: optimal path selection; Distribution mode and node: determine to use unicast, multicast or groupcast mode, and preferentially select critical nodes for distribution; Key distribution time and frequency: determine the time and frequency of key distribution according to network status, node demand; Pass the final distribution scheme to the distribution execution module to perform the key distribution task.

10. The improved GAN-RL based dynamic key generation and distribution method of claim 9, wherein: Key distribution execution and feedback optimization in step S5 includes the following steps: S51, according to the distribution strategy output by the reinforcement learning model, distribute the generated key to the target node and monitor the node response and distribution result in real time during the distribution process; First, obtain the optimized distribution strategy of the reinforcement learning model from step S4; this strategy selects the optimal target node and path according to the network status, node demand and key characteristics; specifically including: Target node selection: The reinforcement learning model determines which nodes should receive the key first by considering factors such as node load and communication requirements; the node set is N={ , ,..., The model will select the set of nodes Ntarget∈N that best meets the requirements; Distribution path selection: select the best transmission path according to the location of the target node and the network topology; path selection is usually determined by factors such as delay d, bandwidth b and stability s; the goal of path selection is to minimize delay and maximize bandwidth to ensure key transmission efficiency; The selection function of the target node is: ； wherein, is the node load, is the communication demand, is the node stability, , , are the weights of the respective factors; Then, according to the selected target node and path, start the key transmission; at this time, the generated key is transmitted to the target node through the selected path; in this process, the following transmission methods can be selected: Unicast: transmit the key directly to a single target node; Multicast: transmit the key to multiple nodes simultaneously; Groupcast: transmit the key to a group of nodes; Next, during the key transmission process, the system monitors the following key indicators in real time to ensure the smooth progress of the distribution process: Node response: monitor whether the target node successfully receives the key; if the node does not respond or communication fails, the system will mark it as abnormal and prepare for the next step of abnormal handling; Distribution delay: Real-time tracking of the delay in key transmission, ensuring that the delay does not exceed the set threshold; the delay is calculated by the following formula: ； wherein, is the time the target node receives the key, is the time the key starts to be sent, is the number of target nodes; Distribution success rate: Calculate the proportion of nodes that successfully distribute; if the proportion of nodes that successfully receive the key is lower than the predetermined threshold, the system will trigger an abnormal alarm and re-evaluate the distribution strategy; ； wherein, is the number of nodes that successfully received the key, is the total number of target nodes; Distribution failure rate: The proportion of nodes that fail to distribute, which also needs to be monitored in a timely manner to avoid high failure rates during key distribution; its calculation method is: ； wherein, is the number of failed distribution nodes, is the total number of target nodes; Finally, during the monitoring process, the system will record the relevant data of the cases where the delay exceeds the standard, the nodes do not respond, or the distribution fails in real time, and feed these exceptions back to the reinforcement learning model for subsequent optimization; the specific feedback content includes: Delay information: Real-time feedback of distribution delay, and optimization of distribution path according to the current network status; Node response failure information: Feedback on the failure of the target node to respond, which may require reselecting the path or node; Distribution failure information: Feedback on the nodes that failed to distribute, for reference in subsequent strategy adjustment; The system optimizes the distribution path and distribution method through these real-time feedbacks, making the next round of distribution more efficient and stable; S52, If an exception occurs during the distribution process, record the exception information and feed it back to the reinforcement learning module, and the reinforcement learning model updates its strategy according to the feedback information; The main steps are as follows: S521, Abnormality detection and recording: During the key distribution process, the system continuously monitors and detects multiple key indicators, including: High delay: If the delay of a distribution path exceeds the set threshold, the system immediately marks it as abnormal; the calculation formula for delay is: ； If greater than a set maximum delay threshold a delay anomaly is considered to have occurred; Node failure: If the target node does not respond to the key reception request within the specified time, or the connection with the target node fails, the system records the node failure information; the node failure rate can be calculated by the following formula: ； If exceeds a set threshold then flag as node failure anomaly; Path interruption: If the path is lost or interrupted, causing the key to fail to be transmitted to the target node, the system records the path interruption event and triggers an alarm; S522, Feedback of exceptions to the reinforcement learning module: Once an exception is detected, the system feeds the exception information back to the reinforcement learning module in step S4; the specific feedback information includes: Delay exception information: The system sends the recorded delay data that exceeds the threshold to the reinforcement learning model, and the model adjusts the selection strategy of the distribution path based on this information; Node failure information: If some nodes fail during the key distribution process, the system will feed back the failure information of these nodes, and adjust the node selection and path optimization strategy through the reinforcement learning model; Path interruption information: When the path is interrupted, the system will record the state of the path and the interruption time, and feed it back to the reinforcement learning model, which will re-evaluate the path selection and backup strategy; S523, The reinforcement learning model adjusts the strategy according to the feedback: After receiving the exception feedback information, the reinforcement learning model updates the strategy based on the current network environment and feedback data; the update method usually includes the following aspects: Path optimization: If the delay or reliability of some paths is poor, the model will optimize the path selection and reduce the dependence on these paths; Node selection optimization: The reinforcement learning model will reselect more reliable nodes based on node failure data; Distribution strategy: If certain distribution patterns result in high latency or failure rates, the model may adjust the distribution method to adapt to changes in the network environment; The specific policy update formula is as follows: For the Q-learning method, the policy update formula is: ； where a is the learning rate, g is the discount factor, is the immediate reward, is the value of the current state-action pair; For the policy gradient method, the formula for updating the policy parameters is: ； wherein, is a parameter of the current policy, is a gradient of the policy; S523, Feedback mechanism loop optimization: Throughout the process, the feedback of abnormal information and the update of the strategy form a closed loop; After each round of key distribution, the system will continuously optimize the strategy according to the execution results to ensure that the next round of distribution can adapt to changes in the network state; S53, After the completion of key distribution, if the communication environment characteristics or distribution efficiency does not meet the expected value, the input conditions of the generative adversarial network (GAN) are optimized through the feedback mechanism to further improve the adaptability of key generation and distribution; The communication environment characteristics and distribution efficiency during the key generation and distribution process are evaluated, and the input conditions of the generative adversarial network (GAN) are optimized through the feedback mechanism to further improve the adaptability of key generation and distribution; Including the following: (1) Evaluate the effect of key distribution and communication environment characteristics: Evaluate key indicators and communication environment characteristics during the distribution process; Including: Distribution delay: Calculate the total time from key generation to receiving by the target node; High delay will affect the real-time needs of the network; Distribution success rate: Evaluate whether the key successfully reaches all target nodes; If there are nodes that have not been successfully distributed, the system needs to be marked as failed and further analyzed; Node response: Check the response speed and success rate of the target node; Path stability: Evaluate the stability and reliability of the key distribution path; If the path is unstable or interrupted, optimize the path selection; The above evaluation indicators are quantified by the following formulas: Distribution delay: ； wherein, is the time the key was received, is the time the key was sent, is the number of target nodes; Distribution success rate: ； wherein, is the number of nodes that successfully received the key, is the total number of target nodes; (2) Optimize the input conditions of GAN through the feedback mechanism: According to the above evaluation results, the system will pass the feedback information to the generative adversarial network (GAN) to optimize its input conditions; The goal of the generative adversarial network is to generate keys that meet the requirements, and its input conditions include the following aspects: network communication environment characteristics, node state requirements, and path selection optimization; The specific optimization methods include: Optimize key generation rules: Optimize the generation rules of GAN through analysis of historical data; Adjust network input features: If certain environmental characteristics have a negative impact on key distribution; Adjust the input features of GAN; Improve adaptability: Continuously optimize the input conditions of GAN to improve the adaptability of the key generation process, ensuring efficient and secure key distribution in different network environments and node requirements; The optimization formula for feedback is as follows: ； wherein, is the optimized GAN input, is the current input condition, is the adjustment amount of the input condition according to the feedback information, and a is the learning rate.

Images