Routing method of quantum key distribution network and related device

Abstract

The invention discloses a routing method of a quantum key distribution network and a related device. The method comprises the steps that the state of a first QKD network is collected, the value of a corresponding reward function is calculated, the state is input into a preset actor network, and a corresponding action is output; the reward function is a linear combination of a plurality of topological state influence parameters; training the actor network by using a training architecture of the actor-commentator network in combination with the state, the action and the reward function value until a training termination condition is reached; taking the trained actor network as a strategy network; inputting the state of the link between the nodes in the second QKD network of which the route is to be determined into the strategy network, and outputting the weight value of the link between the nodes in the second QKD network through the strategy network; and calculating the optimal route of the second QKD network according to the weight value output by the strategy network. The invention provides a strategy network which is more intelligent and stronger in robustness so as to realize a routing strategy which is higher in distribution efficiency and more balanced in load.

Description

Technical Field

[0001] The present invention relates to the field of quantum communication technology, and in particular to a routing method and related devices for a quantum key distribution network. Background Art

[0002] In recent years, the Quantum Key Distribution (QKD) network has developed rapidly due to its theoretical unconditional security. The trusted relay QKD network is a QKD network based on trusted nodes. It solves the limitation of single-photon signal transmission distance through the hop-by-hop relay of key data above the quantum key generation layer, thereby realizing a QKD network that can be used by more users and meet longer-distance transmission. The network can theoretically realize global quantum key distribution and is considered to be the most feasible wide-area quantum key network networking method under current technical conditions.

[0003] The implementation of QKD network routing is different from the architecture of classical communication networks in that the routing of QKD networks must not only consider indicators similar to classical communication networks such as routing delay, but also need to consider evaluation indicators related to quantum channels. Therefore, the routing implementation of QKD networks needs to be evaluated and designed in combination with parameters related to classical channels and quantum channels. Summary of the invention

[0004] The inventor of this application has found that for the implementation of QKD network routing, there are a variety of routing algorithms that can meet different network environments and needs, such as the open shortest path first protocol, the routing algorithm based on supervised learning, and the routing algorithm based on traditional deep reinforcement learning, etc. In terms of quantum key distribution networks, they have obvious shortcomings and are difficult to effectively cope with the challenges of modern quantum key distribution networks. The prior art usually considers the quantum key amount in the QKD network as a single influencing factor. Some existing routing algorithms require artificial experience to preset parameters for modeling and cannot adapt to more complex network scenarios. In addition, the routing algorithm based on traditional deep reinforcement learning has a significant increase in the computing resources and training time required as the network scale grows.

[0005] In view of the above problems, the present invention is proposed to provide a routing method and related devices for a quantum key distribution network that overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present invention provides a routing method for a quantum key distribution network, comprising:

[0007] Through a preset simulation environment, the following steps are iteratively performed: the state of the current time step of the first quantum key distribution QKD network for training is collected, and the value of the corresponding reward function is calculated, the state is input into a preset actor network, and the corresponding action is output, where the action is the updated weight value of the link between each node in the QKD network; the reward function is a linear combination of the influencing parameters of the topological states of multiple quantum key distribution networks;

[0008] Using a preset actor network-critic network training architecture, as well as the state, action, and reward function value at each time step, the actor network is trained until a preset training termination condition is reached; and the trained actor network is used as a policy network;

[0009] Obtaining the status of links between nodes in the second QKD network to be routed;

[0010] Using the state of the links between the nodes in the second QKD network as the input of the policy network, and outputting the weight value of the links between the nodes in the second QKD network through the policy network;

[0011] The optimal route of the second QKD network is calculated according to the weight value of each link output by the policy network.

[0012] In one embodiment, before training the actor network, the method further includes:

[0013] Constructing a reward function and corresponding constraints; the reward function is a linear combination of the following parameters: the total path time of all quantum key distributions through the routing strategy, the quantum key freshness, the average local quantum key consumption rate, the average local quantum key generation rate, and the path quantum key utilization rate;

[0014] The total time of the paths through which all quantum key distributions pass through the routing strategy is the sum of the propagation delay, transmission delay, processing delay and queuing delay of the paths through which all quantum key distribution processes pass through the routing selection strategy;

[0015] The quantum key freshness represents the ratio of the amount of remaining quantum keys in all nodes in the quantum key distribution network to the maximum capacity of the quantum key pool of the node;

[0016] The average local quantum key consumption rate represents the average value of the quantum key consumption rate of all inter-node links per unit time;

[0017] The average local quantum key generation rate represents the average value of the quantum key generation rates of all inter-node links per unit time;

[0018] The path quantum key utilization rate represents the proportion of the capacity of all quantum keys distributed on the link to the total capacity of the entire link;

[0019] The constraints include:

[0020] The total time of all quantum key distribution paths through the routing strategy is less than or equal to the delay required by the preset quality of service QoS;

[0021] The minimum value of the key utilization of all quantum keys passing from one node to another is greater than or equal to the path quantum utilization required by the preset QoS;

[0022] The average local quantum key consumption rate is less than the average local quantum key generation rate.

[0023] In one embodiment, calculating the reward function value includes:

[0024] Based on the constraints, for the first QKD network after performing the simulated quantum key distribution operation, the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate and path quantum key utilization rate of all quantum key distributions through the routing strategy are counted, and based on the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate and path quantum key utilization rate of all quantum key distributions through the routing strategy, the corresponding reward function value is calculated.

[0025] In one embodiment, the following steps are iteratively performed through a preset simulation environment: collecting the state of the current time step of the first quantum key distribution QKD network for training, and calculating the corresponding reward function value, inputting the state into a preset actor network, and outputting the corresponding action, wherein the action is the updated weight value of the link between each node in the QKD network; including:

[0026] In a preset simulation environment, initializing the weight value of the link between each node in the first QKD network and initializing the state of the link between each node in the first QKD network;

[0027] The state of the link between each node is used as the input of the actor network, and the weight value corresponding to the link between each node is output through the actor network; the weight value is the action of the current time step output by the routing actor network; the architecture of the actor network is a hybrid neural network composed of quantum-classical neural network;

[0028] Determine the shortest quantum key distribution path from the source node to each node in the first QKD network using a preset quantum shortest path search algorithm according to the weight values ​​of the links between the nodes output by the actor network, and the source node and the destination node of each key byte packet;

[0029] According to the shortest quantum key distribution path from the source node to each node in the first QKD network, determine the shortest quantum key distribution path from the source node to the destination node, perform a simulated quantum key distribution operation in the first QKD network, and calculate the reward function value corresponding to the action at the current time step according to the result of the simulated quantum key distribution operation;

[0030] Repeat the following steps: determine the updated state of the links between the nodes according to the simulated quantum key distribution result of the previous time step in the first QKD network; input the updated state to the actor network, output the updated weight value corresponding to the link between the nodes through the actor network, and the weight value is the action of the next time step; update the shortest quantum key distribution path based on the weight value, and perform the simulated quantum key distribution operation again, and calculate the reward function value corresponding to the next action according to the result of the simulated quantum key distribution operation, until the training of the actor network is terminated;

[0031] The conditions for training termination include any of the following:

[0032] The reward function value reaches convergence;

[0033] The number of iterations has reached the preset training round threshold.

[0034] In one embodiment, the actor network architecture includes:

[0035] A first angle encoding unit, at least one layer of a first parameterized quantum circuit unit including parameters to be optimized, and a first classical neural network connected in sequence; the first parameterized quantum circuit includes a quantum circuit composed of a plurality of quantum logic gates;

[0036] The architecture of the critic network includes:

[0037] A second angle encoding unit, at least one layer of second parameterized quantum circuit unit with parameters to be optimized, and a second classical neural network are connected in sequence; the second parameterized quantum circuit includes a quantum circuit composed of multiple quantum logic gates.

[0038] In one embodiment, the actor network is trained using a preset actor network-critic network training architecture, and the state, action, and reward function value at each time step until a preset training termination condition is reached, including:

[0039] Through the critic network, for each action at each time step output by the actor network, calculate the corresponding value function value;

[0040] Calculate the Q-value of the action; the Q-value represents the expected return of the action;

[0041] Calculate the value of the Advantage function, where the Advantage function is equal to the difference between the Q-value and the value function value;

[0042] Take the Advantage function as the loss function of the actor network, and according to the value of the Advantage function, perform gradient descent and backpropagation to update the parameters of the actor network until a preset training termination condition is reached.

[0043] In one embodiment, according to the weight values of each link output by the policy network, calculating the optimal routing of the second quantum key distribution network includes:

[0044] According to the weight values of each link output by the policy network, through a preset quantum shortest path search algorithm, determine the shortest quantum key distribution path from the source node to each target node in the second QKD network.

[0045] In one embodiment, the quantum shortest path search algorithm is implemented through the following steps:

[0046] Step 1: Initialize a list h of size N; the N data in the list respectively represent the total weight corresponding to the shortest path from the source node v s to each node v in the network structure diagram i ; and initialize the total weight corresponding to all shortest paths to positive infinity;

[0047] Step 2: Determine the source node v s , and set the s-th data in the list h to 0, that is, h[v s = 0;

[0048] Step 3: Determine the node i, and run the quantum minimum search algorithm j times. The quantum minimum search algorithm is used to calculate the total weight P corresponding to the shortest path that can reach the current node i from the source node v s through the adjacent node h of node i, and compare it with the total weight h[i] corresponding to the shortest path of the current node i; if h[i] < P, the value of h[i] remains unchanged; otherwise h[i] = P. Where

[0049] Step 4: Traverse the source node ν sAll subsequent nodes, that is, i = {s+1, ....N}, and run step 3 again, and finally get the source node ν s A list h of the sum of the weights of the shortest paths to each node in the network structure graph.

[0050] In a second aspect, an embodiment of the present invention provides a device for quantum key distribution network routing, including:

[0051] The training module is used to iteratively perform the following steps through a preset simulation environment: collect the state of the current time step of the first quantum key distribution QKD network used for training, and calculate the corresponding reward function value, input the state into the preset actor network, and output the corresponding action, wherein the action is the weight value of the link between each node in the updated QKD network; use the preset actor network-critic network training architecture, as well as the state, action and reward function value of each time step, to train the actor network until the preset training termination condition is reached; use the trained actor network as the policy network;

[0052] An acquisition module, used to acquire the status of links between nodes in the second QKD network to be routed;

[0053] A weight value prediction module, used to use the state of each link between nodes in the current quantum key distribution network as the input of the policy network, and output the weight value of the link between each node in the second QKD network through the policy network;

[0054] A routing module is used to calculate the optimal route of the second quantum key distribution network according to the weight value of each link output by the policy network.

[0055] In a third aspect, an embodiment of the present invention provides a computing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the routing method of the quantum key distribution network as described above is implemented.

[0056] In a fourth aspect, an embodiment of the present invention provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the routing method of the quantum key distribution network as described above.

[0057] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0058] The routing method of the quantum key distribution network provided in the embodiment of the present invention utilizes the architecture of quantum reinforcement learning to train the policy network used to guide the quantum key distribution strategy. During the training process, the state of the quantum key distribution network at different time steps is collected in a simulated environment through the training architecture of the actor network-critic network in quantum reinforcement learning, and the reward function value is calculated (the reward function takes into account the possible influencing parameters of various quantum key distribution network topologies), and the critic network evaluates the actions of the actor network to guide the training of the actor network. This scheme applies a dynamic learning mechanism to continuously extract information from environmental feedback, update the weights of links between nodes, and continuously optimize the policy network. It can train a more intelligent and robust policy network, improve the model accuracy, and provide a routing strategy with higher distribution efficiency and more balanced load for the actual quantum key distribution network. In addition, the embodiment of the present invention fully considers the various factors that may affect the topological state of the quantum key distribution network, which can improve the comprehensiveness and accuracy of the evaluation decision, and is suitable for large-scale networking scenarios of quantum key distribution technology.

[0059] In the embodiment of the present invention, multiple influencing parameters of the topological state of the quantum key distribution network include multiple considerations that affect the quantum key distribution route selection from multiple dimensions such as transmission delay, key utilization, freshness, generation rate and consumption rate, which can more fully improve the comprehensiveness and accuracy of routing strategy evaluation.

[0060] In the embodiment of the present invention, both the actor network and the critic network adopt the architecture of quantum-classical hybrid neural network. Compared with the classical neural network, this network architecture can make full use of the quantum superposition state encoding state, reduce the resource usage and training time in the training process, improve the model accuracy, and improve the training efficiency.

[0061] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0062] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0064] Figure 1A schematic diagram of a quantum cryptography network and a QKD network therein in an embodiment of the present invention;

[0065] Figure 2 1 is a structural block diagram of a routing method for a quantum key distribution network in an embodiment of the present invention;

[0066] Figure 3A and 3B It is a schematic diagram of the architecture of the actor network-critic network in an embodiment of the present invention;

[0067] Figure 4A and Figure 4B A specific example diagram of a network architecture in an embodiment of the present invention;

[0068] Figure 5 A schematic diagram of an example of a routing method for a quantum key distribution network in an embodiment of the present invention;

[0069] Figure 6 It is a schematic diagram of the structure of a device for quantum key distribution network routing in an embodiment of the present invention;

[0070] Reference numerals:

[0071] 31-first angle encoding unit 32-first parameterized quantum circuit unit 33-first classical neural network;

[0072] 34-second angle encoding unit 35-second parameterized quantum circuit unit 36-second classical neural network;

[0073] 61 - training module 62 - acquisition module 63 - weight value prediction module 64 - routing module. DETAILED DESCRIPTION

[0074] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0075] In order to facilitate the understanding of this application, the present invention first briefly describes the architecture of the existing quantum cryptography network and the quantum key distribution (QKD) network therein:

[0076] The quantum cryptography network is a secure communication network that uses quantum cryptography. Figure 1As shown in the figure, the quantum cryptography network is constructed by the classical communication network and the QKD network. The QKD network is mainly composed of QKD terminal equipment and quantum links for quantum key distribution. The classical communication network uses quantum keys to realize data encryption and decryption and the transmission of encrypted data. A quantum cryptography network node is generally composed of a classical communication terminal connected to the classical communication network and a QKD device terminal connected to the quantum communication network. The network nodes of the quantum cryptography network are generally divided into two types: terminal nodes and relay nodes.

[0077] Specifically, classical communication links are used to complete tasks such as control signaling transmission and quantum key negotiation. Quantum links are used to transmit quantum bit signals carried by quantum states, and can be physical media such as optical fibers or free space (such as satellite links). For QKD networks, QKD terminals include QKD transmitters and QKD receivers. The QKD transmitter is used to generate quantum keys, generate, modulate and send quantum bit strings. The QKD receiver is used to receive, measure and parse quantum bit strings from the transmitter. It has functions such as key sequence comparison, screening, and error rate detection, and obtains the same symmetric quantum key as the transmitter. The two can be connected through a QKD relay device.

[0078] The routing method of the quantum key distribution network provided in the embodiment of the present invention provides an intelligent quantum key distribution routing implementation scheme for the QKD network (such as a trusted relay QKD network) to provide a network routing with a higher quantum key distribution success rate and a more balanced load. The specific implementation of the routing method of the quantum key distribution network provided in the embodiment of the present invention is described in detail below.

[0079] The embodiment of the present invention provides a routing method for a quantum key distribution network, referring to Figure 2 As shown, the following steps are included:

[0080] S21. Iteratively perform the following steps through a preset simulation environment: collect the state of the current time step of the first quantum key distribution QKD network for training, calculate the value of the corresponding reward function, input the state into the preset actor network, and output the corresponding action, where the action is the weight value of the link between each node in the updated QKD network; train the actor network using the preset actor network-critic network training architecture, as well as the state, action and reward function value of each time step, until the preset training termination condition is reached;

[0081] In the above steps, the reward function is a linear combination of the influencing parameters of multiple quantum key distribution network topological states.

[0082] S22, use the trained actor network as the strategy network;

[0083] In an embodiment of the present invention, the policy network is a neural network for providing a routing strategy for a quantum key distribution network.

[0084] S23, obtaining the status of the links between the nodes in the second QKD network to be determined;

[0085] S24, using the state of the link between each node in the second QKD network as the input of the strategy network, and outputting the weight value of the link between each node in the second QKD network through the strategy network;

[0086] S25. Calculate the optimal route of the second QKD network according to the weight value of each link output by the policy network.

[0087] In one embodiment, in the above steps S21 to S25, in order to conveniently distinguish the quantum key distribution network used by the policy network in the training process, and the quantum key distribution network that needs to be guided by the trained policy network to implement the actual routing scheme, the former is called the first quantum key distribution network, that is, the first QKD network, and the latter is called the second quantum key distribution network, that is, the second QKD network.

[0088] The routing method of the quantum key distribution network provided in the embodiment of the present invention utilizes the architecture of quantum reinforcement learning to train the policy network used to guide the quantum key distribution strategy. During the training process, the state of the quantum key distribution network at different time steps is collected in a simulated environment through the training architecture of the actor network-critic network in quantum reinforcement learning, and the reward function value is calculated (the reward function takes into account the possible influencing parameters of various quantum key distribution network topologies), and the critic network evaluates the actions of the actor network to guide the training of the actor network. The embodiment of the present invention applies a dynamic learning mechanism to continuously extract information from environmental feedback, update the weights of the links between nodes, and continuously optimize the policy network. A more intelligent and robust policy network can be trained, the model accuracy is improved, and a routing strategy with higher distribution efficiency and more balanced load is provided for the actual quantum key distribution network. In addition, the embodiment of the present invention fully considers the various factors that may affect the topological state of the quantum key distribution network, which can further improve the comprehensiveness and accuracy of the evaluation decision, and is suitable for large-scale networking scenarios of quantum key distribution technology.

[0089] Moreover, in an embodiment of the present invention, the training of the policy network can be completed offline, and the process is independent of the step of guiding the routing of the second QKD network through the policy network, which can greatly reduce the error process in the online environment, thereby avoiding interference with the stable operation of the second QKD network.

[0090] In one embodiment, the preset simulation environment includes, for example, simulation tools such as a quantum key distribution simulator.

[0091] A quantum key distribution simulator is a tool that simulates the quantum key distribution process through software or computer programs. Since the actual cost of building a quantum key distribution system is high and the experimental operation is relatively complicated, a quantum key distribution simulator can provide researchers with an environment for studying and testing quantum key distribution while saving money and time. Of course, the embodiments of the present invention are not limited to such simulation tools.

[0092] In one embodiment, the influencing parameters of the topological state of the multiple quantum key distribution networks may include, for example: the total path time of all quantum key distributions through the routing strategy, the quantum key freshness, the average local quantum key consumption rate, the average local quantum key generation rate and the path quantum key utilization rate; wherein:

[0093] 1) The total path time of all quantum key distributions through the routing strategy. This parameter represents the sum of the path times of all quantum key distribution processes through the routing selection strategy.

[0094] Specifically, the path time that each quantum key distribution process passes through the routing selection strategy is the sum of the propagation delay, transmission delay, processing delay and queuing delay of the path that the quantum key distribution process passes through the routing selection strategy.

[0095] In other words, for the calculation of the total path time for all quantum key distributions through the routing strategy, we can first sum up the propagation delay, transmission delay, processing delay and queuing delay of each quantum key distribution process to obtain the path time for the quantum key distribution process through the routing selection strategy, and then sum up the path times for all quantum key distribution processes through the routing selection strategy to obtain the total path time for all quantum key distributions through the routing strategy.

[0096] The total path time of all quantum key distributions through the routing strategy can be calculated by the following formula, for example:

[0097]

[0098] In the above formula, T is the total time of all quantum key distribution paths through the routing strategy, T k is the path time of the kth quantum key distribution through the routing selection strategy, that is, the sum of propagation delay, transmission delay, processing delay and queuing delay, that is:

[0099]

[0100] in Indicates whether the kth quantum key passes through path l ij To distribute, This means that the kth quantum key passes through path l ij To distribute, This means that the kth quantum key does not pass through path l ij for distribution. is the propagation delay of the kth quantum key through the routing strategy, is the transmission delay of the kth quantum key through the routing selection strategy, is the processing delay of the kth quantum key passing through the routing selection strategy, is the queuing delay of the kth quantum key passing through the routing selection strategy, where k∈K.

[0101] 2) Quantum key freshness, which represents the ratio of the amount of remaining quantum keys in all nodes in the quantum key distribution network to the maximum capacity of the quantum key pool of the node;

[0102] The quantum key freshness can be expressed, for example, by the following formula:

[0103]

[0104] where ∑ j R ij is the sum of the remaining quantum keys of all adjacent nodes of the i-th node, C is the maximum capacity of the quantum key pool, d(i) is the degree of node i, and the degree is the number of neighboring nodes.

[0105] 3) Average local quantum key consumption rate, which represents the average value of the quantum key consumption rate of all inter-node links per unit time;

[0106] The average local quantum key consumption rate represents the average value of the quantum key consumption rate of all links between nodes in the quantum key distribution network per unit time.

[0107] 4) Average local quantum key generation rate, which represents the average value of the quantum key generation rate of all inter-node links per unit time;

[0108] The average local quantum key generation rate represents the average value of the quantum key generation rate of all links between nodes in the quantum key distribution network per unit time.

[0109] 5) Path quantum key utilization, which represents the proportion of the capacity of all quantum keys distributed on the link to the total capacity of the entire link;

[0110] The path quantum key utilization rate can be expressed as:

[0111]

[0112] where d k represents the size of the kth quantum key, It means that the kth quantum key passes through path l ij The total capacity, B ij Represents the entire path l ij Total capacity; η = ∑ ij η ij .

[0113] In the process of quantum key distribution, assuming that K packets (key byte packets) may be sent at a time, a utilization rate can be calculated for a single packet. ij is through the path l ij The sum of quantum key utilization for distributing K packets.

[0114] On the path ij In the process, not only quantum keys but also other information are transmitted. The path quantum key utilization rate calculates the proportion of quantum keys to the total capacity (bandwidth) of the entire channel.

[0115] The above-mentioned multiple influencing parameters of the topological states of the quantum key distribution network include multiple considerations that affect the selection of quantum key distribution routes from multiple dimensions such as transmission delay, key utilization, freshness, generation rate, and consumption rate, and can more fully improve the comprehensiveness and accuracy of routing strategy evaluation. Of course, the embodiments of the present invention are not limited to the above-mentioned influencing parameters of the topological states of the quantum key distribution network.

[0116] Accordingly, before steps S21 to S25 of the embodiment of the present invention, the steps of constructing a reward function and defining its constraints are also included.

[0117] Specifically, the reward function can be a linear combination of several parameters: the total path time of all the above quantum key distributions through the routing strategy, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate and path quantum key utilization.

[0118] For example, the reward function can be defined as follows:

[0119]

[0120] The parameters α, β, γ, λ, τ, and μ are all coefficients greater than zero; T represents the total path time of all quantum key distributions through the routing selection strategy; ζ represents the quantum key freshness; η represents the sum of the quantum key utilization rates of the paths; represents the average local quantum key consumption rate; represents the average local quantum key generation rate; T maxT represents the maximum total time of all quantum key distribution paths through the routing strategy. Qos represents the delay required by the Quality of Service (QoS), η QoS Indicates the quantum key utilization required by QoS.

[0121] The constraints of the above reward function include:

[0122] Constraint 1: The maximum total time of all quantum key distribution paths through the routing strategy must be less than or equal to the delay required by QoS, that is,

[0123] Constraint 2: The minimum utilization of all quantum keys from node i to node j must be greater than or equal to the quantum key utilization required by QoS, that is,

[0124] Constraint 3: The average local quantum key consumption rate needs to be less than the average local quantum key generation rate, that is,

[0125] In the above actor network training process, when the reward function needs to be calculated, the quantum key distribution operation can be simulated in the simulation environment (sending several key byte packets), and the above T, ζ, η, and η QoS , substitute the values ​​of these parameters into the formula of the above reward function, and calculate the value of the reward function based on the above constraints.

[0126] In one embodiment, in the training architecture of the actor network-critic network in the embodiment of the present invention, the actor network and the critic network can adopt similar network architectures, respectively referring to Figure 3A and Figure 3B As shown, specifically, refer to Figure 3A As shown, the architecture of the actor network may include: a first angle encoding unit 31 connected in sequence, at least one layer of a first parameterized quantum circuit unit 32 containing parameters to be optimized ( Figure 3A The first parameterized quantum circuit unit 32 includes a quantum circuit composed of multiple quantum logic gates.

[0127] Reference Figure 3B As shown, the architecture of the critic network includes: a second angle encoding unit 34, at least one layer of second parameterized quantum circuit unit 35 (M layer) of parameters to be optimized, and a second classical neural network 36 connected in sequence; the second parameterized quantum circuit unit 35 includes a quantum circuit composed of multiple quantum logic gates.

[0128] The output of the first angle encoding unit is connected to the input of the first parameterized quantum circuit unit 32 of the first layer, and the output of the last parameterized quantum circuit unit 32 is connected to the input of the first classical neural network 33. The structural relationship among the second angle encoding unit 34, the second parameterized quantum circuit unit 35 and the second classical neural network 36 is the same.

[0129] The first classical neural network 33 and the second classical neural network 36 may be the same type of neural network, or different neural networks. The classical neural network may be, for example, a convolutional neural network (CNN), a recurrent neural network (RNN), etc., which is not limited in the embodiment of the present invention.

[0130] Examples of the network architecture of actor networks and critic networks can be found in Figure 4A and Figure 4B As shown, Figure 4A This is a schematic diagram of the Actor network architecture; Figure 4B This is a schematic diagram of the Critic network architecture.

[0131] Figure 4A The number of layers of the first parameterized quantum circuit unit 32 in the Actor network shown is D A , Figure 4B The number of layers of the second parameterized quantum circuit unit 35 in the Critic network shown is D C After building the Actor network and the Critic network, you also need to initialize a set of parameters to be optimized for the two networks. and That is to and Set the initial value.

[0132] The first angle encoding module 31 and the second angle encoding module 34 are used to encode the input state information.

[0133] from Figure 4A and Figure 4B As shown in the figure, both the Actor network and the Critic network adopt the architecture of quantum-classical hybrid neural network. Compared with the classical neural network, this network architecture can represent more information with fewer bits (1 quantum bit can represent 2 N information), can make full use of the quantum superposition state encoding state, can effectively reduce the resource usage and training time during the training process, and improve the model accuracy and training efficiency.

[0134] In one embodiment, the process of training the actor network in step S21 may be performed offline. Specifically, it may be implemented through the following iterative process:

[0135] In a preset simulation environment, initializing the weight value of the link between each node in the first QKD network and initializing the state of the link between each node in the first QKD network;

[0136] The state of the link between each node is used as the input of the actor network, and the weight value corresponding to the link between each node is output through the actor network; the weight value is the action of the current time step output by the routing actor network;

[0137] According to the weight values ​​of the links between the nodes output by the actor network, and the source node and the destination node of each key byte packet, a preset quantum shortest path search algorithm is used to determine the shortest quantum key distribution path from the source node to each target node in the first QKD network;

[0138] According to the shortest quantum key distribution path from the source node to each node in the first QKD network, determine the shortest quantum key distribution path from the source node to the destination node, perform a simulated quantum key distribution operation in the first QKD network, and calculate the reward function value corresponding to the action at the current time step according to the result of the simulated quantum key distribution operation;

[0139] Repeat the following steps: determine the updated state of the links between the nodes according to the simulated quantum key distribution result of the previous time step in the first QKD network; input the updated state to the actor network, and output the weight value corresponding to the updated link between the nodes through the actor network, and the weight value is the action of the next time step; update the shortest quantum key distribution path based on the weight value, and perform the simulated quantum key distribution operation again, and calculate the reward function value corresponding to the next action according to the result of the simulated quantum key distribution operation, until the training of the actor network is terminated.

[0140] The above-mentioned preset quantum shortest path search algorithm is used to determine "each node" in the shortest quantum key distribution path from the source node to each node in the first QKD network, including the destination node and the relay node. In a trusted relay QKD network, from the source node to the destination node, there may be several hops of relay nodes. Therefore, the quantum shortest path search algorithm can calculate the shortest quantum key distribution path from the source node to any node in the first QKD network. After determining the source node and the destination node, the quantum shortest path from the source node→relay node→destination node can be determined.

[0141] In one embodiment, the above training termination condition for the actor network may be, for example:

[0142] The reward function value changes slightly and has reached a convergence state; for example, after several iterations, the reward function value continues to change within a smaller range than the reward function value in the previous time step, which can be regarded as convergence.

[0143] The number of iterations has reached the preset training round threshold.

[0144] For the actor network-critic network architecture of quantum reinforcement learning, the operation of simulating quantum key distribution is to simulate the operation of sending a quantum key package.

[0145] In the actor-critic architecture, training data for the actor network is collected by interacting with the environment. This is done by performing actions, observing states, and receiving rewards.

[0146] In one embodiment, the state of the above inter-node link can be defined in the following manner:

[0147] The number of received packets, the number of sent packets and the remaining quantum key of the node in the link are taken as the state, that is, state = {in, out, R ij}. In which, in is the number of received packets of the node in the link, out is the number of sent packets, R ij is the remaining quantum key quantity as the state.

[0148] At the beginning of training, both in and out can be set to 0, R ij Set to non-zero.

[0149] The above action is: at each time step, the weight of the link between nodes corresponding to the input state is output.

[0150] After the actor network outputs the weights of the links between nodes, the preset quantum shortest path search algorithm can be used to update the shortest quantum key distribution path. When the quantum key distribution operation is simulated again in the first QKD network, the shortest path of quantum key distribution may have changed. For example, in the previous time step, the shortest path from the key from node i to node j is the shortest path. In the next time step, after the link weights are adjusted, the shortest path may change to other links.

[0151] In one embodiment, the above quantum shortest path search algorithm is an algorithm based on dynamic programming and quantum minimum search, and its goal is to calculate the source node v s To each node ν in the network structure i Specifically, this can be achieved through the following steps:

[0152] Step 1: Initialize a list h of size N; the N data in the list represent the data from the source node v sto each node ν in the network structure diagram i The sum of the weights corresponding to the shortest path; and initialize the sum of the weights corresponding to all the shortest paths to positive infinity, that is, h = {+∞,... +∞};

[0153] Step 2: Determine the source node ν s , and set the s-th data in the list h to 0, that is, h[v s = 0;

[0154] Step 3: Determine the node i, and run the quantum minimum search algorithm j times. The quantum minimum search algorithm is used to calculate the sum of the weights P corresponding to the shortest path from the source node v s to the current node i that can pass through the adjacent node j of the node i, and compare it with the sum of the weights h[i] corresponding to the shortest path of the current node i; if h[i] < P, the value of h[i] remains unchanged; otherwise, h[i] = P. Where

[0155] Step 4: Traverse all the nodes after the source node v s , that is, i = {s + 1,... N}, and run the above Step 3 again. Finally, obtain the list h of the sum of the weights corresponding to the shortest paths from the source node v s to each node in the network structure diagram.

[0156] In the embodiment of the present invention, using the above quantum shortest path search algorithm to calculate the optimal link path can reduce the algorithm complexity and improve the calculation efficiency.

[0157] In the training framework of the actor network - critic network, the action usually means an operation that can be executed in a specific state. In the embodiment of the present invention, the action is the weight between the output node links.

[0158] The state describes the characteristics or situations at a certain moment. In the embodiment of the present invention, the state represents the state information such as the number of received packets, the number of sent packets, and the remaining quantum key amount of each node in the network after implementing a certain routing policy.

[0159] The reward represents the feedback signal given by the environment to the actor network, and is used to measure the goodness or effectiveness of the actions taken. In the embodiment of the present invention, after implementing the distribution of quantum key packets through the updated shortest path between nodes, calculate each parameter in the first QKD network and finally calculate the corresponding reward function value.

[0160] In reinforcement learning, the actor network and the critic network usually cooperate with each other. The actor network is responsible for selecting actions, and the critic network is responsible for evaluating the goodness or badness of these actions. Through continuous interaction and learning, the actor network can gradually improve its strategy to obtain better performance in the environment.

[0161] In the embodiment of the present invention, the critic network calculates the corresponding value function value for each time step of the action output by the actor network;

[0162] Calculate the Q value (i.e. Quality value) of the action, which represents the expected return of the action at each time step;

[0163] Calculate the value of the Advantage function, where the Advantage function is equal to the difference between the Q value and the value function;

[0164] The Advantage function is used as the loss function of the actor network. According to the value of the Advantage function, gradient descent is performed and back propagation is used to update the parameters of the actor network until the preset training termination condition is reached.

[0165] By calculating the Advantage function, the critic network can evaluate the relative advantage of each action and provide guidance to the actor network to help it learn better strategies. The actor network adjusts its behavior based on the Advantage information provided by the critic network to maximize the long-term accumulated rewards.

[0166] Advantage functions can use temporal difference (TD) error, Monte Carlo sampling, or neural network-based approximation, etc. It can help the actor network better understand the value of actions, thereby guiding it to learn more effective strategies.

[0167] For the critic network, it is also necessary to update the parameters of the critic network using the mean square error function at each time step.

[0168] In one embodiment, in the above step S25, according to the weight value of each link output by the policy network, the shortest quantum key distribution path from the source node to each target node in the second QKD network is determined by the aforementioned quantum shortest path search algorithm.

[0169] In the embodiment of the present invention, the process of the second quantum key distribution network using the policy network to determine the quantum key distribution route can be multiple times, and the training process of the policy network can also be performed multiple times (the data of each training may be different, and the network parameters obtained by training are also different) to gradually optimize the policy network, and the two processes are independent of each other. Each time, the second quantum key distribution network can use the policy network obtained by the most recent training to guide its own quantum key distribution route to achieve the optimal routing distribution effect at the moment.

[0170] The following is a specific example to illustrate the routing method of the above-mentioned quantum key distribution network.

[0171] Reference Figure 5 As shown, the method includes constructing a directed graph according to the environment network (the first QKD network), using reinforcement learning to train the actor network to obtain a policy network, and then obtaining the optimal path through the trained policy network through quantum shortest path search, and updating the routing table based on this.

[0172] Specifically, the implementation process of this example is described as follows:

[0173] 1. Construct a network topology diagram. Model the environment network (i.e., the first QKD network) as a directed graph G(V,E). The sets V and E represent the quantum node set in the network and the edge set between the connected nodes, respectively. ij Represents the edge between node i and node j. ij Represents the link between node i and node j.

[0174] 2. Determine the reward function and constraints.

[0175] For example, the reward function is constructed by a linear combination of the total path time through which all quantum key distributions pass through the routing selection strategy, the quantum key freshness, the average local quantum key consumption rate, the average local quantum key generation rate, and the path quantum key utilization rate.

[0176] As mentioned above, the formula of the reward function is set as:

[0177] The parameters α, β, γ, λ, τ, and μ are all coefficients greater than zero; T represents the total path time of all quantum key distributions through the routing selection strategy; ζ represents the quantum key freshness; η represents the sum of the quantum key utilization rates of the paths; represents the average local quantum key consumption rate; represents the average local quantum key generation rate; T max T represents the maximum total time of all quantum key distribution paths through the routing strategy. Qos represents the delay required by the Quality of Service (QoS), η QoS Indicates the quantum key utilization required by QoS.

[0178] The constraint function is as mentioned above and is set as:

[0179] Constraint 1: The maximum total time of the path that quantum key distribution passes through the routing selection strategy needs to be less than or equal to the delay required by the Quality of Service (QoS);

[0180] Constraint 2: The minimum utilization of all quantum keys from node i to node j needs to be greater than or equal to the quantum key utilization of the path required by QoS;

[0181] Constraint 3: The average local quantum key consumption rate needs to be less than the average local quantum key generation rate.

[0182] 3. Offline training of the policy network. In a simulated environment, the actor-critic architecture in quantum reinforcement learning is used to train the policy network. Specifically, the following steps 3.1 to 3.4 are included;

[0183] 3.1 Initialize the environment. Initialize the weight w between node i and node j in the routing table ij , where if there is no link between node i and node j, the weight w ij Initialized to positive infinity.

[0184] In this step, using the directed graph G(V,E) mentioned above, the weight of the link between each node and other nodes can be obtained, such as the link weight between node 1 and node 2, node 3, and so on, node N, that is, the routing table is obtained. If there is no link between the nodes, the weight of the link between them is positive infinity.

[0185] 3.2 Determine and initialize the state. The number of received packets, the number of sent packets, and the remaining quantum key of the node in the link are taken as the state, that is, state = {in, out, K rest}, and set the parameters in and out to 0, K rest Set to non-zero.

[0186] 3.3 Quantum state encoding state. Use angle encoding to encode the initial state state.

[0187] 3.4 Training strategy network. Based on the current state and Figure 4A and 4B The action generated by the quantum-classical hybrid neural network shown in Figure 1 selects the next action to be performed in the continuous action space and executes the action, that is, updates the weight w of the path ij .

[0188] 3.41 Reference Figure 4A and 4B As shown in the figure, two quantum circuits are constructed to implement the Actor network and the Critic network respectively, and a set of parameters to be optimized for the two networks are initialized respectively. and

[0189] 3.42 Collect experience samples to train the actor network. Explore according to the strategy of the Actor network, iteratively execute at different time steps: obtain experience samples from the environment, including state, action, and reward. Specifically, the quantum key distribution simulator (i.e., the environment in quantum reinforcement learning) reads the updated routing table and uses the quantum shortest path search algorithm to find the shortest quantum key distribution path based on the starting point and target point of each key byte packet, records the state value reflecting the quantum key distribution of the link, and returns the calculated reward function value.

[0190] 3.43 Use the Critic network to calculate the expected return and Advantage function.

[0191] The Critic network is used to estimate the value of each state-action pair, and the expected return of each time step is calculated based on the actual reward received. Then, the Advantage function value is calculated, which is the difference between the expected return of the action and the state value predicted by the Critic network.

[0192] 3.44 Update the parameters of the Actor network That is, the Advantage function value is used as the loss function, the policy gradient is calculated, and the parameters of the Actor network are updated.

[0193] 3.45 Update the parameters of the Critic network That is, the mean square error loss function is used to update the parameters of the Critic network to reduce the error between the expected return and the actual return.

[0194] 3.46 Check whether the termination condition is met. If the current time step meets the termination condition, the learning process ends. Otherwise, return to step 3.42 to continue. The termination condition can be:

[0195] Condition 1: The value of the reward function changes little;

[0196] Condition 2: reaching the preset number of training rounds.

[0197] 3.47 Output the Actor network in step 3.44 to obtain the policy network.

[0198] 4. According to the trained strategy network, guide the actual routing decision of the second QKD, update the link weight according to the routing status, and use the quantum shortest search path algorithm to calculate the network routing with higher key distribution success rate and more balanced load.

[0199] Based on the same inventive concept, an embodiment of the present invention also provides a device and a computing device for quantum key distribution network routing. Since the principles of the problems solved by these devices and computing devices are similar to the aforementioned routing method of the quantum key distribution network, the implementation of the device and computing device can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0200] An embodiment of the present invention provides a device for quantum key distribution network routing, referring to Figure 6 As shown, including:

[0201] The training module 61 is used to iteratively perform the following steps through a preset simulation environment: collect the state of the first quantum key distribution QKD network for training at the current time step, and calculate the corresponding reward function value, input the state into the preset actor network, and output the corresponding action, wherein the action is the weight value of the link between each node in the updated QKD network; use the preset actor network-critic network training architecture, as well as the state, action and reward function value of each time step, to train the actor network until the preset training termination condition is reached; use the trained actor network as the policy network;

[0202] An acquisition module 62 is used to acquire the state of the link between each node in the second QKD network whose route is to be determined;

[0203] A weight value prediction module 63, used to use the state of each link between nodes in the current quantum key distribution network as the input of the policy network, and output the weight value of the link between each node in the second QKD network through the policy network;

[0204] The routing module 64 is used to calculate the optimal route of the second quantum key distribution network according to the weight value of each link output by the policy network.

[0205] Regarding the device for quantum key distribution network routing in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method and will not be elaborated here.

[0206] A computing device provided by an embodiment of the present invention includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the routing method of the quantum key distribution network as described above is implemented.

[0207] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the routing method of the quantum key distribution network as described above is implemented.

[0208] An embodiment of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the routing method of the quantum key distribution network as described above.

[0209] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0210] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0213] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A routing method for a quantum key distribution network, characterized in that: include: Through a preset simulation environment, the following steps are iteratively performed: the state of the current time step of the first quantum key distribution QKD network for training is collected, and the value of the corresponding reward function is calculated, the state is input into a preset actor network, and the corresponding action is output, where the action is the updated weight value of the link between each node in the QKD network; the reward function is a linear combination of the influencing parameters of the topological states of multiple quantum key distribution networks; Using a preset actor network-critic network training architecture, as well as the state, action, and reward function value at each time step, the actor network is trained until a preset training termination condition is reached; and the trained actor network is used as a policy network; Obtaining the status of links between nodes in the second QKD network to be routed; Using the state of the links between the nodes in the second QKD network as the input of the policy network, and outputting the weight value of the links between the nodes in the second QKD network through the policy network; The optimal route of the second QKD network is calculated according to the weight value of each link output by the policy network.

2. The method according to claim 1, characterized in that Before training the actor network, also include: Constructing a reward function and corresponding constraints; the reward function is a linear combination of the following parameters: the total path time of all quantum key distributions through the routing strategy, the quantum key freshness, the average local quantum key consumption rate, the average local quantum key generation rate, and the path quantum key utilization rate; The total time of the paths through which all quantum key distributions pass through the routing strategy is the sum of the propagation delay, transmission delay, processing delay and queuing delay of the paths through which all quantum key distribution processes pass through the routing selection strategy; The quantum key freshness represents the ratio of the amount of remaining quantum keys in all nodes in the quantum key distribution network to the maximum capacity of the quantum key pool of the node; The average local quantum key consumption rate represents the average value of the quantum key consumption rate of all inter-node links per unit time; The average local quantum key generation rate represents the average value of the quantum key generation rates of all inter-node links per unit time; The path quantum key utilization rate represents the proportion of the capacity of all quantum keys distributed on the link to the total capacity of the entire link; The constraints include: The total time of all quantum key distribution paths through the routing strategy is less than or equal to the delay required by the preset quality of service QoS; The minimum value of the key utilization of all quantum keys passing from one node to another is greater than or equal to the path quantum utilization required by the preset QoS; The average local quantum key consumption rate is less than the average local quantum key generation rate.

3. The method according to claim 2, characterized in that The calculating the reward function value comprises: Based on the constraints, for the first QKD network after performing the simulated quantum key distribution operation, the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate and path quantum key utilization rate of all quantum key distributions through the routing strategy are counted, and based on the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate and path quantum key utilization rate of all quantum key distributions through the routing strategy, the corresponding reward function value is calculated.

4. The method according to claim 1, characterized in that Through a preset simulation environment, the following steps are iteratively performed: the state of the current time step of the first quantum key distribution QKD network for training is collected, and the corresponding reward function value is calculated, the state is input into a preset actor network, and the corresponding action is output, and the action is the weight value of the link between each node in the QKD network after the update, specifically including: In a preset simulation environment, initializing the weight value of the link between each node in the first QKD network and initializing the state of the link between each node in the first QKD network; The state of the link between each node is used as the input of the actor network, and the weight value corresponding to the link between each node is output through the actor network; the weight value is the action of the current time step output by the routing actor network; the architecture of the actor network is a hybrid neural network composed of quantum-classical neural network; Determine the shortest quantum key distribution path from the source node to each node in the first QKD network using a preset quantum shortest path search algorithm according to the weight values ​​of the links between the nodes output by the actor network, and the source node and the destination node of each key byte packet; According to the shortest quantum key distribution path from the source node to each node in the first QKD network, determine the shortest quantum key distribution path from the source node to the destination node, perform a simulated quantum key distribution operation in the first QKD network, and calculate the reward function value corresponding to the action at the current time step according to the result of the simulated quantum key distribution operation; Repeat the following steps: determine the updated state of the links between the nodes according to the simulated quantum key distribution result of the previous time step in the first QKD network; input the updated state to the actor network, output the updated weight value corresponding to the link between the nodes through the actor network, and the weight value is the action of the next time step; update the shortest quantum key distribution path based on the weight value, perform the simulated quantum key distribution operation again, and calculate the reward function value corresponding to the next action according to the result of the simulated quantum key distribution operation, until the training of the actor network is terminated; The conditions for training termination include any of the following: The reward function value reaches convergence; The number of iterations has reached the preset training round threshold.

5. The method according to claim 4, characterized in that The architecture of the actor network includes: A first angle encoding unit, at least one layer of a first parameterized quantum circuit unit including parameters to be optimized, and a first classical neural network connected in sequence; the first parameterized quantum circuit includes a quantum circuit composed of a plurality of quantum logic gates; The architecture of the critic network includes: A second angle encoding unit, at least one layer of second parameterized quantum circuit unit with parameters to be optimized, and a second classical neural network are connected in sequence; the second parameterized quantum circuit includes a quantum circuit composed of multiple quantum logic gates.

6. The method according to claim 1, characterized in that Using the preset actor network-critic network training architecture, and the state, action, and reward function value at each time step, the actor network is trained until the preset training termination conditions are reached, including: Calculate the corresponding value function value for each time step action output by the actor network through the critic network; Calculating a Q value of the action; the Q value represents the expected return of the action; Calculating a value of an Advantage function, wherein the Advantage function is equal to a difference between the Q value and the value of the value function; The Advantage function is used as the loss function of the actor network, and according to the value of the Advantage function, gradient descent is performed, and back propagation is performed to update the parameters of the actor network until a preset training termination condition is reached.

7. The method according to claim 1, characterized in that Calculating the optimal route of the second quantum key distribution network according to the weight value of each link output by the policy network, including: According to the weight value of each link output by the strategy network, the shortest quantum key distribution path from the source node to each target node in the second QKD network is determined by a preset quantum shortest path search algorithm.

8. The method according to any one of claims 4, 5 and 7, characterized in that The quantum shortest path search algorithm is implemented by the following steps: Step 1: Initialize a list h of size N; the N data in the list represent the data from the source node v s To each node v in the network structure diagram i The sum of weights corresponding to the shortest path; and initialize the sum of weights corresponding to all shortest paths to positive infinity; Step 2: Determine the source node v s , and set the sth data in list h to 0, that is, h[v s ]=0; Step 3: Determine node i and run the quantum minimum search algorithm j times. The quantum minimum search algorithm is used to calculate the total weight P corresponding to the shortest path that can reach the current node i from the source node v through the adjacent node j of node i, and compare it with the total weight h[i] corresponding to the shortest path of the current node i. If h[i] < P, the value of h[i] remains unchanged; otherwise, h[i] = P. Among them s Arriving at the current node i, the sum of the weights P corresponding to the shortest path, and compare it with the sum of the weights h[i] corresponding to the shortest path of the current node i; if h[i] < P, the value of h[i] remains unchanged; otherwise, h[i] = P. Wherein Step 4: Traverse the source node v s All subsequent nodes, that is, i = {s+1, ... N}, and run step 3 again, and finally get the source node v s A list h of the sum of the weights of the shortest paths to each node in the network structure graph.

9. A device for quantum key distribution network routing, characterized in that: include: The training module is used to iteratively perform the following steps through a preset simulation environment: collect the state of the current time step of the first quantum key distribution QKD network used for training, and calculate the corresponding reward function value, input the state into the preset actor network, and output the corresponding action, wherein the action is the weight value of the link between each node in the updated QKD network; use the preset actor network-critic network training architecture, as well as the state, action and reward function value of each time step, to train the actor network until the preset training termination condition is reached; use the trained actor network as the policy network; An acquisition module, used to acquire the status of links between nodes in the second QKD network to be routed; A weight value prediction module, used to use the state of each link between nodes in the current quantum key distribution network as the input of the policy network, and output the weight value of the link between each node in the second QKD network through the policy network; A routing module is used to calculate the optimal route of the second quantum key distribution network according to the weight value of each link output by the policy network.

10. A computing device, characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the routing method for the quantum key distribution network as described in any one of claims 1 to 8 is implemented.

Images