A routing method of a quantum key distribution network and a related device

Abstract

The application discloses a kind of quantum key distribution network routing method and related device.The method includes: the state of first QKD network is collected, and the value of corresponding reward function is calculated, input state into the preset actor network, output corresponding action;Reward function is the linear combination of multiple topological state influence parameters;Using the training architecture of actor-critic network, in combination with state, action and reward function value, the actor network is trained until the training termination condition is reached;The trained actor network is used as a policy network;The state of link between each node in the second QKD network to be determined routing is input into the policy network, and the weight value of link between each node in the second QKD network is output by the policy network;According to the weight value output by the policy network, the optimal routing of the second QKD network is calculated.The application provides more intelligent, more robust policy network, to realize the routing strategy with higher distribution efficiency and more balanced load.

Description

Technical Field

[0001] This invention relates to the field of quantum communication technology, and in particular to a routing method and related apparatus for a quantum key distribution network. Background Technology

[0002] In recent years, quantum key distribution (QKD) networks have developed rapidly due to their theoretically unconditional security. Trusted relay QKD networks are QKD networks based on trusted nodes. By relaying key data hop-by-hop above the quantum key generation layer, they overcome the limitation of single-photon signal transmission distance, thus enabling QKD networks that can be used by more users and meet the needs of longer-distance transmission. Theoretically, this network can achieve global quantum key distribution and is considered the most feasible wide-area quantum key distribution network architecture under current technological conditions.

[0003] Unlike classical communication networks, QKD network routing implementations must consider not only metrics similar to those in classical communication networks, such as routing delay, but also evaluation metrics related to quantum channels. Therefore, QKD network routing implementations require evaluation and design that combine parameters related to both classical and quantum channels. Summary of the Invention

[0004] The inventors of this application have discovered that, to date, various routing algorithms have been developed to meet different network environments and requirements for QKD network routing implementations. These include open shortest path first (BSF) protocols, supervised learning-based routing algorithms, and traditional deep reinforcement learning-based routing algorithms. However, these algorithms have significant shortcomings in quantum key distribution networks (QKD networks) and struggle to effectively address the challenges of modern QKD networks. Existing technologies typically consider the amount of quantum key in a QKD network as a single influencing factor. Some existing routing algorithms require manually preset parameters for modeling, making them unsuitable for more complex network scenarios. Furthermore, the computational resources and training time required for traditional deep reinforcement learning-based routing algorithms increase significantly with network size.

[0005] In view of the above problems, the present invention is proposed to provide a routing method and related apparatus for a quantum key distribution network that overcomes or at least partially solves the above problems.

[0006] In a first aspect, embodiments of the present invention provide a routing method for a quantum key distribution network, comprising:

[0007] The following steps are executed iteratively in a preset simulation environment: the state of the first quantum key distribution (QKD) network used for training at the current time step is collected, and the corresponding reward function value is calculated. The state is input into a preset actor network, and the corresponding action is output. The action is the weight value of the link between each node in the updated QKD network. The reward function is a linear combination of the influence parameters of multiple quantum key distribution network topology states.

[0008] The actor network is trained using a pre-defined actor-critic network training architecture, along with the state, action, and reward function value at each time step, until a pre-defined training termination condition is met; the trained actor network is then used as the policy network.

[0009] Obtain the status of links between nodes in the second QKD network with routes to be determined;

[0010] The state of the links between each node in the second QKD network is used as the input to the policy network, and the weight values ​​of the links between each node in the second QKD network are output through the policy network.

[0011] The optimal route for the second QKD network is calculated based on the weight values ​​of each link output by the policy network.

[0012] In one embodiment, prior to training the actor network, the method further includes:

[0013] Construct a reward function and corresponding constraints; the reward function is a linear combination of the following parameters: total path time for all quantum key distributions through the routing strategy, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate.

[0014] Wherein, the total time of all quantum key distribution paths through the routing strategy is the sum of the propagation delay, transmission delay, processing delay and queuing delay of all quantum key distribution processes through the routing strategy;

[0015] The quantum key freshness is the ratio of the remaining amount of quantum keys in all nodes of the quantum key distribution network to the maximum capacity of the node's quantum key pool.

[0016] The average local quantum key consumption rate is the average value of the quantum key consumption rate of all inter-node links per unit time.

[0017] The average local quantum key generation rate is the average value of the quantum key generation rate of all inter-node links per unit time.

[0018] The path quantum key utilization rate characterizes the proportion of the capacity of all quantum keys distributed on the link to the total capacity of the entire link;

[0019] The constraints include:

[0020] The total path time for all quantum key distributions through the routing policy is less than or equal to the latency required by the preset Quality of Service (QoS) requirement.

[0021] The minimum key utilization rate of all quantum keys passing from one node to another is greater than or equal to the path quantum utilization rate of the preset QoS requirement.

[0022] The average local quantum key consumption rate is less than the average local quantum key generation rate.

[0023] In one embodiment, calculating the reward function value includes:

[0024] Based on the aforementioned constraints, for the first QKD network after performing simulated quantum key distribution, the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate of all quantum key distributions through the routing strategy are statistically analyzed. Based on the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate of all quantum key distributions through the routing strategy, the corresponding reward function value is calculated.

[0025] In one embodiment, the following steps are iteratively executed in a preset simulation environment: acquiring the state of the first quantum key distribution (QKD) network used for training at the current time step, calculating the corresponding reward function value, inputting the state into a preset actor network, and outputting the corresponding action, wherein the action is the updated weight value of the links between nodes in the QKD network; including:

[0026] In a preset simulation environment, initialize the weight values ​​of the links between each node in the first QKD network and initialize the state of the links between each node in the first QKD network.

[0027] The state of the links between each node is used as the input of the actor network, and the actor network outputs the weight values ​​corresponding to the links between each node; the weight values ​​are the actions of the current time step output by the routing actor network; the architecture of the actor network is a hybrid neural network composed of quantum-classical neural networks;

[0028] Based on the weight values ​​of the links between nodes output by the actor network, and the source and destination nodes of each key byte packet, the shortest quantum key distribution path from the source node to each node in the first QKD network is determined using a preset quantum shortest path search algorithm.

[0029] Based on the shortest quantum key distribution path from the source node to each node in the first QKD network, determine the shortest quantum key distribution path from the source node to the destination node, perform simulated quantum key distribution in the first QKD network, and calculate the reward function value corresponding to the action at the current time step based on the result of the simulated quantum key distribution operation.

[0030] Repeat the following steps: determine the updated state of the links between nodes based on the simulated quantum key distribution result of the first QKD network in the previous time step; input the updated state into the actor network, and output the updated weight values ​​corresponding to the links between nodes through the actor network, where the weight values ​​are the actions for the next time step; update the shortest quantum key distribution path based on the weight values, and perform the simulated quantum key distribution operation again, and calculate the reward function value corresponding to the next action based on the result of the simulated quantum key distribution operation, until the training of the actor network is terminated;

[0031] Training may be terminated under any of the following conditions:

[0032] The reward function value has reached convergence.

[0033] The number of iterations has reached the preset threshold for the number of training rounds.

[0034] In one embodiment, the architecture of the actor network includes:

[0035] The system consists of a first angle encoding unit, at least one layer of a first parameterized quantum circuit unit containing parameters to be optimized, and a first classical neural network connected in sequence; the first parameterized quantum circuit contains a quantum circuit composed of multiple quantum logic gates.

[0036] The architecture of the critic network includes:

[0037] The second angle encoding unit, at least one layer of second parameterized quantum circuit unit with parameters to be optimized, and the second classical neural network are connected in sequence; the second parameterized quantum circuit contains a quantum circuit composed of multiple quantum logic gates.

[0038] In one embodiment, the actor network is trained using a pre-defined actor-critic network training architecture, along with the state, action, and reward function values ​​at each time step, until a pre-defined training termination condition is met, including:

[0039] Through the critic network, for each action at each time step output by the actor network, calculate the corresponding value function value;

[0040] Calculate the Q-value of the action; the Q-value represents the expected return of the action;

[0041] Calculate the value of the Advantage function, where the Advantage function is equal to the difference between the Q-value and the value function value;

[0042] Take the Advantage function as the loss function of the actor network, and according to the value of the Advantage function, perform gradient descent and backpropagation to update the parameters of the actor network until the preset training termination condition is reached.

[0043] In one embodiment, according to the weight values of each link output by the policy network, calculating the optimal routing of the second quantum key distribution network includes:

[0044] According to the weight values of each link output by the policy network, through a preset quantum shortest path search algorithm, determine the shortest quantum key distribution path of the second quantum key distribution network from the source node to each target node in the second QKD network.

[0045] In one embodiment, the quantum shortest path search algorithm is implemented through the following steps:

[0046] Step 1: Initialize a list h of size N; the N data in the list respectively represent the sum of the weights corresponding to the shortest paths from the source node v s to each node v i in the network structure diagram; and initialize the sum of the weights corresponding to all the shortest paths to positive infinity;

[0047] Step 2: Determine the source node v s , and set the s-th data in the list h to 0, that is, h[v s = 0;

[0048] Step 3: Determine the node i, and run the quantum minimum search algorithm j times. The quantum minimum search algorithm is used to calculate the sum of the weights P corresponding to the shortest paths that can reach the current node i from the source node v s through the adjacent nodes h of the node i, and compare it with the sum of the weights h[i] corresponding to the shortest path of the current node i; if h[i] < P, the value of h[i] remains unchanged; otherwise h[i] = P. Where

[0049] Step 4: Traverse the source node ν sFor all subsequent nodes, i.e., i = {s+1, ..., N}, step three is run again to finally obtain the result from the source node ν. s The list h of the total weights corresponding to the shortest paths to each node in the network structure graph.

[0050] In a second aspect, embodiments of the present invention provide an apparatus for routing quantum key distribution networks, comprising:

[0051] The training module iteratively executes the following steps in a preset simulation environment: It collects the state of the first quantum key distribution (QKD) network used for training at the current time step and calculates the corresponding reward function value; inputs the state into a preset actor network and outputs the corresponding action, where the action is the updated weight value of the links between nodes in the QKD network; trains the actor network using a preset actor-critic network training architecture, along with the state, action, and reward function value at each time step, until a preset training termination condition is met; and uses the trained actor network as the policy network.

[0052] The acquisition module is used to acquire the status of the links between nodes in the second QKD network where the route to be determined is to be determined.

[0053] The weight prediction module is used to take the state of each link between nodes in the current quantum key distribution network as the input of the policy network, and output the weight values ​​of each link between nodes in the second QKD network through the policy network.

[0054] The routing module is used to calculate the optimal route for the second quantum key distribution network based on the weight values ​​of each link output by the policy network.

[0055] Thirdly, embodiments of the present invention provide a computing device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the routing method of the quantum key distribution network as described above.

[0056] Fourthly, embodiments of the present invention provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the routing method for a quantum key distribution network as described above.

[0057] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following:

[0058] The routing method for quantum key distribution networks provided in this invention utilizes a quantum reinforcement learning architecture to train a policy network to guide quantum key distribution strategies. During training, the state of the quantum key distribution network at different time steps is collected in a simulated environment using an actor-commentator network training architecture in quantum reinforcement learning. The reward function value is calculated (the reward function considers possible influence parameters of various quantum key distribution network topologies), and the commentator network evaluates the actions of the actor network to guide its training. This scheme applies a dynamic learning mechanism to continuously extract information from environmental feedback, update the weights of links between nodes, and continuously optimize the policy network. This allows for the training of a more intelligent and robust policy network, improving model accuracy and providing a routing strategy with higher distribution efficiency and more balanced load for practical quantum key distribution networks. Furthermore, this invention fully considers various factors that may affect the topology state of the quantum key distribution network, improving the comprehensiveness and accuracy of evaluation decisions, making it suitable for large-scale networking scenarios of quantum key distribution technology.

[0059] In this embodiment of the invention, multiple parameters affecting various quantum key distribution network topologies encompass various factors influencing quantum key distribution routing selection from multiple dimensions such as transmission latency, key utilization, freshness, generation rate, and consumption rate, which can more fully improve the comprehensiveness and accuracy of routing strategy evaluation.

[0060] In this embodiment of the invention, both the Actor network and the Critic network adopt a quantum-classical hybrid neural network architecture. Compared with classical neural networks, this network architecture can make full use of quantum superposition state encoding states, thereby reducing resource usage and training time during training, improving model accuracy, and enhancing training efficiency.

[0061] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0062] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0063] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0064] Figure 1This is a schematic diagram of a quantum cryptography network and its QKD network in an embodiment of the present invention;

[0065] Figure 2 This is a structural block diagram of the routing method of the quantum key distribution network in an embodiment of the present invention;

[0066] Figure 3A and 3B This is a schematic diagram of the actor network-critic network architecture in an embodiment of the present invention;

[0067] Figure 4A and Figure 4B This is a specific example diagram of the network architecture in an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram illustrating an example of a routing method for a quantum key distribution network in an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of the structure of the quantum key distribution network routing device in an embodiment of the present invention;

[0070] Figure label:

[0071] 31-First angle encoding unit; 32-First parameterized quantum circuit unit; 33-First classical neural network;

[0072] 34-Second Angle Encoding Unit; 35-Second Parameterized Quantum Circuit Unit; 36-Second Classical Neural Network;

[0073] 61-Training Module; 62-Acquisition Module; 63-Weight Prediction Module; 64-Route Module. Detailed Implementation

[0074] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0075] To facilitate understanding of this application, the present invention first provides a brief description of the architecture of existing quantum cryptography networks and their quantum key distribution (QKD) networks:

[0076] Quantum cryptography networks are secure communication networks that employ quantum cryptography. (See attached image.) Figure 1As shown, a quantum cryptography network is constructed from both a classical communication network and a QKD network. The QKD network mainly consists of QKD terminal devices and quantum links, used for quantum key distribution. The classical communication network uses quantum keys to encrypt and decrypt data and transmit encrypted data. A quantum cryptography network node typically consists of a classical communication terminal connected to the classical communication network and a QKD device terminal connected to the quantum communication network. Quantum cryptography network nodes are generally divided into two types: terminal nodes and relay nodes.

[0077] Specifically, classical communication links are used for control signaling transmission and quantum key negotiation. Quantum links are used to transmit qubit signals carried by quantum states, and can be physical media such as optical fibers or free space (e.g., satellite links). For a QKD network, the QKD terminal includes a QKD transmitter and a QKD receiver. The QKD transmitter is used to generate quantum keys and generate, modulate, and transmit qubit strings. The QKD receiver is used to receive, measure, and parse the qubit strings from the transmitter, and has functions such as key sequence comparison, filtering, and error rate detection. It also obtains the same symmetric quantum key as the transmitter, and the two can be connected through QKD relay devices.

[0078] The routing method for quantum key distribution networks provided in this embodiment of the invention offers an intelligent implementation scheme for QKD networks (e.g., trusted relay QKD networks) to provide network routing with higher quantum key distribution success rates and more balanced load. The specific implementation of the routing method for quantum key distribution networks provided in this embodiment of the invention will be described in detail below.

[0079] This invention provides a routing method for a quantum key distribution network, referring to... Figure 2 As shown, it includes the following steps:

[0080] S21. Through a preset simulation environment, iteratively execute the following steps: collect the state of the first quantum key distribution (QKD) network used for training at the current time step, calculate the value of the corresponding reward function, input the state into the preset actor network, and output the corresponding action, wherein the action is the weight value of the link between each node in the updated QKD network; train the actor network using the preset actor network-critic network training architecture, as well as the state, action and reward function value at each time step, until the preset training termination condition is reached;

[0081] In the above steps, the reward function is a linear combination of the influence parameters of the topological states of multiple quantum key distribution networks.

[0082] S22. Use the trained actor network as the policy network;

[0083] In this embodiment of the invention, the policy network is a neural network used to provide routing policies for a quantum key distribution network.

[0084] S23. Obtain the status of the links between nodes in the second QKD network where the route to be determined is to be determined;

[0085] S24. The state of the links between each node in the second QKD network is used as the input of the policy network, and the weight values ​​of the links between each node in the second QKD network are output through the policy network.

[0086] S25. Calculate the optimal route for the second QKD network based on the weight values ​​of each link output by the policy network.

[0087] In one embodiment, in order to easily distinguish between the quantum key distribution network used by the policy network during the training process and the quantum key distribution network that needs to be guided by the trained policy network to implement the actual routing scheme, the former is referred to as the first quantum key distribution network, i.e., the first QKD network, and the latter is referred to as the second quantum key distribution network, i.e., the second QKD network.

[0088] The routing method for quantum key distribution networks provided in this invention utilizes a quantum reinforcement learning architecture to train a policy network to guide quantum key distribution strategies. During training, the state of the quantum key distribution network at different time steps is collected in a simulated environment using an actor-commentator network training architecture in quantum reinforcement learning. The reward function value is calculated (the reward function considers possible influence parameters of various quantum key distribution network topologies), and the commentator network evaluates the actions of the actor network to guide its training. This invention employs a dynamic learning mechanism to continuously extract information from environmental feedback, update the weights of links between nodes, and continuously optimize the policy network. This allows for the training of a more intelligent and robust policy network, improving model accuracy and providing a routing strategy with higher distribution efficiency and more balanced load for practical quantum key distribution networks. Furthermore, this invention fully considers various factors that may affect the topology state of the quantum key distribution network, further improving the comprehensiveness and accuracy of evaluation decisions, making it suitable for large-scale networking scenarios of quantum key distribution technology.

[0089] Furthermore, in this embodiment of the invention, the training of the policy network can be completed offline, and its process is independent of the step of guiding the routing of the second QKD network through the policy network. This can greatly reduce the error process in the online environment, thereby avoiding interference with the stable operation of the second QKD network.

[0090] In one embodiment, the preset simulation environment includes simulation tools such as a quantum key distribution simulator.

[0091] A quantum key distribution simulator is a tool that uses software or computer programs to simulate the quantum key distribution process. Because building a real quantum key distribution system is costly and the experimental operation is complex, a quantum key distribution simulator can provide researchers with an environment to study and test quantum key distribution while saving money and time. Of course, the embodiments of this invention are not limited to this simulation tool.

[0092] In one embodiment, the parameters influencing the topology of various quantum key distribution networks may include, for example: the total path time for all quantum key distributions through the routing strategy, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate; wherein:

[0093] 1) The total path time for all quantum key distributions through the routing strategy. This parameter represents the sum of the path times for each quantum key distribution process through the routing strategy.

[0094] Specifically, the path time for each quantum key distribution process through the routing strategy is the sum of the propagation delay, transmission delay, processing delay, and queuing delay of the path through the routing strategy for that quantum key distribution process.

[0095] In other words, to calculate the total path time of all quantum key distributions through the routing strategy, we can first sum the propagation delay, transmission delay, processing delay, and queuing delay of each quantum key distribution process to obtain the total path time of that quantum key distribution process through the routing strategy. Then, we can sum the total path time of all quantum key distributions through the routing strategy to obtain the total path time of all quantum key distributions through the routing strategy.

[0096] The total path time for all quantum key distributions through routing strategies can be calculated, for example, using the following formula:

[0097]

[0098] In the above formula, T represents the total path time for all quantum key distributions through the routing strategy. k The path time for the k-th quantum key distribution through the routing strategy is the sum of propagation delay, transmission delay, processing delay, and queuing delay, i.e.:

[0099]

[0100] in Indicates whether the k-th quantum key passes through path l ij Distribute. This indicates that the k-th quantum key is obtained through path l. ij Distribute. This indicates that the k-th quantum key does not pass through path l. ij Distribute it. Let be the propagation delay of the k-th quantum key through the routing strategy. Let be the transmission delay of the k-th quantum key through the routing strategy. Let be the processing delay of the k-th quantum key through the routing strategy. Let be the queuing delay for the k-th quantum key to pass through the routing strategy. Where k∈K.

[0101] 2) Quantum key freshness, which is the ratio of the remaining amount of quantum keys in all nodes of the quantum key distribution network to the maximum capacity of the quantum key pool of the node;

[0102] Quantum key freshness can be expressed, for example, by the following formula:

[0103]

[0104] Where ∑ j R ij Let d(i) be the sum of the remaining quantum key quantities of all neighboring nodes of the i-th node, C be the maximum capacity of the quantum key pool, and d(i) be the degree of node i, where the degree is the number of neighboring nodes.

[0105] 3) Average local quantum key consumption rate, which represents the average quantum key consumption rate of all inter-node links per unit time;

[0106] The average local quantum key consumption rate characterizes the average quantum key consumption rate of all inter-node links in a quantum key distribution network per unit time.

[0107] 4) Average local quantum key generation rate, which represents the average quantum key generation rate of all inter-node links per unit time;

[0108] The average local quantum key generation rate characterizes the average quantum key generation rate of all inter-node links in a quantum key distribution network per unit time.

[0109] 5) Path quantum key utilization rate, which represents the proportion of the capacity of all quantum keys distributed on the link to the total capacity of the entire link;

[0110] The path quantum key utilization rate can be expressed by the formula:

[0111]

[0112] Where d k This represents the size of the k-th quantum key. Indicates that the k-th quantum key is obtained through path l ij Total capacity, B ij Indicates the entire path l ij Total capacity; η = ∑ ij η ij .

[0113] In quantum key distribution, assuming K packets (key byte packets) may be sent at a time, a utilization rate can be calculated for each packet. η ij via path l ij The total quantum key utilization rate when distributing K packets.

[0114] In path l ij In this process, not only quantum keys can be transmitted, but other information can also be transmitted. The path quantum key utilization rate is the proportion of quantum keys to the total capacity (bandwidth) of the entire channel.

[0115] The aforementioned parameters influencing various quantum key distribution network topologies encompass multiple dimensions, including transmission latency, key utilization, freshness, generation rate, and consumption rate, covering various factors affecting quantum key distribution routing selection. This can more comprehensively improve the evaluation of routing strategies and enhance their accuracy. However, the embodiments of this invention are not limited to the aforementioned parameters influencing quantum key distribution network topologies.

[0116] Accordingly, before steps S21 to S25 in the embodiments of the present invention, the steps of constructing a reward function and defining its constraints are also included.

[0117] Specifically, the reward function can be a linear combination of several parameters, including the total path time for all quantum key distributions through the routing strategy, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate.

[0118] For example, the reward function can be defined by the following formula:

[0119]

[0120] Where the parameters α, β, γ, λ, τ, and μ are all coefficients greater than zero; T represents the total path time for all quantum key distributions through the routing strategy; ζ represents the quantum key freshness; and η represents the sum of the quantum key utilization rates of the paths. This represents the average local quantum key consumption rate; T represents the average local quantum key generation rate; maxT represents the maximum total time of all paths through which quantum key distributions can be completed using routing strategies. Qos η represents the latency required to achieve Quality of Service (QoS) requirements. QoS This represents the quantum key utilization rate required for QoS.

[0121] The constraints of the above reward function include:

[0122] Constraint 1: The maximum total time of all paths accessible through the routing strategy for all quantum key distributions must be less than or equal to the latency required by the QoS (Quality of Service) requirement.

[0123] Constraint 2: The minimum utilization rate of all quantum keys passing through node i to node j must be greater than or equal to the QoS-required path quantum key utilization rate, i.e.

[0124] Constraint 3: The average local quantum key consumption rate must be less than the average local quantum key generation rate, i.e.

[0125] During the aforementioned actor network training process, when it is necessary to calculate the reward function, a quantum key distribution operation (sending several key byte packets) can be simulated in a simulation environment. Based on the results of the simulated quantum key distribution operation, the aforementioned T, ζ, η, and... η QoS The values ​​of the parameters are then substituted into the formula for the reward function, and the value of the reward function is calculated based on the constraints described above.

[0126] In one embodiment, in the actor network-critic network training architecture of the present invention, the actor network and the critic network can adopt similar network architectures, respectively referring to... Figure 3A and Figure 3B As shown, specifically, refer to Figure 3A As shown, the architecture of the actor network may include: a first angle encoding unit 31 connected in sequence, and at least one layer of first parameterized quantum circuit unit 32 containing the parameters to be optimized. Figure 3A The first classical neural network 33 consists of N layers and a first parameterized quantum circuit unit 32 containing a quantum circuit composed of multiple quantum logic gates.

[0127] Reference Figure 3B As shown, the architecture of the critic network includes: a second angle encoding unit 34, at least one layer of second parameterized quantum circuit unit 35 (M layer) with parameters to be optimized, and a second classical neural network 36 connected in sequence; the second parameterized quantum circuit unit 35 contains a quantum circuit composed of multiple quantum logic gates.

[0128] The output of the first angle encoding unit is connected to the input of the first-layer first parameterized quantum circuit unit 32, and the output of the last-layer first parameterized quantum circuit unit 32 is connected to the input of the first classical neural network 33. The structural relationship between the second angle encoding unit 34, the second parameterized quantum circuit unit 35, and the second classical neural network 36 is similar.

[0129] The first classical neural network 33 and the second classical neural network 36 mentioned above can be of the same type of neural network or different neural networks. Classical neural networks can be, for example, convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc., and the embodiments of the present invention are not limited thereto.

[0130] Examples of network architectures for actor and critic networks can be found in [reference needed]. Figure 4A and Figure 4B As shown, where, Figure 4A This is a schematic diagram of the Actor network architecture; Figure 4B This is a schematic diagram of the Critic network architecture.

[0131] Figure 4A The first parameterized quantum circuit unit 32 in the Actor network shown has D layers. A , Figure 4B The second parameterized quantum circuit unit 35 in the Critic network shown has a layer number of D. C After constructing the Actor and Critic networks, it is also necessary to initialize a set of parameters to be optimized for each network. and That is to and Set initial values.

[0132] The first angle encoding module 31 and the second angle encoding module 34 are used to encode the input state information.

[0133] from Figure 4A and Figure 4B As shown, both the Actor and Critic networks employ a quantum-classical hybrid neural network architecture. Compared to classical neural networks, this architecture, due to its quantum superposition encoding states, can represent more information with fewer bits (1 qubit can represent 2^365 qubits). N This method (using information from quantum superposition) can fully utilize the quantum superposition state encoding state, effectively reducing resource usage and training time during the training process, while improving model accuracy and training efficiency.

[0134] In one embodiment, the process of training the actor network in step S21 above can be performed offline. Specifically, this can be achieved through the following iterative process:

[0135] In a preset simulation environment, initialize the weight values ​​of the links between each node in the first QKD network and initialize the state of the links between each node in the first QKD network.

[0136] The state of the links between each node is used as the input to the actor network, and the actor network outputs the weight values ​​corresponding to the links between each node; the weight values ​​are the actions of the current time step output by the routing actor network.

[0137] Based on the weight values ​​of the links between nodes output by the actor network, and the source and destination nodes of each key byte packet, the shortest quantum key distribution path from the source node to each target node in the first QKD network is determined using a preset quantum shortest path search algorithm.

[0138] Based on the shortest quantum key distribution path from the source node to each node in the first QKD network, determine the shortest quantum key distribution path from the source node to the destination node, perform simulated quantum key distribution in the first QKD network, and calculate the reward function value corresponding to the action at the current time step based on the result of the simulated quantum key distribution operation.

[0139] Repeat the following steps: determine the updated state of the links between nodes based on the simulated quantum key distribution result of the previous time step in the first QKD network; input the updated state into the actor network, and output the updated weight values ​​corresponding to the links between nodes through the actor network. These weight values ​​are the actions for the next time step; update the shortest quantum key distribution path based on the weight values, and perform the simulated quantum key distribution operation again. Calculate the reward function value corresponding to the next action based on the result of the simulated quantum key distribution operation, until the training of the actor network is terminated.

[0140] The above-described quantum shortest path search algorithm determines "each node" in the shortest quantum key distribution path from the source node to each node in the first QKD network, including the destination node and relay nodes. In a trusted relay QKD network, the path from the source node to the destination node may involve several relay nodes. Therefore, the quantum shortest path search algorithm can calculate the shortest quantum key distribution path from the source node to any node in the first QKD network. After determining the source node and the destination node, the quantum shortest path from the source node → relay node → destination node can be determined.

[0141] In one embodiment, the termination condition for training the actor network described above could be, for example:

[0142] The reward function value changes little, indicating convergence. For example, after several iterations, if the reward function value changes within a small range compared to the previous time step, it can be considered convergent.

[0143] The number of iterations has reached the preset threshold for the number of training rounds.

[0144] For the actor-commentator network architecture of quantum reinforcement learning, the operation of simulating quantum key distribution is to simulate the operation of sending quantum key packets.

[0145] Within the actor-critic network architecture, training data for the actor network is collected through interaction with the environment. This is accomplished by performing actions, observing states, and receiving rewards.

[0146] In one embodiment, the state of the inter-node link can be defined in the following way:

[0147] The number of packets received, the number of packets sent, and the remaining quantum key quantity of each node in the link are used as the state, i.e., state = {in, out, R}. ij} Where in is the number of packets received by the node in the link, out is the number of packets sent, and R ij The remaining quantum key quantity is taken as the state.

[0148] At the beginning of training, both in and out can be set to 0, R ij Set to a non-zero value.

[0149] The above action means that at each time step, the weight of the link between the nodes corresponding to the input state is output.

[0150] After the weights of the links between nodes are output in the actor network, the preset quantum shortest path search algorithm can be used to update the shortest quantum key distribution path. When the quantum key distribution operation is simulated again in the first QKD network, the shortest path of quantum key distribution may have changed. For example, in the previous time step, the shortest path from node i to node j is the key, but in the next time step, after the link weights are adjusted, the shortest path may change to other links.

[0151] In one embodiment, the aforementioned quantum shortest path search algorithm is based on dynamic programming and quantum minimum search, and its goal is to calculate the source node v. s To each node in the network structure ν i A list of shortest paths. Specifically, this can be achieved through the following steps:

[0152] Step 1: Initialize a list h of size N; the N data in the list represent the data from the source node ν. sto each node ν in the network structure diagram i the sum of weights corresponding to the shortest path; and initialize the sum of weights corresponding to all shortest paths to positive infinity, i.e., h = {+∞,... +∞};

[0153] Step 2: Determine the source node ν s , and set the s-th data in the list h to 0, i.e., h[v s = 0;

[0154] Step 3: Determine the node i, run the quantum minimum search algorithm j times. The quantum minimum search algorithm is used to calculate the sum of weights P corresponding to the shortest path from the source node v s to the current node i through the adjacent node j of the node i, and compare it with the sum of weights h[i] corresponding to the shortest path of the current node i; if h[i] < P, the value of h[i] remains unchanged; otherwise, h[i] = P. Where

[0155] Step 4: Traverse all nodes after the source node v s , i.e., i = {s + 1,... N}, and run the above Step 3 again. Finally, obtain the list h of the sum of weights corresponding to the shortest paths from the source node v s to each node in the network structure diagram.

[0156] In the embodiment of the present invention, using the above quantum shortest path search algorithm to calculate the optimal link path can reduce the algorithm complexity and improve the calculation efficiency.

[0157] In the training framework of the actor network - critic network, the action usually means an operation that can be executed in a specific state. In the embodiment of the present invention, the action is the weight between the output node links.

[0158] The state describes the characteristics or situations at a certain moment. In the embodiment of the present invention, the state represents the state information such as the number of received packets, the number of sent packets, and the remaining quantum key amount of each node in the network after implementing a certain routing strategy.

[0159] The reward represents the feedback signal given by the environment to the actor network, and is used to measure the goodness or effectiveness of the actions taken. In the embodiment of the present invention, after implementing the quantum key packet distribution through the updated shortest path between nodes, calculate each parameter in the first QKD network and finally calculate the corresponding reward function value.

[0160] In reinforcement learning, the actor network and the critic network usually cooperate with each other. The actor network is responsible for selecting actions, and the critic network is responsible for evaluating the goodness or badness of these actions. Through continuous interaction and learning, the actor network can gradually improve its strategy to obtain better performance in the environment.

[0161] In this embodiment of the invention, a critic network is used to calculate the corresponding value function value for the action output by the actor network at each time step.

[0162] Calculate the Q-value (i.e., Quality value) of the action, which represents the expected reward of the action at each time step;

[0163] Calculate the value of the Advantage function, where the Advantage function is equal to the difference between the Q value and the value function value;

[0164] The Advantage function is used as the loss function of the actor network. Based on the value of the Advantage function, gradient descent is performed, and backpropagation is used to update the parameters of the actor network until the preset training termination condition is reached.

[0165] By calculating the Advantage function, the critic network can evaluate the relative advantage of each action, providing guidance to the actor network and helping it learn better strategies. The actor network then adjusts its behavior based on the Advantage information provided by the critic network to maximize long-term cumulative rewards.

[0166] Advantage functions can utilize temporal difference (TD) error, Monte Carlo sampling, or neural network-based approximation. They can effectively help actor networks better understand the value of actions, thereby guiding them to learn more effective strategies.

[0167] For the commentator network, it is also necessary to update the parameters of the commentator network at each time step using the mean squared error function.

[0168] In one embodiment, in step S25 above, based on the weight values ​​of each link output by the policy network, the shortest quantum key distribution path from the source node to each target node in the second quantum key distribution network is determined by the aforementioned quantum shortest path search algorithm.

[0169] In this embodiment of the invention, the process of the second quantum key distribution network determining the quantum key distribution route using the policy network can be repeated multiple times, and the training process of the policy network can also be executed multiple times (each training may use different data, and the network parameters obtained from the training may also be different) to gradually optimize the policy network. The two processes are independent of each other. Each time, the second quantum key distribution network can use the policy network obtained from the most recent training to guide its own quantum key distribution route in order to achieve the optimal routing distribution effect at that time.

[0170] The following specific example illustrates the routing method of the quantum key distribution network described above.

[0171] Reference Figure 5 As shown, the method includes the following steps: constructing a directed graph based on the environment network (first QKD network); training the actor network using reinforcement learning to obtain a policy network; then, through the trained policy network, performing quantum shortest path search to obtain the optimal path; and updating the routing table accordingly.

[0172] Specifically, the implementation process of this example is described as follows:

[0173] 1. Construct the network topology graph. Model the environment network (i.e., the first QKD network) as a directed graph G(V,E). Here, sets V and E represent the set of quantum nodes in the network and the set of edges connecting the nodes, respectively. ij This represents the edge between node i and node j. ij This represents the link between node i and node j.

[0174] 2. Determine the reward function and constraints.

[0175] For example, a reward function can be constructed by linearly combining several parameters, including the total path time for all quantum key distributions through the routing strategy, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate.

[0176] As mentioned above, the formula for the reward function is set as follows:

[0177] Where the parameters α, β, γ, λ, τ, and μ are all coefficients greater than zero; T represents the total path time for all quantum key distributions through the routing strategy; ζ represents the quantum key freshness; and η represents the sum of the quantum key utilization rates of the paths. This represents the average local quantum key consumption rate; T represents the average local quantum key generation rate; max T represents the maximum total time of all paths through which quantum key distributions can be completed using routing strategies. Qos η represents the latency required to achieve Quality of Service (QoS) requirements. QoS This represents the quantum key utilization rate required for QoS.

[0178] The constraint function, as described above, is set as follows:

[0179] Constraint 1: The maximum total time of all paths through which quantum key distribution is completed by the routing strategy must be less than or equal to the latency required by the Quality of Service (QoS).

[0180] Constraint 2: The minimum utilization of all quantum keys from node i to node j must be greater than or equal to the QoS requirement of the path quantum key utilization.

[0181] Constraint 3: The average local quantum key consumption rate must be less than the average local quantum key generation rate.

[0182] 3. Offline training of the policy network. The policy network is trained in a simulated environment using the actor-commentator architecture from quantum reinforcement learning. Specifically, this includes steps 3.1–3.4 below;

[0183] 3.1 Initialize the environment. Initialize the weight w between node i and node j in the routing table. ij If node i and node j are not linked, then the weight w is... ij Initialize to positive infinity.

[0184] In this step, using the aforementioned directed graph G(V,E), the weights of the links between each node and other nodes can be obtained, such as the link weights between node 1 and node 2, node 3, ..., node N, thus obtaining the routing table. If there are no links between nodes, the weights of the links between them are positive infinity.

[0185] 3.2 Determine and initialize the state. The number of packets received, the number of packets sent, and the remaining quantum key quantity of each node in the link are used as the state, i.e., state = {in, out, K}. rest}, and set both parameters in and out to 0, K rest Set to a non-zero value.

[0186] 3.3 Quantum state encoding of the state. The initial state is encoded using angle encoding.

[0187] 3.4 Training the policy network. Based on the current state and such... Figure 4A and 4B The quantum-classical hybrid neural network shown generates actions, selects the next action to be executed in a continuous action space, and then executes that action, i.e., updates the weights w of the pathway. ij .

[0188] 3.41 Reference Figure 4A and 4B As shown, two quantum circuits are constructed to implement the Actor network and the Critic network respectively, and a set of parameters to be optimized for each network is initialized. and

[0189] 3.42 Collecting experience samples to train the Actor network. Following the Actor network's policy, exploration is performed iteratively at different time steps: acquiring experience samples from the environment, including states, actions, and rewards. Specifically, the quantum key distribution simulator (i.e., the environment in quantum reinforcement learning) reads the updated routing table and, based on the start and destination points of each key byte packet, uses a quantum shortest path search algorithm to find the shortest quantum key distribution path, records state values ​​reflecting the link's quantum key distribution status, and simultaneously returns the calculated reward function value.

[0190] 3.43 Calculate the expected return and Advantage function using the Critic network.

[0191] The value of each state-action pair is estimated using a Critic network, and the expected return at each time step is calculated by combining this with the actual reward received. Then, the Advantage function value is calculated, which is the difference between the expected return of the action and the state value predicted by the Critic network.

[0192] 3.44 Updates the parameters of the Actor network That is, the Advantage function value is used as the loss function to calculate the policy gradient and update the parameters of the Actor network.

[0193] Version 3.45 updates the parameters of the Critic network. That is, the parameters of the Critic network are updated using the mean squared error loss function to reduce the error between the expected return and the actual return.

[0194] 3.46 Check if the termination condition is met. If the current time step meets the termination condition, end the learning process. Otherwise, return to step 3.42 and continue. The termination condition can be:

[0195] Condition 1: The value of the reward function changes relatively little;

[0196] Condition 2: Reach the preset number of training rounds.

[0197] 3.47 Output the Actor network from step 3.44, which is the policy network.

[0198] 4. Based on the trained policy network, guide the actual routing decisions of the second QKD, update the link weights according to the routing status, and use the quantum shortest search path algorithm to calculate the network route with higher key distribution success rate and more balanced load.

[0199] Based on the same inventive concept, embodiments of the present invention also provide a device and computing device for routing quantum key distribution networks. Since the principle of solving the problem by these devices and computing devices is similar to the aforementioned routing method for quantum key distribution networks, the implementation of these devices and computing devices can refer to the implementation of the aforementioned method, and repeated details will not be repeated.

[0200] This invention provides an apparatus for routing quantum key distribution networks, referring to... Figure 6 As shown, it includes:

[0201] Training module 61 is used to iteratively execute the following steps in a preset simulation environment: acquiring the state of the first quantum key distribution (QKD) network for training at the current time step and calculating the corresponding reward function value; inputting the state into a preset actor network and outputting the corresponding action, wherein the action is the updated weight value of the link between each node in the QKD network; training the actor network using a preset actor network-critic network training architecture, as well as the state, action, and reward function value at each time step, until a preset training termination condition is reached; and using the trained actor network as the policy network.

[0202] The acquisition module 62 is used to acquire the status of the links between nodes in the second QKD network with routes to be determined;

[0203] The weight prediction module 63 is used to take the state of each link between nodes in the current quantum key distribution network as the input of the policy network, and output the weight value of each link between nodes in the second QKD network through the policy network.

[0204] The routing module 64 is used to calculate the optimal route for the second quantum key distribution network based on the weight values ​​of each link output by the policy network.

[0205] Regarding the quantum key distribution network routing apparatus in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0206] An embodiment of the present invention provides a computing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the routing method of the quantum key distribution network as described above.

[0207] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the routing method for a quantum key distribution network as described above.

[0208] An embodiment of the present invention provides a computer program product, which includes a computer program that, when executed by a processor, implements the routing method for a quantum key distribution network as described above.

[0209] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0210] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0211] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0212] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0213] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A routing method for a quantum key distribution network, characterized in that, include: The following steps are executed iteratively in a preset simulation environment: the state of the first quantum key distribution (QKD) network used for training at the current time step is collected, and the corresponding reward function value is calculated. The state is input into a preset actor network, and the corresponding action is output. The action is the weight value of the link between each node in the updated QKD network. The reward function is a linear combination of the influence parameters of multiple quantum key distribution network topology states. The actor network is trained using a pre-defined actor-critic network training architecture, along with the state, action, and reward function value at each time step, until a pre-defined training termination condition is met; the trained actor network is then used as the policy network. Obtain the status of links between nodes in the second QKD network with routes to be determined; The state of the links between each node in the second QKD network is used as the input to the policy network, and the weight values ​​of the links between each node in the second QKD network are output through the policy network. Based on the weight values ​​of each link output by the policy network, calculate the optimal route for the second QKD network; Before training the actor network, the following is also included: Construct a reward function and corresponding constraints; the reward function is a linear combination of the following parameters: total path time for all quantum key distributions through the routing strategy, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate. Wherein, the total time of all quantum key distribution paths through the routing strategy is the sum of the propagation delay, transmission delay, processing delay and queuing delay of all quantum key distribution processes through the routing strategy; The quantum key freshness is the ratio of the remaining amount of quantum keys in all nodes of the quantum key distribution network to the maximum capacity of the node's quantum key pool. The average local quantum key consumption rate is the average value of the quantum key consumption rate of all inter-node links per unit time. The average local quantum key generation rate is the average value of the quantum key generation rate of all inter-node links per unit time. The path quantum key utilization rate characterizes the proportion of the capacity of all quantum keys distributed on the link to the total capacity of the entire link; The constraints include: The total path time for all quantum key distributions through the routing policy is less than or equal to the latency required by the preset Quality of Service (QoS) requirement. The minimum key utilization rate of all quantum keys passing from one node to another is greater than or equal to the path quantum utilization rate of the preset QoS requirement. The average local quantum key consumption rate is less than the average local quantum key generation rate.

2. The method as described in claim 1, characterized in that, The calculation of the reward function value includes: Based on the aforementioned constraints, for the first QKD network after performing simulated quantum key distribution, the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate of all quantum key distributions through the routing strategy are statistically analyzed. Based on the total path time, quantum key freshness, average local quantum key consumption rate, average local quantum key generation rate, and path quantum key utilization rate of all quantum key distributions through the routing strategy, the corresponding reward function value is calculated.

3. The method as described in claim 1, characterized in that, The following steps are iteratively executed in a preset simulation environment: The state of the first quantum key distribution (QKD) network used for training at the current time step is acquired, and the corresponding reward function value is calculated. The state is then input into a preset actor network, and the corresponding action is output. The action is the updated weight value of the links between nodes in the QKD network, specifically including: In a preset simulation environment, initialize the weight values ​​of the links between each node in the first QKD network and initialize the state of the links between each node in the first QKD network. The state of the links between each node is used as the input of the actor network, and the actor network outputs the weight values ​​corresponding to the links between each node; the weight values ​​are the actions of the current time step output by the routing actor network; the architecture of the actor network is a hybrid neural network composed of quantum-classical neural networks; Based on the weight values ​​of the links between nodes output by the actor network, and the source and destination nodes of each key byte packet, the shortest quantum key distribution path from the source node to each node in the first QKD network is determined using a preset quantum shortest path search algorithm. Based on the shortest quantum key distribution path from the source node to each node in the first QKD network, determine the shortest quantum key distribution path from the source node to the destination node, perform simulated quantum key distribution in the first QKD network, and calculate the reward function value corresponding to the action at the current time step based on the result of the simulated quantum key distribution operation. Repeat the following steps: determine the updated state of the links between nodes based on the simulated quantum key distribution result of the first QKD network in the previous time step; input the updated state into the actor network, and output the updated weight values ​​corresponding to the links between nodes through the actor network, where the weight values ​​are the actions for the next time step; update the shortest quantum key distribution path based on the weight values, and perform the simulated quantum key distribution operation again, and calculate the reward function value corresponding to the next action based on the result of the simulated quantum key distribution operation, until the training of the actor network is terminated; Training may be terminated under any of the following conditions: The reward function value has reached convergence. The number of iterations has reached the preset threshold for the number of training rounds.

4. The method as described in claim 3, characterized in that, The architecture of the actor network includes: The system consists of a first angle encoding unit, at least one layer of a first parameterized quantum circuit unit containing parameters to be optimized, and a first classical neural network connected in sequence; the first parameterized quantum circuit contains a quantum circuit composed of multiple quantum logic gates. The architecture of the critic network includes: The second angle encoding unit, at least one layer of second parameterized quantum circuit unit with parameters to be optimized, and the second classical neural network are connected in sequence; the second parameterized quantum circuit contains a quantum circuit composed of multiple quantum logic gates.

5. The method as described in claim 1, characterized in that, The actor network is trained using a pre-defined actor-critic network training architecture, along with the state, action, and reward function values ​​at each time step, until a pre-defined training termination condition is met, including: Using the critic network, calculate the corresponding value function value for the action at each time step output by the actor network; Calculate the Q-value of the action; the Q-value represents the expected reward of the action; Calculate the value of the Advantage function, which is equal to the difference between the Q value and the value function value; The Advantage function is used as the loss function of the actor network. Gradient descent is performed based on the value of the Advantage function, and backpropagation is used to update the parameters of the actor network until the preset training termination condition is reached.

6. The method as described in claim 1, characterized in that, Based on the weight values ​​of each link output by the policy network, the optimal route for the second QKD network is calculated, including: Based on the weight values ​​of each link output by the policy network, the shortest quantum key distribution path from the source node to each target node in the second QKD network is determined by a preset quantum shortest path search algorithm.

7. The method according to any one of claims 3, 4, and 6, characterized in that, The quantum shortest path search algorithm is implemented through the following steps: Step 1: Initialize a space of size 1 list The N data points in the list represent data from the source node. To each node in the network structure graph The sum of weights corresponding to the shortest paths; and initialize the sum of weights corresponding to all shortest paths to positive infinity; Step 2: Determine the source node and list The Middle Each data point is set to 0, that is... ; Step 3: Determine the nodes ,run The sub-quantum minimum search algorithm is used to calculate all nodes that can be accessed. Adjacent nodes From the source node to the current node The sum of weights corresponding to the shortest path and with the current node The sum of weights corresponding to the shortest path Compare; if ,but The value remains unchanged; otherwise, the value remains unchanged. ;in ; Step 4: Traverse the source node All subsequent nodes, i.e. Then run step three again to finally obtain the result from the source node. List of weights corresponding to the shortest paths to each node in the network structure graph .

8. A device for routing in a quantum key distribution network, characterized in that, include: The training module iteratively executes the following steps in a preset simulation environment: It collects the state of the first quantum key distribution (QKD) network used for training at the current time step and calculates the corresponding reward function value; inputs the state into a preset actor network and outputs the corresponding action, where the action is the updated weight value of the links between nodes in the QKD network; trains the actor network using a preset actor-critic network training architecture, along with the state, action, and reward function value at each time step, until a preset training termination condition is met; and uses the trained actor network as the policy network. The acquisition module is used to acquire the status of the links between nodes in the second QKD network where the route to be determined is to be determined; The weight prediction module is used to take the state of each link between nodes in the current quantum key distribution network as the input of the policy network, and output the weight values ​​of each link between nodes in the second QKD network through the policy network. The routing module is used to calculate the optimal route for the second QKD network based on the weight values ​​of each link output by the policy network.

9. A computing device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the routing method for a quantum key distribution network as described in any one of claims 1-7.

Images