Quantum algorithm compiling method and system based on multi-agent reinforcement learning

Abstract

This invention relates to the field of quantum computing technology and discloses a quantum algorithm compilation method and system based on multi-agent reinforcement learning. The method simplifies high-level quantum algorithms by resolving them as unitarily equivalent ZX-graphs, ensuring their unitary equivalence. Furthermore, it constructs a Markov decision process environment for multiple agents using multiple graph features, graph feature vectors, and quantum mapping information of the target quantum hardware. This environment is then used to train the multiple agents through reinforcement learning, resulting in a well-trained multi-agent model. Through multi-agent collaborative optimization of the quantum circuit structure and hardware mapping, the optimal compilation scheme is obtained while maintaining unitary equivalence. This approach balances the structure of the quantum algorithm itself with the dynamic characteristics of the target hardware, avoiding the accumulation of execution errors and exhibiting strong adaptability and generalization capabilities, thus supporting the stable operation of high-level quantum algorithms on real hardware.

Description

Technical Field

[0001] This invention relates to the field of quantum computing technology, and in particular to a quantum algorithm compilation method and system based on multi-agent reinforcement learning. Background Technology

[0002] High-level quantum algorithms (such as quantum Fourier transform and quantum approximation optimization algorithms) are theoretical computational logic that cannot be directly executed by quantum hardware. They need to be compiled into native instructions that quantum hardware can recognize. Quantum algorithm compilation is the core link connecting high-level quantum algorithms and low-level quantum hardware. It is the hub for realizing the execution of high-level quantum algorithms. Therefore, the effectiveness of quantum algorithm compilation directly determines the execution efficiency and accuracy of quantum algorithms on hardware.

[0003] Currently, quantum algorithm compilation only considers the deployment effect in a single dimension. In actual hardware deployment, it is difficult to take into account the multi-dimensional deployment effect, resulting in poor actual deployment performance. Furthermore, existing compilation methods on NISQ devices often ignore the spatiotemporal heterogeneity of T1 / T2 decoherence fluctuations and gate error rates, neglect the dynamic characteristics of the hardware, and the mapping scheme does not prioritize high-fidelity hardware resources, leading to the accumulation of execution errors, poor adaptability, and weak generalization ability, making it difficult to support the stable operation of high-level quantum algorithms on real hardware. Summary of the Invention

[0004] In view of this, in order to solve the above-mentioned technical problems, the present invention provides a quantum algorithm compilation method and system based on multi-agent reinforcement learning. It solves the technical problems that existing compilation methods often fail to support the stable operation of high-level quantum algorithms on real hardware because they ignore the spatiotemporal heterogeneity of T1 / T2 decoherence fluctuations and gate error rates, ignore the dynamic characteristics of hardware, and fail to prioritize high-fidelity hardware resources in the mapping scheme.

[0005] The first aspect of this invention provides a quantum algorithm compilation method based on multi-agent reinforcement learning, comprising:

[0006] The high-level quantum algorithm resolves the ZX-graph into a unitary equivalent, extracts multiple graph features from the ZX-graph, and fuses these multiple graph features into a graph feature vector.

[0007] Obtain the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm, and construct a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors and quantum mapping information;

[0008] Based on the Markov decision process environment of the multi-agent, the multi-agent is trained by reinforcement learning to obtain a trained multi-agent model.

[0009] Obtain the quantum algorithm to be compiled in the current computing period and the current quantum mapping information of the target quantum hardware corresponding to the quantum algorithm to be compiled. Based on the quantum algorithm to be compiled and the current quantum mapping information, and in combination with the trained multi-agent model, output the optimal compilation scheme of the quantum algorithm to be compiled.

[0010] In one example, the process of resolving high-level quantum algorithms into unitarily equivalent ZX-graphs, extracting multiple graph features from the ZX-graph, and fusing these multiple graph features into a graph feature vector includes:

[0011] The single quantum gate and the two quantum gate in the high-level quantum algorithm are respectively mapped to spider nodes and directed edges between spider nodes in the ZX-graph to generate the initial ZX-graph;

[0012] The initial ZX-graph is preprocessed and simplified based on the ZX calculus rules to obtain a preprocessed ZX-graph; wherein the ZX calculus rules include at least one of the following: spider fusion, zero node removal, and phase merging.

[0013] The preprocessed ZX-graph is subjected to unitary equivalence verification. After feature encoding of the preprocessed ZX-graph that passes the unitary equivalence verification, multiple graph features of the preprocessed ZX-graph that passes the unitary equivalence verification are extracted.

[0014] After standardizing multiple graph features, the standardized graph features are merged into a graph feature vector.

[0015] In one example, constructing a multi-agent Markov decision process environment based on multiple graph features, the graph feature vector, and the quantum mapping information includes:

[0016] A smart agent framework is constructed, consisting of a route optimization smart agent, a mapping scheduling smart agent, and a global coordination super smart agent. The state space, action space, and reward function of each smart agent are defined based on multiple graph features, graph feature vectors, and quantum mapping information.

[0017] Based on the state space, action space, and reward function defined by each agent, a Markov decision process environment for the multi-agent system is constructed.

[0018] In one example, the graph features include line depth, total number of gates, number of two-qubit gates, number of nodes, and number of edges;

[0019] The quantum mapping information includes a hardware topological adjacency matrix, a logical-physical mapping matrix, hardware calibration parameters, and two quantum gate queues to be executed;

[0020] The step of defining the state space, action space, and reward function for each agent based on multiple graph features, the graph feature vector, and the quantum mapping information includes:

[0021] The state space of the circuit optimization agent includes graph feature vectors, circuit depth, total number of gates, and number of two-qubit gates.

[0022] The action space of the line optimization agent includes multiple ZX calculus rewriting actions that satisfy unitary equivalence constraints.

[0023] The reward function of the line optimization agent is determined by a weighted combination of changes in the total number of gates, changes in line depth, and equivalence fidelity deviation.

[0024] The state space of the mapping scheduling agent includes line dependency graph features, hardware topology adjacency matrix, logical-physical mapping matrix, hardware calibration parameters, and two quantum gate queues to be executed; wherein, the line dependency graph features are obtained by parsing the quantum gate execution order of the ZX-graph, constructing a directed acyclic graph, and extracting features from the directed acyclic graph using a graph attention network;

[0025] The action space of the mapping scheduling agent includes multiple hardware mapping actions;

[0026] The reward function of the mapping scheduling agent is determined by a weighted combination of the cumulative number of SWAP gates inserted after the action is executed, the compilation delay of the action, and the predicted value of the execution fidelity of the mapping scheme.

[0027] The state space of the global coordination super agent is a global state composed of the states of the route optimization agent and the mapping scheduling agent.

[0028] The action space of the global coordination super agent is the global action composed of the actions of the route optimization agent and the actions of the mapping scheduling agent.

[0029] The reward function of the global coordination super agent is a weighted sum of the reward function of the circuit optimization agent, the reward function of the mapping scheduling agent, and the global performance reward. The global performance reward is a weighted combination of the circuit depth, the number of two-qubit gates, the execution fidelity, and the compilation latency of the quantum circuit compiled by the quantum algorithm on the target hardware.

[0030] In one example, the Markov decision process environment based on the multi-agent agent is used to train the multi-agent agent through reinforcement learning to obtain a trained multi-agent model, including:

[0031] Based on a centralized training and distributed execution architecture, a policy network for each agent and a centralized critic network shared by all agents are constructed.

[0032] Based on the MAPPO algorithm, and combined with the Markov decision process environment of the route optimization agent and the mapping scheduling agent, independent policy grid updates are performed on the route optimization agent and the mapping scheduling agent respectively to obtain the basic route optimization agent and the basic mapping scheduling agent.

[0033] Based on the MAPPO algorithm, and combined with the Markov decision process environment of the route optimization agent and the mapping scheduling agent, the policy grid of the basic route optimization agent and the basic mapping scheduling agent is jointly optimized through the centralized critic network to obtain the cooperative route optimization agent and the cooperative mapping scheduling agent.

[0034] Based on the MAPPO algorithm and combined with the Markov decision process environment of each agent, joint policy optimization is performed on the policy grid of the cooperative route optimization agent, the cooperative mapping scheduling agent, and the global coordination super agent to obtain the trained route optimization agent, the trained mapping scheduling agent, and the trained coordination super agent.

[0035] The trained multi-agent model is constructed based on the trained route optimization agent, the trained mapping scheduling agent, and the trained coordination super-agent.

[0036] In one example, the step of outputting the optimal compilation scheme for the quantum algorithm to be compiled, based on the quantum algorithm to be compiled and the current quantum mapping information, combined with the trained multi-agent model, includes:

[0037] The quantum algorithm to be compiled is parsed into the current ZX-graph, multiple current graph features of the current ZX-graph are extracted, and the multiple current graph features are fused into a current graph feature vector;

[0038] Based on the trained multi-agent model, and combining the current ZX-graph, the multiple current graph features, the current graph feature vector, and the current quantum mapping information, the optimal compilation scheme for the quantum algorithm to be compiled is obtained; wherein, the optimal compilation scheme includes the optimal circuit optimization action sequence and the optimal hardware mapping action matrix;

[0039] The step of outputting the optimal compilation scheme for the quantum algorithm to be compiled based on the quantum algorithm to be compiled and the current quantum mapping information, combined with the trained multi-agent model, further includes:

[0040] The current ZX-graph is rewritten according to the optimal route optimization action sequence to obtain the optimized ZX-graph;

[0041] Based on the mapping rules between ZX-graphs and quantum gates, the optimized ZX-graph is transformed into the native quantum gate sequence of the target quantum hardware corresponding to the quantum algorithm to be compiled;

[0042] The logical bits in the original quantum gate sequence are replaced with the physical bit numbers in the optimal hardware mapping action matrix, and a compilation instruction set is generated based on the physical bit numbers, combined with the added SWAP gate instructions and coupler control instructions.

[0043] In one example, this method also includes:

[0044] Obtain compilation execution data after executing the optimal compilation scheme, and determine the performance index of the optimal compilation scheme based on the compilation execution data; wherein, the performance index includes at least one of the following: circuit depth reduction rate, two-qubit gate reduction rate, execution fidelity, compilation latency, and robustness;

[0045] Determine whether the performance indicators meet the preset performance indicator conditions;

[0046] If all the performance metrics meet the preset performance metric conditions, then the final compilation scheme is output.

[0047] If any of the performance indicators fails to meet the preset performance indicator conditions, a deviation value is calculated based on the unmet performance indicator and the preset performance indicator conditions. The deviation value is then normalized, and the normalized deviation value is incorporated into the reward function of the multi-agent as an additional reward signal. The weights in the reward function are then adjusted.

[0048] Based on the adjusted reward function and weights, the multi-agent model is retrained and a new optimal compilation scheme is output. Based on the new optimal compilation scheme, the compilation execution data after obtaining the optimal compilation scheme is re-executed. The performance indicators of the optimal compilation scheme are determined based on the compilation execution data until all performance indicators meet the preset performance indicator conditions.

[0049] Secondly, the present invention also provides a quantum algorithm compilation system based on multi-agent reinforcement learning, comprising:

[0050] The quantum algorithm parsing module is used to parse high-level quantum algorithms into unitary equivalent ZX-graphs, extract multiple graph features of the ZX-graphs, and fuse the multiple graph features into a graph feature vector;

[0051] The multi-agent conversion module is used to obtain the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm, and to construct a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors and quantum mapping information.

[0052] A multi-agent training module is used to perform reinforcement learning training on the multi-agent based on the Markov decision process environment of the multi-agent to obtain a trained multi-agent model.

[0053] The compilation scheme determination module is used to obtain the quantum algorithm to be compiled in the current computing period and the current quantum mapping information of the target quantum hardware corresponding to the quantum algorithm to be compiled. Based on the quantum algorithm to be compiled and the current quantum mapping information, and in combination with the trained multi-agent model, the module outputs the optimal compilation scheme for the quantum algorithm to be compiled.

[0054] Thirdly, the present invention also provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the quantum algorithm compilation method based on multi-agent reinforcement learning as described in the first aspect.

[0055] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the steps of the quantum algorithm compilation method based on multi-agent reinforcement learning as described in the first aspect.

[0056] As can be seen from the above technical solutions, this invention simplifies high-level quantum algorithms by resolving them into unitarily equivalent ZX-graphs, ensuring their unitary equivalence. Furthermore, it constructs a multi-agent Markov decision process environment using multiple graph features, graph feature vectors, and quantum mapping information of the target quantum hardware from the ZX-graph. This environment is used to train the multi-agents through reinforcement learning, resulting in a well-trained multi-agent model. Through multi-agent collaborative optimization of the quantum circuit structure and hardware mapping, the optimal compilation scheme is obtained while maintaining unitary equivalence. This approach balances the structure of the quantum algorithm itself with the dynamic characteristics of the target hardware, avoiding the accumulation of execution errors. It exhibits strong adaptability and generalization ability, supporting the stable operation of high-level quantum algorithms on real hardware. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 An application environment diagram of a quantum algorithm compilation method based on multi-agent reinforcement learning provided in an embodiment of the present invention;

[0059] Figure 2 A flowchart illustrating a quantum algorithm compilation method based on multi-agent reinforcement learning, provided for an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of the structure of a quantum algorithm compilation system based on multi-agent reinforcement learning provided in an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0062] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The quantum algorithm compilation method based on multi-agent reinforcement learning provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102 or placed on a cloud or other network server. Terminal 101 or server 102 executes a quantum algorithm compilation method based on multi-agent reinforcement learning. This method includes: resolving the high-level quantum algorithm into a unitarily equivalent ZX-graph, extracting multiple graph features of the ZX-graph, and fusing these features into a graph feature vector; obtaining the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm; constructing a multi-agent Markov decision process environment based on the multiple graph features, graph feature vector, and quantum mapping information; performing reinforcement learning training on the multi-agent agents based on the multi-agent Markov decision process environment to obtain a trained multi-agent model; obtaining the quantum algorithm to be compiled in the current computation period and the current quantum mapping information of the target quantum hardware corresponding to the quantum algorithm to be compiled; and outputting the optimal compilation scheme for the quantum algorithm to be compiled based on the quantum algorithm to be compiled and the current quantum mapping information, combined with the trained multi-agent model.

[0064] Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets.

[0065] Server 102 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0066] like Figure 2 As shown, this application provides a quantum algorithm compilation method based on multi-agent reinforcement learning, which is applied to... Figure 1 Taking terminal 101 or server 102 as an example, the explanation includes the following steps S1 to S4. Wherein:

[0067] Step S1: The high-level quantum algorithm resolves the ZX-graph into a unitary equivalent, extracts multiple graph features of the ZX-graph, and fuses the multiple graph features into a graph feature vector.

[0068] Among them, high-level quantum algorithms (the industry-standard QASM / OpenQASM format) are theoretical computational logics that convert various quantum gates in high-level quantum algorithms into graphical forms of ZX calculus, namely ZX-graphs.

[0069] ZX-graphs are a graphical formalism-based quantum computing representation method that can concisely describe the unitary transformation of quantum gates and the entanglement between qubits. Its core rewriting rules (spider fusion, local complementarity, phase merging, etc.) can simplify quantum circuits while maintaining unitary equivalence.

[0070] Among them, unitary equivalence means that the ZX-graph strictly maintains the unitary evolution property of the original quantum circuit in mathematics, which is verified by the fidelity of the ZX-graph.

[0071] Step S2: Obtain the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm, and construct a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors and quantum mapping information.

[0072] Among them, the target quantum hardware is the physical quantum hardware deployed by the high-level quantum algorithm, and the quantum mapping information is a structured representation of the prior knowledge of the quantum hardware, such as topological connection constraints, gate set restrictions, noise parameters and calibration data, including the hardware topological adjacency matrix, logical-physical mapping matrix, hardware calibration parameters and two quantum gate queues to be executed.

[0073] In this application, a multi-agent system is constructed, with each agent undertaking a specific sub-task: Agent-1 performs circuit simplification based on the ZX calculus rewriting rules, Agent-2 is responsible for the dynamic remapping of logical bits to physical bits, and SuperAgent coordinates the overall situation and fine-tunes the collaborative state of Agent-1 and Agent-2 to achieve global optimization.

[0074] Step S3: Based on the Markov decision process environment of multi-agent agents, perform reinforcement learning training on the multi-agent agents to obtain a trained multi-agent model.

[0075] Step S4: Obtain the current quantum algorithm to be compiled and the current quantum mapping information of the target quantum hardware corresponding to the quantum algorithm to be compiled during the current computing period. Based on the quantum algorithm to be compiled and the current quantum mapping information, and combined with the trained multi-agent model, output the optimal compilation scheme of the quantum algorithm to be compiled.

[0076] For example, by acquiring the current quantum mapping information of the quantum algorithm to be compiled and the target quantum hardware to be deployed during the current computing period, the quantum algorithm to be compiled is preprocessed using a ZX-graph to extract multiple current graph features of the ZX-graph and fused into a current graph feature vector. The multiple current graph features, the current graph feature vector, and the current quantum mapping information are input into a trained multi-agent model, and each agent generates a cooperative action accordingly: Agent-1 outputs a circuit optimization action sequence for rewriting the quantum circuit to be compiled using ZX calculus; Agent-2 outputs a logical bit remapping scheme to adapt to the current hardware topology and noise distribution; SuperAgent, based on global state evaluation, globally optimizes and adjusts the output results of Agent-1 and Agent-2, and finally outputs the globally optimized optimal compilation scheme, which is then converted into compilation instructions that the target quantum hardware can directly execute for application compilation of the quantum algorithm to be compiled.

[0077] It should be noted that the embodiments of this application simplify the high-level quantum algorithm by resolving it into a unitarily equivalent ZX-graph, thus ensuring its unitary equivalence. Furthermore, a multi-agent Markov decision process environment is constructed using multiple graph features, graph feature vectors, and quantum mapping information of the target quantum hardware from the ZX-graph. This environment is used to train the multi-agents through reinforcement learning, resulting in a well-trained multi-agent model. Through multi-agent collaborative optimization of the quantum circuit structure and hardware mapping, the optimal compilation scheme is obtained while ensuring unitary equivalence. This approach takes into account both the structure of the quantum algorithm itself and the dynamic characteristics of the target hardware, avoiding the accumulation of execution errors. It also exhibits strong adaptability and generalization ability, thus supporting the stable operation of the high-level quantum algorithm on real hardware.

[0078] In some embodiments, the high-level quantum algorithm is resolved into a unitarily equivalent ZX-graph, multiple graph features of the ZX-graph are extracted, and the multiple graph features are fused into a graph feature vector. This includes: mapping the single quantum gate and the two quantum gates in the high-level quantum algorithm to spider nodes and directed edges between spider nodes in the ZX-graph, respectively, to generate an initial ZX-graph; preprocessing and simplifying the initial ZX-graph based on ZX calculus rules to obtain a preprocessed ZX-graph; wherein the ZX calculus rules include at least one of spider fusion, zero node removal, and phase merging; verifying the unitary equivalence of the preprocessed ZX-graph; after feature encoding of the preprocessed ZX-graph that passes the unitary equivalence verification, extracting multiple graph features of the preprocessed ZX-graph that passes the unitary equivalence verification; and after standardizing the multiple graph features, fusing the standardized multiple graph features into a graph feature vector.

[0079] The high-level quantum algorithm includes single quantum gates and two quantum gates. The high-level quantum algorithm (QASM / OpenQASM format) is parsed line by line. The single quantum gate (X / Y / Z / H / S / T) is transformed into spider nodes (green / red nodes, labeled with phase information) of the ZX-graph. The two quantum gates generate a correlation relationship between two qubits, that is, an entanglement relationship. Therefore, the two quantum gates (CNOT / CZ / SWAP) are transformed into directed edges between nodes (representing entanglement relationship), generating the initial ZX-graph D0.

[0080] ZX calculus rules include:

[0081] Spider fusion rule: Adjacent spider nodes of the same type and phase are merged into a single node, eliminating redundant edges;

[0082] Zero node removal rule: Isolated zero-phase nodes with no entanglement are deleted directly;

[0083] Phase merging rule: Multiple phases of the same node are merged into an equivalent phase, reducing the number of phase operations.

[0084] This application applies the aforementioned rules to the initial ZX-graph D0 using the ZX calculus rules to obtain a preprocessed ZX-graph D1, and then performs unitary equivalence verification on the preprocessed ZX-graph D1. The unitary equivalence passes the ZX-graph fidelity verification. The conditions for passing the unitary equivalence verification are as follows:

[0085]

[0086] In the formula, To ensure the fidelity of the preprocessed ZX-image D1 to the original ZX-image D0, if it is greater than or equal to 0.999, then the preprocessed ZX-image D1 and the original ZX-image D0 are unitarily equivalent; otherwise, the initial ZX-image generation steps need to be repeated.

[0087] After obtaining the preprocessed ZX-graph that has passed the unitary equivalence verification, feature encoding is performed on the preprocessed ZX-graph to extract graph features including circuit depth, total number of gates, number of two-qubit gates, number of nodes, and number of edges. Among them, the circuit depth is the maximum number of parallel levels in the quantum circuit, which directly reflects the lower bound of the algorithm execution time; the total number of gates and the number of two-qubit gates together characterize the intensity of hardware resource consumption; the number of nodes is the total number of spider nodes in the ZX-graph, and the number of edges is the number of directed edges in the ZX-graph.

[0088] Min-Max normalization is used to map each type of graph feature to the [0,1] interval. Then, a fully connected layer is used to fuse the normalized graph features, ultimately forming a fixed-dimensional (128-dimensional) graph feature vector. During the feature fusion process, the weights of the fully connected layer are initialized through pre-training (based on 1000ZX graph samples from the QC-v1 benchmark kit) to ensure the effectiveness of the feature mapping.

[0089] In some embodiments, a multi-agent Markov decision process environment is constructed based on multiple graph features, graph feature vectors, and quantum mapping information, including: constructing agent frameworks for a route optimization agent, a mapping scheduling agent, and a global coordination super-agent, respectively; defining a state space, action space, and reward function for each agent based on multiple graph features, graph feature vectors, and quantum mapping information; and constructing a multi-agent Markov decision process environment based on the state space, action space, and reward function defined by each agent.

[0090] Among them, the circuit optimization agent is used to optimize the quantum circuit structure, the mapping and scheduling agent is used to optimize the quantum hardware mapping, and the global coordination super agent is responsible for cross-agent collaborative optimization and final scheme verification.

[0091] This application constructs a Markov Decision Process Environment (MAMDP) that reflects the characteristics of compilation tasks and the cooperative relationship among multiple agents by defining the state space, action space, and reward function of each agent.

[0092] Generally, MAMDP uses the six-tuple MA=({S i},{A i},P,{R i}, The formal definition of ω is given, where {S} i Let {A} be the state space of the i-th agent. i Let {R} be the action space of the i-th agent, P be the joint state transition function, and {R} be the action space of the i-th agent. i} is the exclusive reward function for the i-th agent. As a discount factor, this application The value is 0.99, and ω is the agent cooperation coefficient (CTDE architecture). In this application, ω is 0.91.

[0093] Specifically, graph features include line depth, total number of gates, number of two-qubit gates, number of nodes, and number of edges. Quantum mapping information includes the hardware topology adjacency matrix, the logic-physical mapping matrix, hardware calibration parameters, and the queue of two quantum gates to be executed.

[0094] The hardware topological adjacency matrix is ​​constructed based on the physical design file (such as EDF format) of the target quantum hardware chip. The dimension of the hardware topological adjacency matrix is ​​m×m (where m is the number of physical qubits). If the number of physical qubits is p... i With p j If a directly coupled link exists (capable of executing two quantum gates), then A ij =1, otherwise A ij =0.

[0095] The logical-physical mapping matrix has a dimension of n×m (n is the number of logical bits). If the logical bits are l i Mapped to physical bits p j Then M ij =1, otherwise M ij =0, satisfying the constraint. (Each logical bit maps to only one physical bit) and (Each physical bit can be mapped to at most one logical bit).

[0096] The hardware calibration parameters (128-dimensional) are periodically acquired through the real-time calibration interface of the quantum chip (calibration cycle is 5 minutes). These parameters include T1 (longitudinal relaxation time, the time for the quantum state to decay from the excited state to the ground state), T2 (lateral relaxation time, the time for the quantum state to maintain phase coherence), single quantum gate error rate, two quantum gate error rate, readout fidelity, etc. After Min-Max normalization, they are concatenated into a 128-dimensional vector.

[0097] The queue of two quantum gates to be executed stores two quantum gates that need to be executed but do not meet the hardware connectivity constraints. They are represented in the form of gate number + action logic bit pair. The state dimension is fixed at 512 (64 + 176 × 176 (adjacency matrix vectorization) + 20 × 176 (mapping matrix vectorization) + 128 + 32 (queue features), which are fixed at 512 dimensions after compression).

[0098] In the above context, the state space, action space, and reward function of each agent are defined based on multiple graph features, graph feature vectors, and quantum mapping information, including:

[0099] The state space of the circuit optimization agent includes graph feature vectors, circuit depth, total number of gates, and number of two-qubit gates, namely:

[0100]

[0101] In the formula, Optimize the state of the intelligent agent for the line. Optimize the state space of the intelligent agent for the circuit. For graph eigenvectors, For line depth, This represents the total number of doors. For the number of two-qubit gates, The vector is the history vector of the most recent t-step rewritten action (t=10 or other), the state dimension is fixed at 256, and T is the matrix transpose.

[0102] The action space of the route optimization agent includes multiple ZX calculus rewriting actions that satisfy unitary equivalence constraints, namely:

[0103]

[0104] In the formula, The action space of the intelligent agent for line optimization includes 7 types of ZX calculus rewriting actions, all of which satisfy unitary equivalence constraints; This is a spider fusion action, which merges adjacent spider nodes of the same color (Z-spider / X-spider) and the same phase into a single node, eliminating redundant edges in the middle; This is a pivot transformation action, which involves introducing auxiliary nodes to achieve phase transitions between spider nodes while maintaining overall unitarity. This is an identity transformation action, which does not change the structure of the ZX-graph and is used as a no-operation action during the exploration process; For local complementary actions, a complementary transformation is performed on the neighborhood structure of a node in the ZX-graph to simplify entanglement relationships; The spider splitting action involves splitting a single spider node into multiple adjacent nodes of the same color and phase, providing flexibility for subsequent fusion optimization. For phase merging operations, multiple phase values ​​at the same node are merged into an equivalent phase, reducing the number of phase operations (e.g., π / 2 + π / 2 = π). This is a zero-phase removal action, which removes zero-phase nodes and associated edges that have no phase contribution, significantly reducing the complexity of the graph structure.

[0105] The reward function of the line optimization agent is determined by a weighted combination of changes in the total number of gates, changes in line depth, and equivalence fidelity deviation.

[0106] Among them, the reward function R1 of the route optimization agent is based on route optimization as the objective, and is specifically as follows:

[0107]

[0108] In the formula, , , All are weight values, satisfying + + =1, , These represent the total number of doors at steps t and t-1, respectively. , These represent the line depths at steps t and t-1, respectively. This represents the equivalence fidelity between the current ZX-graph and the original graph.

[0109] The state space of the mapping scheduling agent includes the characteristics of the line dependency graph, the hardware topology adjacency matrix, the logical-physical mapping matrix, the hardware calibration parameters, and two quantum gate queues to be executed.

[0110] The path dependency graph features are obtained by analyzing the quantum gate execution order of the ZX-graph, constructing a directed acyclic graph (DAG), and then using a graph attention network to extract features from the DAG. Specifically, the DAG is constructed by analyzing the quantum gate execution order of the ZX-graph D1, with each quantum gate as a node. If gate g... j Need to be at the door g i If execution is required before execution can proceed, then a directed edge g will be constructed. i →g j A directed acyclic graph (DAG) is formed. A graph attention network (GAT) is used to extract features from the DAG. The fixed-dimensional line dependency graph features are obtained through two GAT layers (input dimension is gate type + number of active bits, output dimension is 64).

[0111] The state space of the mapping and scheduling agent is:

[0112]

[0113] In the formula, To map the state of the scheduling agent, To map the state space of the scheduling agent, For features of the path dependency graph, This is the hardware topology adjacency matrix. For logical-physical mapping matrix, For hardware calibration parameters, For the two quantum gate queues to be executed.

[0114] The action space of the mapping scheduling agent includes multiple hardware mapping actions.

[0115] The action space of the mapping and scheduling agent is as follows:

[0116]

[0117] In the formula, To map the action space of the scheduling agent, Logical-physical bit allocation action: Allocate logical bit l i Allocated to physical bits p j All legal combinations; For the adjacent physical bit swap operation, perform a swap operation on A in the adjacency matrix A. ij =1 physical bit pair (p i ,p j Perform a SWAP operation to swap the mapped logical bits; For coupler control actions: turn physical bit pairs on / off (p i ,p j Couplers between them, or adjusting the coupling strength; No operation: This means that there is no need to perform mapping / swap / coupler control actions at present. Only the gate execution queue is pushed forward to avoid invalid actions occupying training resources.

[0118] The reward function of the mapping scheduling agent is determined by a weighted combination of the cumulative number of SWAP gates inserted after the action is executed, the compilation delay of the action, and the predicted value of the execution fidelity of the mapping scheme.

[0119] The reward function of the mapping and scheduling agent is set based on hardware mapping optimization as the objective. for:

[0120]

[0121] In the formula, , All are weights. + + =1, The cumulative number of SWAP gates inserted after the execution of step t is determined by counting the number of times agent 2 performs the Aswap action. The compilation delay for the t-th action is determined by recording the action execution time (including policy reasoning time + action application time); The execution fidelity prediction value for the current mapping scheme is obtained through a pre-trained fidelity prediction model (with φ as input). hard The model is calculated using the mapping matrix M, the line depth D, and the output is the fidelity in the [0,1] interval. This model is generated based on 10,000 sets of hardware test data (including the correspondence between different mapping schemes, line parameters and actual fidelity).

[0122] The state space of the global coordination super agent is a global state composed of the states of the route optimization agent and the mapping scheduling agent.

[0123] The state space of the globally coordinated super-intelligent agent is as follows:

[0124]

[0125] In the formula, The global state vector (128-dimensional) consists of three parts: the policy entropy of the route optimization agent (1-dimensional), the policy entropy of the mapping and scheduling agent (1-dimensional), and the coordination coefficient of the two agents' actions (obtained by calculating the correlation between the two agents' actions and the global reward, 126-dimensional). These are fused into a 128-dimensional vector through a fully connected layer to represent the agent's cooperative state. The state dimension is fixed at 896 (256+512+128).

[0126] The action space of the global coordination super agent is the global action composed of the actions of the route optimization agent and the actions of the mapping and scheduling agent.

[0127] The action space of the globally coordinated super-intelligent agent is as follows:

[0128] A super =A1×A2

[0129] The action space of the global coordination super agent includes all action combinations of the route optimization agent and the mapping scheduling agent, totaling 7×(m×n+|E|+c+1) combinations, where |E| is the number of coupling edges and c is the number of couplers.

[0130] Through action mask M a Filtering illegal combinations is specifically implemented as follows: 1) Constructing a rule base for illegal action combinations: such as the route optimization agent executing f fuse (Needs adjacent spider nodes) When the mapping scheduling agent executes non-adjacent bit SWAP, the mapping scheduling agent executes A. assign Time-based route optimization agent executes f zero-removal (Removing a node is ineffective when there is no mapping), etc.; 2) Regarding the current state s super Generate binary action mask vector M a If the action combination is valid, then M a,k =1, otherwise M a,k =0; 3) After the policy network outputs the action probability distribution, it is compared with M a Multiply each element and then normalize to obtain the probability distribution of legal actions, ensuring that actions are selected only from legal combinations.

[0131] The reward function of the global coordination super agent is a weighted sum of the reward function of the circuit optimization agent, the reward function of the mapping scheduling agent, and the global performance reward. The global performance reward is a weighted combination of the circuit depth, the number of two-qubit gates, the execution fidelity, and the compilation latency of the quantum circuit compiled by the quantum algorithm on the target hardware.

[0132] Among them, the reward function R of the globally coordinated super-intelligent agent super for:

[0133]

[0134] In the formula, For weights, such as =0.91, The global performance bonus value is calculated by considering circuit depth, number of gates, fidelity, and compilation time.

[0135]

[0136] In the formula, , , , As weight, + + + =1, For the reduction rate of line depth, For the rate of reduction of the number of two-qubit gates, This is due to compilation delay.

[0137] Furthermore, in a Markov decision process environment, to ensure the legality of state transitions after an agent's action (e.g., the SWAP action only allows adjacent physical bits), a joint state transition function P is constructed as follows:

[0138]

[0139] In some embodiments, reinforcement learning training is performed on multiple agents within a multi-agent Markov decision process environment to obtain a trained multi-agent model. This includes: constructing a policy network for each agent and a centralized critic network shared by all agents based on a centralized training and distributed execution architecture; updating the policy grid independently for the route optimization agent and the mapping scheduling agent respectively based on the MAPPO algorithm and the Markov decision process environment of the route optimization agent and the mapping scheduling agent to obtain a basic route optimization agent and a basic mapping scheduling agent; and updating the policy grid independently for the route optimization agent and the mapping scheduling agent based on the MAPPO algorithm and the Markov decision process environment of the route optimization agent and the mapping scheduling agent. In this context, a centralized critic network is used to jointly optimize the policy grids of the basic route optimization agent and the basic mapping scheduling agent, resulting in a cooperative route optimization agent and a cooperative mapping scheduling agent. Based on the MAPPO algorithm and combined with the Markov decision process environment of each agent, the policy grids of the cooperative route optimization agent, the cooperative mapping scheduling agent, and the global coordination super-agent are jointly optimized, resulting in trained route optimization agents, trained mapping scheduling agents, and trained coordination super-agents. Based on the trained route optimization agents, trained mapping scheduling agents, and trained coordination super-agents, a trained multi-agent model is constructed.

[0140] Based on the centralized training and distributed execution (CTDE) architecture, each agent has an independent policy network (distributed execution) and shares a centralized critic network (centralized training). This allows the agent to evaluate the value of actions using global information while preserving the decision-making autonomy of each agent.

[0141] For example, the policy network of the route optimization agent takes input s1 (256 dimensions), passes through two layers of MLP (256→128, ReLU activation function) and an action mask layer, and outputs... (7-dimensional action probability distribution); The policy network for mapping and scheduling agents takes input s2 (512-dimensional), passes through a GraphSAGE layer (processing the hardware topology adjacency matrix, outputting 256-dimensional), two MLP layers (512→256, ReLU), and an action masking layer, and outputs... (m×n+|E|+c+1 dimensional action probability distribution); the policy network of the globally coordinated super-agent is based on the input s. super (896-dimensional), after passing through a Cross-Attention layer, two MLP layers, and an action masking layer, the output is πθ. super (a super |s super (Probability distribution of joint actions).

[0142] Centralized critic network: Input global state ssuper (896 dimensions), through a Cross-Attention layer (incorporating agent role embeddings), and a 2-layer MLP, output state value V. φ (s super ).

[0143] Based on the MAPPO (Multi-Agent Proximal Policy Optimization) algorithm, the objective function is set as follows:

[0144]

[0145] In the formula, These are the policy network parameters for the three agents. For strategy ratio, For editing parameters, =0.2, used to limit the fluctuation range of the strategy ratio. To avoid strategy crashes caused by excessively large strategy updates, For the clipping function, Here is the entropy regularization coefficient. =0.2, This is the strategy entropy (which encourages exploration).

[0146] The objective function is used in each iteration of subsequent training: trajectory data of the multi-agent is collected in each training round. Calculate the advantage function Ait for each agent (based on their respective reward r and the state value V output by the centralized critic network). φ (s super ); Fix the previous round's strategy parameter θ old =θ, calculate the strategy ratio r t (θ); 4. r t (θ) Substitute into the objective function and minimize L using gradient descent. CLIP Update the policy network parameters θ of each agent. i ; Simultaneously update the parameters φ of the centralized critic network (minimize the state value prediction error); Repeat the above steps until the policy converges.

[0147] For example, the multi-agent training process is divided into three stages, including:

[0148] Phase 1: Single-agent independent pre-training: Agent 1: In an ideal hardware environment (no noise, φ) hard Training was conducted with all-one vectors, optimizing only the path depth and number of gates. The number of training epochs was N1 = 1.8 × 10⁶, and the learning rate was... Agent 2: Trained under a fixed hardware topology (e.g., Tianyan-176 adjacency matrix), optimizing only the number of swaps and mapping validity. Training epochs N² = 2.1 × 10⁶, learning rate... SuperAgent: No training is required yet; only network parameters are initialized (the policy network weights of half the agent are copied + the Cross-Attention layer is randomly initialized); Convergence criterion: the average reward fluctuation of each agent over 100 consecutive rounds is <0.01;

[0149] Phase 2: Dual-Agent Collaborative Training: Enable information interaction between Agent 1 and Agent 2, sharing an experience pool (storing samples (s1, s2, a1, a2, r1, r2) from both); the centralized critic network begins training, based on the global state s super Evaluate the value of the combined actions; training epochs N3 = 2.5 × 10⁶, learning rate decayed to Convergence criterion: The combined reward R1+R2 of the two agents fluctuates by less than 0.01 for 100 consecutive rounds;

[0150] Phase 3: Joint Training of Three Agents: SuperAgent officially participates in training, and its action space Asuper covers the action combinations of Agent 1 / 2; the training objective is to optimize the joint reward Rsuper of SuperAgent, and the policy networks of Agent 1 / 2 are updated synchronously through gradient backpropagation; the number of training rounds N4 = 3.6 × 106, and the learning rate decays to... Collaboration mechanism: SuperAgent assigns task weights through role vector zit. If the route optimization is insufficient, the action weight of agent 1 is increased. If the mapping fidelity is low, the action weight of agent 2 is increased. Convergence judgment: The joint reward of SuperAgent is 87.3, and the fluctuation is <0.01 for 1000 consecutive rounds.

[0151] After training, validate the core metrics:

[0152] Agent 1: Policy entropy stabilized at 0.42 bits, and the average line depth decreased by 29; Agent 2: The average number of swaps decreased by 60; SuperAgent: Joint reward 87.3.

[0153] In some embodiments, based on the quantum algorithm to be compiled and the current quantum mapping information, and combined with a trained multi-agent model, the optimal compilation scheme of the quantum algorithm to be compiled is output, including: parsing the quantum algorithm to be compiled into the current ZX-graph, extracting multiple current graph features of the current ZX-graph, and fusing the multiple current graph features into a current graph feature vector; based on the trained multi-agent model, and combining the current ZX-graph, multiple current graph features, the current graph feature vector, and the current quantum mapping information, the optimal compilation scheme of the quantum algorithm to be compiled is obtained; wherein, the optimal compilation scheme includes the optimal circuit optimization action sequence and the optimal hardware mapping action matrix.

[0154] Optionally, the quantum algorithm to be compiled is parsed into the current ZX-graph, multiple current graph features are extracted from the current ZX-graph, and these features are fused into a current graph feature vector. These multiple current graph features and the current graph feature vector are then input into a trained line optimization agent to generate an optimized ZX-graph rewriting action sequence. The current ZX-graph is updated based on the optimized ZX-graph rewriting action sequence to obtain the updated current ZX-graph, and the current line dependency graph features of the updated current ZX-graph are extracted. The current line dependency graph features and the current quantum mapping information are input into a trained mapping scheduling agent to generate an optimized hardware mapping action sequence, i.e., a hardware mapping action matrix. Finally, SuperAgent integrates the updated ZX-graph and the mapping action matrix, performs fine-tuning on the updated current ZX-graph and the mapping action matrix, and re-verifies the fidelity to be greater than 0.999 and the mapping matrix constraints to ensure validity. The optimal line optimization action sequence and the optimal hardware mapping action matrix are then output.

[0155] For example, 1. Initialization: Combine the preprocessed ZX-image D1, the target hardware topology A, and the real-time calibration parameters φ hard Input the MAMDP environment and initialize the mapping matrix M0 (using a random uniform distribution strategy); the preprocessing of D1 includes: 1. Format parsing: converting the QASM / OpenQASM quantum algorithm into a node-edge structure of a ZX-graph; 2. Basic simplification: performing ZX calculus rules such as spider fusion, zero node removal, and phase merging; 3. Unitary verification: ensuring F(D0,D1)≥0.999; 4. Feature extraction: generating φ zx Feature vector; Line optimization (executed by agent 1): Agent 1 outputs the optimal rewrite action sequence {a} based on state s1. 1,1 ,a1,2,...,a 1,T}, applied to D1 to obtain the optimized ZX-graph D2, verifying F(D0,D2)≥0.999; Hardware mapping (executed by agent 2): Agent 2 generates a DAG dependency graph based on the path dependency graph of D2 (construction method: parsing the execution order of quantum gates in D2, using gates as nodes and causal relationships as directed edges), outputting the optimal mapping action sequence {a 2,1 ,a 2,2 ,...,a 2,T}, generate hardware mapping action matrix M * And SWAP insertion scheme; joint adjustment (SuperAgent execution): SuperAgent based on global state s super For D2 and M * Fine-tuning is performed as follows: 1) Performance evaluation: Calculate the current depth D of D2 and the number of gates N between two qubits. 2G and M * The corresponding number of swaps N swap And the fidelity prediction value F; 2) Local rewriting: if D or N 2G Target not achieved (D reduced by 30) 3) SWAP optimization: If N swap Too much or F t Target not achieved (70) 4) Verification and confirmation: After fine-tuning, re-verify F(D0,D2)≥0.999 and the mapping matrix constraint to ensure legality.

[0156] In some embodiments, based on the quantum algorithm to be compiled and the current quantum mapping information, combined with a trained multi-agent model, the optimal compilation scheme of the quantum algorithm to be compiled is output. This further includes: rewriting the current ZX-graph according to the optimal circuit optimization action sequence to obtain an optimized ZX-graph; based on the mapping rules between the ZX-graph and quantum gates, converting the optimized ZX-graph into the native quantum gate sequence of the target quantum hardware corresponding to the quantum algorithm to be compiled; replacing the logical bits in the native quantum gate sequence with the physical bit numbers in the optimal hardware mapping action matrix, and generating a compilation instruction set based on the physical bit numbers, combined with added SWAP gate instructions and coupler control instructions.

[0157] The mapping rules between the ZX-graph and quantum gates are: Z-spider → Z-gate, X-spider → X-gate, directed edge → CNOT / CZ gate, phase node → phase gate. This transforms the optimized node-edge structure of the ZX-graph into a native quantum gate sequence supported by the target hardware (e.g., Tianyan-176 supports CZ gate, X gate, Z gate, and H gate); combined with the mapping matrix M... * Generate instruction set: Replace the logical bits in the original quantum gate sequence with the physical bits corresponding to M* (such as logical bit l). i Replace with p j ,in =1), add SWAP gate instructions (according to the SWAP insertion scheme) and coupler control instructions, and finally generate the QASM instruction set or native instruction set (such as the binary instruction format of Tianyan-176) that can be executed by the quantum chip.

[0158] Among them, the SWAP gate instruction is dynamically inserted based on the physical bit coupling topology constraints to ensure that each SWAP operation meets the hardware connectivity requirements; the coupler control instruction precisely configures the frequency offset and pulse timing to suppress crosstalk and improve gate fidelity.

[0159] In some embodiments, after obtaining the optimal compilation scheme, the method further includes: acquiring compilation execution data after executing the optimal compilation scheme; determining the performance indicators of the optimal compilation scheme based on the compilation execution data; wherein the performance indicators include at least one of the following: circuit depth reduction rate, two-qubit gate reduction rate, execution fidelity, compilation latency, and robustness; determining whether the performance indicators meet the preset performance indicator conditions; if all performance indicators meet the preset performance indicator conditions, outputting the final compilation scheme; if any performance indicator does not meet the preset performance indicator conditions, calculating the deviation value based on the unmet performance indicator and the preset performance indicator conditions, normalizing the deviation value, and incorporating the normalized deviation value as an additional reward signal into the reward function of the multi-agent, and adjusting the weights in the reward function; retraining the multi-agent model based on the adjusted reward function and weights and outputting a new optimal compilation scheme; and re-executing the acquisition of compilation execution data after executing the optimal compilation scheme based on the new optimal compilation scheme, determining the performance indicators of the optimal compilation scheme based on the compilation execution data, until all performance indicators meet the preset performance indicator conditions.

[0160] The system constructs a five-dimensional performance evaluation framework to quantify the overall performance of the compilation scheme. Deviations in unmet metrics are used as additional reward signals to adjust the reward function weights of the corresponding agents, leading to retraining and the generation of new schemes. Specifically, the line depth reduction rate is:

[0161]

[0162] In the formula, D orig D opt The original route and the optimized route are respectively the route depths, with a target reduction rate of 30%.

[0163] The two-qubit gate reduction rate is:

[0164]

[0165] In the formula, N 2G,opt N 2G,origThese represent the number of two-qubit gates before and after optimization, with a target reduction rate of 40%.

[0166] The execution fidelity is:

[0167]

[0168] In the formula, K is the number of evaluation samples (preferably K=10~20). The actual circuit fidelity of the k-th evaluation sample is obtained through quantum chip measurement or high-precision quantum simulation; the target value for circuit fidelity is 0.7.

[0169] The compilation delay is:

[0170]

[0171] In the formula, N is the total number of quantum circuit samples used for evaluation. The compilation time for compiling the i-th quantum circuit is a single compilation operation, with a target compilation latency of 0.9.

[0172] The robustness index is:

[0173]

[0174] In the formula, Evaluate the fidelity of the circuit under the i-th perturbation scenario (perturbation scenarios include qubit decoherence time fluctuations, gate error rate fluctuations, etc.). To evaluate the fidelity of the circuit in the baseline scenario, the robustness index is set at 0.9.

[0175] The performance metrics after executing the optimal compilation scheme are calculated based on the above performance metrics calculation formulas. If all performance metrics meet the preset performance metric conditions, the compilation scheme is determined and output as the final compilation scheme.

[0176] If any metric fails to meet the target, a feedback optimization mechanism is triggered:

[0177] The deviation value △ is obtained by subtracting the unmet performance index from the preset performance index condition (target value). After normalizing △, it is used as an additional reward signal △r=△×0.5 (weight 0.5, to avoid excessive additional reward from interfering with the original reward system) and integrated into the reward function of the corresponding agent.

[0178] For example, if △D fails to meet the target: update the reward function R1 to R1 + △ rD (△) rD (For the normalization bonus of depth bias).

[0179] If △N 2GNot met: Update the reward function R1 to R1 + ΔrN 2G ;

[0180] If F eval Not met: Update the reward function R2 to R2 + Δr F Update the reward function R super For ω·(R1+R * 2)+(1-ω)·R global Among them, R * 2 represents the updated reward function R2;

[0181] If Teval is not met: Update the reward function R2 to R2 + Δr T ;

[0182] If R rob Not met: Update reward function R super ω·(R1+R2)+(1-ω)·(R global +△r R );

[0183] Reward function weight adjustment: For indicators that are not met, adjust the corresponding weights. The specific rules are as follows: If the line depth reduction rate is not met (D<30%): increase the weight of agent 1. d Weight to 0.4, reduce w g Up to 0.2, w f Keep it at 0.4;

[0184] The two-qubit gate reduction rate did not meet the target (ΔN). 2G <40%): Increase agent 1's w g Weight to 0.4, reduce w d Up to 0.2, w f Keep it at 0.4;

[0185] Execution fidelity not met (F) eval <70%): Increase agent 2's w f Weight to 0.6, reduce w s Up to 0.2, w t Maintain 0.2; simultaneously increase SuperAgent's R... global Medium fidelity weighted to 0.35;

[0186] Compilation delay not met (T) eval >0.2s): Increase agent 2's w t Weight to 0.3, reduce w s Up to 0.2, w f Keep it at 0.5;

[0187] Robustness not met (R rob<0.9: Increase SuperAgent's ω to 0.95, enhancing the contribution of agent cooperation to robustness;

[0188] Retraining: Retrain the agent 5×10 using the adjusted reward function and weights. 5 In each round (only the fine-tuning phase, no need to train from scratch), a new compilation scheme is generated and evaluated again until all performance metrics are met.

[0189] To further illustrate the implementation process of the present invention, a specific embodiment is provided below, all of which are verified based on the 176-bit Tianyan superconducting quantum processor.

[0190] 1. Input: QFT algorithm from the QC-v1 benchmark kit (60 logic bits, line depth 600).

[0191] 2. ZX-graph preprocessing: Convert to ZX-graph and perform preliminary simplification (spider fusion + phase merging), reducing the circuit depth to 520 and the number of two-qubit gates to 15.

[0192] 3. MAMDP environment initialization: Load the Tianyan-176 topology (adjacency matrix A∈{0,1}176×176) and real-time calibration parameters (T1=80µs, T2=60µs, single quantum gate error rate=0.001, two quantum gate error rate=0.005).

[0193] 4. Agent Execution: - Agent 1 outputs the rewritten action sequence (12 steps, including 8 f's). fuse 3 times f phase-merger 1 time f local-comp The line depth was reduced to 412 (a decrease of 31.3%); Agent 2 outputs a mapped action sequence (8 steps, including 6 A's). swap 2nd time A coupler The number of SWAP insertions is only 38.3% of that in an industrial production line; - SuperAgent fine-tuning: Due to the two-qubit gate count not meeting the target (reduction of 38% < 40%), agent 1's second f is triggered. fuse Action, final line depth 352 (reduced by 41.3)

[0194] 5. Compilation instruction generation: Generates the native Tianyan-176 instruction set with a compilation delay of 0.13 seconds;

[0195] 6. Performance evaluation: Execution fidelity 70.9%, robustness 0.92, all indicators meet the standards.

[0196] The quantum algorithm compilation method based on multi-agent reinforcement learning proposed in this application achieves end-to-end joint optimization of quantum circuit optimization and hardware mapping by constructing a multi-agent collaborative decision-making framework and deeply integrating ZX calculus symbolic reasoning with MARL. This method solves the global suboptimal problem of staged compilation through the CTDE architecture, improves execution fidelity through hardware-aware state space design, and balances core performance indicators through a multi-objective reward function. Experimental results show that this method achieves a 31.4% reduction in circuit depth, a 43.2% reduction in two-qubit gates, and a 70.9% improvement in execution fidelity on the Tianyan-176 processor, significantly outperforming existing compilation methods. It can be widely applied to the implementation of quantum algorithms in quantum chemical simulation, quantum optimization, quantum machine learning, and other fields, and has significant engineering application value.

[0197] Based on the same inventive concept, this application also provides a quantum algorithm compilation system based on multi-agent reinforcement learning for implementing the quantum algorithm compilation method based on multi-agent reinforcement learning mentioned above.

[0198] The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the quantum algorithm compilation system based on multi-agent reinforcement learning provided below can be found in the limitations of the quantum algorithm compilation method based on multi-agent reinforcement learning described above, and will not be repeated here.

[0199] like Figure 3 As shown in the embodiments of this application, a quantum algorithm compilation system based on multi-agent reinforcement learning is also provided, including:

[0200] The quantum algorithm parsing module 100 is used to parse high-level quantum algorithms into unitary equivalent ZX-graphs, extract multiple graph features of the ZX-graphs, and fuse multiple graph features into a graph feature vector.

[0201] The multi-agent conversion module 200 is used to acquire the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm, and to construct a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors and quantum mapping information.

[0202] The multi-agent training module 300 is used to perform reinforcement learning training on multiple agents in a multi-agent Markov decision process environment to obtain a trained multi-agent model.

[0203] The compilation scheme determination module 400 is used to obtain the current quantum mapping information of the quantum algorithm to be compiled and the target quantum hardware corresponding to the quantum algorithm to be compiled during the current computing period. Based on the quantum algorithm to be compiled and the current quantum mapping information, combined with the trained multi-agent model, the optimal compilation scheme of the quantum algorithm to be compiled is output.

[0204] In some embodiments, the quantum algorithm analysis module 100 is used for:

[0205] The single quantum gate and the two quantum gate in the high-level quantum algorithm are respectively mapped to spider nodes and directed edges between spider nodes in the ZX-graph to generate the initial ZX-graph;

[0206] The initial ZX-graph is preprocessed and simplified based on the ZX calculus rules to obtain a preprocessed ZX-graph; wherein the ZX calculus rules include at least one of the following: spider fusion, zero node removal and phase merging.

[0207] The preprocessed ZX-graph is subjected to unitary equivalence verification. After feature encoding of the preprocessed ZX-graph that passes the unitary equivalence verification, multiple graph features of the preprocessed ZX-graph that passes the unitary equivalence verification are extracted.

[0208] After standardizing multiple graph features, the standardized graph features are merged into a graph feature vector.

[0209] In some embodiments, the multi-agent conversion module 200 is used for:

[0210] We construct agent frameworks for route optimization agent, mapping scheduling agent and global coordination super agent respectively, and define the state space, action space and reward function for each agent based on multiple graph features, graph feature vectors and quantum mapping information.

[0211] Based on the state space, action space, and reward function defined by each agent, a multi-agent Markov decision process environment is constructed.

[0212] In some embodiments, graph features include line depth, total number of gates, number of two-qubit gates, number of nodes, and number of edges;

[0213] Quantum mapping information includes the hardware topological adjacency matrix, the logical-physical mapping matrix, hardware calibration parameters, and two quantum gate queues to be executed;

[0214] Based on multiple graph features, graph feature vectors, and quantum mapping information, the state space, action space, and reward function are defined for each agent, including:

[0215] The state space of the circuit optimization agent includes graph feature vectors, circuit depth, total number of gates, and number of two-qubit gates;

[0216] The action space of the route optimization agent includes multiple ZX calculus rewriting actions that satisfy unitary equivalence constraints.

[0217] The reward function of the route optimization agent is determined by a weighted combination of changes in the total number of gates, changes in route depth, and equivalence fidelity deviation.

[0218] The state space of the mapping scheduling agent includes line dependency graph features, hardware topology adjacency matrix, logical-physical mapping matrix, hardware calibration parameters, and two quantum gate queues to be executed. Among them, the line dependency graph features are obtained by analyzing the quantum gate execution order of the ZX-graph, constructing a directed acyclic graph, and extracting features from the directed acyclic graph using a graph attention network.

[0219] The action space of the mapping and scheduling agent includes multiple hardware-mapped actions;

[0220] The reward function of the mapping scheduling agent is determined by a weighted combination of the cumulative number of SWAP gates inserted after the action is executed, the compilation delay of the action, and the predicted value of the execution fidelity of the mapping scheme.

[0221] The state space of the global coordination super agent is the global state jointly constituted by the state of the route optimization agent and the state of the mapping scheduling agent.

[0222] The action space of the global coordination super agent is the global action composed of the actions of the route optimization agent and the actions of the mapping and scheduling agent.

[0223] The reward function of the global coordination super agent is a weighted sum of the reward function of the circuit optimization agent, the reward function of the mapping scheduling agent, and the global performance reward. The global performance reward is a weighted combination of the circuit depth, the number of two-qubit gates, the execution fidelity, and the compilation latency of the quantum circuit compiled by the quantum algorithm on the target hardware.

[0224] In some embodiments, the multi-agent training module 300 is used for:

[0225] Based on a centralized training and distributed execution architecture, a policy network for each agent and a centralized critic network shared by all agents are constructed.

[0226] Based on the MAPPO algorithm, and combined with the Markov decision process environment of the route optimization agent and the mapping scheduling agent, independent policy grid updates are performed on the route optimization agent and the mapping scheduling agent respectively to obtain the basic route optimization agent and the basic mapping scheduling agent.

[0227] Based on the MAPPO algorithm, and combined with the Markov decision process environment of the route optimization agent and the mapping scheduling agent, the policy grid of the basic route optimization agent and the basic mapping scheduling agent is jointly optimized through a centralized critic network to obtain a cooperative route optimization agent and a cooperative mapping scheduling agent.

[0228] Based on the MAPPO algorithm and combined with the Markov decision process environment of each agent, joint policy optimization is performed on the policy grid of the cooperative route optimization agent, the cooperative mapping scheduling agent, and the global coordination super agent to obtain the trained route optimization agent, the trained mapping scheduling agent, and the trained coordination super agent.

[0229] The trained multi-agent model is constructed from the trained route optimization agent, the trained mapping scheduling agent, and the trained coordination super-agent.

[0230] In some embodiments, the compilation scheme determination module 400 is used for:

[0231] The quantum algorithm to be compiled is parsed into the current ZX-graph, multiple current graph features of the current ZX-graph are extracted, and multiple current graph features are fused into a current graph feature vector;

[0232] Based on the trained multi-agent model, and combining the current ZX-graph, multiple current graph features, current graph feature vectors, and current quantum mapping information, the optimal compilation scheme for the quantum algorithm to be compiled is obtained; among which, the optimal compilation scheme includes the optimal circuit optimization action sequence and the optimal hardware mapping action matrix.

[0233] In some embodiments, the system further includes an instruction compilation module, used for:

[0234] The current ZX-graph is rewritten based on the optimal route optimization action sequence to obtain the optimized ZX-graph;

[0235] Based on the mapping rules between ZX-graphs and quantum gates, the optimized ZX-graph is transformed into the native quantum gate sequence of the target quantum hardware corresponding to the quantum algorithm to be compiled;

[0236] The logical bits in the original quantum gate sequence are replaced with the physical bit numbers in the optimal hardware mapping action matrix. Based on the physical bit numbers, the added SWAP gate instructions and coupler control instructions are combined to generate a compilation instruction set.

[0237] In some embodiments, the system further includes: a performance evaluation module, used for:

[0238] Obtain compilation and execution data after implementing the optimal compilation scheme, and determine the performance indicators of the optimal compilation scheme based on the compilation and execution data; among them, the performance indicators include at least one of the following: circuit depth reduction rate, two-qubit gate reduction rate, execution fidelity, compilation latency, and robustness;

[0239] Determine whether the performance indicators have met the preset performance indicator conditions;

[0240] If all performance metrics meet the preset performance metric conditions, the final compilation scheme will be output.

[0241] If any performance indicator fails to meet the preset performance indicator conditions, the deviation value is calculated based on the unmet performance indicator and the preset performance indicator conditions. The deviation value is normalized and then incorporated into the reward function of the multi-agent as an additional reward signal, and the weights in the reward function are adjusted.

[0242] Based on the adjusted reward function and weights, the multi-agent model is retrained and a new optimal compilation scheme is output. The compilation execution data after the optimal compilation scheme is obtained is re-executed based on the new optimal compilation scheme. The performance indicators of the optimal compilation scheme are determined based on the compilation execution data until all performance indicators meet the preset performance indicator conditions.

[0243] like Figure 4 As shown, this application provides an electronic device. The electronic device 10 includes a memory 20 and a processor 30. The memory 20 stores a computer program. When the computer program is executed by the processor 30, the processor 30 performs the steps of the quantum algorithm compilation method based on multi-agent reinforcement learning as described in the above embodiment.

[0244] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements the steps of the quantum algorithm compilation method based on multi-agent reinforcement learning as described in the above embodiments.

[0245] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, electronic devices, and computer storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0246] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0247] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0248] In the embodiments provided by this invention, it should be understood that the disclosed systems, electronic devices, computer storage media, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0249] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0250] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0251] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of the present invention through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0252] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A quantum algorithm compilation method based on multi-agent reinforcement learning, characterized in that, include: The high-level quantum algorithm resolves the ZX-graph into a unitary equivalent, extracts multiple graph features from the ZX-graph, and fuses these multiple graph features into a graph feature vector. Obtain the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm, and construct a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors and quantum mapping information; Based on the Markov decision process environment of the multi-agent, the multi-agent is trained by reinforcement learning to obtain a trained multi-agent model. Obtain the quantum algorithm to be compiled in the current computing period and the current quantum mapping information of the target quantum hardware corresponding to the quantum algorithm to be compiled. Based on the quantum algorithm to be compiled and the current quantum mapping information, and in combination with the trained multi-agent model, output the optimal compilation scheme of the quantum algorithm to be compiled.

2. The quantum algorithm compilation method based on multi-agent reinforcement learning according to claim 1, characterized in that, The process of resolving high-level quantum algorithms into unitarily equivalent ZX-graphs, extracting multiple graph features from the ZX-graph, and fusing these multiple graph features into a graph feature vector includes: The single quantum gate and the two quantum gate in the high-level quantum algorithm are respectively mapped to spider nodes and directed edges between spider nodes in the ZX-graph to generate the initial ZX-graph; The initial ZX-graph is preprocessed and simplified based on the ZX calculus rules to obtain a preprocessed ZX-graph; wherein the ZX calculus rules include at least one of the following: spider fusion, zero node removal, and phase merging. The preprocessed ZX-graph is subjected to unitary equivalence verification. After feature encoding of the preprocessed ZX-graph that passes the unitary equivalence verification, multiple graph features of the preprocessed ZX-graph that passes the unitary equivalence verification are extracted. After standardizing multiple graph features, the standardized graph features are merged into a graph feature vector.

3. The quantum algorithm compilation method based on multi-agent reinforcement learning according to claim 1, characterized in that, The construction of a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors, and quantum mapping information includes: A smart agent framework is constructed, consisting of a route optimization smart agent, a mapping scheduling smart agent, and a global coordination super smart agent. The state space, action space, and reward function of each smart agent are defined based on multiple graph features, graph feature vectors, and quantum mapping information. Based on the state space, action space, and reward function defined by each agent, a Markov decision process environment for the multi-agent system is constructed.

4. The quantum algorithm compilation method based on multi-agent reinforcement learning according to claim 3, characterized in that, The graph features include line depth, total number of gates, number of two-qubit gates, number of nodes, and number of edges; The quantum mapping information includes a hardware topological adjacency matrix, a logical-physical mapping matrix, hardware calibration parameters, and two quantum gate queues to be executed; The step of defining the state space, action space, and reward function for each agent based on multiple graph features, the graph feature vector, and the quantum mapping information includes: The state space of the circuit optimization agent includes graph feature vectors, circuit depth, total number of gates, and number of two-qubit gates. The action space of the line optimization agent includes multiple ZX calculus rewriting actions that satisfy unitary equivalence constraints. The reward function of the line optimization agent is determined by a weighted combination of changes in the total number of gates, changes in line depth, and equivalence fidelity deviation. The state space of the mapping scheduling agent includes line dependency graph features, hardware topology adjacency matrix, logical-physical mapping matrix, hardware calibration parameters, and two quantum gate queues to be executed; wherein, the line dependency graph features are obtained by parsing the quantum gate execution order of the ZX-graph, constructing a directed acyclic graph, and extracting features from the directed acyclic graph using a graph attention network; The action space of the mapping scheduling agent includes multiple hardware mapping actions; The reward function of the mapping scheduling agent is determined by a weighted combination of the cumulative number of SWAP gates inserted after the action is executed, the compilation delay of the action, and the predicted value of the execution fidelity of the mapping scheme. The state space of the global coordination super agent is a global state composed of the states of the route optimization agent and the mapping scheduling agent. The action space of the global coordination super agent is the global action composed of the actions of the route optimization agent and the actions of the mapping scheduling agent. The reward function of the global coordination super agent is a weighted sum of the reward function of the circuit optimization agent, the reward function of the mapping scheduling agent, and the global performance reward. The global performance reward is a weighted combination of the circuit depth, the number of two-qubit gates, the execution fidelity, and the compilation latency of the quantum circuit compiled by the quantum algorithm on the target hardware.

5. The quantum algorithm compilation method based on multi-agent reinforcement learning according to claim 3, characterized in that, The Markov decision process environment based on the multi-agent agent is used to train the multi-agent agent through reinforcement learning to obtain a trained multi-agent agent model, including: Based on a centralized training and distributed execution architecture, a policy network for each agent and a centralized critic network shared by all agents are constructed. Based on the MAPPO algorithm, and combined with the Markov decision process environment of the route optimization agent and the mapping scheduling agent, independent policy grid updates are performed on the route optimization agent and the mapping scheduling agent respectively to obtain the basic route optimization agent and the basic mapping scheduling agent. Based on the MAPPO algorithm, and combined with the Markov decision process environment of the route optimization agent and the mapping scheduling agent, the policy grid of the basic route optimization agent and the basic mapping scheduling agent is jointly optimized through the centralized critic network to obtain the cooperative route optimization agent and the cooperative mapping scheduling agent. Based on the MAPPO algorithm and combined with the Markov decision process environment of each agent, joint policy optimization is performed on the policy grid of the cooperative route optimization agent, the cooperative mapping scheduling agent, and the global coordination super agent to obtain the trained route optimization agent, the trained mapping scheduling agent, and the trained coordination super agent. The trained multi-agent model is constructed based on the trained route optimization agent, the trained mapping scheduling agent, and the trained coordination super-agent.

6. The quantum algorithm compilation method based on multi-agent reinforcement learning according to claim 1, characterized in that, The step of outputting the optimal compilation scheme for the quantum algorithm to be compiled based on the quantum algorithm to be compiled and the current quantum mapping information, combined with the trained multi-agent model, includes: The quantum algorithm to be compiled is parsed into the current ZX-graph, multiple current graph features of the current ZX-graph are extracted, and the multiple current graph features are fused into a current graph feature vector; Based on the trained multi-agent model, and combining the current ZX-graph, the multiple current graph features, the current graph feature vector, and the current quantum mapping information, the optimal compilation scheme for the quantum algorithm to be compiled is obtained; wherein, the optimal compilation scheme includes the optimal circuit optimization action sequence and the optimal hardware mapping action matrix; The step of outputting the optimal compilation scheme for the quantum algorithm to be compiled based on the quantum algorithm to be compiled and the current quantum mapping information, combined with the trained multi-agent model, further includes: The current ZX-graph is rewritten according to the optimal route optimization action sequence to obtain the optimized ZX-graph; Based on the mapping rules between ZX-graphs and quantum gates, the optimized ZX-graph is transformed into the native quantum gate sequence of the target quantum hardware corresponding to the quantum algorithm to be compiled; The logical bits in the original quantum gate sequence are replaced with the physical bit numbers in the optimal hardware mapping action matrix, and a compilation instruction set is generated based on the physical bit numbers, combined with the added SWAP gate instructions and coupler control instructions.

7. The quantum algorithm compilation method based on multi-agent reinforcement learning according to claim 1, characterized in that, Also includes: Obtain compilation execution data after executing the optimal compilation scheme, and determine the performance index of the optimal compilation scheme based on the compilation execution data; wherein, the performance index includes at least one of the following: circuit depth reduction rate, two-qubit gate reduction rate, execution fidelity, compilation latency, and robustness; Determine whether the performance indicators meet the preset performance indicator conditions; If all the performance metrics meet the preset performance metric conditions, then the final compilation scheme is output. If any of the performance indicators fails to meet the preset performance indicator conditions, a deviation value is calculated based on the unmet performance indicator and the preset performance indicator conditions. The deviation value is then normalized, and the normalized deviation value is incorporated into the reward function of the multi-agent as an additional reward signal. The weights in the reward function are then adjusted. Based on the adjusted reward function and weights, the multi-agent model is retrained and a new optimal compilation scheme is output. Based on the new optimal compilation scheme, the compilation execution data after obtaining the optimal compilation scheme is re-executed. The performance indicators of the optimal compilation scheme are determined based on the compilation execution data until all performance indicators meet the preset performance indicator conditions.

8. A quantum algorithm compilation system based on multi-agent reinforcement learning, characterized in that, include: The quantum algorithm parsing module is used to parse high-level quantum algorithms into unitary equivalent ZX-graphs, extract multiple graph features of the ZX-graphs, and fuse the multiple graph features into a graph feature vector; The multi-agent conversion module is used to obtain the quantum mapping information of the target quantum hardware corresponding to the high-level quantum algorithm, and to construct a multi-agent Markov decision process environment based on multiple graph features, graph feature vectors and quantum mapping information. A multi-agent training module is used to perform reinforcement learning training on the multi-agent based on the Markov decision process environment of the multi-agent to obtain a trained multi-agent model. The compilation scheme determination module is used to obtain the quantum algorithm to be compiled in the current computing period and the current quantum mapping information of the target quantum hardware corresponding to the quantum algorithm to be compiled. Based on the quantum algorithm to be compiled and the current quantum mapping information, and in combination with the trained multi-agent model, the module outputs the optimal compilation scheme for the quantum algorithm to be compiled.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the quantum algorithm compilation method based on multi-agent reinforcement learning as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the quantum algorithm compilation method based on multi-agent reinforcement learning as described in any one of claims 1-7.

Images