Apparatus and method for unitary synthesis of quantum circuits with reinforcement learning

Abstract

An apparatus for determining a gate representation of a target unitary matrix according to an embodiment is provided. The apparatus comprises a selection unit (110) configured for conducting a selection step to select a selected operation out of a plurality of possible operations to append at least one quantum gate to an intermediate quantum circuit. Moreover, the apparatus comprises an appending unit (120) configured for conducting an appending step, which updates a gate representation of the intermediate quantum circuit, so that the at least one quantum gate is appended to the intermediate quantum circuit in the gate representation of the intermediate quantum circuit. The apparatus is configured to repeat the selection step of the selection unit (110) and the appending step of the appending unit (120) until a unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

Description

[0001] Apparatus and Method for Unitary Synthesis of Quantum Circuits with Reinforcement Learning

[0002] Description

[0003] The present invention relates to an apparatus and a method for unitary synthesis of quantum circuits with reinforcement learning, in particular, using a Monte Carlo tree search, and, more particularly, to an apparatus and a method for unitary synthesis of Clifford+T circuits with reinforcement learning.

[0004] The advent of quantum computing foreshadows a paradigm shift in our computational capabilities, sparking significant efforts towards scalable implementations of quantum algorithms on actual quantum hardware. Like classical computers, executing a quantum algorithm on a device requires converting a high-level algorithmic representation into low- level, hardware-dependent operations. For gate-model quantum computers, which process and manipulate quantum information using quantum gates that operate on qubits, a quantum compiler is required that can map quantum algorithms to efficient quantum circuits utilizing gates from a predefined gate set (see [1]).

[0005] Quantum compilation is an umbrella term for various sub-routines, which make it possible for a high-level abstract quantum algorithm to be efficiently executed on a quantum hardware. One of the subproblems in quantum compilation is that of circuit optimisation, which involves sub-routines to reduce the depth of the given circuit by either locally optimising the circuit as done in peephole optimisation and template optimisation or globally optimising the circuit. An overview on quantum compilation is provided in

[0064] ,

[0006] Another interesting method for local circuit optimisation is PyZX (see

[0017] ), a Python tool which implements the theory of the ZX-calculus for the creation, visualisation, and automated rewriting of large-scale quantum circuits, and which maps circuits to their optimized equivalent.

[0007] In unitary synthesis, synthesizing quantum circuits from unitaries is conducted. In particular, in unitary synthesis, fundamental one and two qubit gates are used to approximate / represent a target circuit which is given in the unitary matrix representation, with the aim to reduce the depth of the circuit, total gate count, a particular gate count or some combinations of these. Although various algorithms are available for continuous gate sets (rotation gates and two qubit entangling gates), efficient algorithms for synthesising exactly representable unitaries in terms of Clifford+T gate sets are comparatively scarce, and especially those that scale well in terms of the execution time, with the number of qubits. Approaching unitary synthesis formally, given a gate set, G it can be said that C is a circuit over C?, if C is a quantum circuit only containing gates in and the set of all such circuits can be denoted as {G) . Given a unitaryEZ / (2"),agafeset G and some precision e, unitary synthesis aims to find a circuit C e S) such that jS adistance metric / distance measure. When the target range of d is P, 1] G R, this inequality can be formulated in terms of similarity, i.e., ’— 0, where d is a similarity threshold. If e = 0 or, similarly, 6 = 1, then the problem is referred to as exact synthesis.

[0008] Most fault-tolerant protocols use Clifford circuits, which comprise gates from the Clifford group, that is, the set of unitaries that map Pauli strings to Pauli strings up to a phase of It does not form a universal gate set, but requires the one-qubit T gate corresponding to a unitary matrix with {l, eJ7r / 4} along the diagonal. This gate set is known as Clifford+T.

[0009] Among other issues, the T gate poses a challenge for implementing the error-correcting codes in fault-tolerant quantum computers (see [3]), which dictates reducing their number in a circuit as much as possible. Of course, not all unitaries can be exactly synthesized using Clifford+T gate sets, and, in this case, one needs to resort to approximations (see

[0018] ). To concretize the idea of the optimal number of T gates required, the definitions provided in

[0019] may, e.g., be used for any unitary U which is known to be exactly synthesizable from the Clifford+T gate set (as defined in

[0020] ), we define its T-count, denoted H^ ), as the minimal number of T gates required to implement U, which may, e.g., only differ in a global phase, and call such unitary a Clifford+T unitary.

[0010] Regarding quantum compilation, the umbrella term quantum compilation accounts for the challenges faced by noisy NISQ devices as well as fault-tolerant devices, on both the software and the hardware level. They have been categorised (see [1]) as: (1) a translation of high level quantum algorithm into hardware specific gate instruction set, (2) a qubit mapping and routing on connectivity constrained architecture, (3) a circuit optimisation to reduce the depth of the circuit, and (4) a scheduling of quantum operations on the real hardware.

[0011] In quantum compilation, the primary aim of quantum circuit optimisation can be described as reducing the depth of the circuit by reducing one or more gate counts in the translated circuit from step 1 and 2. For example, for the NISQ devices which use 2-qubit entangling gates and 1 qubit rotation gates, reducing the 2-qubit gate count is essential due to their high noise threshold (see [4]), whereas for fault-tolerant systems reducing the number of T gates in the circuit takes priority (see [3]).

[0012] There are various approaches one can take in order to optimise a given quantum circuit. Circuit optimisation may, e.g., be conducted as a local optimisation or as a global optimisation. Local optimisation concepts are, e.g., peephole optimization, template optimisation, and ZX-calculus based optimization. Global optimisation concepts are, e.g., unitary synthesis (e.g., MITM or MIN-T-SYNTH) and unitary decomposition (e.g., cosinesine or quantum Shannon).

[0013] In local optimisation, the original quantum circuit is kept intact and modifications are done to a subset of qubits and small number of gates on those qubits, at a given time. Examples of such optimization algorithms are the peephole optimisation, template optimisation (see

[0052] and

[0053] ) and ZX-calculus based algorithm (see

[0017] ). A major advantage of local optimisation algorithms is that they can be applied for quantum circuit containing hundreds of qubits and gates without the use of extensive computational resources, but this also leads to a drawback in the sense that there is no way to guarantee if the optimised circuit is actually optimal.

[0014] On the other hand, global optimisation algorithms take the entire high level un-optimised quantum circuit in the form of its unitary matrix representation. Global optimisation algorithms are further divided into unitary synthesis and decomposition. One of the first papers on decomposing various unitaries as well as controlled unitaries was

[0050] , where provable decompositions were provided. Similarly, 2-qubit decomposition (see

[0054] and

[0055] ), general 3-qubit quantum gate decomposition (see

[0056] ), cosine-sine decomposition (see

[0057] and

[0058] ) and quantum Shannon decomposition (see

[0059] ) for general multi-qubit gates and exact decomposition of multi-qubit Clifford+T circuits (see

[0020] ) have been provided. A common theme across all these approaches is that one can derive a relation between the number of qubits and total number of unitaries / particular gate. Or, in other words, the ’decomposition’ of the arbitrary target unitary has a known count in terms of the elementary unitaries, which are well known. Previous work on unitary synthesis such as single qubit unitary synthesis using Clifford+T (see

[0027] ), depth-optimal synthesis using meet-in-the-middle (MITM) (see [5]), matroid partitioning algorithm (see

[0060] ), COUNT-T algorithm in

[0019] and in [6], QSEARCH, QFAST and LEAP (see

[0061] ,

[0062] and

[0063] ) have been provided. There, the primary aim is to use known elementary gate sets (such as Clifford+T or CX and rotation gates) to approximate / represent the arbitrary target unitary, such that the total number of gates or the depth of the circuit, or a specific gate count (number of T gates or CX gates) is as small as possible, without going into getting the exact number of gates in a general sense. This distinction between unitary synthesis (approximate / exact representation of a given unitary in term of the elementary gate sets) and decomposition (a mathematical relationship between the total number of either all / particular elementary gate and the number of qubits) is crucial.

[0015] In a fault-tolerant setting, see [2], the Clifford+T gate set plays a crucial role due to its relevance in realizing universal, fault-tolerant quantum computation. However, implementing special gates like the T-gate is associated with significant costs, see [3], analogous to 2-qubit entangling gates tending to be more susceptible to noise, see [4], Thus, besides optimising towards shallow circuits, compilation strategies explicitly considering the number of specific gates like the T-gate in the fault-tolerant Clifford+T regiment or the count of 2-qubit gates on near-term devices are required.

[0016] Although exact and provably optimal compilation methods for Clifford+T exist (see [5], [6]), their practicality is limited due to runtime complexities that grow exponentially in the number of qubits.

[0017] This highlights the need for heuristic approaches to make quantum compilation more applicable to real-world problems. In recent developments, reinforcement learning (RL) has emerged as a highly successful framework for learning such heuristics through trial and error. Notably, Monte Carlo tree search (MCTS) (see [7] and [8]) approaches, exemplified by AlphaGo (see [9]) and AlphaZero (see

[0010] ), have demonstrated great potential for solving complex problems, see

[0011] and

[0012] ,

[0018] Along the advances of reinforcement learning, researchers in the quantum computing field have started using this technology for quantum compilation, quantum circuit optimisation and quantum routing (see

[0013] ,

[0021] ,

[0022] ,

[0023] ,

[0024] ,

[0025] ,

[0026] ).

[0019] Reinforcement learning is a paradigm of learning algorithms, where an agent interacts with an environment and, by itself, learns how to behave in order to optimize a reward signal. Typically, the agent can perform a set of discrete or continuous actions, or a mixture of both. In particular, the agent’s goal is to learn an action policy that, in every state, maximizes the expected return, meaning the expected sum of cumulative rewards when acting according to a policy from a state.

[0020] In classical quantum compilation, the earliest theoretical work on the exact synthesis of single-qubit unitaries using Clifford+T has been provided in

[0027] , which has focused on efficiently deciding, if a unitary can be synthesised exactly and, if so, finding an efficient gate sequence that implements it. In

[0020] , another exact synthesis algorithm has been provided, subsequently improved by

[0028] , though the number of gates required by both algorithms has been shown to be exponential in the number of qubits. In

[0029] , a highly efficient algorithm for approximating arbitrary single-qubit unitaries using the Clifford+T gate set has been proposed.

[0021] The first classical approach with respect to Clifford+T synthesis regarding T gate minimization was provided in

[0019] , That approach builds on the channel representation of a unitary (see

[0019] ) and a technique capable of deciding m G N. The complexity of this algorithm was shown to be O(2nrnoly (m., )),were n is the number of qubits. The work in [6], herein referred to as MIN-T-SYNTH, improved this algorithm, building on the MITM method (see [5]) through novel ways to prune their method’s search tree in order to find the T-count of a given unitary and to synthesize the T-count optimal circuit. It achieved a space complexity ofe lpoly(m, 2" )), where c > 2, and a time complexity 1 1.

[0022] QuantumCircuitOpt (see

[0030] ) is one of the few frameworks towards practical unitary synthesis. It formulates the quantum circuit design task as crafted mixed-integer programs (MIP) (see

[0031] ) that are solved using the Gurobi (see

[0032] ) optimisation software. Moreover, considerable research has been done on quantum circuit optimisation to minimize T-gate utilization. Out of these methods, (see

[0066] ) presented a practical optimisation scheme built on the rules of ZX-calculus, which is publicly available through the PyZX package (see

[0017] ).

[0023] In

[0033] ,

[0034] ,

[0014] ,

[0035] ,

[0036] ,

[0037] ,

[0038] ,

[0039] ,

[0040] ,

[0041] ,

[0042] and

[0043] , contributions regarding machine learning quantum compilation have been provided.

[0024] For approximate quantum synthesis, in

[0013] , the formulation of compiling quantum circuits from unitaries as a sequential decision-making process has been described (see also

[0065] ). There the successful compilation of general single-qubit Haar-unitaries under the HRC efficient universal set has been described. Moreover, in the prior art, a rotational gate set leveraging DQN (see

[0044] ) and PPO (see

[0045] ) has been provided. In

[0046] , this setting could be scaled to 2-qubit unitaries for the special case of the HRC gate set using a combination of DQN and A* search, which, however, suffers from a search complexity that grows exponentially in the number of gates required for the circuit. The object of the present invention is to provide improved concepts for unitary synthesis of quantum circuits. The object of the present invention is solved by the subject-matter of the independent claims. Particular embodiments are provided in the dependent claims.

[0025] An apparatus for determining a gate representation of a target unitary matrix according to an embodiment is provided. The apparatus comprises a selection unit configured for conducting a selection step to select a selected operation out of a plurality of possible operations to append at least one quantum gate to an intermediate quantum circuit. Moreover, the apparatus comprises an appending unit configured for conducting an appending step, which updates a gate representation of the intermediate quantum circuit, so that the at least one quantum gate is appended to the intermediate quantum circuit in the gate representation of the intermediate quantum circuit. The apparatus is configured to repeat the selection step of the selection unit and the appending step of the appending unit until a unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

[0026] E.g., a target unitary matrix exhibits a unitary matrix representation.

[0027] Moreover, a method for determining a gate representation of a target unitary matrix according to an embodiment is provided. The method comprises:

[0028] Conducting a selection step to select a selected operation out of a plurality of possible operations to append at least one quantum gate to an intermediate quantum circuit. And:

[0029] Conducting an appending step, which updates a gate representation of the intermediate quantum circuit, so that the at least one quantum gate is appended to the intermediate quantum circuit in the gate representation of the intermediate quantum circuit.

[0030] The method comprises repeating the selection step and the appending step until a unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

[0031] Furthermore, a computer program for implementing the above-described method when being executed on a computer or signal processor according to an embodiment is provided. Embodiments relate to unitary synthesis using Clifford+T circuits with the target to minimize the number of T gates employed while obtaining an exact solution circuit, e.g., which may, e.g., only differ in a global phase.

[0032] According to embodiments, a deep reinforcement learning method with a novel treesearch approach utilizing Gumbel AlphaZero method for the problem of synthesising unitaries is provided, which can exactly be synthesised using Clifford+T gate sets. In embodiments, scaling the synthesis task up to 4 qubits can be achieved, with execution times on the order of 10 - 100 seconds on a single GPU, beating state-of-the-art algorithms, such as QuantumCircuitOpt and MIN-T-SYNTH for higher qubits.

[0033] Some embodiments address the unitary synthesis approach to globally optimising the circuit using the Clifford+T gate set, which can then be handed to the quantum compiler afterwards for execution.

[0034] According to embodiments, policies for compiling different unitaries may, e.g., be learned, in contrast to previous approaches that rely on training as the compilation routine for one specific unitary, see

[0014] and

[0016] ,

[0035] Embodiments scale such RL-based unitary synthesis algorithms past the one or two-qubit setting for arbitrary but exactly synthesisable unitaries. Moreover, embodiments are not limited to any specific parameter-free gate set, high-level algorithm representation, or the exact synthesis regime.

[0036] Improving previous reinforcement learning-based compilers (see

[0013] ), embodiments model quantum compilation as a sequential decision-making problem where an RL agent appends one gate at a time to the quantum circuit. Even though model-free reinforcement learning has proven successful on simple quantum compilation tasks (see

[0013] and

[0014] ), embodiments provide improved concepts for more challenging settings.

[0037] In particular, embodiments provide a novel tree-search approach for compiling quantum circuits utilizing Gumbel AlphaZero (see

[0015] ). In particular, embodiments realize an exact synthesis of unitaries using the Clifford+T gate set, thereof referred to as unitary synthesis, and can synthesise unitaries into shallow circuits with a reduced number of T- gates, compared to that used to originally build the target quantum circuit for circuits of up to five qubits. The global optimisation method of embodiments performs similar local optimisations as PyZX in order to reduce the effective depth of the circuit.

[0038] Some embodiments focus on unitary synthesis using a Clifford+T gate set as the elementary gate set to construct an arbitrary target unitary which is known to be exactly representable using Clifford+T gate set.

[0039] In further embodiments, quantum mechanical identities may, e.g., be incorporated into the knowledge base of the agent, so that it does not waste training time exploring fruitless paths.

[0040] Embodiments may, e.g., employ a beam search instead of a parallel search to improve results.

[0041] In the following, embodiments of the present invention are described in more detail with reference to the figures, in which:

[0042] Fig. 1 illustrates an apparatus according to an embodiment.

[0043] Fig. 2 depicts a schematic illustration of a reinforcement learning environment according to an embodiment.

[0044] Fig. 3 illustrates an algorithm for parallel search using Gumbel AlphaZero according to an embodiment.

[0045] Fig. 4 depicts a schematic illustration of the evaluation performed by an reinforcement learning agent according to an embodiment, and further illustrates how the success ratio is defined.

[0046] Fig. 5 illustrates a table which shows a benchmarking of the method according to an embodiment regarding specific unitaries

[0047] Fig. 6 illustrates a 2-qubit gate true decomposition, wherein Fig. 6(a) depicts a decomposition of a controlled-Y Gate, wherein Fig. 6(b) illustrates a decomposition of a controlled-Z gate, wherein Fig. 6(c) shows a decomposition of Controlled-H gate, wherein Fig. 6(d) depicts a decomposition of a Controlled-P gate, wherein Fig. 6(e) illustrates a decomposition of a controlled-V gate, and wherein Fig. 6(f) depicts a decomposition of a W gate.

[0048] Fig. 7 illustrates a 3-qubit circuits decomposition, wherein Fig. 7(a) depicts a Toffoli gate decomposition, wherein Fig. 7(b) depicts a single negated Toffoli decomposition, wherein Fig. 7(c) depicts a quantum OR decomposition, and wherein Fig. 7(d) depicts a Peres gate decomposition.

[0049] Fig. 8 illustrates 4-qubit unitaries used in the table of Fig. 5, wherein Fig. 8(a) depicts a Toffoli with one Ancilla, wherein Fig. 8(b) depicts two Toffoli, wherein Fig. 8(c) depicts three Toffoli, wherein Fig. 8(d) depicts a controlled-T with 2 Ancilla, and wherein Fig. 8(e) depicts a reversible 1-but full adder.

[0050] Fig. 9 illustrates a first histogram of the percentage of circuits, an RL agent according to an embodiment was able to evaluate for each T-gate count in the source circuit, compared to PyZX.

[0051] Fig. 10 illustrates a second histogram of the percentage of circuits, an RL agent according to another embodiment was able to evaluate as a function of total gate count in the source circuit, compared to PyZX.

[0052] Fig. 11 illustrates an average and median number of T gates for both PyZX and for an apparatus according to an embodiment.

[0053] Fig. 12 illustrates an average and median total gate count after optimization, and illustrates the curves of PyZX and reinforcement learning according to an embodiment.

[0054] Fig. 1 illustrates an apparatus for determining a gate representation of a target unitary matrix according to an embodiment.

[0055] The apparatus comprises a selection unit 110 configured for conducting a selection step to select a selected operation out of a plurality of possible operations to append at least one quantum gate to an intermediate quantum circuit. Moreover, the apparatus comprises an appending unit 120 configured for conducting an appending step, which updates a gate representation of the intermediate quantum circuit, so that the at least one quantum gate is appended to the intermediate quantum circuit in the gate representation of the intermediate quantum circuit.

[0056] The apparatus is configured to repeat the selection step of the selection unit 110 and the appending step of the appending unit 120 until a unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

[0057] According to an embodiment, the selection unit 110 may, e.g., be configured to select the selected operation out of the plurality of possible operations by selecting a quantum gate of a set of two or more quantum gates.

[0058] In an embodiment, the selection unit 110 may, e.g., be configured to select the quantum gate out of the set of two or more quantum gates which comprise at least two of the following gates: a Hadamard gate, a T gate, a Hermitian conjugate of T gate, a phase gate a Hermitian conjugate of a phase gate, a NOT gate, a CNOT gate.

[0059] According to an embodiment, the selection unit 110 may, e.g., be configured to select the quantum gate out of a Clifford+T gate set.

[0060] In an embodiment, the apparatus may, e.g., be configured to start with a gate representation of an empty quantum circuit which does not comprise any quantum gates as the representation of the intermediate quantum circuit. Starting with the representation of the empty quantum circuit, the apparatus may, e.g., be configured to repeat the selection step of the selection unit 110 and the appending step of the appending unit 120 until the unitary matrix of the intermediate quantum circuit may, e.g., be equal to the target unitary matrix.

[0061] According to an embodiment, the apparatus may, e.g., be configured to start with a gate representation of a random quantum circuit which comprises one or more randomly or quasi-randomly chosen quantum gates as the representation of the intermediate quantum circuit. Starting with the representation of the random quantum circuit, the apparatus may, e.g., be configured to repeat the selection step of the selection unit 110 and the appending step of the appending unit 120 until the unitary matrix of the intermediate quantum circuit may, e.g., be equal to the target unitary matrix.

[0062] In an embodiment, the apparatus may, e.g., be configured to receive the gate representation of the target quantum circuit being a gate representation of a problem dependent quantum circuit. The apparatus may, e.g., be configured to conduct a preprocessing step in which the gate representation of the target quantum circuit may, e.g., be converted into a target unitary matrix.

[0063] According to an embodiment, the apparatus may, e.g., be configured to repeat the selection step of the selection unit 110 and the appending step of the appending unit 120 until the unitary matrix of the gate representation of the intermediate quantum circuit may, e.g., be equal to the target unitary matrix, depending on a distance measure.

[0064] In an embodiment, the apparatus may, e.g., be configured to employ the distance measure by determining a distance between the unitary matrix of the intermediate quantum circuit and the target unitary matrix.

[0065] According to an embodiment, the apparatus may, e.g., be configured to determine the distance using at least one of:

[0066] Hilbert-Schmidt norm,

[0067] Frobenius norm,

[0068] Diamond norm, Spectral norm,

[0069] - Average gate fidelity.

[0070] In an embodiment, the apparatus may, e.g., be configured to determine the distance depending on wherein U indicates the unitary matrix of the intermediate quantum circuit, wherein Vindicates the target unitary matrix, wherein d indicates a real number, wherein V1indicates a Hermitian conjugate of the target unitary matrix.

[0071] According to an embodiment, the apparatus may, e.g., be configured to determine a reward depending on the distance.

[0072] In an embodiment, the apparatus may, e.g., be configured to determine a plurality of gate representations for a same target unitary matrix.

[0073] According to an embodiment, the apparatus may, e.g., be configured to determine the plurality of gate representations for the same target unitary matrix in parallel.

[0074] In an embodiment, the apparatus may, e.g., be configured to determine the plurality of gate representations for the same target unitary matrix by conducting a plurality of parallel searches.

[0075] According to an embodiment, the apparatus may, e.g., be configured to conduct a Monte Carlo tree search to determine the plurality of gate representations of a plurality of target unitary matrices.

[0076] In an embodiment, the apparatus may, e.g., be configured to determine the plurality of gate representations for the same target unitary matrix by applying a Gumbel AlphaZero algorithm.

[0077] According to an embodiment, the apparatus may, e.g., be configured to train and / or to test a set of one or more neural networks for determining a gate representation of the target unitary matrix.

[0078] In an embodiment, the apparatus comprises the neural network.

[0079] According to an embodiment, the apparatus may, e.g., be configured to train the neural network depending on the reward.

[0080] In the following, particular embodiments are provided.

[0081] In particular embodiment, exact Clifford+T synthesis may, e.g., realized using a Monte Carlo tree search (MCTS) based on the method Gumbel AlphaZero. An apparatus / agent according to an embodiment learns in episodical trial and error interactions with its (e.g., software) environment to find an optimal solution.

[0082] The environment of the agent / apparatus may, e.g., be described as follows:

[0083] At the start of an episode an arbitrary / random quantum circuit may, e.g., be generated based on Clifford+T. The start state may, e.g., be initialised based on an empty quantum circuit.

[0084] The agent may, e.g., select an append operation (an action) out of all possible append operations (possible placements of each gate of the Clifford+T gate alphabet (a Clifford+T gate set) at the end of the quantum circuit). The append operation is then applied on the quantum circuit. The agent obtains / determines the matrix product of the unitary matrix of the updated quantum circuit multiplied by the Hermitian conjugate of the target unitary (in other words: a residual unitary matrix is obtained). A reward of -1 (in other words: a penalty) is obtained, if the unitary matrix of the intermediate quantum circuit is not yet equal to the target unitary matrix; otherwise, a reward of 0 is obtained.

[0085] A method according to an embodiment can be separated into a training phase and into a testing phase.

[0086] Training: A Gumbel AlphaZero agent may, e.g., be trained in the above-described embodiment and may, e.g., provide as a result, e.g., two neural networks, which represent the trained state of the agent.

[0087] Testing may, e.g., comprise:

[0088] 1. The trained state of the agent may, e.g., be loaded.

[0089] 2. The unitary matrix (e.g., externally defined) is replicated N times.

[0090] 3. The agent may, e.g., executed N times in parallel to generate a set of potential solutions, e.g., based on statistic properties.

[0091] 4. The best circuit may, e.g., be selected out of the set of potential solutions.

[0092] In the following, particular embodiments are described.

[0093] In particular, reinforcement learning for unitary synthesis according to embodiments is described. Fig. 2 illustrates a reinforcement learning (RL) setup according to an embodiment. In particular, Fig. 2 depicts a schematic illustration of a reinforcement learning environment according to an embodiment. Some elements of that RL setup are based on

[0013] ,

[0094] At the beginning of an episode, a random quantum circuit is created by sampling uniformly from the set of possible gate append operations and by calculating the episode’s target unitary, which may, e.g., be denoted as V. Here unitaries of random circuits are employed instead of sampling them, for example, from the Haar measure, as the exact synthesisability of the training unitaries using Clifford+T shall be guaranteed.

[0095] Given the discrete action space representing the set of all possible append operations, the agent’s goal is to successively build up a circuit C with unitary U that is equal to the target unitary V, which may, e.g., only differ in a global phase. For an exact compilation, 0 may, e.g., be set to 0 = 1, and the Hilbert-Schmidt Test (HST) may, e.g., be employed to define the distance between U and V as

[0096] Instead of the concatenation of U and V , UV^ may, e.g., be employed as observation for the agent. This is expected to stabilize training as it normalizes the goal-state observation to the identity matrix independent of the target V and, secondly, halves the size of the observation space as, otherwise, both U and V would have to be provided to the agent. Intuitively, UVfcan be interpreted as the unitary residual toward the target unitary V, meaning the unitary of the yet unbuilt part of the final circuit.

[0097] At the start of every episode, a random circuit may, e.g., be generated to derive a feasible Clifford+T target unitary. At each time step:

[0098] (1) the agent’s action may, e.g., be correlated with its corresponding append operation,

[0099] (2) the append operation may, e.g., be applied to the circuit,

[0100] (3) the circuit unitary U may, e.g., be updated,

[0101] (4) U and V may, e.g., be compared, e.g., using the Hilbert-Schmidt Test (HST)

[0102] (5) the appropriate reward may, e.g., be calculated,

[0103] (6) observation, reward and terminal signals may, e.g., be returned to the agent. In the following, a search, e.g., a parallel search, using, e.g., (Gumbel) AlphaZero, according to embodiments is described.

[0104] AlphaZero employs an approximate policy iteration in conjunction with a Monte Carlo tree search (MCTS) to enhance its decision-making capabilities. When using a small number of simulations to construct the search tree, AlphaZero may encounter limitations, leading to certain actions being undersampled and insufficiently improving the policy network guiding the search. In scenarios with a number of discrete actions denoted as k, and an MCTS simulation budget denoted as n, the Gumbel AlphaZero algorithm may, e.g., be specifically designed for settings where the simulation budget n is much smaller than the number of discrete actions k. Embodiments may, e.g., leverage this method to perform a parallel search for the optimal circuit inside the solution space.

[0105] Gumbel AlphaZero utilizes the Gumbel-Top-^ trick at the root node to sample a subset of m actions without replacement. This may, e.g., involve a vector of independent Gumbel-distributed variables, and a policy network = logits e for the root state 5. The Gumbel-Top-^ trick selects the actions that maximize the expression -' / logit* , denoted asi i rptop\ tj + logit*. j Subsequently, using the same Gumbel variables, actions from the subsampled set may, e.g., be selected as argmax a where empirical state-action values, and <7 is a monotonically increasing function. It can be shown that selecting following Equation 2 guarantees policy improvement, assuming accurate estimation of state-action values at the root state. As a concrete instantiation of a, in embodiments:

[0106] (based on the MPO policy update in

[0067] ), where N(s, d) is the action visitation counter for root state 5, and cv / OTfand cscaie are hyperparameters. The Sequential Halving approach may, e.g., be employed to effectively distribute the simulation budget among the sampled actions, providing a bandit algorithm that minimises simple regret as opposed to the cumulative regret in AlphaZero. Finally, to minimise unwanted variance in the Q-value estimates, actions may, e.g., be deterministically selected at non-root nodes, in contrast to the stochastic non-root action selection in vanilla AlphaZero. Some embodiments may, e.g., employ a parallel search. Assuming a fully trained Gumbel AlphaZero agent with policy network and value network , a trick to find optimal solutions is to leverage the stochasticity of the Gumbel AlphaZero method and search across multiple solutions produced by independent MCTS runs. Instead of greedily sampling actions during evaluation, an embodiment may, e.g., employ the same strategy of Gumbel-based sampling during training. An implementation according to an embodiment may, e.g., be build on Jax (see

[0047] ) and / or may, e.g., employ a vectorized Gumbel AlphaZero implementation, for example, provided my DeepMind’s mctx package, which incurs relatively little extra computational cost.

[0107] Fig. 3 illustrates an algorithm for parallel search using Gumbel AlphaZero according to an embodiment. In particular, the algorithm of Fig. 3 represents an algorithm for parallel evaluation of a single target unitary. It should be noted that it is assumed that the intermediate quantum circuit is built at time step t as part of the environment state stto extract and score the solution circuit using the terminal states saved inside S of the parallel environment runs.

[0108] In the following, further particular embodiment are described.

[0109] Regarding the setup, most of the code has been implemented in Jax with full JIT- compilability to circumvent impractical training complexity. As of this date, no JIT quantum simulation framework exists; therefore, the circuit simulation and unitary calculation were also implemented manually but cross-verified with Qiskit (see

[0068] ) during evaluation. For Gumbel AlphaZero, the mctx package from DeepMind (see

[0047] ) has been used, which is also written in Jax. Two simple feed-forward neural networks have been used for the actor and critic network, where the real and imaginary parts of the observation UV^ have been split into separate entries, and the result for the forward-pass has been flattened. Every network comprises five layers, 1024 hidden units per layer, and intermediate layer normalization, see

[0048] , To stabilize training, the critic output has been normalized into the [-1, 1] range.

[0110] Regarding training, apparatuses / agents according to embodiments have been trained for two million time steps using a replay buffer fitting 100000 transitions, a batch size of 1024, and step 256 environments in parallel. Therefore, every transition sample has an expected sampling rate of four before it gets replaced with a new sample. This setup allows to train an apparatus / agent using a single GPU (Tesla V100 and RTX 3080) in less than seven days. Concerning the environment, the number of gates has been sampled in the random circuits uniformly from the interval [3, p\, where fi has been decreased with growing qubit numbers to ease training due to the exponentially growing size of the observation and action space. More specifically, fJ> = 120, 100, 80, 60 for the two- to five- qubit models respectively. The gate set T,T 5, S\ , C - } hasbeenused, if not stated otherwise.

[0111] Fig. 4 depicts a schematic illustration of the evaluation performed by an reinforcement learning agent according to an embodiment, and further illustrates how the success ratio is defined.

[0112] Regarding evaluation, for parallel search, a given unitary is repeated 128 times into a single batch. The synthesis is done synchronously parallel employing Batch MCTS (see

[0049] ), in the same manner as during training. The synthesised circuit with the lowest gate count or number of T gates (depending on the experiment) is selected among all of the solution circuits. The fraction of successful runs that found a solution as the success ratio has been denominated.

[0113] Fig. 5 illustrates a table which shows a benchmarking of the method according to an embodiment regarding specific unitaries against MIN-T-SYNTH and MIP as implemented in

[0038] , The metric of comparison is the number of T gates in the synthesised circuits and the time for synthesis. The success ratio, in the column for our RL agent, provides the number of successful runs. The actual decompositions of some unitaries are shown in B.

[0114] Regarding synthesising structured unitaries, to assess the performance of the apparatus / agent of embodiments in synthesizing quantum circuits from a given unitary, unitaries have been selected that are commonly encountered in various high-level algorithms. These particular unitaries have also served as benchmarks for other synthesis algorithms, allowing for a comparison between the RL apparatus / agent of embodiments and classical synthesis algorithms. These unitaries, such as Toffoli, Fredkin, Controlled- Phase (CP), Controlled-T (CT), and Controlled-v f (CV) gates, are crucial for decomposing multi-qubit multi-controlled unitaries efficiently, see

[0050] , The RL agent of embodiments has been compared with the MIN-T-SYNTH algorithm (see [6]), which is an enhancement of the MITM algorithm (see [5]) based on the work in

[0019] ,

[0115] Additionally, the apparatus / agent of embodiments has been compared against QuantumCircuitOpt (see

[0030] ), an open-source synthesis framework implemented in Julia (see

[0051] ). The underlying algorithm of QuantumCircuitOpt is based on the Mixed-Integer Programming (MIP) method, see

[0031] , Fig. 5 illustrates a table which shows a comparison of the performance of the apparatus / agent of embodiments in synthesising quantum circuits from specific unitaries. The first column indicates the number of qubits in the input circuit. The second column lists the target unitaries, which have been extensively studied in previous synthesis algorithms and are commonly used in complex algorithms. The third column (RL agent) provides the results obtained by the apparatus / agent of embodiments, including the success rate, number of T gates of the synthesized circuit, and the time taken for synthesis. The fourth and fifth column are the results of the MIN-T-SYNTH and MIP methods.

[0116] As shown in the table, the success rate for 2-qubit unitaries is 1.0, indicating that the apparatus / agent successfully synthesises the entire batch of these relatively simple unitaries. However, for 3-qubit unitaries, there is a slight drop in the success ratio, and for the more complex 4-qubit unitaries, the performance.

[0117] For comparison, the code provided by the authors of [6] has been employed. While the evaluation was successful for some of the chosen examples, manual code adjustment were required for others. The true decomposition has been provided as presented in [5], For higher qubit circuits, the synthesis using the MIP method took more than three hours without finding a solution in the basic Clifford+T gate set.

[0118] The results indicate that for the apparatus / agent of embodiments, the number T gates for the Toffoli (1 Ancilla), 3 Toffoli (U2), and 1-bit adder circuits is optimal, which demonstrates the promising performance of the RL agent of embodiments. A slight drop in performance for higher qubits can be attributed to the vast number of possibilities for gate placement on the qubit lines.

[0119] Comparing the runtime of an MCTS apparatus / agent of an embodiment to the two classical methods, highly promising results are achieved, as for embodiments, it is not necessary to handcraft rules to handle the complexities of quantum computing. The agent of embodiments learned to account for these rules over time by having a single incentive to reduce the total gate count.

[0120] In the following, solution circuits for specific unitaries are provided. Fig. 6 and Fig. 7 illustrate some known decompositions of specific and structured unitaries, which were also the target for the RL agent of an embodiment as depicted in the table of Fig. 5. The decompositions are from [5] and from [6],

[0121] In Fig. 8, the structure of the target 4-qubit unitaries with the number of T gates found by an RL agent according to an embodiment shown in the table of Fig. 5 is provided.

[0122] In particular, Fig. 6 illustrates a 2-qubit gate true decomposition as shown in [5], wherein Fig. 6(a) depicts a decomposition of a controlled-Y Gate, wherein Fig. 6(b) depicts a decomposition of a controlled-Z gate, wherein Fig. 6(c) depicts a decomposition of controlled-H gate, wherein Fig. 6(d) depicts a decomposition of a controlled-P gate, wherein Fig. 6(e) depicts a decomposition of a controlled-V gate, and wherein Fig. 6(f) depicts a decomposition of a W gate.

[0123] Moreover, Fig. 7 illustrates a 3-qubit circuits decomposition as shown in [5] and in [6], wherein Fig. 7(a) depicts a Toffoli gate decomposition, wherein Fig. 7(b) depicts a single negated Toffoli decomposition, wherein Fig. 7(c) depicts a quantum OR decomposition, and wherein Fig. 7(d) depicts a Peres gate decomposition.

[0124] Furthermore, Fig. 8 illustrates 4-qubit unitaries used in the table of Fig. 5, wherein Fig. 8(a) depicts a Toffoli with one ancilla, wherein Fig. 8(b) depicts two Toffoli (U 1), wherein Fig. 8(c) depicts three Toffoli (U2), wherein Fig. 8(d) depicts a controlled-T with 2 ancillae, and wherein Fig. 8(e) depicts a reversible 1-bit full adder.

[0125] Embodiments allow to prioritize certain properties of the found solution circuits by employing a reward function / signal. As an RL agent may, e.g., aim to maximise the reward signal, a reward of -1 may, e.g., have the effect that the agent attempts to find circuits with few gates. Other reward functions, however, may, e.g., also or alternatively be employed. In embodiments, it is possible to search for circuits with a minimum number of T gates, for example to search for such circuits among a set of circuits generated when conducting tests.

[0126] Embodiments may, e.g., exhibit an advantage that the run time better scales with the number of qubits compared to heuristic methods. Compared to 2-qubit or 5-qubit settings, the run time difference is only caused by the additional processing effort of a larger unitary matrix of the neural networks. On the other side, agents have to be trained before they are employed. Embodiments may, e.g., be employed in a software application for executing circuits in quantum computers based on unitary matrix presentations with high relevance for fault- tolerant quantum computers based on Clifford+T. Moreover, embodiments may, e.g., be employed for circuit optimiser, e.g., for a transformation of a circuit into a unitary matrix, out of which an optimized circuit may, e.g., be calculated.

[0127] In the following, the results of quantitative evaluations are discussed.

[0128] To conduct a more robust test of the method according to an embodiment, its capabilities in synthesising random circuits have been evaluated. For this purpose, 1500 circuits have been generated by uniformly sampling gates from the Clifford+T gate set, ensuring that the total gate count of the target circuit fell within the range of 3 to 80.

[0129] The results of embodiments have been compared against the MIN-T-SYNTH (see [6]) and QuantumCircuitOpt (see

[0030] ) with respect to a set of structured unitaries, such as Controlled-Hadamard, Toffoli, and Peres gate. Due to difficulties scaling these methods to higher qubit numbers in a quantitative evaluation regiment, the PyZX circuit optimizer has been used to derive an upper-bound on of randomized evaluation circuits.

[0130] To assess the performance of the agent, PyZX (see

[0066] ) has been employed, an optimisation algorithm based on the ZX-calculus (see

[0017] ), which can reduce the number of T gates of a circuit, to obtain an upper bound on the number of T gates and total gates required to more effectively represent the target unitary.

[0131] Fig. 9 illustrates a first histogram of the percentage of circuits, an reinforcement learning (RL) agent according to an embodiment was able to evaluate for each T-gate count in the source circuit, compared to PyZX (regarding the definition of succ runs, see the explanations regarding success ratio). In particular, in Fig. 9, the distribution of optimised circuits binned by the number of T gates in the original circuit is provided. It can be observed that the RL agent of an embodiment is only able to successfully synthesise (a successful synthesis is defined in term of success ratio) a small fraction of all the test circuits shown to it, in the majority of the cases. In the ideal case, the successful runs may, e.g., be more than 80% of all the circuits shown to it as a test circuit.

[0132] Fig. 10 illustrates a second histogram of the percentage of circuits, an RL agent according to another embodiment was able to evaluate as a function of total gate count in the source circuit, compared to PyZX. Since the RL agent of embodiments is incentivized to reduce the total gate count rather than the T-gate count of the circuit, better performance can be expected compared to the embodiments of the previous plot depicted in Fig. 9. In particular, similarly as in Fig. 9, in Fig. 10, the distribution of successful RL evaluations binned by the number of total gates in the original circuit is depicted. Also in Fig. 10, the reduction in total gate count is compared against the reduction in the total gate count by PyZX.

[0133] Fig. 11 illustrates a comparison of the count of T gates in the source circuit (x-axis) and the average and median count of T gates (y-axis) over the successful circuits of PyZX (see

[0066] ) and RL of embodiments respectively (pyzx_Avg:1110, RL_Avg:1120, pyzx_Median:1130, RL_median:1140, x=y:1150). For PyZX only those circuits are considered, which also have a solution using our RL method, for fair comparison. In particular, in Fig. 11, the average and median number of T gates for both PyZX and for an apparatus / a method according to an embodiment is plotted. It can be observed that the apparatus / agent according to an embodiment closely follows the curve of PyZX, indicating that the apparatus of an embodiment, on average, is capable of synthesising quantum circuits with a performance concerning T gates comparable to those achieved after PyZX optimisation.

[0134] Fig. 12 illustrates a comparison of the total gate count in the source circuit (x-axis) and the average and median count of T gates (y-axis) over the successful circuits of PyZX and of an apparatus / a method according to an embodiment, respectively (pyzx_Avg:1110, RL_Avg:1120, pyzx_Median:1130, RL_median:1140, x=y:1150). In particular, in Fig. 12, the average and median total gate count after optimisation is depicted, and a comparison of the curves of PyZX and RL according to an embodiment is shown. There, fluctuations can be observed in both curves due to the small number of unitaries over which the metrics are calculated. Nevertheless, a similar behavior of the RL agent of embodiments closely aligning with the PyZX curve can be observed.

[0135] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus. Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0136] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0137] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0138] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0139] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0140] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitory.

[0141] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet. A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0142] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0143] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0144] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0145] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0146] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0147] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein. References

[0148] [1] Marco Maronese, Lorenzo Moro, Lorenzo Rocutto, and Enrico Prati. Quantum compiling. In Quantum Computing Environments, pages 39-74. Springer, 2022.

[0149] [2] Peter W Shor. Fault-tolerant quantum computation. In Proceedings of 37th conference on foundations of computer science, pages 56-65. IEEE, 1996.

[0150] [3] Bryan Eastin and Emanuel Knill. Restrictions on transversal encoded quantum gate sets. Physical review letters, 102 (11): 110502, 2009.

[0151] [4] LI Masanes, Guifre Vidal, and Jose Ignacio Latorre. Time-optimal hamiltonian simulation and gate synthesis using homogeneous local unitaries. arXiv preprint quant-ph / 0202042, 2002.

[0152] [5] Matthew Amy, Dmitri Maslov, Michele Mosca, and Martin Roetteler. A meet-in-the- middle algorithm for fast synthesis of depth-optimal quantum circuits. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 32(6):818-830, 2013.

[0153] [6] Michele Mosca and Priyanka Mukhopadhyay. A polynomial time and space heuristic algorithm for t-count. Quantum Science and Technology, 7(1):015003, 2021.

[0154] [7] Remi Coulom. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In H. Jaap van den Herik, Paolo Ciancarini, and H. H. L. M. (Jeroen) Donkers, editors, Computers and Games, Lecture Notes in Computer Science, pages 72-83, Berlin, Heidelberg, 2007. Springer. ISBN 978-3-540-75538-8. doi: 10.1007 / 978-3-540-75538-8_7.

[0155] [8] Levente Kocsis and Csaba Szepesvari. Bandit Based Monte-Carlo Planning. In

[0156] Johannes Furnkranz, Tobias Scheffer, and Myra Spiliopoulou, editors, Machine Learning: ECML 2006, Lecture Notes in Computer Science, pages 282-293, Berlin, Heidelberg, 2006. Springer. ISBN 978-3-540-46056-5. doi:

[0157] 10.1007 / 11871842_29.

[0158] [9] David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, loannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484-489, January 2016. ISSN 1476-4687. doi: 10.1038 / nature16961.

[0159]

[0010] David Silver, Thomas Hubert, Julian Schrittwieser, loannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419): 1140-1144, December 2018. doi: 10.1126 / science.aar6404.

[0160]

[0011] Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera- Paredes, Mohammadamin Barekatain, Alexander Novikov, Francisco J. R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, David Silver, Demis Hassabis, and Pushmeet Kohli. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610 (7930):47-53, October 2022. ISSN 1476- 4687. doi: 10.1038 / s41586-022-05172-4.

[0161]

[0012] Daniel J. Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco Selvi, Cosmin Paduraru, Edouard Leurent, Shariq Iqbal, Jean-Baptiste Lespiau, Alex Ahern, Thomas Koppe, Kevin Millikin, Stephen Gaffney, Sophie Elster, Jackson Broshear, Chris Gamble, Kieran Milan, Robert Tung, Minjae Hwang, Taylan Cemgil, Mohammadamin Barekatain, Yujia Li, Amol Mandhane, Thomas Hubert, Julian Schrittwieser, Demis Hassabis, Pushmeet Kohli, Martin Riedmiller, Oriol Vinyals, and David Silver. Faster sorting algorithms discovered using deep reinforcement learning. Nature, 618(7964):257-263, June 2023. ISSN 1476-4687. doi: 10.1038 / S41586-023-06004-9.

[0162]

[0013] Lorenzo Moro, Matteo G. A. Paris, Marcello Restelli, and Enrico Prati. Quantum compiling by deep reinforcement learning. Communications Physics, 4(1): 178, August 2021. ISSN 2399-3650. doi: 10.1038 / s42005-021-00684-3.

[0163]

[0014] Zhimin He, Lvzhou Li, Shenggen Zheng, Yongyao Li, and Haozhen Situ. Variational quantum compiling with double Q-learning. New Journal of Physics, 23(3):033002, March 2021. ISSN 1367-2630. doi: 10.1088 / 1367-2630 / abe0ae.

[0015] Ivo Danihelka, Arthur Guez, Julian Schrittwieser, and David Silver. Policy improvement by planning with Gumbel. In International Conference on Learning Representations. OpenReview.net, 2022.

[0164]

[0016] Shunsuke Daimon, Kakeru Tsunekawa, Ryoto Takeuchi, Takahiro Sagawa, Naoki Yamamoto, and Eiji Saitoh. Quantum Circuit Distillation and Compression, September 2023, arXiv preprint arXiv:2309.01911.

[0165]

[0017] Aleks Kissinger and John van de Wetering. PyZX: Large Scale Automated Diagrammatic Reasoning. In Bob Coecke and Matthew Leifer, editors, Proceedings 16th International Conference on Quantum Physics and Logic, Chapman University, Orange, CA, USA., 10-14 June 2019, volume 318 of Electronic Proceedings in Theoretical Computer Science, pages 229-241. Open Publishing Association, 2020. doi: 10.4204 / EPTCS.318.14.

[0166]

[0018] Christopher M Dawson and Michael A Nielsen. The solovay-kitaev algorithm. arXiv preprint quant-ph / 0505030, 2005.

[0167]

[0019] David Gosset, Vadym Kliuchnikov, Michele Mosca, and Vincent Russo. An algorithm for the t-count. arXiv preprint arXiv: 1308.4134, 2013.

[0168]

[0020] Brett Giles and Peter Selinger. Exact synthesis of multiqubit clifford+ t circuits. Physical Review A, 87(3):032332, 2013.

[0169]

[0021] Florian Furrutter, Gorka Munoz-Gil, and Hans J Briegel. Quantum circuit synthesis with diffusion models. arXiv preprint arXiv:2311.02041 , 2023.

[0170]

[0022] Mathias Weiden, Ed Younis, Justin Kalloor, John Kubiatowicz, and Costin lancu. Improving quantum circuit synthesis with machine learning. arXiv preprint arXiv:2306.05622, 2023.

[0171]

[0023] Zhiding Liang, Jinglei Cheng, Rui Yang, Hang Ren, Zhixin Song, Di Wu, Xuehai Qian, Tongyang Li, and Yiyu Shi. Unleashing the potential of Urns for quantum computing: A study in quantum architecture design. arXiv preprint arXiv:2307.08191, 2023.

[0172]

[0024] Francesco Preti, Michael Schilling, Sofiene Jerbi, Lea M Trenkwalder, Hendrik Poulsen Nautrup, Felix Motzoi, and Hans J Briegel. Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning. arXiv preprint arXiv:2307.05744, 2023a.

[0173]

[0025] Alexandru Paler, Lucian Sasu, Adrian-C“at“alin Florea, and R“azvan Andonie. Machine learning optimization of quantum circuit layouts. ACM Transactions on Quantum Computing, 4(2): 1-25, 2023.

[0174]

[0026] Thomas Fosel, Murphy Yuezhen Niu, Florian Marquardt, and Li Li. Quantum circuit optimization with deep reinforcement learning. arXiv preprint arXiv:2103.07585, 2021.

[0175]

[0027] Vadym Kliuchnikov, Dmitri Maslov, and Michele Mosca. Fast and efficient exact synthesis of single qubit unitaries generated by clifford and t gates. arXiv preprint arXiv: 1206.5236, 2012.

[0176]

[0028] Vadym Kliuchnikov. Synthesis of unitaries with clifford+ t circuits. arXiv preprint arXiv: 1306.3200, 2013.

[0177]

[0029] Peter Selinger. Efficient clifford+ t approximation of single-qubit operators. arXiv preprint arXiv:1212.6253, 2012.

[0178]

[0030] Harsha Nagarajan, Owen Lockwood, and Carleton Coffrin. QuantumCircuitOpt: An open-source framework for provably optimal quantum circuit design. In SC21 : The International Conference for High Performance Computing, Networking, Storage, and Analysis, Second Workshop on Quantum Computing Software, pages 55-63. IEEE Computer Society, 2021a. doi: 10.1109 / QCS54837.2021.00010.

[0179]

[0031] Robert E Bixby, Mary Fenelon, Zonghao Gu, Ed Rothberg, and Roland Wunderling. Mixed-integer programming: A progress report. In The sharpest cut: the impact of Manfred Padberg and his work, pages 309-325. SIAM, 2004.

[0180]

[0032] Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2023. URL https: / / www.gurobi.com.

[0181]

[0033] M. S. Alam, Noah F. Berthusen, and P. P. Orth. Quantum Logic Gate Synthesis as a Markov Decision Process, December 2019.

[0034] Yuval Baum, Mirko Amico, Sean Howell, Michael Hush, Maggie Liuzzi, Pranav Mundada, Thomas Merkh, Andre R.R. Carvalho, and Michael J. Biercuk. Experimental Deep Reinforcement Learning for Error-Robust Gate-Set Design on a Superconducting Quantum Computer. PRX Quantum, 2(4):040324, November 2021. ISSN 2691-3399. doi: 10.1103 / PRXQuantum.2.040324.

[0182]

[0035] Tomoaki Kimura, Kodai Shiba, Chih-Chieh Chen, Masaru Sogabe, Katsuyoshi Sakamoto, and Tomah Sogabe. Quantum circuit architectures via quantum observable Markov decision process planning. Journal of Physics Communications, 6(7):075006, July 2022. ISSN 2399-6528. doi: 10.1088 / 2399- 6528 / ac7d39.

[0183]

[0036] En-Jui Kuo, Yao-Lung L. Fang, and Samuel Yen-Chi Chen. Quantum Architecture Search via Deep Reinforcement Learning, April 2021.

[0184]

[0037] Yangzhi Li, Wen Liu, Maoduo Li, and Yugang Li. Quantum circuit compilation for nearest-neighbor architecture based on reinforcement learning. Quantum Information Processing, 22(8):295, July 2023. ISSN 1573-1332. doi: 10.1007 / sl 1128-023-04050-w.

[0185]

[0038] Harsha Nagarajan, Owen Lockwood, and Carleton Coffrin. QuantumCircuitOpt: An Open-source Framework for Provably Optimal Quantum Circuit Design. 2021 IEEE / ACM Second International Workshop on Quantum Computing Software (QCS), pages 55-63, November 2021b. doi: 10.1109 / QCS54837.2021.00010.

[0186]

[0039] Tirthak Patel, Ed Younis, Costin lancu, Wibe De Jong, and Devesh Tiwari. QUEST: Systematically approximating Quantum circuits for higher output fidelity. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pages 514-528, Lausanne Switzerland, February 2022. ACM. ISBN 978-1-4503-9205-1. doi: 10.1145 / 3503222.3507739.

[0187]

[0040] Francesco Preti, Michael Schilling, Sofiene Jerbi, Lea M. Trenkwalder, Hendrik Poulsen Nautrup, F. Motzoi, and H. Briegel. Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning, July 2023b.

[0041] Stan van der Linde, Willem de Kok, Tariq Bontekoe, and Sebastian Feld. Qgym: A Gym for Training and Benchmarking RL-Based Quantum Compilation. 2023. doi: 10.48550 / ARXIV.2308.02536.

[0188]

[0042] Pei-Yong Wang, Muhammad Usman, Udaya Parampalli, Lloyd C. L. Hollenberg, and Casey R. Myers. Automated Quantum Circuit Design with Nested Monte Carlo Tree Search. IEEE Transactions on Quantum Engineering, 4: 1-20, 2023. ISSN 2689-1808. doi: 10.1109 / TQE.2023.3265709.

[0189]

[0043] Jiahao Yao, Haoya Li, Marin Bukov, Lin Lin, and Lexing Ying. Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits. In Proceedings of Mathematical and Scientific Machine Learning, pages 49-64. PMLR, September 2022.

[0190]

[0044] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, loannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.

[0191]

[0045] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017.

[0192]

[0046] Qiuhao Chen, Yuxuan Du, Qi Zhao, Yuling Jiao, Xiliang Lu, and XingyaoWu. Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning, April 2022.

[0193]

[0047] DeepMind, Igor Babuschkin, Kate Baumli, Alison Bell, Surya Bhupatiraju, Jake Bruce, Peter Buchlovsky, David Budden, Trevor Cai, Aidan Clark, Ivo Danihelka, Antoine Dedieu, Claudio Fantacci, Jonathan Godwin, Chris Jones, Ross Hemsley, Tom Hennigan, Matteo Hessel, Shaobo Hou, Steven Kapturowski, Thomas Keck, lurii Kemaev, Michael King, Markus Kunesch, Lena Martens, Hamza Merzic, Vladimir Mikulik, Tamara Norman, George Papamakarios, John Quan, Roman Ring, Francisco Ruiz, Alvaro Sanchez, Laurent Sartran, Rosalia Schneider, Eren Sezener, Stephen Spencer, Srivatsan Srinivasan, Milos Stanojevi'c, Wojciech Stokowiec, Luyu Wang, Guangyao Zhou, and Fabio Viola. The DeepMind JAX Ecosystem, 2020. URL http: / / github.com / deepmind.

[0194]

[0048] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016.

[0049] Tristan Cazenave. Batch monte carlo tree search. In Computers and Games: International Conference, CG 2022, Virtual Event, November 22-24, 2022, Revised Selected Papers, page 146-162, Berlin, Heidelberg, 2023. Springer- Verlag. ISBN 978-3-031-34016-1. doi: 10.1007 / 978-3-031 -34017-8_13. URL https: / / doi.Org / 10.1007 / 978-3-031-34017-8_13.

[0195]

[0050] Adriano Barenco, Charles H Bennett, Richard Cleve, David P DiVincenzo, Norman Margolus, Peter Shor, Tycho Sleator, John A Smolin, and Harald Weinfurter. Elementary gates for quantum computation. Physical review A, 52 (5):3457, 1995.

[0196]

[0051] Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. Julia: A fresh approach to numerical computing. SIAM review, 59(1):65-98, 2017. URL https: / / doi.org / 10.1137 / 141000671.

[0197]

[0052] Md Mazder Rahman, Gerhard W Dueck, and Joseph D Horton. An algorithm for quantum template matching. ACM Journal on Emerging Technologies in Computing Systems (JETC), 11 (3): 1-20, 2014.

[0198]

[0053] Raban Iten, Romain Moyard, Tony Metger, David Sutter, and Stefan Woerner. Exact and practical pattern matching for quantum circuit optimization. ACM Transactions on Quantum Computing, 3(1): 1-41, 2022.

[0199]

[0054] Guifre Vidal and Christopher M Dawson. Universal quantum circuit for two-qubit transformations with three controlled not gates. Physical Review A, 69(1):010301, 2004.

[0200]

[0055] Farrokh Vatan and Colin Williams. Optimal quantum circuits for general two-qubit gates. Physical Review A, 69(3): 032315, 2004a.

[0201]

[0056] Farrokh Vatan and Colin P Williams. Realization of a general three-qubit quantum gate. arXiv preprint quantph / 0401178, 2004b.

[0202]

[0057] Robert R Tucci. A rudimentary quantum compiler (2cnd ed.). arXiv preprint quant- ph / 9902062, 1999.

[0058] Mikko Mottonen, Juha J Vartiainen, Ville Bergholm, and Martti M Salomaa. Quantum circuits for general multiqubit gates. Physical review letters, 93(13): 130502, 2004.

[0203]

[0059] Vivek V Shende, Stephen S Bullock, and Igor L Markov. Synthesis of quantum logic circuits. In Proceedings of the 2005 Asia and South Pacific Design Automation Conference, pages 272-275, 2005.

[0204]

[0060] Matthew Amy, Dmitri Maslov, and Michele Mosca. Polynomial-time t-depth optimization of clifford+ t circuits via matroid partitioning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 33(10): 1476-1489, 2014.

[0205]

[0061] Marc G Davis, Ethan Smith, Ana Tudor, Koushik Sen, Irfan Siddiqi, and Costin lancu. Towards optimal topology aware quantum circuit synthesis. In 2020 IEEE International Conference on Quantum Computing and Engineering (QCE), pages 223-234. IEEE, 2020.

[0206]

[0062] Ed Younis, Koushik Sen, Katherine Yelick, and Costin lancu. Qfast: Conflating search and numerical optimization for scalable quantum circuit synthesis. In 2021 IEEE International Conference on Quantum Computing and Engineering (QCE), pages 232-243. IEEE, 2021.

[0207]

[0063] Ethan Smith, Marc Grau Davis, Jeffrey Larson, Ed Younis, Lindsay Bassman Oftelie, Wim Lavrijsen, and Costin lancu. Leap: Scaling numerical optimization based synthesis using an incremental approach. ACM Transactions on Quantum Computing, 4(1): 1-23, 2023.

[0208]

[0064] Saeedi, M., & Markov, I. L. (2013). Synthesis and optimization of reversible circuits — a survey. ACM Computing Surveys (CSUR), 45(2), 1-34.

[0209]

[0065] WO 2022 / 195494 A1 , A computer implemented method for real time quantum compiling based on artificial intelligence, published 2022.

[0210]

[0066] Aleks Kissinger and John van de Wetering. Reducing t-count with the zx-calculus. arXiv preprint arXiv: 1903.10477, 2019.

[0067] Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Remi Munos, Nicolas Heess, and Martin Riedmiller. Maximum a Posteriori Policy Optimisation, June 2018.

[0068] Qiskit contributors. Qiskit: An open-source framework for quantum computing,

[0211] 2023.

Claims

Claims1. An apparatus for determining a gate representation of a target unitary matrix, wherein the apparatus comprises: a selection unit (110) configured for conducting a selection step to select a selected operation out of a plurality of possible operations to append at least one quantum gate to an intermediate quantum circuit, and an appending unit (120) configured for conducting an appending step, which updates a gate representation of the intermediate quantum circuit, so that the at least one quantum gate is appended to the intermediate quantum circuit in the gate representation of the intermediate quantum circuit, wherein the apparatus is configured to repeat the selection step of the selection unit (110) and the appending step of the appending unit (120) until a unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

2. An apparatus according to claim 1, wherein the selection unit (110) is configured to select the selected operation out of the plurality of possible operations by selecting a quantum gate of a set of two or more quantum gates.

3. An apparatus according to claim 2, wherein the selection unit (110) is configured to select the quantum gate out of the set of two or more quantum gates which comprises at least two of the following gates: a Hadamard gate, a T gate, a Hermitian conjugate of T gate, a phase gate a Hermitian conjugate of a phase gate, a NOT gate, a CNOT gate.

4. An apparatus according to claim 2 or 3, wherein the selection unit (110) is configured to select the quantum gate out of a Clifford+T gate set.

5. An apparatus according to one of the preceding claims, wherein the apparatus is configured to start with a gate representation of an empty quantum circuit which does not comprise any quantum gates as the representation of the intermediate quantum circuit, and wherein, starting with the gate representation of the empty quantum circuit, the apparatus is configured to repeat the selection step of the selection unit (110) and the appending step of the appending unit (120) until the unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

6. An apparatus according to one of claims 1 to 4, wherein the apparatus is configured to start with a gate representation of a random quantum circuit which comprises one or more randomly or quasi-randomly chosen quantum gates as the representation of the intermediate quantum circuit, and wherein, starting with the gate representation of the random quantum circuit, the apparatus is configured to repeat the selection step of the selection unit (110) and the appending step of the appending unit (120) until the unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

7. An apparatus according to one of the preceding claims, wherein the apparatus is configured to receive the gate representation of the target quantum circuit being a gate representation of a problem dependent quantum circuit, wherein the apparatus is configured to conduct a pre-processing step in which the gate representation of the target quantum circuit is converted into a target unitary matrix.

8. An apparatus according to one of the preceding claims,wherein the apparatus is configured to repeat the selection step of the selection unit (110) and the appending step of the appending unit (120) until the unitary matrix of the representation of the intermediate quantum circuit is equal to the target unitary matrix, depending on a distance measure.

9. An apparatus according to claim 8, wherein the apparatus is configured to employ the distance measure by determining a distance between the unitary matrix of the intermediate quantum circuit and the target unitary matrix.

10. An apparatus according to claim 9, wherein the apparatus is configured to determine the distance using at least one of:Hilbert-Schmidt norm,Frobenius norm,Diamond norm, Spectral norm,- Average gate fidelity.

11. An apparatus according to claim 10, wherein the apparatus is configured to determine the distance depending onwherein U indicates the unitary matrix of the intermediate quantum circuit, wherein Vindicates the target unitary matrix, wherein d indicates a real number, whereinindicates a Hermitian conjugate of the target unitary matrix.

12. An apparatus according to one of claims 9 to 11 ,wherein the apparatus is configured to determine a reward depending on the distance.

13. An apparatus according to one of the preceding claims, wherein the apparatus is configured to determine a plurality of gate representations for a same target unitary matrix.

14. An apparatus according to claim 13, wherein the apparatus is configured to determine the plurality of gate representations for the same target unitary matrix in parallel.

15. An apparatus according to claim 14, wherein the apparatus is configured to determine the plurality of gate representations for the same target unitary matrix by conducting a plurality of parallel searches.

16. An apparatus according to one of claims 13 to 15, wherein the apparatus is configured to conduct a Monte Carlo tree search to determine the plurality of gate representations of a plurality of target unitary matrices.

17. An apparatus according to one of claims 13 to 16, wherein the apparatus is configured to determine the plurality of gate representations for the same target unitary matrix by applying a Gumbel AlphaZero algorithm.

18. An apparatus according to one of the preceding claims, wherein the apparatus is configured to train and / or to test a set of one or more neural networks for determining a gate representation of the target unitary matrix.

19. An apparatus according to claim 18,wherein the apparatus comprises the neural network.

20. An apparatus according to claim 18 and 19, further depending on claim 12, wherein the apparatus is configured to train the neural network depending on the reward.

21. A method for determining a gate representation of a target unitary matrix, wherein the method comprises: conducting a selection step to select a selected operation out of a plurality of possible operations to append at least one quantum gate to an intermediate quantum circuit, and conducting an appending step, which updates a gate representation of the intermediate quantum circuit, so that the at least one quantum gate is appended to the intermediate quantum circuit in the gate representation of the intermediate quantum circuit, wherein the method comprises repeating the selection step and the appending step until a unitary matrix of the intermediate quantum circuit is equal to the target unitary matrix.

22. A computer program for implementing the method of claim 21 when being executed on a computer or signal processor.

Images