A method, apparatus, device and medium for generating a QAOA quantum circuit

Abstract

The application discloses a QAOA quantum circuit generation method, device and equipment and a medium, and belongs to the technical field of quantum computing. The method comprises the following steps: determining, according to a mapping relationship between variables of a target optimization problem and quantum bits on a quantum chip, interaction terms between quantum bits corresponding to the variables and having no connection relationship as target terms; all the target terms form a target vector; an action is selected by using a trained first reinforcement learning model, the selected action is performed on a first matrix representing the target optimization problem, and a second matrix corresponding to a case where the number of element values of the target vector that are first numerical values reaches a preset value is obtained; one row of the first matrix corresponds to one variable of the target optimization problem, and the action is an operation of performing modular addition on quantum states representing two quantum bits generated based on a connection relationship between quantum bits on the quantum chip; an action of restoring the second matrix to the first matrix is obtained by using a trained second reinforcement learning model; and a QAOA quantum circuit is generated by applying a CNOT gate to quantum bits corresponding to the obtained action. By applying the embodiment of the application, the quantum resources occupied for implementing QAOA can be reduced.

Description

Technical Field

[0001] This application belongs to the field of quantum computing technology, and in particular to a method, apparatus, device and medium for generating QAOA quantum circuits. Background Technology

[0002] The Quantum Approximate Optimization Algorithm (QAOA) is a hybrid quantum-classical variational algorithm that combines parameterized quantum circuits with classical optimization methods to approximate solutions to combinatorial optimization problems using quantum computers. As a highly efficient quantum optimization algorithm, QAOA has already demonstrated enormous application potential in multiple fields. With continuous advancements in quantum hardware and algorithm optimization, QAOA is expected to solve even more complex real-world problems in the future.

[0003] Implementing QAOA requires executing corresponding quantum circuits on a quantum chip. However, due to limitations in the physical connections of qubits on the quantum chip, not all qubits can interact directly. Therefore, for edges that cannot be directly connected, a SWAP operation is inserted using a shortest path algorithm to reduce the total number of SWAPs. Each SWAP gate requires three CNOT gates (or native gate implementations in specific hardware), significantly increasing the gate count and depth. The increased gate count and depth in quantum circuits lead to higher complexity, requiring more quantum resources to execute these operations, resulting in higher quantum resource consumption. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, and medium for generating QAOA quantum circuits, aiming to reduce the quantum resources required to realize QAOA.

[0005] One embodiment of this application provides a QAOA quantum circuit generation method, the method comprising:

[0006] Based on the mapping relationship between the variables of the objective optimization problem and the qubits, the interaction terms that are not connected between the qubits corresponding to the determined variables are taken as objective terms, and all objective terms form an objective vector;

[0007] Using a trained first reinforcement learning model, an action is selected and executed on a first matrix representing the target optimization problem to obtain a second matrix when the number of elements of the target vector with the first value reaches a preset value; wherein, a row of the first matrix corresponds to a variable of the target optimization problem, and the action is a modular addition operation based on the connection relationship between qubits on the quantum chip, representing the quantum states of two qubits;

[0008] By using a well-trained second reinforcement learning model, the action of restoring the second matrix to the first matrix can be obtained;

[0009] A QAOA quantum circuit is generated by applying a CNOT gate to the qubit corresponding to the obtained action.

[0010] Optionally, the step of selecting actions using a trained first reinforcement learning model and executing the selected actions on a first matrix representing the target optimization problem to obtain a second matrix corresponding to the condition that the number of elements of the target vector with the first value reaches a preset value includes:

[0011] Based on the reward value and evaluation Q value corresponding to each action determined by the first training reinforcement learning model, an action is selected using a heuristic search algorithm. The selected action is executed on the first matrix representing the target optimization problem, and a second matrix is ​​obtained when the number of elements of the target vector with the first value reaches a preset value.

[0012] Optionally, the heuristic search algorithm is the A* search algorithm.

[0013] Optionally, the step of selecting actions based on the reward value and evaluation Q-value corresponding to each action determined by the trained first reinforcement learning model, using a heuristic search algorithm, and executing the selected actions on the first matrix representing the target optimization problem to obtain the second matrix corresponding to the number of elements of the target vector with the first value reaching a preset value, includes:

[0014] Using the trained first reinforcement learning model, determine the action corresponding to the maximum evaluation Q value and add the selected action to the OPEN table;

[0015] The A* search algorithm selects an action from the OPEN table using its evaluation function, transforms the first matrix, updates the target vector based on the transformation result, and adds the action to the CLOSE table. When the search cutoff condition is met, the latest matrix obtained based on the transformation of the first matrix is ​​used as the second matrix. The evaluation function of the A* search algorithm consists of the evaluation Q-value and the reward value corresponding to the action output by the trained first reinforcement learning model. The search cutoff condition is that the number of elements in the target vector with the first value reaches a preset value.

[0016] The step involves selecting an action from the OPEN table using the A* search algorithm's evaluation function, transforming the first matrix, updating the target vector based on the transformation result of the first matrix, and adding the action to the CLOSE table. When the search cutoff condition is met, the latest matrix obtained based on the transformation of the first matrix is ​​used as the second matrix, including:

[0017] The action with the largest function value in the OPEN table is used to transform the current matrix to obtain a new matrix. The target vector is then updated using the new matrix.

[0018] If the search cutoff criteria have not been met and the currently selected action is in the COLSE table, determine whether the reward value for performing the action is greater than the reward value for the action stored in the COLSE table.

[0019] If the value is greater than the target value, update the matrix and reward value corresponding to the action in the COLSE table, and update the OPEN table until the search cutoff condition is met. Then, the latest matrix is ​​obtained based on the transformation of the first matrix and used as the second matrix.

[0020] Optionally, the reward value corresponding to the action is calculated by dividing the number of the first value in the target vector by the length of the target vector.

[0021] Optionally, the method further includes:

[0022] Obtain the initial state and target state, action space, and maximum number of actions to restore the target state to the initial state for training the second reinforcement learning model for the target optimization problem;

[0023] Select several target actions from the action space to transform the target state. When the loss function converges, update the number of target actions and return to the previous step of selecting several target actions from the action space to transform the target state. Continue this process until the target state is transformed to obtain the initial state or the maximum number of actions is reached, and obtain the trained second reinforcement learning model. Here, the number of target actions is the number of actions required to restore the target state to the initial state.

[0024] Another embodiment of this application provides a QAOA quantum circuit generation device, the device comprising:

[0025] The objective term determination module is used to determine the interaction terms between the variables and qubits corresponding to the variables, based on the mapping relationship between the variables and qubits in the objective optimization problem. These interaction terms, which are not connected between the qubits, are used as objective terms. All objective terms form an objective vector.

[0026] The module is used to select an action using a trained first reinforcement learning model, execute the selected action on a first matrix representing the target optimization problem, and obtain a second matrix corresponding to the number of elements of the target vector with the first value reaching a preset value; wherein, a row of the first matrix corresponds to a variable of the target optimization problem, and the action is a modular addition operation based on the connection relationship between qubits on the quantum chip, representing the quantum states of two qubits;

[0027] The restoration module is used to obtain the action of restoring the first matrix to the second matrix through the trained second reinforcement learning model;

[0028] The circuit generation module is used to generate QAOA quantum circuits by applying CNOT gates to the qubits corresponding to the obtained actions.

[0029] Another embodiment of this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the QAOA quantum circuit generation method in any of the above embodiments.

[0030] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the QAOA quantum circuit generation method in any of the above embodiments.

[0031] In this application, based on the mapping relationship between variables and qubits in the target optimization problem, interaction terms between qubits without connections corresponding to the variables are identified as target terms. This approach can more accurately focus on the key parts of the optimization problem, avoiding the processing of unnecessary connections, improving the efficiency and specificity of problem solving, and allowing subsequent computational resources to be concentrated on the parts that directly affect the target. A first reinforcement learning model is used to learn the optimal operation strategy in a complex state space. Through continuous trial and error and learning, the best action sequence that makes the target vector satisfy preset conditions is found, thereby effectively exploring paths between qubits without physical connections and avoiding inefficient methods such as exhaustive search. A second reinforcement learning model is then used to explore paths, and the determined actions achieve the shortest possible path. Finally, CNOT gates are applied to the path to generate QAOA quantum circuits. Compared to using SWAP gates, this reduces the number of quantum gates and the circuit depth, thus effectively reducing the consumption of quantum resources. Attached Figure Description

[0032] Figure 1 Example system block diagram for the QAOA quantum circuit generation method provided in the embodiments of this application;

[0033] Figure 2 This application provides a schematic diagram of the Cost layer structure corresponding to the QAOA algorithm in an embodiment of the present application.

[0034] Figure 3 A flowchart of a QAOA quantum circuit generation method provided in this application embodiment;

[0035] Figure 4 A structural diagram of a QAOA quantum circuit generation device provided in an embodiment of this application;

[0036] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0037] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0039] Classical computers use transistors to encode information in binary data, such as bits, where each bit can represent a value of 1 or 0. These 1s and 0s act as switches to drive the functions of a classical computer. If there are n bits of data, there are 2^n possible classical states, and one state is represented at a time.

[0040] Quantum computers use quantum processors that operate on data represented by qubits, also known as quantum bits. A qubit can represent the classical binary states "0" or "1", or a superposition of "0" and "1". Because it can represent a superposition of "0" and "1", a qubit can represent both "0" and "1" states simultaneously. For example, if there are n bits of data, n quantum states can be represented simultaneously. Furthermore, qubits in a superposition can be correlated with each other, a phenomenon known as entanglement. The state of one qubit (whether it's 1, 0, or both) can depend on the state of another qubit, and more information can be encoded within two entangled qubits. Based on the principles of superposition and entanglement, qubits enable quantum computers to perform functions that might be relatively complex and time-consuming for classical computers.

[0041] Please refer to Figure 1 This illustrates an example system block diagram of a QAOA quantum circuit generation method according to an embodiment of this application. System 100 may be a hybrid computing system comprising a combination of one or more quantum computers, quantum systems, and / or classical computers. Figure 1In the example shown, system 100 may include a quantum system 110 and a classical computer 120. In one implementation, the quantum system 110 and the classical computer 120 may be configured to communicate via one or more wired and / or wireless connections (e.g., wireless networks). The quantum system 110 may include a quantum chipset consisting of one or more quantum chips, comprising various hardware components for processing data encoded in qubits. The quantum chipset may be a quantum computing core surrounded by infrastructure to protect the quantum chips from electromagnetic noise sources, mechanical vibration sources, heat sources, and other noise sources that can degrade the performance of the quantum chips. The classical computer 120 may be electronically integrated with the quantum system 110 via any suitable wired and / or wireless electronic connection.

[0042] exist Figure 1 In the example shown, quantum system 110 can be any suitable set of components capable of performing quantum operations on a physical system. Quantum operations, for example, can be quantum logic gate operations that manipulate the quantum states of qubits to evolve and / or become entangled. Figure 1 In the illustrated example embodiment, the quantum system 110 may include a measurement and control unit 111, an interface 112, and a quantum chip 113. In some embodiments, all or part of each of the measurement and control unit 111, interface 112, and quantum chip 113 may be located in a cryogenic environment to facilitate the performance of quantum operations. The quantum chip 113 may be any hardware capable of processing information using quantum states. This hardware may include multiple qubits and means for coupling or entanglement of the qubits to process information using quantum states. Qubits may include, but are not limited to, charge qubits, flux qubits, phase qubits, spin qubits, and ion qubits. The quantum chip may include a set of quantum logic gates configured to perform quantum logic operations on the qubits stored in a quantum register. The quantum gates may include one or more single-qubit gates, two-qubit gates, and / or other multi-qubit gates.

[0043] The measurement and control unit 111 can be any combination of digital computing devices capable of performing quantum computing (e.g., executing quantum circuits) in conjunction with interface 112. This digital computing device may include a digital processor and memory for storing and executing quantum instructions using interface 112. The digital computing device may also include a communication protocol device for receiving instructions and sending the results of the performed quantum computing to a classical computer. Additionally, the digital computing device may include a communication interface having interface 112. In one embodiment, the measurement and control unit 111 may be configured to receive classical instructions (e.g., from classical computer 120) and convert these classical instructions into measurement and control instructions for interface 112. The measurement and control instructions provided by the measurement and control unit 111 to interface 112 may be, for example, digital signals indicating which quantum gates in a quantum gate array need to be applied to the qubits to perform a specific function. Interface 112 may be configured to convert these digital signals into analog signals (e.g., analog pulses of microwave pulses), which can be used to apply quantum gates to the qubits to manipulate the interactions between the qubits.

[0044] Interface 112 may be a classical-quantum interface, comprising a combination of devices capable of receiving instructions from the integrated measurement and control unit 111 and converting those instructions into a means for implementing quantum operations. In one embodiment, interface 112 may convert instructions from the integrated measurement and control unit 111 into drive signals capable of driving or manipulating qubits, and / or applying quantum gates to qubits. Additionally, interface 112 may be configured to convert signals received from the quantum chip 113 into digital signals capable of being processed and transmitted by the integrated measurement and control unit 111. Devices included in interface 112 may include, but are not limited to, digital-to-analog converters, analog-to-digital converters, waveform generators, attenuators, amplifiers, optical fibers, lasers, and filters. Interface 112 may further include circuitry configured to measure multiple qubits after the application of quantum gates, wherein the measurements may produce results represented in classical bits. Each measurement performed by interface 112 may be read out to a device connected to the quantum system 110, such as a classical computer 120. The multiple measurement results provided by interface 112 may represent probabilistic results.

[0045] The classical computer 120 can include hardware components such as a processor and storage devices (e.g., including memory devices and classical registers) for processing data encoded in classical bits. In one embodiment, the classical computer 120 can be configured to provide the quantum system 110 with various control signals, instructions, and data encoded in classical bits. Further, quantum states measured by the quantum system 110 can be read out by the classical computer 120, and the classical computer 120 can store the measured quantum states as classical bits in classical registers. In one embodiment, the classical computer 120 can be any suitable combination of computer-executable hardware and / or computer-executable software capable of executing the preparation module 121 to perform quantum computation using data stored in the data storage module 122 as part of the construction and computation. The data storage module 122 can be a repository for data to be analyzed using quantum computing algorithms and the results of that analysis. The preparation module 121 can be a program or module capable of preparing classical data from the data storage module 122 as part of a quantum circuit implementation. Preparation module 121 can be instantiated as part of a larger algorithm, such as an application programming interface (API) function call, or by resolving hybrid classical-quantum computing into aspects of quantum and classical computing. For example, preparation module 121 can generate instructions for creating quantum circuits using quantum gates. In an embodiment, such instructions can be stored by the measurement and control unit 111 and can be instantiated by components of interface 112 to execute, enabling quantum operations of quantum gates to be performed on quantum chip 113.

[0046] The classic computer 120 may be a laptop computer, desktop computer, vehicle-integrated computer, smart mobile device, tablet device, and / or any other suitable classic computing device. Additionally or alternatively, the classic computer 120 may also operate as part of a cloud computing service model, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). The classic computer 120 may also reside in a cloud computing deployment model, such as a private cloud, community cloud, public cloud, or hybrid cloud.

[0047] The following section will first explain how to use the QAOA algorithm to solve optimization problems and the key terms involved in reinforcement learning.

[0048] In quantum mechanics, all measurable mechanical quantities can be described by a Hermitian matrix. A Hermitian matrix is ​​defined as the matrix itself, where its transpose and conjugate are equal.

[0049]

[0050] Such matrices are usually called measurement operators. Non-zero operators will have at least one non-zero eigenvalue λ and a corresponding eigenstate |ψ>.

[0051] H|ψ>=λ|ψ>

[0052] If the eigenvalues ​​of the operator H correspond to the energy level distribution of a certain system, then such an operator can also be called a Hamiltonian.

[0053] According to the time-dependent Schrödinger equation, the evolution from one state |ψ(t=0)> to another state |ψ(t=T)> is accomplished using unitary operators:

[0054] U(0,T)|ψ(t=0)>=|ψ(t=T)>

[0055] The relationship between the Hamiltonian and the unitary operator is that if a quantum state evolves naturally in a system, and the energy describing the system is the Hamiltonian, then the unitary operator can be written from the Hamiltonian:

[0056]

[0057] When the system starts from time 0 and the Hamiltonian does not change with time, the unitary operator is U = exp(-iHt).

[0058] In quantum computing within a closed system, all quantum operations, except for measurement, can be described by a unitary matrix. A unitary matrix is ​​defined as the inverse of its transpose and conjugate. That is:

[0059]

[0060] Generally speaking, unitary operators are also referred to as quantum logic gates in quantum computing.

[0061] In some cases, we need to start from the initial Hamiltonian H B ground state |ψ B Starting from this point, we search for the lowest energy level eigenstate (ground state) |ψ of a certain target Hamiltonian H. P However, by directly using this Hamiltonian for evolution, we can only obtain...

[0062] exp(-itH)|ψ B >=|ψ′ P >

[0063] Where, |ψ′ P >≠|ψ P >, but |ψ′ P >Contains a part of |ψ P The components of >, that is, their inner product <ψ′ P |ψ P >≠0, but this has a poor effect because if |ψ B> corresponds to the Hamiltonian H B (initial Hamiltonian), then |ψ B > cannot evolve to |ψ in one step P >, therefore, a part of H can be mixed into H first, and the proportion is gradually increased over time. The composition of the two parts changes with time, which gives the following: B

[0064]

[0065] where t changes starting from 0, t0 approaches 0, and t i approaches T. As long as the total time T is sufficiently large and the number of t values is sufficiently large, the component of |ψ in |ψ′ P > B > will always decrease slowly, further making <ψ′ P |ψ P >→1. Such a process is similar to slowly "annealing" from one state to another, so this algorithm is called a quantum annealing algorithm. Around this type of algorithm, an important class of quantum computers that use physical fields to simulate the annealing process has emerged — quantum annealers. Quantum annealers can easily produce quantum chips with a huge number of qubits, but quantum annealers only perform relatively well in a specific class of problems, and their application scenarios are difficult to expand, which led to the later more general-purpose quantum computers — gate-based quantum computers. However, in a quantum circuit composed of quantum logic gates, directly preparing a corresponding quantum logic gate is relatively difficult, so a special decomposition is often required for simplification — Trotter decomposition:

[0066] exp(-i(A+B))|ψ>≈exp(-iA)exp(-iB)|ψ>

[0067] The above gives the form of the first-order Trotter decomposition. If A and B commute, the approximate equal sign in the above formula becomes an equal sign, that is, exp(-i(A+B))|ψ>=exp(-iA)exp(-iB)|ψ>. If higher precision is desired, there are other more Trotter decomposition forms such as second-order and third-order. Thus the original:

[0068]

[0069] Thus:

[0070]

[0071] In the evolution of quantum annealing, the evolution path, that is, t0…t i …t T ​The settings are important; whether they are set correctly will directly affect the final |ψ′. P The same effect can be achieved by adjusting the hyperparameters of QAOA.

[0072] Combinatorial optimization problems are a class of problems that seek optimal solutions in a finite or countably infinite solution space. These problems typically involve selecting the best combination of a set of discrete variables to maximize or minimize a certain objective function. When solving these problems using QAOA, the first step is to transform the combinatorial optimization problem into a quantum-processable form, usually by mapping the objective or cost function to a Hamiltonian H. The specific mapping method may vary depending on the problem and the function, but it mainly involves replacing the variables in the function with corresponding Pauli operators. Taking the QAOA algorithm applied to molecular crystal structure prediction as an example, the objective function is:

[0073]

[0074] in, Describe the objective function. Let i and j represent the molecule numbers, e represent the set of all interacting molecules, and a and b represent the orientations of the corresponding molecules. E represents the set of all interacting molecules. i,j,a,b ν represents the dipole energy when molecule i is at position a and molecule j is at position b, ν represents the orientation number of the molecular crystal to be predicted, and k represents the total number of orientations of the molecular crystal to be predicted.

[0075] The Hamiltonian H corresponding to the prediction of molecular crystal structure C It can be represented as:

[0076]

[0077] Where I represents the identity matrix, n represents the total number of molecules in the molecular crystal to be predicted, and Z represents the Pauli Z operator.

[0078] The core of QAOA is the design of a parameterized quantum circuit, which consists of two unitary operators with adjustable parameters, alternating between two Hamiltonians. Typically, these two alternating unitary operators are... and The components are the Cost layer and the Mixer layer, where H... C For the objective optimization problem, H represents the Hamiltonian. B As a mixed Hamiltonian, γ is responsible for performing random walks in the solution space. i and β iLet be the variational parameters, and ...

[0079] In quantum circuit construction, constructing the Mixer layer is a natural step, typically achieving the desired function using RX gates. Therefore, the Cost layer is the primary source of depth for QAOA circuits. The Cost layer is constructed by using CNOT gates to accumulate data, and then constructing the final bit. That is, an RZ gate is applied to the last bit to construct the phase, and finally a CNOT gate is used to restore the quantum state back to the original state. For example, H C =Z1Z2+Z2Z3+Z3Z4+Z4Z1, the corresponding Cost layer is as follows Figure 2 As shown, the depth of the circuit is related to the number of interaction terms. An interaction term is a subterm consisting of two Pauli Z operators. If the qubits corresponding to the interaction terms are not directly connected, a SWAP gate is needed to establish the connection. The SWAP gate can also be constructed using three CNOT gates. Therefore, reducing the number of CNOT gates can save logic gate hardware resources.

[0080] Reinforcement learning is an important field in machine learning, primarily studying how an agent takes a series of actions in an environment to maximize cumulative rewards. This application may refer to a reinforcement learning model, specifically DQN (Deep Q-Network). DQN is an algorithm that combines deep learning with reinforcement learning, aiming to solve the challenges faced by traditional reinforcement learning algorithms in handling high-dimensional state and action spaces.

[0081] DQN is based on a mechanism called "Q-Learning". For any state, a Q-value is used to determine whether the reward obtained after taking an action is greater or less, and the action with greater reward is prioritized. To obtain an accurate Q-value table, a large amount of "learning" is required. This process is carried out using the Bellman equation and a γ value is used to penalize longer results. In practice, because there are too many Q-values ​​(each state + each action has its own Q-value), it is impossible to store the Q-values ​​in a table. Therefore, two neural networks (target network and evaluation network) are used to approximate the Q-values. In each step, starting from the current state s, s is input into the target network, and the target network's value is calculated. This value represents the target network's suggestion for the next action. Using a certain strategy (e.g., taking the maximum value, or random selection with a certain probability), an action is selected, and the next state s_ and reward r after this action are obtained. These results (s, a, r, s_) are stored in memory, and the loss function is calculated.

[0082]

[0083] In the above formula, q eval (s) a It is the evaluation Q-value, q, of "performing action a in state s" using an evaluation network. next(s-)a It uses the target network to evaluate the Q-value obtained through action 'a' plus the Q-value corresponding to state 's', where γ is a pre-set penalty weight, and the loss function is used to determine whether the evaluated Q-value is consistent with the target network. If the values ​​are close, the Q-value table can be considered stable, and at each step, the action that maximizes the total Q-value can be chosen.

[0084] When training DQN, an experience replay reinforcement learning strategy can be employed. Experience replay prioritizes specific "experiences" during each training iteration. In the scenario defined above, only achieving a perfect result in the final step is considered a 1, while all others are considered 0. Most experiences are unimportant; only successful experiences matter. By prioritizing successful cases for learning, reinforcement learning can acquire more useful knowledge. This is achieved by using a method called SumTree to store memories during training and employing Q-reality-Q estimation as the priority level for learning.

[0085] See Figure 3 , Figure 3 A QAOA quantum circuit generation method provided in this application includes the following steps:

[0086] S301: Based on the mapping relationship between the variables of the objective optimization problem and the qubits, the interaction terms between the qubits corresponding to the determined variables that have no connection relationship are taken as objective terms, and all objective terms form an objective vector.

[0087] In objective optimization problems, variables are the decision factors, specifically unknowns or adjustable parameters in the decision-making process. Their values ​​need to be determined through optimization to achieve a certain objective or satisfy certain constraints. Variables can be continuous or discrete, depending on the nature of the problem. When mapping the optimization problem onto a quantum chip, each variable needs to correspond to a qubit on the quantum chip. For example, if the objective optimization problem has variables x1, x2, and x3, these variables will be mapped to qubits q1, q2, and q3 on the quantum chip. This mapping relationship is the first step in solving the problem because it determines how the variables of the problem are represented on the quantum hardware. The number of variables in the objective optimization problem and the meaning that the variables represent are determined by the objective optimization problem itself.

[0088] There are many ways to determine the mapping relationship between variables and qubits. Specifically, random mapping can be used, or the optimal solution can be obtained using a bit mapping algorithm based on the graph corresponding to the objective optimization problem and the bit connection topology of the quantum chip. In this embodiment, a one-to-one correspondence is established between the vertices of the objective optimization problem and the vertices of the bit connection topology, with the goal of maximizing the overlap between the edges of the graph corresponding to the objective optimization problem and the bit connections. Mathematically, this means constructing a mapping f: V1→V2 between the graph G(V1,E1) corresponding to the objective optimization problem and the bit connection topology T(V2,E2) such that the mapped {(f(i),f(j))|(i,j)∈E1,i,j∈V1}∪E2 is maximized. This operation reduces the need for additional quantum gates to implement the QAOA algorithm. To achieve this, the SABRE (Swap-based Automatic BReakeven) algorithm or heuristic search algorithms can be used, etc. The SABRE algorithm is a mapping algorithm for compiling quantum computing circuits. It primarily solves the problem of qubit mapping, that is, mapping logical bits in a quantum circuit to qubits on an actual quantum chip. The goal of the SABRE algorithm is to reduce the depth of quantum circuits, thereby reducing the time required to execute the quantum circuits and the error rate.

[0089] There is a correspondence between variables, logical bits, and physical bits. In existing methods, the mapping relationship between variables and logical bits is first determined. Based on this mapping relationship, a logic circuit is constructed using Hamiltonians. Then, through the mapping relationship between logical bits and physical bits, a quantum circuit that can be directly executed on a quantum chip is obtained based on the logic circuit. In this application, it is not necessary to establish a mapping relationship between variables and logical bits and then generate a logic circuit. Instead, the mapping relationship is directly determined using the relationship between variables and physical bits, eliminating the need to generate a logic circuit.

[0090] The objective optimization problem can have action terms consisting of two or more variables. These action terms can be processed into combinations of interaction terms, specifically by transforming them into multiple multiplicative interaction terms.

[0091] Quantum bits on a quantum chip interact through physical connections (such as couplers). However, the physical structure of the quantum chip may restrict direct connections between some qubits. Therefore, it is necessary to identify the interaction terms between qubits corresponding to variables that are not directly connected on the quantum chip; these identified interaction terms are called target interaction terms.

[0092] S302: Using the trained first reinforcement learning model to select an action, execute the selected action on the first matrix representing the target optimization problem, and obtain the second matrix corresponding to the number of elements of the target vector with the first value reaching a preset value; wherein, one row of the first matrix corresponds to a variable of the target optimization problem, and the action is a modular addition operation based on the connection relationship between qubits on the quantum chip, representing the quantum state of two qubits.

[0093] Each row of the first matrix represents a variable in the objective optimization problem. Because there is a one-to-one mapping between variables and qubits, each row of the first matrix also represents a qubit. The elements and structure of the first matrix reflect relevant information about the objective optimization problem at a certain moment or under certain conditions. Specifically, the first matrix is ​​derived from the objective optimization problem and can represent the initial state of solving the objective optimization problem using the QAOA algorithm; it can be an identity matrix. Actions are generated based on the connection relationships between qubits on the quantum chip. An action simply involves the modular addition of the quantum states of two qubits, i.e., performing modular addition on the corresponding row of the first matrix.

[0094] The target vector is a key indicator for measuring the achievement of the target. Changes in its element values ​​reflect the progress of matrix transformation, i.e., the progress of path exploration and the traversal of the target quantum state corresponding to the target item. Each element in the target vector corresponds one-to-one with a target item. When the corresponding action is performed on the first matrix, the elements in the first matrix change. Performing an action on the first matrix explores the path between the two qubits corresponding to the target item. If, after performing the selected action, the first matrix shows the result of the modulo addition of the two variables corresponding to the target item (i.e., the value corresponding to the target item appears), it indicates that the sequence of actions can establish a connection between the qubits corresponding to the target item. Initially, the target vector can be composed of a second value. The first and second values ​​in the target vector represent whether the value corresponding to the target item appears, i.e., whether the corresponding target quantum state appears. When an action is performed and the value corresponding to the target item appears, the element value at the corresponding position in the target vector is changed from the second value to the first value. That is, when the value corresponding to the target item is obtained during the transformation from the first matrix to the second matrix, the corresponding position in the target vector changes from the second value to the first value.

[0095] For reinforcement learning, the initial state includes a first matrix and a target vector containing a preset number of first values. When the selected action is applied to the current state, the state changes and the first matrix is ​​transformed into a new matrix. If there is a row in the new matrix that corresponds to the value of the target item, then the second value at the corresponding position of the target vector is changed to the first value.

[0096] For example, if the target terms are x0⊕x1 and x0⊕x2, then the first matrix is ​​the identity matrix, and the action is [0,1], which means performing a modulo addition on the first and second rows. For a quantum chip, this action means using the qubit mapped to x0 as the control bit and the qubit mapped to x1 as the target bit, applying a CNOT gate, and the resulting new matrix can be represented as [[1,0,0],[1,1,0],[0,0,1]]. Because the new matrix has the row [1,1,0], which corresponds to the value of the target term x0⊕x1, the target vector can change from [0,0] to [1,0]. When the target vector has the first value, it means that the target quantum state has been obtained. When the target vector has a preset number of first values, it means that all the desired target states have been traversed. That is, after a series of operations, all the target quantum states have appeared in this process, but the initial state has not been returned. The matrix obtained at this time is the second matrix. The goal of obtaining the second matrix using the first reinforcement learning model is to minimize the number of operations in the entire process.

[0097] In this application, a pre-trained first reinforcement learning model is used to obtain the evaluation result of each action in the action space for each matrix. The evaluation result may include reward value, evaluation Q value, etc. An action is selected based on the evaluation result of each action, and the selected action is executed on the first matrix. At this point, the first matrix becomes a new matrix. It is determined whether the value corresponding to the target item appears in the new matrix. If it does, the target vector is updated. Then, an action is selected for the new matrix, and the above process is repeated until the number of elements in the target vector that have the first value reaches a preset value. The preset value is a threshold number of the first value that the target vector is expected to reach.

[0098] S303: Obtain the action of restoring the second matrix to the first matrix through the trained second reinforcement learning model.

[0099] Similarly, this is a trained reinforcement learning model whose task is to learn how to restore the second matrix to the first matrix. Here, "restoration" actually means finding a series of appropriate actions that allow the matrix to transition from the state of the second matrix to the state of the first matrix.

[0100] By combining the first and second reinforcement learning models, the optimal gate sequence combination strategy is automatically discovered, avoiding the subjective bias of manually designed heuristic rules and demonstrating stronger generalization ability in NP-hard combinatorial optimization problems.

[0101] S304: Generate a QAOA quantum circuit by applying a CNOT gate to the qubit corresponding to the obtained action.

[0102] The essence of a CNOT gate is to implement a modular addition operation, similar to the essence of an action. By applying a CNOT gate to the qubit corresponding to the obtained action, a QAOA quantum circuit for solving objective optimization problems can be constructed without additional operations or the addition of other quantum gates. This quantum circuit can leverage the advantages of quantum computing to find approximate optimal solutions to objective optimization problems.

[0103] In this application, based on the mapping relationship between variables and qubits in the target optimization problem, the action terms that are not connected between the corresponding qubits are determined as the target terms. This approach can more accurately focus on the key parts of the optimization problem, avoid processing unnecessary connections, improve the efficiency and specificity of problem solving, and allow subsequent computational resources to be concentrated on the parts that directly affect the target. A first reinforcement learning model is used to learn the optimal operation strategy in a complex state space. Through continuous trial and error and learning, the best action sequence that makes the target vector satisfy preset conditions is found, thereby effectively exploring paths between unconnected qubits and avoiding inefficient methods such as exhaustive search. A second reinforcement learning model is then used to explore paths, and the determined actions achieve the shortest possible path. Finally, CNOT gates are applied to the path to generate QAOA quantum circuits. Compared with using SWAP gates, this reduces the number of quantum gates and the circuit depth, thus effectively reducing the consumption of quantum resources.

[0104] In this application, the action space design is directly mapped to the quantum chip connection topology, and the generated CNOT gate sequence naturally conforms to hardware coupling constraints, avoiding the complex mapping and transformation steps in the quantum compilation process. Numerical monitoring of the target vector ensures that key constraints are satisfied, matrix transformations maintain the mathematical equivalence of the problem semantics, and bidirectional reinforcement learning guarantees the physical realizability of the operation sequence. Based on these operations, solution quality is guaranteed. The scheme in this application uses a reinforcement learning model to guide the construction process of the quantum circuit. Combined with consideration of the quantum bit connection relationships and the rational use of CNOT gates, it can automatically optimize and simplify the construction of the quantum circuit. This helps reduce the number of quantum gates and the circuit depth in the quantum circuit, thereby reducing error accumulation in the quantum computing process, improving the stability and reliability of quantum computing, and making the constructed QAOA quantum circuit more practically valuable. Therefore, the scheme in this application can be used to handle large-scale combinatorial optimization problems with sparse inter-variable connections. In the era of NISQ with limited quantum resources, by establishing the shortest mapping channel from problem characteristics to quantum resources to compress the quantum circuit depth, it provides a new technical path for demonstrating practical quantum advantage.

[0105] In some embodiments of this application, the step of selecting an action using a trained first reinforcement learning model, performing the selected action on a first matrix representing the target optimization problem, and obtaining a second matrix corresponding to the condition that the number of elements of the target vector with the first value reaches a preset value includes:

[0106] Based on the reward value and evaluation Q value corresponding to each action determined by the first training reinforcement learning model, a heuristic search algorithm is used to perform corresponding operations on the first matrix representing the target optimization problem to obtain the second matrix corresponding to the number of elements of the target vector with the first value reaching a preset value.

[0107] Heuristic search algorithms are algorithms that use heuristic information (i.e., prior knowledge or experience about the problem) to guide the search direction during the search process. Unlike blind search, which searches all possible paths comprehensively, heuristic search algorithms explore the search space in a targeted manner based on some prior knowledge or experience, thereby improving search efficiency and finding the target solution faster.

[0108] The first reinforcement learning model determines the reward value and evaluation Q-value for each possible action based on the current state. The Q-value represents the expected cumulative reward obtained after performing an action in a specific state, and is a crucial indicator for evaluating the quality of actions in reinforcement learning. The heuristic search algorithm uses the reward value and Q-value provided by the reinforcement learning model as components of a heuristic function to guide the search direction. In each search step, the algorithm evaluates various possible subsequent states based on the current state and available actions, combined with the heuristic function, and selects the most promising action path for expansion. The second matrix is ​​the result obtained by performing a series of actions selected by the heuristic search algorithm on the first matrix.

[0109] By leveraging the efficient search capabilities of heuristic search algorithms and combining them with the decision-making abilities of a trained reinforcement learning model, the first matrix is ​​manipulated, continuously changing its state until the number of elements in the target vector that are the first numerical values ​​reaches a preset value, thus obtaining the second matrix. This second matrix is ​​the result of the combined action of heuristic search and the reinforcement learning model; it represents the state of the optimization problem under certain conditions.

[0110] In some embodiments of this application, the heuristic search algorithm is the A* search algorithm. The A* search algorithm combines the systematic search of Dijkstra's algorithm with the advantages of heuristic search, reducing the search space by using a heuristic function while ensuring the shortest path is found. The rewards and evaluation Q-values ​​of the first reinforcement learning model essentially express the "distance" of each operation from reaching the goal; the shortest path is found by combining reinforcement learning and path search algorithms.

[0111] In some possible implementations of this application, the step of selecting actions based on the reward value and evaluation Q-value corresponding to each action determined by the trained first reinforcement learning model, executing the selected actions on the first matrix representing the target optimization problem, and obtaining the second matrix corresponding to the number of target vector elements with the first value reaching a preset value includes:

[0112] Using the trained first reinforcement learning model, determine the action corresponding to the maximum evaluation Q value and add the selected action to the OPEN table;

[0113] The A* search algorithm selects an action from the OPEN table using its evaluation function, transforms the first matrix, updates the target vector based on the transformation result, and adds the action to the CLOSE table. When the search cutoff condition is met, the latest matrix obtained based on the transformation of the first matrix is ​​used as the second matrix. The evaluation function of the A* search algorithm consists of the evaluation Q-value and the reward value corresponding to the action output by the trained first reinforcement learning model. The search cutoff condition is that the number of elements in the target vector with the first value reaches a preset value.

[0114] The first reinforcement learning model evaluates a Q-value for each possible action. The Q-value represents an estimate of the long-term cumulative reward obtained by taking a certain action in the current state. The evaluated Q-value is the Q-value obtained by the evaluation network mentioned above. From all possible actions, the action with the largest evaluated Q-value is selected. This is because in reinforcement learning, the action with the largest Q-value is generally considered the optimal action choice for the current state.

[0115] The OPEN table is a set used in the A* search algorithm to store nodes to be expanded. The algorithm then selects appropriate actions from this table for processing. The A* search algorithm uses an evaluation function to assess the priority of each action. The Q-value in the evaluation function reflects the action's potential for long-term cumulative reward, while the reward value is the immediate feedback obtained after executing the action. By considering both factors, the merits of each action can be more accurately determined. During execution, the actions in the OPEN table are sorted according to the evaluation function, and then the action with the smallest (or largest, depending on the function's definition) evaluation function value is selected. This action is considered the most promising to reach the goal. Based on the actions selected from the OPEN table, the first matrix is ​​transformed accordingly, and the actions are added to the CLOSE table.

[0116] The A* search algorithm uses heuristic information to guide the search direction, selecting the next action based on estimates of the current state and the target state, and can quickly find the optimal path from the starting state to the target state. Reinforcement learning, on the other hand, learns the optimal policy through continuous trial and error by interacting with the environment. Combining the two allows the agent to consider both the current optimal estimate and the heuristic information from the A* search algorithm, which provides a better exploration direction for reinforcement learning, reducing the search space and accelerating the learning speed. Furthermore, reinforcement learning continuously optimizes the policy, improving the quality and accuracy of decisions, enabling the agent to make better decisions in different environments and tasks. In addition, reinforcement learning has a certain degree of dynamic environmental adaptability, while the A* search algorithm can quickly replan the path when facing environmental changes. Combining the two, when the environment changes, reinforcement learning can adjust its strategy based on the new environmental feedback, while the A* search algorithm can recalculate the optimal path using the updated information, allowing the agent to better adapt to dynamically changing environments and maintain good performance. In some embodiments of this application, the reward value corresponding to the action is calculated by dividing the number of first numerical elements in the target vector by the length of the target vector.

[0117] In this application, the relevant information of the target vector is used as the basis for calculating the reward value. Specifically, the reward value corresponding to an action is calculated by dividing the number of first values ​​in the target vector by the length of the target vector. The target vector is a vector with a certain length (i.e., containing a certain number of elements), some of which are first values. By counting the number of first values ​​in the target vector and then dividing this number by the total length of the target vector (i.e., the total number of elements), the result is determined as the reward value corresponding to the action. For example, if the target vector obtained after performing an action is [1,0,1], the corresponding reward value is 2 / 3 = 0.67.

[0118] In some embodiments of this application, the step of selecting an action in the OPEN table using the evaluation function of the A* search algorithm, transforming the first matrix, updating the target vector and adding the action to the CLOSE table based on the transformation result of the first matrix, and when the search cutoff condition is met, using the latest matrix obtained based on the transformation of the first matrix as the second matrix includes:

[0119] The action with the largest function value in the OPEN table is used to transform the current matrix to obtain a new matrix. The target vector is then updated using the new matrix.

[0120] If the search cutoff criteria have not been met and the currently selected action is in the COLSE table, determine whether the reward value for performing the action is greater than the reward value for the action stored in the COLSE table.

[0121] If the value is greater than the target value, update the matrix and reward value corresponding to the action in the COLSE table, and update the OPEN table until the search cutoff condition is met. Then, the latest matrix is ​​obtained based on the transformation of the first matrix and used as the second matrix.

[0122] The CLOSE table stores actions, the corresponding matrix for each action, and the corresponding reward value for each action. The matrix corresponding to an action is a new matrix obtained by performing the corresponding action on the current matrix. The current matrix is ​​the first matrix during the first search; otherwise, it is a matrix obtained by transforming the first matrix using the selected action.

[0123] The A* search algorithm is used to operate on the OPEN table. The OPEN table stores some actions. Specifically, the action with the largest evaluation function value in the OPEN table is selected, and then this action is used to transform the current matrix to obtain a new matrix. Here, the evaluation function value is an evaluation index of the action after comprehensively considering multiple factors; the larger the value, the better the action is in the current situation.

[0124] If the reward value for the currently executed action is greater than the reward value stored in the CLOSE table for that action, then the matrix corresponding to that action in the CLOSE table needs to be updated (updated to the new matrix obtained after the current action is executed) and the reward value needs to be updated (updated to the larger reward value calculated now). Simultaneously, the OPEN table also needs to be updated, possibly including recalculating the evaluation function values ​​of actions in the OPEN table to reflect the latest situation. This process continues until the search cutoff condition is reached. When the search cutoff condition is reached, the latest matrix obtained after a series of transformations based on the first matrix is ​​determined as the second matrix. The second matrix represents a state of the objective optimization problem under specific conditions. After updating the OPEN table, the action with the largest reward is selected from the updated OPEN table as the chosen action. The reward and evaluation Q-value for each action are obtained using the second reinforcement learning model.

[0125] In this embodiment, when obtaining the second matrix using the A* search algorithm and the first reinforcement learning model, an OPEN table and a CLOSE table can be constructed first, defining the node structure as (s, a, r). The first reinforcement learning model is used to select the action 'a' that maximizes the evaluation Q-value, along with the corresponding reward. (s, a, r) ​​is input into the OPEN table. As long as the OPEN table is not empty, the following steps are performed:

[0126] Extract the node(s,a,r) with the largest value r from OPEN, execute a on the current s, and obtain s',r';

[0127] If the executed s' satisfies the search termination condition, add (s', a, r') to the CLOSE table, and return the CLOSE table, where s' is the second matrix at this time.

[0128] If the search termination condition is not satisfied, calculate r = r + r'. If s' is not in the CLOSE table, or r_c of CLOSE[s'] < r, where r_c represents the corresponding reward value obtained when s' is obtained in the CLOSE table, then add (s', a, r) to the CLOSE table. Calculate f(a') = Qeval(s'), and store (s', a', r + f(a')) in the OPEN table for all actions a'.

[0129] In the above process, for the currently selected action of s, if the obtained s', that is, the new matrix, has a value corresponding to the target item, the position corresponding to the target vector is modified from the second value to the first value.

[0130] In some implementations of the present application, the method for training the second reinforcement learning model is as follows:

[0131] Obtain the initial state and target state for training the second reinforcement learning model for the target optimization problem, the action space, and the maximum number of actions to restore the target state to the initial state;

[0132] Select a target number of actions from the action space to transform the target state. When the loss function converges, update the target number of actions, return to the step of selecting the target number of actions from the action space to transform the target state, until the target state is transformed to the initial state or the maximum number of actions is reached, so as to obtain the trained second reinforcement learning model, where the target number of actions is the set expected number of actions required to restore the target state to the initial state.

[0133] The initial state is the starting situation of the target optimization problem, and the target state is the situation that is desired to be transformed from the initial state through a series of operations. In the training of the second reinforcement learning model, it is necessary to learn how to restore the target state back to the initial state. The second reinforcement learning model performs a process opposite to that of the first reinforcement learning model. The first reinforcement learning model goes from the initial state to the target state, except that the target state is unknown during training, and the identification of the completion of training is that the number of the first values in the target vector is a preset number, and the state at this time is the target state. The action spaces of the first reinforcement learning model and the second reinforcement learning model are generated based on the same quantum chip, that is, generated based on the same qubit topology. The action spaces of the two reinforcement learning models may be the same, and the second reinforcement learning model executes the actions in the action space in reverse; the actions in the action spaces of the two reinforcement learning models may also be opposite.

[0134] The maximum number of actions is a preset upper limit, representing the maximum number of actions that can be used to restore the target state to the initial state. If this number is reached before the target state is restored to the initial state, the training process will stop. The target number of actions is a preset value representing the expected number of actions required to restore the target state to the initial state. An initial target number of actions is set initially. Actions are selected from the action space: The target number of actions is randomly selected from the action space or selected according to a certain strategy. For example, if the target number of actions is 3, 3 actions are selected from the action space. The selected actions are applied sequentially to the target state, changing the target state. The state is updated each time an action is applied. When the target state has undergone a series of transformations and the resulting state is exactly the same as the initial state or the difference is within an acceptable range, the training process stops. If the number of actions used during training reaches the preset maximum number of actions, training will stop even if the target state has not been restored to the initial state. When training on a given number of actions, if the loss function converges, the current number of actions is incremented by 1, and training continues. For example, if the initial number of target actions is 1, one action is selected to operate on the target state. It is determined whether the operation returns the target state to its initial state. If not, the loss function is checked for convergence. If convergence occurs, the target number of actions is incremented by 1. Two actions are then selected, and it is determined whether these two actions can restore the target state to its initial state. It should be noted that during each update of the target number of actions, the input to the second reinforcement learning model is always the target state. Within the same target work loop, each selected number of target actions is executed on the target state.

[0135] See Figure 4 , Figure 4 A QAOA quantum circuit generation device provided in this application embodiment may include:

[0136] The target term determination module 401 is used to determine the interaction terms between the variables and qubits that are not connected according to the mapping relationship between the variables of the target optimization problem and the qubits, and to use them as target terms. All target terms form a target vector.

[0137] The module 402 is used to select an action using a trained first reinforcement learning model, execute the selected action on a first matrix representing the target optimization problem, and obtain a second matrix corresponding to the number of elements of the target vector with the first value reaching a preset value; wherein, a row of the first matrix corresponds to a variable of the target optimization problem, and the action is an operation of adding the quantum states of two qubits based on the connection relationship between qubits on the quantum chip.

[0138] The restoration module 403 is used to obtain the action of restoring the second matrix to the first matrix through the trained second reinforcement learning model;

[0139] The circuit generation module 404 is used to generate a QAOA quantum circuit by applying a CNOT gate to the qubit corresponding to the obtained action.

[0140] The specific functions and effects of the QAOA quantum circuit generation device described above can be explained by referring to other embodiments of this application, and will not be repeated here. Each sub-circuit in the QAOA quantum circuit generation device can be implemented entirely or partially through software, hardware, or a combination thereof. Each sub-circuit can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each of the above sub-circuits.

[0141] Please see Figure 5 This application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the QAOA quantum circuit generation method in any of the above embodiments. Please refer to [link to relevant documentation]. Figure 5 The computer device can be a classical computer or a quantum computer.

[0142] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, causes the computer to perform the QAOA quantum circuit generation method in any of the above embodiments.

[0143] This application also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the QAOA quantum circuit generation method in any of the above embodiments.

[0144] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not limit the implementation process of the embodiments of this application in any way.

[0145] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.

[0146] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0147] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software sub-circuits in the decoding processor. The software sub-circuits can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0148] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0153] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0154] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for generating QAOA quantum circuits, characterized in that, The method includes: Based on the mapping relationship between the variables of the objective optimization problem and the qubits on the quantum chip, the interaction terms between the qubits corresponding to the determined variables that have no connection relationship are taken as objective terms; all objective terms form an objective vector. Using a trained first reinforcement learning model, an action is selected and executed on a first matrix representing the target optimization problem to obtain a second matrix when the number of elements of the target vector with the first value reaches a preset value; wherein, a row of the first matrix corresponds to a variable of the target optimization problem, and the action is a modular addition operation based on the connection relationship between qubits on the quantum chip, representing the quantum states of two qubits; By using a well-trained second reinforcement learning model, the action of restoring the second matrix to the first matrix can be obtained; A QAOA quantum circuit is generated by applying a CNOT gate to the qubit corresponding to the obtained action.

2. The method as described in claim 1, characterized in that, The step of selecting actions using a trained first reinforcement learning model, executing the selected actions on a first matrix representing the target optimization problem, and obtaining a second matrix corresponding to the condition where the number of elements in the target vector that are first values ​​reaches a preset value includes: Based on the reward value and evaluation Q value corresponding to each action determined by the first training reinforcement learning model, an action is selected using a heuristic search algorithm. The selected action is executed on the first matrix representing the target optimization problem, and a second matrix is ​​obtained when the number of elements of the target vector with the first value reaches a preset value.

3. The method as described in claim 2, characterized in that, The heuristic search algorithm is the A* search algorithm.

4. The method as described in claim 3, characterized in that, The reward value and evaluation Q-value corresponding to each action determined based on the trained first reinforcement learning model are used to select actions using a heuristic search algorithm. The selected actions are then executed on a first matrix representing the target optimization problem to obtain a second matrix corresponding to the number of elements in the target vector that have the first value as their value, which includes: Using the trained first reinforcement learning model, determine the action corresponding to the maximum evaluation Q value and add the selected action to the OPEN table; The A* search algorithm selects an action from the OPEN table using its evaluation function, transforms the first matrix, updates the target vector based on the transformation result, and adds the action to the CLOSE table. When the search cutoff condition is met, the latest matrix obtained based on the transformation of the first matrix is ​​used as the second matrix. The evaluation function of the A* search algorithm consists of the evaluation Q-value and the reward value corresponding to the action output by the trained first reinforcement learning model. The search cutoff condition is that the number of elements in the target vector with the first value reaches a preset value.

5. The method as described in claim 4, characterized in that, The step involves selecting an action from the OPEN table using the A* search algorithm's evaluation function, transforming the first matrix, updating the target vector based on the transformation result of the first matrix, and adding the action to the CLOSE table. When the search cutoff condition is met, the latest matrix obtained based on the transformation of the first matrix is ​​used as the second matrix, including: The action with the largest function value in the OPEN table is used to transform the current matrix to obtain a new matrix. The target vector is then updated using the new matrix. If the search cutoff criteria have not been met and the currently selected action is in the COLSE table, determine whether the reward value for performing the action is greater than the reward value for the action stored in the COLSE table. If the value is greater than the target value, update the matrix and reward value corresponding to the action in the COLSE table, and update the OPEN table until the search cutoff condition is met. Then, the latest matrix is ​​obtained based on the transformation of the first matrix and used as the second matrix.

6. The method as described in claim 2, characterized in that, The reward value corresponding to the action is calculated by dividing the number of the first value in the target vector by the length of the target vector.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Obtain the initial state and target state, action space, and maximum number of actions to restore the target state to the initial state for training the second reinforcement learning model for the target optimization problem; Select several target actions from the action space to transform the target state. When the loss function converges, update the number of target actions and return to the previous step of selecting several target actions from the action space to transform the target state. Continue this process until the target state is transformed to obtain the initial state or the maximum number of actions is reached, and obtain the trained second reinforcement learning model. Here, the number of target actions is the number of actions required to restore the target state to the initial state.

8. A QAOA quantum circuit generation device, characterized in that, The method includes: The objective term determination module is used to determine the interaction terms between the variables and qubits corresponding to the variables, based on the mapping relationship between the variables and qubits in the objective optimization problem. These interaction terms, which are not connected between the qubits, are used as objective terms. All objective terms form an objective vector. The module is used to select an action using a trained first reinforcement learning model, execute the selected action on a first matrix representing the target optimization problem, and obtain a second matrix corresponding to the number of elements of the target vector with the first value reaching a preset value; wherein, a row of the first matrix corresponds to a variable of the target optimization problem, and the action is a modular addition operation based on the connection relationship between qubits on the quantum chip, representing the quantum states of two qubits; The restoration module is used to obtain the action of restoring the second matrix to the first matrix through the trained second reinforcement learning model; The circuit generation module is used to generate QAOA quantum circuits by applying CNOT gates to the qubits corresponding to the obtained actions.

9. A computer device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the QAOA quantum circuit generation method of any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the QAOA quantum circuit generation method of any one of claims 1-7.

Images