Hamiltonian simulation computing method and device based on reinforcement learning and electronic equipment

Abstract

This application discloses a Hamiltonian simulation computation method, apparatus, and electronic device based on reinforcement learning. The method includes: initializing a task to be processed, which is a quantum computing task received by a quantum cloud platform; determining the quantum computing encoding strategy corresponding to the task to be processed using a Markov decision process through reinforcement learning, and performing operator transformation processing on the Hamiltonian corresponding to the task to be processed according to the quantum computing encoding strategy to obtain the Hamiltonian in Pauli representation; performing quantum computing corresponding to the task to be processed on the Hamiltonian in Pauli representation; measuring the obtained output wavefunction, and determining the output result of the quantum cloud platform based on the corresponding measurement result. Implementing this application embodiment can optimize the selection decision for the quantum computing encoding strategy to adapt to the actual needs of computing tasks with different complexities, thereby helping to optimize the overall computing framework and improve the actual computing efficiency of quantum computing.

Description

Technical Field

[0001] This application relates to the field of quantum computing technology, specifically to a Hamiltonian simulation computing method, apparatus, and electronic device based on reinforcement learning. Background Technology

[0002] Currently, quantum computing offers efficient computational solutions for many-body and strongly correlated system problems that are difficult to handle using traditional computational methods. These solutions are particularly relevant in fields such as quantum materials, superconducting systems, and biosciences, where the interactions of multiple particles (or molecules, particles, etc.) can be challenging. However, current quantum computing solutions often suffer from complex coding and excessive reliance on manual design, making them ill-suited to the practical needs of computational tasks of varying complexity. This results in the underutilization of limited quantum computing resources, significantly reducing the actual computational efficiency of quantum computing. Summary of the Invention

[0003] This application discloses a Hamiltonian simulation computing method, apparatus, and electronic device based on reinforcement learning, which can optimize the selection decision for quantum computing encoding strategies to adapt to the actual needs of computing tasks with different complexities, thereby helping to optimize the overall computing framework and improve the actual computing efficiency of quantum computing.

[0004] The first aspect of this application discloses a Hamiltonian simulation computation method based on reinforcement learning, applied to a quantum cloud platform, the method comprising: Initialization processing is performed on the task to be processed, which is the quantum computing task received by the quantum cloud platform; By using reinforcement learning, a Markov decision process is used to determine the quantum computing encoding strategy corresponding to the task to be processed. Based on the quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is subjected to operator transformation to obtain the Hamiltonian in Pauli representation. Perform quantum computation corresponding to the task to be processed on the Hamiltonian represented by the Pauli symbol; The obtained output wavefunction is measured, and the output result of the quantum cloud platform is determined based on the corresponding measurement results.

[0005] The second aspect of this application discloses a Hamiltonian simulation computing device based on reinforcement learning, applied to a quantum cloud platform, the device comprising: An initialization unit is used to perform initialization processing on a task to be processed, which is a quantum computing task received by the quantum cloud platform. The decision unit is used to determine the quantum computing encoding strategy corresponding to the task to be processed by using a Markov decision process through reinforcement learning, and to perform operator transformation processing on the Hamiltonian corresponding to the task to be processed according to the quantum computing encoding strategy to obtain the Hamiltonian in Pauli representation. The computing unit is used to perform quantum computation corresponding to the task to be processed on the Hamiltonian represented by the Pauli symbol; The measurement output unit is used to measure the obtained output wave function and determine the output result of the quantum cloud platform based on the corresponding measurement results.

[0006] The third aspect of this application discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor enables the processor to implement any of the reinforcement learning-based Hamiltonian simulation computation methods disclosed in the first aspect of this application.

[0007] The fourth aspect of this application discloses a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the reinforcement learning-based Hamiltonian simulation computation methods disclosed in the first aspect of this application.

[0008] Compared with related technologies, the embodiments of this application have the following beneficial effects: In this embodiment, the quantum cloud platform initializes the task to be processed, and then uses reinforcement learning and Markov decision processes to determine the quantum computing encoding strategy corresponding to the task. Based on this encoding strategy, the Hamiltonian corresponding to the task is further transformed using operator transformations to obtain the Hamiltonian in Pauli representation. Then, quantum computation corresponding to the task can be performed on the Hamiltonian in Pauli representation, and the resulting output wavefunction can be measured. Based on the measurement results, the output of the quantum cloud platform is determined. Therefore, implementing this embodiment enables the optimization of quantum computing encoding strategies through reinforcement learning (RL) based on Markov decision processes (MDPs). This allows for the dynamic selection of the most suitable encoding strategy for different quantum computing tasks, adapting to the actual needs of each task (e.g., different computational requirements corresponding to different task complexities). This Hamiltonian simulation method can provide quantum computing solutions for many-body and strongly correlated system problems that are difficult to handle by traditional computing methods, based on the Hamiltonian corresponding to the quantum computing task. At the same time, by adopting appropriate quantum computing encoding strategies, it can help optimize the overall computing framework, reduce the demand for quantum computing resources, and give full play to the practical advantages of quantum computing in specific scenarios, thereby improving the actual computing efficiency of quantum computing. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating a Hamiltonian simulation computation method based on reinforcement learning disclosed in an embodiment of this application. Figure 2 This is a flowchart illustrating another Hamiltonian simulation computation method based on reinforcement learning disclosed in an embodiment of this application. Figure 3 This is a schematic diagram of a framework for implementing reinforcement learning based on a target Markov decision process disclosed in an embodiment of this application; Figure 4 This is a flowchart illustrating another Hamiltonian simulation computation method based on reinforcement learning disclosed in the embodiments of this application; Figure 5 This is a schematic diagram of a system flow diagram of the Hamiltonian simulation computing method based on reinforcement learning disclosed in the embodiments of this application applied to a quantum cloud platform; Figure 6 This is a modular schematic diagram of a Hamiltonian simulation computing device based on reinforcement learning disclosed in an embodiment of this application; Figure 7 This is a modular schematic diagram of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0011] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0012] This application discloses a Hamiltonian simulation computing method, apparatus, and electronic device based on reinforcement learning, which can optimize the selection decision for quantum computing encoding strategies to adapt to the actual needs of computing tasks with different complexities, thereby helping to optimize the overall computing framework and improve the actual computing efficiency of quantum computing.

[0013] The following will be described in detail with reference to the accompanying drawings.

[0014] In this embodiment of the application, the reinforcement learning-based Hamiltonian simulation computation method can be applied to a quantum cloud platform. This quantum cloud platform can receive quantum computing tasks submitted by users, use the corresponding Hamiltonian to realize system modeling and simulation for the task, and after certain computational processing, can provide the corresponding output results to the user.

[0015] Among them, the aforementioned quantum computing tasks can include computational tasks related to many-body systems, which are used to address many-body and strongly correlated system problems that are difficult to handle by traditional computing methods. For example, in fields such as quantum materials, superconducting systems, and bioscience, appropriate quantum computing solutions can be provided for related problems that may be involved due to the interaction of multiple particles (or molecules, particles, etc.).

[0016] For example, users can directly submit computational tasks to the quantum cloud platform, which will then convert them from classical computing tasks (i.e., computational tasks based on classical mathematics, classical physics, etc., and implemented using traditional computing methods) to quantum computing tasks, allowing for subsequent computational processing. In some embodiments, users can also pre-convert and adjust the computational tasks to be processed according to quantum computing requirements before submitting them to the quantum cloud platform for subsequent conversion and computational processing.

[0017] Based on this, the quantum cloud platform can perform initialization processing on the received tasks, including corresponding system modeling and parameter configuration. Then, it can determine a suitable quantum computing encoding strategy through reinforcement learning, and based on the quantum computing encoding strategy, perform the operator transformation required for quantum computing on the Hamiltonian corresponding to the task, so that the quantum cloud platform can perform actual quantum computing and corresponding output.

[0018] Please see Figure 1 , Figure 1 This is a flowchart illustrating a reinforcement learning-based Hamiltonian simulation computation method disclosed in an embodiment of this application. This method can be applied to the aforementioned quantum cloud platform. Figure 1 As shown, the method may include the following steps: S102. Initialize the task to be processed, which is a quantum computing task received by the quantum cloud platform.

[0019] In this embodiment of the application, after receiving a quantum computing task submitted by a user, the quantum cloud platform can treat it as a task to be processed and perform certain initialization processes to facilitate the corresponding quantum computing encoding in subsequent steps, and then perform the quantum computing actually required for the task to be processed.

[0020] For example, the initialization process described above may include various preprocessing steps required for quantum computing of the task to be processed, such as system modeling and parameter configuration.

[0021] In some embodiments, system modeling for a task to be processed may include a process of constructing a corresponding model based on the system problem to be solved by the task or the system scenario to which it is applied. For example, if the task to be processed is a quantum computing task related to a many-body system, then system modeling for the task to be processed may include constructing a corresponding crystal model, such as defining its spatial structure and unit cell (i.e., the smallest repeating unit in a crystal lattice), superlattice structure (which may include a combination of multiple unit cells), electron orbitals, and electron interactions, etc.

[0022] In some embodiments, parameter configuration for the task to be processed may include providing necessary system configuration parameters for the task on the quantum cloud platform, so that the quantum cloud platform can perform effective quantum simulation calculations. For example, the above-mentioned system configuration parameters may include the quantum initial state (i.e., the initial state of the quantum system before the execution of the quantum circuit corresponding to the task to be processed), evolution time (i.e., the time taken for the quantum system to evolve from the above-mentioned quantum initial state to the quantum final state during quantum computation) and / or the number of time slices (i.e., the number of time slices to which the above-mentioned evolution time is divided), etc., which are not specifically limited in the embodiments of this application.

[0023] S104. Through reinforcement learning, a Markov decision process is used to determine the quantum computing encoding strategy corresponding to the task to be processed. Based on the quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is subjected to operator transformation to obtain the Hamiltonian in Pauli representation.

[0024] In this embodiment of the application, in order to complete the above-mentioned task using quantum computing, it is necessary to perform operator transformation processing on the Hamiltonian corresponding to the task based on a certain quantum computing encoding strategy to obtain the Hamiltonian (i.e., Pauli operator) represented by Pauli required for quantum computing.

[0025] In some embodiments, the problem of selecting a quantum computing encoding strategy can be modeled as a learnable and optimizeable decision process using reinforcement learning to determine the most suitable quantum computing encoding strategy for the task from a variety of candidate strategies. For example, the aforementioned variety of candidate strategies may include encoding strategies based on the Jordan-Wigner Transformation, encoding strategies based on the Bravyi-Kitaev Transformation, encoding strategies based on Parity Encoding, etc., and are not specifically limited in this embodiment.

[0026] For example, a Markov Decision Process (MDP) can be used to implement the reinforcement learning described above, continuously optimizing decisions and ultimately determining the quantum computing encoding strategy corresponding to the task to be processed. The MDP can include a state space (representing different characteristics of the quantum system), an action space (including multiple different candidate strategies), a transition function (describing the system state transitions of the quantum system under a specific quantum computing encoding strategy), and a reward function (defining actions, i.e., the merits of the candidate strategies, to maximize system performance). Through MDP-based reinforcement learning, the quantum computing encoding strategy that maximizes the reward function calculation result can be determined from the action space, serving as the quantum computing encoding strategy corresponding to the task to be processed.

[0027] Based on this, the quantum cloud platform can transform the Hamiltonian corresponding to the task to be processed into a Pauli representation of the Hamiltonian based on the quantum computing encoding strategy corresponding to the task to be processed. Then, in subsequent steps, the corresponding quantum circuit can be compiled (i.e. constructed) based on the Pauli representation of the Hamiltonian to perform the corresponding quantum computing.

[0028] S106. Perform quantum computation on the Hamiltonian represented by Pauli and corresponding to the task to be processed.

[0029] In this embodiment, an executable quantum circuit for completing the aforementioned task can be constructed based on the Hamiltonian (i.e., the Pauli operator) in Pauli representation. For example, by decomposing the Pauli operator (further performing necessary basis transformations) and mapping it to a corresponding single-qubit rotation gate, circuit synthesis can be achieved through the parallel execution of each bit operation, thus realizing the quantum computation corresponding to the task. It can be understood that the aforementioned quantum computation can be represented as a time evolution simulation based on the Hamiltonian. In subsequent steps, by measuring the corresponding final evolution state (i.e., the quantum final state, which can be described by the obtained output wavefunction), the computational result corresponding to the quantum computation can be obtained for feedback output from the quantum cloud platform.

[0030] In some embodiments, the execution of the aforementioned quantum computing can also be achieved within a quantum-classical hybrid computing framework. For example, necessary Hamiltonian preprocessing and quantum circuit optimization can be performed using classical computing. Then, the optimized quantum circuit can be used to execute the core quantum simulation task (i.e., time evolution simulation based on the Hamiltonian) based on the preprocessed Pauli representation of the Hamiltonian. This helps to further optimize the overall computing framework, reduce the demand for quantum computing resources, leverage the practical advantages of quantum computing in the aforementioned quantum simulation tasks, and improve the actual computational efficiency of quantum computing.

[0031] S108. Measure the obtained output wave function and determine the output result of the quantum cloud platform based on the corresponding measurement results.

[0032] In this embodiment of the application, after completing the quantum computation corresponding to the above-mentioned task to be processed (i.e., executing the corresponding quantum circuit), the quantum cloud platform can perform the required basis measurement on the obtained output wave function to obtain the corresponding measurement result.

[0033] For example, the reference substrate used for the above measurement may include at least the Pauli Basis (the corresponding Pauli measurement can be used to obtain specific component information of the quantum state) or the Hadamard Basis (the corresponding Hadamard measurement can be used to determine the phase of the quantum state), and no specific limitation is made in the embodiments of this application.

[0034] Based on this, the quantum cloud platform can further calculate the relevant probabilities or physical quantities required for the task to be processed according to the above measurement results, and then collect them as the output results corresponding to the task to be processed, so as to provide corresponding feedback output for the task submitted by the user through the quantum cloud platform.

[0035] As can be seen, the Hamiltonian simulation computing method described in the above embodiments can optimize quantum computing encoding strategies through reinforcement learning based on Markov decision processes. This allows for the dynamic selection of the most suitable encoding strategy for different quantum computing tasks, adapting to varying task complexities and other practical needs. Based on the Hamiltonian corresponding to the quantum computing task, this Hamiltonian simulation computing method can provide quantum computing solutions for many-body and strongly correlated system problems that are difficult to handle using traditional computing methods. Furthermore, employing appropriate quantum computing encoding strategies helps optimize the overall computing framework, reduce quantum computing resource requirements, and leverage the practical advantages of quantum computing in specific scenarios, thereby improving the actual computational efficiency of quantum computing.

[0036] Please see Figure 2 , Figure 2 This is a flowchart illustrating another Hamiltonian simulation computation method based on reinforcement learning disclosed in an embodiment of this application. This method can be applied to the aforementioned quantum cloud platform. Figure 2 As shown, the method may include the following steps: S202. Initialize the task to be processed, which is a quantum computing task received by the quantum cloud platform.

[0037] Step S202 is similar to step S102 above, and will not be described again here.

[0038] S204. For the task to be processed, construct a target Markov decision process; wherein, the target Markov decision process includes a state space, action space, transition function and reward function corresponding to the task to be processed, and the action space includes multiple different quantum computing encoding strategies.

[0039] In this embodiment of the application, the quantum cloud platform can construct a target Markov decision process (hereinafter referred to as target MDP) for the task to be processed, so as to determine the most suitable quantum computing encoding strategy for the task to be processed by reinforcement learning of the target MDP in subsequent steps.

[0040] In some embodiments, the state space S constituting the target MDP can be as shown in Formula 1 below, to describe different characteristics of the quantum system, such as task complexity, type of crystal model (for tasks to be processed related to many-body systems), representation of Hamiltonians, etc.

[0041] Formula 1:

[0042] in, ( (And n is a positive integer) can represent a specific state corresponding to the task to be processed, such as a specific model complexity, coupling coefficient, etc.

[0043] Furthermore, the action space A constituting the target MDP can include multiple different candidate strategies to represent all possible quantum computing encoding strategies that can be selected in a specific state, as shown in Equation 2 below.

[0044] Formula 2:

[0045] Furthermore, the transfer function T constituting the target MDP can be expressed as shown in Equation 3 below, which describes how the quantum system transitions from one specific state to another given the selected quantum computing encoding strategy.

[0046] Formula 3:

[0047] in, and The current state and the next state after the transition can be represented separately. This represents the action taken to achieve the transfer (i.e., the selected quantum computing encoding strategy). This indicates starting from the current state. Take action Then transition to the next state The probability of.

[0048] Furthermore, the reward function R constituting the target MDP can be used, as shown in Formula 4, to define the merits of the actions taken, in order to evaluate and determine the quantum computing encoding strategy that maximizes the performance of the quantum system. For example, This can represent the current state. Take action below The advantages and disadvantages of corresponding quantum computing encoding strategies.

[0049] Formula 4:

[0050] In other words, the above reward function It can be constructed based on at least the following metrics: number of qubits used, computation time, and computational accuracy. This indicates the number of qubits used in the task to be processed; the fewer the better. This indicates the computation time for quantum computing; the faster the better. This indicates the computational precision of quantum computing; the higher the precision, the better. , , This is a hyperparameter used to balance the weights of the above indicators.

[0051] Please refer to further information. Figure 3 , Figure 3 This is a schematic diagram of a framework for reinforcement learning based on a target Markov decision process, as disclosed in an embodiment of this application. Figure 3 As shown, the target MDP used in the reinforcement learning framework can act on the interaction between the agent and the environment, based on the current environment (i.e., the current state of the quantum system). and corresponding feedback rewards The intelligent agent can make decisions and take actions. This is to enable the current environment to undergo a state transition (i.e., from the current state). Transition to the next state The probability corresponding to this process is expressed as ), and calculate the corresponding feedback rewards. Based on this, the data generated by the above interaction process can be used by the intelligent agent to continuously learn and make decisions that are directed towards maximizing the performance of the quantum system, thereby determining the most suitable quantum computing encoding strategy for the task at hand.

[0052] S206. Based on the objective Markov decision process, a quantum computing encoding strategy that maximizes the reward function calculation result is determined from the action space through reinforcement learning, which serves as the quantum computing encoding strategy corresponding to the above-mentioned task to be processed.

[0053] In this embodiment of the application, reinforcement learning based on the target MDP can be as follows: Figure 3 As shown, this is achieved through the interaction between the agent and the environment. In this process, the objective of the reward function R can include maximizing the performance of the quantum system. Specifically, it can include maximizing the computational result of the reward function R.

[0054] In some embodiments, the agent can base its actions on the current environmental state corresponding to the state space S described above. (and corresponding feedback rewards) (The first loop may not include this step) to determine the target encoding strategy from action space A. This will be the next action taken, based on the target encoding strategy. Current environmental status The transition can be achieved through a transfer function T, that is, a state transition occurs to obtain the target environment state. The corresponding system performance results are calculated using the reward function R to determine the feedback reward. .

[0055] Based on this, the agent can further analyze the target environment state. As the new current environment state, the process of determining the target encoding strategy and implementing state transitions is repeated until the system performance result recalculated through the reward function R satisfies the preset maximization condition (e.g., the system performance result satisfies the preset performance threshold condition, or the determined feedback reward satisfies the preset reward threshold condition, or the maximum number of iterations is reached, etc.). Furthermore, the latest target encoding strategy after exiting the above loop can be determined as the quantum computing encoding strategy corresponding to the task to be processed, thereby enabling the optimal encoding decision for the specific model or quantum system corresponding to the task to be processed.

[0056] As an optional implementation, the decision-making process of the aforementioned intelligent agent can employ Q-Learning to maximize long-term feedback rewards in order to learn the corresponding optimal decision-making method. For example, the Q-Learning update formula (i.e., the corresponding Q-value function) can be as shown in Equation 5 below, based on the current environmental state corresponding to the aforementioned state space S. The target encoding strategy is determined from the action space A using a Q-Learning reinforcement learning strategy (i.e., reinforcement learning based on optimizing the Q-value function). The goal in this process is to maximize the system performance result calculated by the reward function R.

[0057] Formula 5:

[0058] in, Indicates the state Next action The current Q value, Then it is the execution action. Then, a state transition occurs to the next state. The instant reward received. Indicates the learning rate. This represents the weighting coefficient for future rewards. This indicates the next state. All actions that can be performed below (via (This represents) the maximum Q value.

[0059] By iteratively updating the Q value using Formula 5, a corresponding reinforcement learning strategy based on the target MDP can be implemented, enabling the agent to learn the optimal decision-making method and ultimately determine the quantum computing encoding strategy most suitable for the task to be processed.

[0060] S208. Based on the above quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is transformed into a linear combination of Pauli operators to obtain the Hamiltonian in Pauli representation.

[0061] Step S208 is similar in some implementations to step S104 described above. It should be noted that, including the selected quantum computing encoding strategy, the multiple different candidate strategies included in the action space can all perform operator transformation processing on the Hamiltonian corresponding to the task to be processed, obtaining the corresponding Pauli operators. It can be understood that a linear combination of Pauli operators represents an observable (Pauli string) composed of a weighted sum of one or more Pauli terms. Therefore, the Hamiltonian represented by Pauli can be compiled term by term into the corresponding target quantum circuit, through which the quantum computation corresponding to the task to be processed can be performed.

[0062] S210. Through classical calculation, the Hamiltonian expressed by Pauli is preprocessed to obtain the corresponding simplified Hamiltonian.

[0063] S212. Using the target quantum circuit, perform quantum computation corresponding to the simplified Hamiltonian to obtain the corresponding output wave function.

[0064] In the embodiments of this application, a quantum-classical hybrid computing framework can be adopted. On the one hand, classical computing is used to complete various preprocessing and optimization tasks for the task to be processed, such as preprocessing of the Hamiltonian represented by Pauli, initial optimization of quantum circuit parameters, and subsequent parameter updates. On the other hand, the optimized quantum circuit can be used to execute the core quantum simulation task (i.e., time evolution simulation based on the Hamiltonian) based on the preprocessed Hamiltonian represented by Pauli.

[0065] In some embodiments, the Hamiltonian is computed classically (via... H Preprocessing (representing the representation) can include methods such as gradient optimization, tensor decomposition, and singular value decomposition (SVD) to extract the main physical features and construct corresponding low-dimensional simplified Hamiltonians. This can effectively reduce the complexity of subsequent quantum computing.

[0066] For example, the preprocessing performed using the preprocessing function P can be expressed as follows: Specifically, taking SVD as an example, the above preprocessing process can be shown in Formula 6 below.

[0067] Formula 6:

[0068] in, Represents a diagonal matrix composed of singular values After truncation or approximation, the resulting singular value matrix can reduce the quantum state dimension required for subsequent simulations.

[0069] Based on this, and using the compiled and preprocessed optimized target quantum circuit, the simplified Hamiltonian described above can be executed. The corresponding quantum computing process can be represented by the following formula 7. The resulting evolutionary final state (i.e., the quantum final state) can be obtained through... (represented) can be described as the corresponding output wavefunction.

[0070] Formula 7:

[0071] in, For the quantum initial state, t This represents the evolution time. Through Trotter-SuzuKi decomposition, it can be... (in The imaginary unit, The reduced Planck constant is approximated as a product of a series of shorter time steps to facilitate implementation in practical quantum circuits.

[0072] S214. Measure the obtained output wave function and determine the output result of the quantum cloud platform based on the corresponding measurement results.

[0073] Step S214 is similar to step S108 above, and will not be described again here.

[0074] As can be seen, the Hamiltonian simulation computing method described in the above embodiments can optimize quantum computing encoding strategies through reinforcement learning based on Markov decision processes. This allows for the dynamic selection of the most suitable encoding strategy for different quantum computing tasks, adapting to varying task complexities and other practical needs. This Hamiltonian simulation computing method can provide quantum computing solutions for many-body and strongly correlated system problems that are difficult to handle using traditional computing methods, based on the Hamiltonian corresponding to the quantum computing task. Simultaneously, employing a suitable quantum computing encoding strategy helps optimize the overall computing framework, reduce quantum computing resource requirements, and leverage the practical advantages of quantum computing in specific scenarios, thereby improving the actual computational efficiency of quantum computing. Furthermore, by using Q-Learning reinforcement learning strategies, the accuracy and effectiveness of dynamically selecting the optimal encoding strategy can be further improved by combining indicators such as qubit usage, computation time, and computational accuracy. Moreover, by adopting a quantum-classical hybrid computing framework, the complexity of quantum computing can be significantly reduced, further decreasing the demand for qubits, thus contributing to further optimization of the overall computing framework and improving the task processing efficiency of the quantum cloud platform.

[0075] Please see Figure 4 , Figure 4 This is a flowchart illustrating another Hamiltonian simulation computation method based on reinforcement learning disclosed in an embodiment of this application. This method can be applied to the aforementioned quantum cloud platform. Figure 4 As shown, the method may include the following steps: S402. When the task to be processed is a quantum computing task related to a many-body system, a corresponding crystal model is constructed for the task to be processed. The crystal model is described based on the Hamiltonian corresponding to the task to be processed.

[0076] In the embodiments of this application, when modeling a quantum computing task related to a many-body system, a corresponding crystal model can be constructed, and its spatial structure, unit cell, superlattice structure, electron orbitals, and electron interactions can be defined.

[0077] For example, the above spatial structure can be determined according to the user requirements corresponding to the task to be processed, and is used to define the form of the crystal structure of the crystal model itself (e.g., simple cubic form, face-centered cubic form, etc.). The smallest repeating unit (basic building unit) in the lattice generated after the model is built is the unit cell, which can contain all the necessary physical information.

[0078] The superlattice structure described above can be used to simulate long-range interactions in large-scale systems. In some embodiments, the superlattice structure may include a combination of multiple unit cells, thereby enabling the introduction of more complex periodic structures into the model. For example, in compound models, materials with specific physical properties can be constructed by changing the arrangement of material layers.

[0079] The aforementioned electron orbitals can refer to multiple electron orbitals contained within each unit cell, and different orbitals determine how electrons move within the unit cell. In some embodiments, depending on the properties of the material being modeled, the type and arrangement of electron orbitals within each unit cell can be defined during modeling to meet the computational requirements of the task at hand.

[0080] The aforementioned electronic interactions can be included in the crystal model as transition interactions, in-situ interactions, etc., between electrons. By defining different interaction strengths and types in the crystal model, the foundation can be laid for the definition, transformation, and calculation of the Hamiltonian of a quantum system.

[0081] S404. Configure the quantum cloud platform according to the task to be processed, using the corresponding configuration parameters; wherein, the configuration parameters include at least the quantum initial state, evolution time and / or the number of time slices.

[0082] In this embodiment of the application, by providing the necessary system configuration parameters for the task to be processed, the quantum cloud platform can be configured accordingly for the task to be processed, so that the quantum cloud platform can perform effective quantum simulation calculations.

[0083] For example, the quantum initial state included in the above configuration parameters can represent the initial state of the quantum system before the execution of the quantum circuit corresponding to the task to be processed. In some embodiments, the quantum initial state can be precisely set according to the specific problem to be solved by the task to be processed, or it can be preset according to the requirements of the actual physical characteristics of the quantum system. No specific limitation is made in the embodiments of this application.

[0084] The evolution time included in the above configuration parameters can represent the time it takes for the quantum system to evolve from the initial quantum state to the final quantum state during quantum computing, that is, the total evolution time of the entire quantum system can be preset.

[0085] The number of time slices included in the above configuration parameters represents the number of time segments into which the evolution time is divided, where each time segment can be used to perform a corresponding quantum operation. It is understood that a larger number of time slices results in higher simulation accuracy, but also increases the depth of the quantum circuitry and the consumption of computational resources.

[0086] S406. Perform a task complexity assessment on the task to be processed and obtain the complexity assessment results.

[0087] In this embodiment, before constructing the target MDP for the task to be processed, a task complexity assessment can be performed on the task to be processed. This allows for a more refined construction of the state space of the target MDP based on the obtained complexity assessment results. For example, the task complexity assessment can be implemented based on task complexity modeling as shown in Formula 8 below, thereby directly calculating the required complexity assessment results. .

[0088] Formula 8:

[0089] in, Indicates the number of layers in a superlattice structure. This indicates the number of time slices mentioned above. This indicates the number of interaction terms; , , These are the weighting factors for the complexity indicators of the above tasks.

[0090] S408. Based on the above complexity assessment results, construct a target Markov decision process for the task to be processed; wherein, the target Markov decision process includes the state space, action space, transition function and reward function corresponding to the above task to be processed, and the above action space includes multiple different quantum computing encoding strategies.

[0091] Step S408 is similar in some implementations to step 204 described above. It should be noted that the complexity evaluation result can be used as a numerical representation of the task complexity corresponding to the task to be processed, i.e., one of the characteristics of the quantum system corresponding to the task to be processed, to construct the state space of the target MDP. Based on this, the intelligent coding strategy decision optimization implemented based on the target MDP can adapt to different task types and complexities, fitting the actual needs of the task to be processed to provide the most suitable quantum computing coding strategy for that task.

[0092] S410. Based on the current environment state corresponding to the state space, determine the target encoding strategy from the action space.

[0093] S412. Based on the target coding strategy, the current environment state is transformed through the transfer function to obtain the target environment state, and the corresponding system performance results are calculated through the reward function.

[0094] S414. Take the target environment state as the new current environment state and repeat the above steps S410 to S412 until the system performance result recalculated by the reward function meets the preset maximization condition. Then, determine the latest target encoding strategy as the quantum computing encoding strategy corresponding to the task to be processed.

[0095] Steps S410, S412, and S414 are similar to some implementations of step S206 described above, and will not be repeated here.

[0096] S416. Based on the above quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is transformed into a linear combination of Pauli operators to obtain the Hamiltonian in Pauli representation.

[0097] Step S416 is similar to step S208 above, and will not be described again here.

[0098] S418. By classical calculation, the Hamiltonian expressed by Pauli is preprocessed to obtain the corresponding simplified Hamiltonian.

[0099] S420. Using the target quantum circuit, perform quantum computation corresponding to the simplified Hamiltonian to obtain the corresponding output wave function.

[0100] Steps S418 and S420 are similar to steps S210 and S212 above, and will not be described again here.

[0101] S422. Obtain a set reference substrate and measure the obtained output wave function using the projection method corresponding to the reference substrate to obtain the corresponding measurement result; wherein, the reference substrate includes at least a Pauli substrate or a Hadamard substrate.

[0102] S424. Based on the above measurement results, calculate the output results corresponding to the task to be processed, and output them through the quantum cloud platform.

[0103] In this embodiment, the user can choose (or specify through the submitted task) whether to perform actual measurement output on the output wavefunction obtained after performing the above quantum computation, and whether to output the output wavefunction as complete quantum state information. If measurement output is required, the observation of the quantum state can be determined by setting a reference basis, i.e., projection and measurement under the corresponding basis.

[0104] In some embodiments, the quantum cloud platform can acquire a reference substrate set by the user or the platform, and then use the projection method corresponding to the reference substrate to measure the output wave function to observe the required quantum state information (such as phase, superposition state, etc.) and obtain the corresponding measurement results.

[0105] For example, when using the Pauli basis as the reference basis, the quantum state measurement based on the Pauli operators (X, Y, Z) can obtain the specific component information of the quantum state; when using the Hadamard basis as the reference basis, the qubit can be converted from the classical Z basis to the superposition state X basis by using the Hadamard gate, and the phase (positive phase or negative phase) of the quantum state can be determined to obtain specific quantum state phase information.

[0106] Based on this, according to the corresponding measurement results obtained from the above measurement process, the quantum cloud platform can further calculate the relevant probabilities or physical quantities required for the task to be processed, and then collect them as the output results corresponding to the task to be processed, so as to provide corresponding feedback output for the task submitted by the user through the quantum cloud platform.

[0107] Please refer to further information. Figure 5 , Figure 5 This is a schematic diagram of a system flow illustrating the application of the reinforcement learning-based Hamiltonian simulation computation method disclosed in this application to a quantum cloud platform. Figure 5 As shown, the quantum cloud platform can treat user-submitted quantum computing tasks as pending tasks. It first categorizes these tasks into Hamiltonian simulation tasks or other types of quantum tasks. For Hamiltonian simulation tasks, initialization and other pre-processing can be performed. Initialization can include system modeling (e.g., establishing a crystal model corresponding to the pending task related to the multi-body system) and parameter configuration (i.e., configuring the quantum cloud platform accordingly for the pending task). Other pre-processing can include task complexity assessment and intelligent encoding strategy decision-making (e.g., using reinforcement learning with the aforementioned target MDP).

[0108] Based on this, and using the quantum computing encoding strategy selected through the aforementioned intelligent encoding strategy decision-making process, a quantum-classical hybrid framework can be employed to schedule and solve the task. Specifically, this can be achieved by performing the aforementioned classical computing-based preprocessing on the Hamiltonian corresponding to the task, followed by subsequent actual quantum computing. Furthermore, by measuring the obtained output wavefunction, the quantum cloud platform can obtain the output result corresponding to the task and output it.

[0109] As can be seen, the Hamiltonian simulation computing method described in the above embodiments can optimize quantum computing encoding strategies through reinforcement learning based on Markov decision processes. This allows for the dynamic selection of the most suitable encoding strategy for different quantum computing tasks, adapting to varying task complexities and other practical needs. This Hamiltonian simulation computing method can provide quantum computing solutions for many-body and strongly correlated system problems that are difficult to handle using traditional computing methods, based on the Hamiltonian corresponding to the quantum computing task. Simultaneously, employing a suitable quantum computing encoding strategy helps optimize the overall computing framework, reduce quantum computing resource requirements, and leverage the practical advantages of quantum computing in specific scenarios, thereby improving the actual computational efficiency of quantum computing. Furthermore, by using Q-Learning reinforcement learning strategies, the accuracy and effectiveness of dynamically selecting the optimal encoding strategy can be further improved by combining indicators such as qubit usage, computation time, and computational accuracy. Moreover, by adopting a quantum-classical hybrid computing framework, the complexity of quantum computing can be significantly reduced, further decreasing the demand for qubits, thus contributing to further optimization of the overall computing framework and improving the task processing efficiency of the quantum cloud platform. Furthermore, by performing multimodal measurements and analyses using different reference substrates, different types of quantum state information can be obtained as output results. This allows for flexible fulfillment of the multidimensional analytical needs of various tasks, such as material phase transitions and entanglement entropy, which is beneficial for broadening the functionality of the quantum cloud platform.

[0110] Please see Figure 6 , Figure 6 This is a modular schematic diagram of a Hamiltonian simulation computing device based on reinforcement learning disclosed in an embodiment of this application. Figure 6 As shown, the device may include an initialization unit 601, a decision-making unit 602, a calculation unit 603, and a measurement output unit 604, wherein: Initialization unit 601 is used to perform initialization processing on the task to be processed, which is a quantum computing task received by the quantum cloud platform. The decision unit 602 is used to determine the quantum computing encoding strategy corresponding to the task to be processed by reinforcement learning and Markov decision process, and to perform operator transformation processing on the Hamiltonian corresponding to the task to be processed according to the quantum computing encoding strategy to obtain the Hamiltonian in Pauli representation. Computation unit 603 is used to perform quantum computation corresponding to the task to be processed on the Hamiltonian represented by Pauli; The measurement output unit 604 is used to measure the obtained output wave function and determine the output result of the quantum cloud platform based on the corresponding measurement results.

[0111] As can be seen, the Hamiltonian simulation computing device described in the above embodiments can optimize quantum computing encoding strategies through reinforcement learning based on Markov decision processes. This allows for the dynamic selection of the most suitable encoding strategy for different quantum computing tasks, adapting to varying task complexities and other practical needs. Based on the Hamiltonian corresponding to the quantum computing task, this enables the implementation of quantum computing solutions for many-body and strongly correlated system problems that are difficult to handle using traditional computational methods. Furthermore, employing appropriate quantum computing encoding strategies helps optimize the overall computational framework, reduce quantum computing resource requirements, and leverage the practical advantages of quantum computing in specific scenarios, thereby improving the actual computational efficiency of quantum computing.

[0112] In some embodiments, the decision unit 602 described above can be specifically used for: For the task to be processed, a target Markov decision process is constructed; wherein, the target Markov decision process may include a state space, action space, transition function and reward function corresponding to the task to be processed, and the action space may include multiple different quantum computing encoding strategies; Based on the objective Markov decision process, a quantum computing encoding strategy that maximizes the reward function calculation result is determined from the action space through reinforcement learning, which serves as the quantum computing encoding strategy for the task to be processed.

[0113] In some embodiments, when the decision unit 602 is used to determine a quantum computing encoding strategy that maximizes the reward function computation result from the action space through reinforcement learning based on a target Markov decision process, it may specifically include: Based on the current environment state corresponding to the state space, the target encoding strategy is determined from the action space; Based on the target coding strategy, the current environment state is transformed through the transfer function to obtain the target environment state, and the corresponding system performance results are calculated through the reward function. The target environment state is taken as the new current environment state. The target encoding strategy is determined from the action space based on the current environment state corresponding to the state space. This process continues until the system performance result obtained by recalculating through the reward function meets the preset maximization condition. The latest target encoding strategy is then determined as the quantum computing encoding strategy corresponding to the task to be processed.

[0114] In some embodiments, when the decision unit 602 determines the target encoding strategy from the action space based on the current environment state corresponding to the state space, it may specifically include: Based on the current environment state corresponding to the state space, the target encoding strategy is determined from the action space using the Q-Learning reinforcement learning strategy. The Q-Learning reinforcement learning strategy is used to maximize the system performance result calculated by the reward function.

[0115] For example, the above reward function can be constructed based at least on the amount of qubits used, computation time, and computational accuracy.

[0116] In some embodiments, the decision unit 602 described above can also be used for: The task complexity is evaluated to obtain the complexity evaluation results.

[0117] Based on this, the decision-making unit 602 can construct a target Markov decision process for the task to be processed according to the above complexity assessment results.

[0118] In some embodiments, the decision unit 602 described above can also be used for: According to the quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is transformed into a linear combination of Pauli operators to obtain the Hamiltonian in Pauli representation. The Hamiltonian represented by Pauli can be used to compile the corresponding target quantum circuit, so as to perform the quantum computation corresponding to the task to be processed through the target quantum circuit.

[0119] In some embodiments, the initialization unit 601 described above can be specifically used for: When the task to be processed is a quantum computing task related to a many-body system, a corresponding crystal model is constructed for the task to be processed. This crystal model can be described based on the Hamiltonian corresponding to the task to be processed. Based on the task to be processed, the quantum cloud platform is configured with the corresponding configuration parameters; the configuration parameters may include at least the quantum initial state, evolution time and / or the number of time slices.

[0120] In some embodiments, the above-described calculation unit 603 may be specifically used for: By preprocessing the Hamiltonian represented by Pauli through classical calculation, the corresponding simplified Hamiltonian is obtained. Using the target quantum circuit, quantum computation corresponding to the simplified Hamiltonian is performed to obtain the corresponding output wave function.

[0121] In some embodiments, the measurement output unit 604 described above can be specifically used for: A predetermined reference substrate is obtained, and the output wavefunction is measured using the projection method corresponding to the reference substrate to obtain the corresponding measurement results; wherein, the reference substrate may include at least a Pauli substrate or a Hadamard substrate; Based on the measurement results, the output results corresponding to the task to be processed are calculated and output through the quantum cloud platform.

[0122] As can be seen, the Hamiltonian simulation computing device described in the above embodiments can further improve the accuracy and effectiveness of dynamically selecting the optimal encoding strategy by using Q-Learning reinforcement learning strategies, combined with indicators such as the number of qubits used, computation time, and computational accuracy. Furthermore, by adopting a quantum-classical hybrid computing framework, the complexity of quantum computing can be significantly reduced, further decreasing the demand for qubits, thereby helping to further optimize the overall computing framework and improve the task processing efficiency of the quantum cloud platform. In addition, multimodal measurement and analysis using different reference substrates can obtain different types of quantum state information as output results, thus flexibly meeting the multi-dimensional analysis needs of different tasks regarding material phase transitions, entanglement entropy, etc., which is beneficial for broadening the functionality of the quantum cloud platform.

[0123] Please see Figure 7 , Figure 7 This is a modular schematic diagram of an electronic device disclosed in an embodiment of this application. For example... Figure 7 As shown, the electronic device may include: Memory 701 storing executable program code; Processor 702 coupled to memory 701; The processor 702 can call the executable program code stored in the memory 701 to execute all or part of the steps in any of the reinforcement learning-based Hamiltonian simulation computation methods described in the above embodiments.

[0124] Furthermore, embodiments of this application disclose a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program enables a computer to execute all or part of the steps in any of the reinforcement learning-based Hamiltonian simulation computation methods described in the above embodiments.

[0125] Furthermore, this application further discloses a computer program product that, when run on a computer, enables the computer to execute all or part of the steps in any of the reinforcement learning-based Hamiltonian simulation computation methods described in the above embodiments.

[0126] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0127] The foregoing has provided a detailed description of a Hamiltonian simulation computation method, apparatus, and electronic device based on reinforcement learning disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A Hamiltonian simulation computation method based on reinforcement learning, characterized in that, Applied to a quantum cloud platform, the method includes: Initialization processing is performed on the task to be processed, which is the quantum computing task received by the quantum cloud platform; By using reinforcement learning, a Markov decision process is used to determine the quantum computing encoding strategy corresponding to the task to be processed. Based on the quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is subjected to operator transformation to obtain the Hamiltonian in Pauli representation. Perform quantum computation corresponding to the task to be processed on the Hamiltonian represented by the Pauli symbol; The obtained output wavefunction is measured, and the output result of the quantum cloud platform is determined based on the corresponding measurement results.

2. The method according to claim 1, characterized in that, The step of determining the quantum computing encoding strategy corresponding to the task to be processed through reinforcement learning and Markov decision processes includes: For the task to be processed, a target Markov decision process is constructed; wherein, the target Markov decision process includes a state space, action space, transition function and reward function corresponding to the task to be processed, and the action space includes multiple different quantum computing encoding strategies; Based on the target Markov decision process, a quantum computing encoding strategy that maximizes the reward function calculation result is determined from the action space through reinforcement learning, and is used as the quantum computing encoding strategy corresponding to the task to be processed.

3. The method according to claim 2, characterized in that, The quantum computing encoding strategy, which determines the reward function computation result from the action space through reinforcement learning based on the target Markov decision process, includes: Based on the current environment state corresponding to the state space, the target encoding strategy is determined from the action space; According to the target coding strategy, the current environment state is transformed by the transfer function to obtain the target environment state, and the corresponding system performance result is calculated by the reward function; The target environment state is taken as the new current environment state. The process of determining the target encoding strategy from the action space based on the current environment state corresponding to the state space is repeated until the system performance result recalculated by the reward function meets the preset maximization condition. The latest target encoding strategy is then determined as the quantum computing encoding strategy corresponding to the task to be processed.

4. The method according to claim 3, characterized in that, The step of determining the target encoding strategy from the action space based on the current environment state corresponding to the state space includes: Based on the current environment state corresponding to the state space, a target encoding strategy is determined from the action space using a Q-Learning reinforcement learning strategy. The Q-Learning reinforcement learning strategy is used to maximize the system performance result calculated by the reward function.

5. The method according to claim 3, characterized in that, The reward function is constructed based at least on the number of qubits used, computation time, and computational accuracy.

6. The method according to claim 2, characterized in that, Before constructing the target Markov decision process for the task to be processed, the method further includes: The task to be processed is evaluated for complexity, and the complexity evaluation result is obtained. The construction of the target Markov decision process for the task to be processed includes: Based on the complexity assessment results, a target Markov decision process is constructed for the task to be processed.

7. The method according to any one of claims 1 to 6, characterized in that, The step of performing operator transformation processing on the Hamiltonian corresponding to the task to be processed according to the quantum computing encoding strategy to obtain the Hamiltonian in Pauli representation includes: According to the quantum computing encoding strategy, the Hamiltonian corresponding to the task to be processed is transformed into a linear combination of Pauli operators to obtain the Hamiltonian in Pauli representation. The Hamiltonian represented by Pauli is used to compile the corresponding target quantum circuit, so as to perform the quantum computation corresponding to the task to be processed through the target quantum circuit.

8. The method according to any one of claims 1 to 6, characterized in that, The initialization process for the task to be processed includes: When the task to be processed is a quantum computing task related to a many-body system, a corresponding crystal model is constructed for the task to be processed, and the crystal model is described based on the Hamiltonian corresponding to the task to be processed. Based on the task to be processed, the quantum cloud platform is configured using corresponding configuration parameters; wherein, the configuration parameters include at least the quantum initial state, evolution time, and / or the number of time slices.

9. The method according to any one of claims 1 to 6, characterized in that, Performing quantum computation corresponding to the task to be processed on the Hamiltonian represented by the Pauli expression includes: The Hamiltonian expressed in Pauli is preprocessed by classical calculation to obtain the corresponding simplified Hamiltonian. Using the target quantum circuit, quantum computation corresponding to the simplified Hamiltonian is performed to obtain the corresponding output wave function.

10. The method according to any one of claims 1 to 6, characterized in that, The process of measuring the obtained output wavefunction and determining the output result of the quantum cloud platform based on the corresponding measurement results includes: A predetermined reference substrate is obtained, and the output wavefunction is measured using the projection method corresponding to the reference substrate to obtain the corresponding measurement results; wherein, the reference substrate includes at least a Pauli substrate or a Hadamard substrate; Based on the measurement results, the output result corresponding to the task to be processed is calculated and output through the quantum cloud platform.

11. A Hamiltonian simulation computing device based on reinforcement learning, characterized in that, The device, applied to a quantum cloud platform, includes: An initialization unit is used to perform initialization processing on a task to be processed, which is a quantum computing task received by the quantum cloud platform. The decision unit is used to determine the quantum computing encoding strategy corresponding to the task to be processed by using a Markov decision process through reinforcement learning, and to perform operator transformation processing on the Hamiltonian corresponding to the task to be processed according to the quantum computing encoding strategy to obtain the Hamiltonian in Pauli representation. The computing unit is used to perform quantum computation corresponding to the task to be processed on the Hamiltonian represented by the Pauli symbol; The measurement output unit is used to measure the obtained output wave function and determine the output result of the quantum cloud platform based on the corresponding measurement results.

12. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 10.

Images