Calibration method and apparatus for quantum computers based on large models and reinforcement learning

Abstract

This application provides a calibration method and apparatus for quantum computers based on large models and reinforcement learning, relating to the field of quantum computing technology. The method includes: S1: Based on the calibration round task and task objective, if a preset judgment index does not meet a preset termination condition, input historical interaction data into a large language model to obtain a decision basis vector; S2: Input the decision basis vector into a decision head to obtain the target action; S3: If the target action is an experimental action, execute the target action and obtain an observation report; S4: Record trajectory information and update historical interaction data, then return to step S1; S5: If the target action is a completed action, execute a preset standard verification process and save the trajectory information of the calibration round task to an experience replay pool; S6: Perform online reinforcement learning on the decision head based on the experience replay pool. This application achieves high-efficiency, high-quality calibration and possesses real-time adaptive capabilities.

Description

Technical Field

[0001] This application relates to the field of quantum computing technology, specifically to a calibration method and apparatus for a quantum computer based on large models and reinforcement learning. Background Technology

[0002] Currently, as the scale of superconducting quantum computers expands, their parameter stability is significantly affected by environmental noise, temperature fluctuations, and hardware drift, necessitating frequent and time-consuming parameter calibrations to maintain computing performance.

[0003] Traditional numerical optimization methods suffer from poor robustness, while data-driven machine learning methods heavily rely on data quality and coverage. Regarding automated process management frameworks, the current mainstream technology for qubit calibration is based on directed acyclic graphs (DAGs). However, the rigid constraints of its static topology and predefined dependencies result in poor robustness and require significant manual intervention. To address this issue, large language models (LLMs) are introduced to construct self-driven laboratory systems. However, the model parameters remain frozen after deployment, resulting in poor flexibility and impacting the final calibration accuracy.

[0004] Existing technologies cannot simultaneously achieve both accuracy and robustness. Summary of the Invention

[0005] Based on this, this application provides a calibration method and apparatus for quantum computers based on large models and reinforcement learning, achieving high-precision and highly robust calibration of quantum computers.

[0006] According to one aspect of this application, a calibration method for a quantum computer based on a large model and reinforcement learning is proposed, comprising: S1: Based on a pre-generated calibration round task and its corresponding task objective, if a preset judgment index does not meet a preset termination condition, inputting historical interaction data into a pre-trained large language model to obtain a decision basis vector; S2: Inputting the decision basis vector into a decision head to obtain the target action at the current time step, wherein the action space of the decision head includes multiple preset calibration actions, and the categories of the preset calibration actions include experimental actions and completion actions; S3: If the target action is an experimental action, executing the target action and obtaining the observation report at the current time step. S4: Record the trajectory information of the current time step, add the target action and observation report to the historical interaction data, and jump to step S1. The trajectory information includes the state representation of the current time step, the target action of the current time step, the single-step reward information, and the state representation of the next time step. The single-step reward information is calculated based on the preset reward function, and the state representation of the next time step includes the historical interaction data updated after the current time step. S5: If the target action is not a completed action, execute the preset standard verification process, obtain the verification result, and save the trajectory information of the calibration round task to the experience replay pool. S6: Perform online reinforcement learning on the decision head based on the experience replay pool.

[0007] According to some embodiments, the method further includes: completing the calibration round task if the verification result is successful; suspending the calibration round task and locking the relevant qubits if the verification result is unsuccessful; and generating a suspension report and / or triggering an alarm based on the suspended calibration round task; and / or the method further includes: suspending the calibration round task and locking the relevant qubits if a preset judgment index meets a preset suspension condition; and generating a suspension report and / or triggering an alarm based on the suspended calibration round task.

[0008] According to some embodiments, step S1 includes: if the preset judgment index does not meet the preset termination condition, generating prompt words based on historical interaction data, task objectives and candidate action lists, and obtaining the state representation of the current time step based on the prompt words; inputting the prompt words into a pre-trained large language model to obtain a decision basis vector.

[0009] According to some embodiments, the prompt words are input into a pre-trained large language model to obtain a decision basis vector, including: segmenting the prompt words based on the large language model to obtain segmentation results; encoding the segmentation results according to a predefined vocabulary of the large language model to obtain an encoding sequence; inputting the encoding sequence into the embedding layer of the large language model to obtain word embedding vectors and their positional codes, which serve as the input sequence; inputting the input sequence into the Transformer layer of the large language model to obtain a feature vector sequence; inputting the feature vector sequence into the output layer of the large language model to obtain a score vector; and using a masking technique to constrain the score vector to a candidate action list to obtain a decision basis vector.

[0010] According to some embodiments, the method further includes: extracting prior knowledge related to the task objective based on historical interaction data; encoding the prior knowledge based on a large language model to obtain prior knowledge encoding; extracting structured numerical features based on the latest observation report in historical interaction data; extracting historical observation features based on historical interaction data; performing temporal aggregation on the historical observation features to obtain observation feature aggregation results; obtaining a text description of the state corresponding to the current time step based on historical interaction data to obtain a text description embedding vector; obtaining a fusion feature for the current time step based on the prior knowledge encoding, structured numerical features, observation feature aggregation results, and / or text description embedding vector; and concatenating the fusion feature into the decision basis vector to obtain a new decision basis vector.

[0011] According to some embodiments, step S6 includes: sampling from the experience replay pool using a preset sampling strategy to obtain training samples; training the decision head using a preset strategy optimization algorithm and a preset reward function based on the training samples; and updating the decision head based on the training results.

[0012] According to some embodiments, the sampling strategies include: an experience replay strategy based on time decay weights, a clustering-based sampling strategy, and / or a priority experience replay strategy based on a fixed-capacity first-in-first-out queue.

[0013] According to one aspect of this application, a calibration device for a quantum computer based on a large model and reinforcement learning includes: a priori model unit for round-based tasks and their corresponding task objectives, which, when a preset judgment index does not meet a preset termination condition, inputs historical interaction data into a pre-trained large language model to obtain a decision basis vector; a real-time decision unit for inputting the decision basis vector into a decision head to obtain the target action at the current time step; wherein the action space of the decision head includes multiple preset calibration actions, the categories of which include experimental actions and completion actions; a calibration observation unit for executing the target action when the target action is an experimental action and obtaining an observation report at the current time step; and a data recording unit. The recording unit records the trajectory information of the current time step, adds the target action and observation report to the historical interaction data, and jumps to step S1. The trajectory information includes the state representation of the current time step, the target action of the current time step, the single-step reward information, and the state representation of the next time step. The single-step reward information is calculated based on a preset reward function, and the state representation of the next time step includes the updated historical interaction data of the current time step. The process control unit executes a preset standard verification process when the target action is not completed, obtains the verification result, and saves the trajectory information of the calibration round task to the experience replay pool. The reinforcement learning unit performs online reinforcement learning on the decision head based on the experience replay pool.

[0014] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described above.

[0015] According to one aspect of this application, a computer-readable medium is provided that stores a computer program or instructions thereon, which, when executed by a processor, implement the method as described above.

[0016] Through the embodiments provided in this application, during the calibration process, a termination check is first performed. If the preset judgment index does not meet the preset termination conditions, the termination check passes, and the agent composed of the large language model and the decision head executes its decision logic. Specifically, utilizing the powerful contextual understanding capability of the large language model, a decision basis vector for the current time step is generated based on historical interaction data. The decision head selects a target action from the action space based on the decision basis vector. If the target action is an experimental action, it is executed, and the agent interacts with the hardware environment to obtain the observation report for the current time step, while repeating the calibration steps. If the target action is a completion action, a preset standardized verification process is executed to finally confirm the calibration quality, and the trajectory information of the entire calibration round is saved to the experience replay pool. The experience replay pool is used to perform online reinforcement learning on the lightweight decision head. This application adopts a collaborative paradigm of providing stable prior knowledge from a large model and online optimization by a lightweight decision head for quantum computer calibration. The large model, as a cross-device knowledge base, remains frozen, providing general guidance for calibration strategies and achieving a balance between high efficiency and high quality calibration. The lightweight decision head, on the other hand, focuses on real-time adaptation to specific devices, ensuring stability while achieving efficient online optimization and enabling real-time adaptation in dynamic environments. Attached Figure Description

[0017] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings, without exceeding the scope of protection claimed by this application.

[0019] Figure 1 A flowchart illustrating a calibration method for a quantum computer based on large models and reinforcement learning, provided for embodiments of this application;

[0020] Figure 2 A flowchart of step S1 of the calibration method for a quantum computer based on large models and reinforcement learning provided in the embodiments of this application; Figure 3 A flowchart illustrating the large model inference decision-making basis vector for a calibration method for quantum computers based on large models and reinforcement learning, provided in an embodiment of this application; Figure 4 A flowchart of step S6 of the calibration method for a quantum computer based on large models and reinforcement learning provided in the embodiments of this application; Figure 5A flowchart illustrating the online reinforcement learning of the decision head in the calibration method for quantum computers based on large models and reinforcement learning, provided in this application embodiment; Figure 6 A flowchart illustrating the deployment architecture-based calibration method for quantum computers based on large models and reinforcement learning, as provided in embodiments of this application. Figure 7 A block diagram of a calibration device for a quantum computer based on large models and reinforcement learning, provided for embodiments of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0023] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0024] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0025] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0026] Superconducting quantum computers face serious parameter stability problems in actual operation. Factors such as ambient temperature fluctuations, electromagnetic noise, and material defects can cause key parameters such as the frequency and coupling strength of qubits to drift over time. This drift can significantly reduce the operational fidelity of quantum gates, thereby affecting the reliability and computational accuracy of the entire quantum computing system.

[0027] Currently, the calibration of quantum computers is mainly divided into two phases: the initial startup calibration phase and the routine maintenance calibration phase. The initial startup calibration steps include, but are not limited to, the following: 1) Scan the readout cavity frequency of each qubit: This step will initially determine the readout frequency and readout power of the qubit; 2) Scan the bias period of each qubit: This step will determine the adjustable bias range of the qubit; 3) Scan the excitation frequency of each qubit. This step will initially determine the excitation frequency of the qubit; 4) Conduct Rabi oscillation experiments: This step will preliminarily determine the qubit. The amplitude and duration of the pulse; 5) Conduct an IQ plane experiment to distinguish the 0 and 1 states of a quantum bit: This step will determine the criteria for distinguishing the 0 and 1 states of a quantum bit by reading back the electrical signal from the readout cavity; 6) Conduct experiments to optimize the criteria for distinguishing the 0 and 1 states of a quantum bit: This step includes optimizing the readout cavity frequency, readout power, and quantum bit excitation frequency. qubit Pulse amplitude and duration are parameters that can affect the readback electrical signal of the readout cavity; 7) Conduct experiments to calibrate the relaxation time of quantum bits: This step includes calibrating the transverse relaxation time of quantum bits. Longitudinal relaxation time Experiments such as; 8) Calibration experiment of tunable coupler: This step includes, but is not limited to, calibrating the modulation range of the tunable coupler, calibrating the spectrum of the tunable coupler, and calibrating the coupling strength between the tunable coupler and the quantum bit.

[0028] Routine maintenance and calibration experiments include, but are not limited to, the following steps: 1) Maintenance and calibration experiment for read fidelity: This step includes monitoring the parameter drift of the qubit read fidelity and the calibration round triggered after the drift is detected; 2) Maintenance and calibration experiment of single-qubit quantum gate: This step includes fidelity calibration of single-qubit quantum gate (fidelity measurement methods such as QPT (quantum process tomography), RB (randomization benchmark), XEB (cross-entropy benchmark), monitoring parameter drift of single-qubit quantum gate, and calibration rounds triggered after drift is detected; 3) Maintenance and calibration experiment of two-qubit quantum gate: This step includes fidelity calibration of the two-qubit quantum gate (fidelity measurement methods such as QPT (quantum process tomography), RB (randomization benchmark), XEB (cross-entropy benchmark), monitoring parameter drift of the two-qubit quantum gate, and calibration rounds triggered after drift is detected; 4) Maintenance and calibration experiments for specified quantum computer parameters: This step includes monitoring the specified quantum computer parameters and triggering a calibration round after a drift is detected.

[0029] During the initial startup calibration phase, after the quantum computer is first put into operation or undergoes major maintenance, all parameters of the chip are comprehensively calibrated using methods such as Bayesian optimization and grid search. This process usually takes several hours to several days and requires a large amount of experimental data. During the routine maintenance calibration phase, as the quantum computer is running, due to parameter drift, it is necessary to periodically fine-tune key parameters locally to restore its performance.

[0030] In recent years, the academic and industrial communities have explored various technical approaches to address the challenges of qubit parameter calibration.

[0031] Traditional numerical optimization methods use classical numerical optimization techniques such as gradient descent, Nelder-Mead, and genetic algorithms to search for optimal parameters in the early stages of calibration. However, these methods generally suffer from problems such as getting trapped in local optima and slow convergence speed in the high-dimensional, non-convex parameter space of quantum systems.

[0032] Machine learning-assisted calibration techniques are becoming increasingly sophisticated. Supervised learning uses massive amounts of historical calibration data to train models such as neural networks to predict the optimal values ​​of parameters. However, the performance of these methods is highly dependent on the quality and coverage of the training data.

[0033] Reinforcement learning treats the calibration process as a decision-making problem, optimizing the calibration strategy through real-time interaction between the decision agent and the hardware environment. However, in practice, it faces challenges such as high initial exploration costs and long learning cycles. In the high-dimensional parameter space of quantum systems, purely random exploration requires a large amount of experimental resources (such as sampling times and time), which is often unbearable in the actual operation and maintenance of quantum hardware.

[0034] In the actual implementation of the above calibration techniques, the mainstream workflow engine for qubit calibration is a calibration framework based on directed acyclic graphs (DAGs).

[0035] Calibration frameworks based on directed acyclic graphs (DAGs) are difficult to adapt to the dynamic changes of quantum systems due to the rigid constraints of static and predefined dependencies. These limitations include: their fixed topology cannot respond to sudden disturbances (such as sudden changes in electromagnetic noise or temperature fluctuations) in real time; they are difficult to effectively handle the cyclic dependencies that are common among qubit parameters (such as the mutual influence between frequency and coupling strength); and they severely lack cross-device generalization capabilities, which requires manual reconstruction on new devices and hinders large-scale deployment.

[0036] Based on this, this application proposes a calibration method and apparatus for quantum computers based on large models and reinforcement learning.

[0037] For specific implementation details, please refer to the following examples.

[0038] Figure 1 A flowchart illustrating a calibration method for a quantum computer based on large models and reinforcement learning, provided for embodiments of this application. Figure 1 As shown, the method includes steps S1-S6.

[0039] In step S1, based on the pre-generated calibration round task and its corresponding task objective, if the preset judgment index does not meet the preset termination condition, the historical interaction data is input into the pre-trained large language model to obtain the decision basis vector.

[0040] Understandably, parameter calibration is typically divided into two phases: initial bring-up and routine upkeep, with the latter requiring dynamic correction of parameter drift during processor operation. To address this need, this application proposes a collaborative architecture based on a Large Language Model (LLM) and a Lightweight Decision Head. The LLM and the Lightweight Decision Head together form an intelligent agent specifically designed to solve the in-situ upkeep calibration problem during continuous operation.

[0041] Specifically, calibration round tasks can be generated in various ways, including pre-built, directly input, or triggered by a signal, and this application does not impose any restrictions on this.

[0042] According to the example implementation, if a decrease in read fidelity is detected, a "maintenance calibration of read fidelity" task is triggered as a calibration round task; when the timer expires, a "routine check" task is triggered as a calibration round task; if the user manually specifies, a "calibration of specified parameters" task is triggered as a calibration round task.

[0043] When generating calibration round tasks in advance, task objectives need to be generated in conjunction with them. The task objectives are the goals that the corresponding calibration round tasks need to achieve, including calibration time, calibration content, etc., for example, "recalibrate the frequency within 20 steps".

[0044] According to the example embodiment, a calibration round task is triggered based on a parameter calibration signal, and a task objective is generated. The triggering conditions for the parameter calibration signal include key performance indicators deviating from a threshold and / or timer timeout. Based on the above embodiments, in some embodiments, key performance indicators are continuously monitored in the background. When an indicator deviates from a threshold, a calibration round task is automatically triggered, and a task objective is defined.

[0045] After obtaining the calibration round task and its corresponding task objective, the first step is to perform an abort check, which checks whether the current state meets the preset abort conditions. Abort conditions are used to ensure safety, prevent catastrophic errors, improve efficiency, and conserve resources.

[0046] The termination conditions can be set according to the situation, and this application does not impose any restrictions on them. These conditions may include key indicators reaching preset thresholds, receiving a failure flag from the algorithm, task timeout, and receiving external signals, etc., and this application does not exhaustively list them. The preset judgment indicators are the relevant indicators involved in the termination conditions.

[0047] If the preset judgment indicators do not meet the preset termination conditions, the termination check passes, and the agent composed of the large language model and the decision head executes its decision logic: inputting historical interaction data into the pre-trained large language model to obtain a decision basis vector. The large language model processes all historical interaction data to generate the decision basis vector for the current step. .

[0048] The historical interaction data includes target actions and observation report interaction tuples for all historical time steps.

[0049] In the method provided in this application, a pre-trained large language model can be selected as appropriate.

[0050] According to the example embodiment, a large model with autoregressive generation capability is selected as the large language model of this application. The large model with autoregressive generation capability is, for example, a decoder-only model based on the Transformer architecture, a sequence model based on the state space model (such as the Mamba architecture), or an improved large parameter model based on recurrent neural networks.

[0051] Structured prompts are constructed based on historical interaction data and input into a large language model. The large language model provides prior knowledge for completing the calibration round task based on historical interaction data and a general knowledge base within the model, and generates a decision-making basis vector.

[0052] In step S2, the decision basis vector is input into the decision head to obtain the target action at the current time step. The action space of the decision head includes multiple preset calibration actions, and the categories of the preset calibration actions include experimental actions and completion actions.

[0053] In existing technologies, to address the poor robustness and high human intervention requirements of calibration frameworks based on Directed Acyclic Graphs (DAGs), some studies have introduced Large Language Models (LLMs) to construct self-driven laboratory systems. While LLM-based self-driven laboratory systems improve cross-device adaptability by incorporating prior knowledge, their core architecture still has significant limitations. Existing solutions typically use LLMs as static knowledge bases, with model parameters frozen after deployment, preventing the system from learning and optimizing policies online based on real-time hardware feedback. This "knowledge solidification" characteristic makes it difficult for the system to effectively track the time-varying drift of quantum computer parameters—for example, when the coherence time of a qubit slowly decays due to material aging, an LLM trained on historical data may continuously recommend calibration schemes that are no longer applicable. More importantly, there is a matching bias between the general policy generated by the LLM and the individualized characteristics of a specific quantum computer. Due to the lack of an online adaptive mechanism for the target device, the output calibration recommendations often fail to adequately adapt to the unique physical characteristics of the specific hardware (such as parameter dispersion caused by manufacturing processes), thus affecting the final calibration accuracy. Furthermore, existing systems lack a reliability verification mechanism for LLM generation strategies, which may result in physically infeasible or inefficient operation sequences, leading to additional trial-and-error costs and time overhead in practical applications.

[0054] Based on this, this application introduces online reinforcement learning (RL) to construct a novel architecture that combines "a large language model representing the state at each step" with "a lightweight decision head performing online policy optimization." This makes the large model the sole "world model" that processes all complex information, while the decision head is a minimally simplistic policy layer that only selects from the high-level semantic space provided by the LLM. This leverages the powerful inference capabilities of the LLM while also enabling rapid adaptation to the real-time state of specific hardware through online learning.

[0055] The decision head receives prior information and real-time hardware status from the large model and outputs specific calibration actions. The decision head can be selected or constructed as needed.

[0056] According to the example implementation, the decision head can be replaced with any model suitable for online learning and fast reasoning, such as: 1) Linear models: such as logistic regression, linear discriminant analysis, etc.; 2) Shallow neural networks: such as multilayer perceptrons (MLP, usually no more than 3 layers); 3) Tree models: such as Gradient Boosting Decision Tree (GBDT), Random Forest, etc.; 4) Other lightweight trainable function approximators.

[0057] In actual implementation, lightweight decision-making head take over As input, and output the final policy distribution. Based on this strategy distribution, sample and select the target action for the current time step. : .

[0058] In some embodiments, temperature parameters may be introduced to regulate the sampling process in order to balance exploration and utilization.

[0059] Action space The multiple preset calibration actions can be set according to the situation.

[0060] In some embodiments, action space It includes all actions that an agent can learn and choose autonomously. It consists of two types of actions: experimental actions and finalized actions.

[0061] The experimental actions include several predefined non-complete actions, such as SPECTROSCOPY_WIDE_SCAN (wide-range spectral scan) and RABI_FINETUNE (Rabi oscillation fine tuning).

[0062] Finish Action: A special action that indicates the agent has determined the current calibration task has been successfully completed and is ready to end the round. The selected action will be executed by the actual calibration system, which will specify the specific calibration algorithm and perform measurement and control operations on the quantum hardware.

[0063] According to an example embodiment, multiple preset calibration actions are set in the action space of a routine maintenance and calibration experiment for a quantum computer.

[0064] Understandably, in this application, each action uniquely defines a calibration experiment category and its specific mode. Specifically, the calibration experiment category is related to the task objective the calibration round task aims to achieve, and the specific mode is implemented through the parameter configuration of the calibration experiment category. By using the calibration experiment category and its specific mode, the calibration path during the execution of each action is uniquely encapsulated, thereby achieving direct task scenario adaptation.

[0065] For example, RABI_WIDE (wide-range Rabi oscillation) and RABI_NARROW (narrow-range Rabi oscillation) both belong to the Rabi oscillation experiment category, but their scanning range and precision are different, and they are different actions.

[0066] In step S3, if the target action is an experimental action, the target action is executed and the observation report of the current time step is obtained.

[0067] If the target action If it is an experimental action, then execute the target action. It also interacts with the hardware to obtain a new observation report. .

[0068] In the specific execution process, according to the example embodiment, the target action (such as RABI_WIDE) is mapped to pulse waveform sequence parameters that can be recognized by the measurement and control system, and sent to the quantum chip through a digital-to-analog converter (ADC / DAC).

[0069] Understandably, the observation report It is structured data, the content of which corresponds to the target action to be performed.

[0070] Receive observation report Then, the state at the next moment can be obtained. And the single-step reward for this step .

[0071] It should be noted that the physical implementation of each start-up calibration step and maintenance calibration step can be achieved by calling the interface provided by the measurement and control system. The measurement and control system includes a database for storing qubit information, a database for storing experimental results, a compiler for controlling the wave order of qubits, a manager for connecting electronic devices and task scheduling, and various electronic devices, including arbitrary waveform generators (AWG), arbitrary function generators (AFG), digital-to-analog converters (ADC), clock generators, triggers, microwave sources, and DC sources.

[0072] In step S4, the trajectory information of the current time step is recorded, and the target action and observation report are added to the historical interaction data. Then, the process jumps to step S1. The trajectory information includes the state representation of the current time step, the target action of the current time step, the single-step reward information, and the state representation of the next time step. The single-step reward information is calculated based on a preset reward function. The state representation of the next time step includes the historical interaction data updated after the current time step.

[0073] Record the trajectory information of this iteration At the same time, the tuples from this interaction Add to historical interaction data, and continue in a loop. Among them, This represents the state at the current time step. The target action for the current time step. For single-step reward information, This serves as the state representation for the next time step. It should be noted that the single-step reward information is calculated based on a preset reward function, and the state representation for the next time step includes the updated historical interaction data from the current time step. The specific generation method will be explained in subsequent embodiments.

[0074] Then, jump to step S1 and repeat the iteration until the target action is completed.

[0075] In this application, a large language model LLM and a lightweight decision head are used. The intelligent agent, composed of LLM prior knowledge and decision heads, Its real-time decision-making capability enables it to quickly locate and correct drift parameters in a low-cost, low-shot manner, achieving efficient and accurate "fine-tuning" calibration.

[0076] In step S5, if the target action is not completed, a preset standard verification process is executed to obtain the verification result, and the trajectory information of the calibration round task is saved to the experience playback pool.

[0077] If the target action The FINISH action indicates that the calibration round has been successfully completed, and the loop terminates. At this point, a pre-defined standardized verification process (referred to as the pre-defined standard verification process) is executed to finally confirm the calibration quality and obtain the verification result.

[0078] After the calibration round is completed, the trajectory information of all time steps during the mission (i.e., the complete trajectory of the current round) will be collected. Updated to the experience replay pool.

[0079] In step S6, the decision head undergoes online reinforcement learning based on the experience replay pool.

[0080] A training dataset is constructed using an experience replay pool, and the decision head is subjected to online reinforcement learning.

[0081] The specific algorithm for reinforcement learning can be selected according to the actual situation, and this application does not impose any restrictions on it.

[0082] The decision head of this application parameters Optimization through online reinforcement learning enables the composite agent, consisting of an LLM and a decision head, to learn decision-making strategies that maximize long-term rewards while following the semantic guidance of the LLM.

[0083] In this application, the calibration process begins with a termination check. If a preset judgment index fails to meet the preset termination conditions, the termination check passes, and the agent composed of a large language model and a decision head executes its decision logic. Specifically, leveraging the powerful contextual understanding capabilities of the large language model, a decision basis vector for the current time step is generated based on historical interaction data. The decision head selects a target action from the action space based on this vector. If the target action is an experimental action, it is executed, and the agent interacts with the hardware environment to obtain the observation report for the current time step, while repeating the calibration steps. If the target action is a completion action, a preset standardized verification process is executed to ultimately confirm the calibration quality. The trajectory information of the entire calibration round is saved to an experience replay pool, which is used to perform online reinforcement learning on the lightweight decision head. This application employs a collaborative paradigm of a large model providing stable prior knowledge and a lightweight decision head performing online optimization for quantum computer calibration. The large model, as a cross-device knowledge base, remains frozen, providing general guidance for calibration strategies and achieving a balance between high efficiency and high quality calibration. The lightweight decision head, on the other hand, focuses on real-time adaptation to specific devices, ensuring stability while achieving efficient online optimization and enabling real-time adaptation in dynamic environments.

[0084] According to some embodiments, the method further includes steps S7-S9 and / or steps S11-S12.

[0085] In step S7, if the verification result is successful, the calibration round task is completed.

[0086] When the preset standard verification process passes, the verification result is "verification passed" and the round ends successfully.

[0087] In step S8, if the verification result is a verification failure, the calibration round task is aborted and the relevant qubits are locked.

[0088] If the verification fails in the preset standard verification process, the verification result is verification failure, the current task is forcibly terminated, and the relevant qubits are locked.

[0089] In step S9, an abort report is generated and / or an alarm is triggered based on the aborted calibration round task.

[0090] Generate a detailed abort report, which includes the complete context history of the failed round and the reason for abort.

[0091] At the same time, the alarm system can be triggered to push the abort report or task termination information to the on-duty quantum engineer and request manual intervention.

[0092] In step S11, if the preset judgment index meets the preset termination condition, the calibration round task is terminated and the relevant qubits are locked.

[0093] When the preset judgment index meets the preset termination condition, the termination check is determined to have failed, the calibration round task is immediately terminated, and the relevant qubits are locked.

[0094] It should be noted that the termination condition is a hard rule independent of the agent's policy, used to determine whether the calibration round task needs to be stopped.

[0095] According to the example embodiment, the preset judgment indicators include the number of round execution steps, the total round time, and / or the number of catastrophic degradation indicators. That is, during the calibration round task execution process, the number of round execution steps and the total round time are recorded, while key performance indicators are monitored.

[0096] Based on this, the abort conditions include the maximum threshold for the number of round execution steps, the maximum threshold for the total round time, and the maximum number of catastrophic degradation metrics. The maximum threshold for the number of round execution steps is used to determine whether the number of round execution steps exceeds the maximum limit (e.g., t max_steps), thereby avoiding resource overruns; the maximum threshold for the total round time is used to determine whether the total round time exceeds the maximum allowed time, thereby avoiding time overruns; the maximum number of catastrophic degradation metrics is used to determine whether there is a sharp and irreversible deterioration in key performance indicators. Typically, the maximum number of catastrophic degradation metrics is set to 1.

[0097] If any of the above termination conditions exist, the calibration round task will terminate immediately and the relevant qubits will be locked.

[0098] In step S12, an abort report is generated and / or an alarm is triggered based on the aborted calibration round task.

[0099] Upon aborting the calibration round, a detailed "abortion report" is generated, containing the complete context history of the failed round and the reason for abortion. Simultaneously, an alarm system can be triggered to push the abortion report or task termination information to the on-duty quantum engineer, requesting manual intervention.

[0100] According to some embodiments, refer to Figure 2 In step S1, based on the pre-generated calibration round task and its corresponding task objective, if the preset judgment index does not meet the preset termination condition, the historical interaction data is input into the pre-trained large language model to obtain the decision basis vector, which can be specifically implemented through steps S210-S220.

[0101] In step 210, if the preset judgment index does not meet the preset termination condition, a prompt word is generated based on the historical interaction data, task objective and candidate action list, and the state representation of the current time step is obtained based on the prompt word.

[0102] It should be emphasized that the core of the calibration method for quantum computers based on large models and reinforcement learning provided in this application is to model the quantum calibration process as a partially observable and non-stationary Markov decision process (POMDP) ​​guided by LLM and thereby construct a composite agent.

[0103] In this framework, the Large Language Model (LLM) serves as the core semantic understanding and reasoning engine, processing all available contextual information at each decision step and guiding a lightweight decision head to make the final action selection.

[0104] In the specific implementation process, all relevant information, including the current task objective, historical interaction data within the round, and the candidate action list, will be integrated into a structured text prompt.

[0105] The candidate action list includes currently available actions in the action space, which can be preset or generated by a large language model.

[0106] According to the example embodiment, in the decision step The process begins by dynamically gathering all relevant information to construct structured prompts. This includes: Current mission objective (GOAL): A macro-level decision objective that the LLM can understand (e.g., "recalibrate frequency in 20 steps"). Complete intra-round history (CALIBRATION_HISTORY): From the start of this round to the end of the current round. The text sequence of all (action, observation report) pairs for each step; The list of executable actions (ACTION_LIST) presents the selectable actions in the action space and their corresponding descriptions in a text list format.

[0107] Based on the collected information, text prompts are generated according to a preset template.

[0108] According to the example embodiment, the prompt word template is as follows: "Given the calibration history of a superconducting quantum processor: {CALIBRATION_HISTORY} The current goal is {GOAL}. What is the next best calibration action? Choose from the actionlist: [ACTION_LIST]. You only need to output the indicator of the action list item."

[0109] Furthermore, the state representation of the current time step can be obtained based on the prompt words. According to the example embodiment, calibration prior knowledge information, historical observation reports, previous round observation reports, the current step (including the state of the current time step in the entire calibration cycle), and a matching list of available action spaces and action numbers (i.e., a candidate action list) are extracted from the prompt words to obtain the state representation of the current time step. .

[0110] In step 220, the prompt words are input into a pre-trained large language model to obtain a decision basis vector.

[0111] The agent's policy is represented by a decision head with parameter θ. (Such as logistic regression or a linear layer) and a fixed LLM jointly determine that at each time step t, the LLM is based on the state representation. Generate decision basis vector The decision-making head is based on Output the final action probability distribution.

[0112] Lightweight decision head take over As input, and output the final policy distribution. .

[0113] Strategy Defined in state Lower selection action space The i-th action The probability of: .

[0114] Among them, the decision basis vector Large Language Model (LLM) analyzes the current state Subsequently, a constrained subset of the original Logits (log probabilities) corresponding to each candidate action is generated from its vocabulary. This vector constitutes the decision head. The input feature space.

[0115] This application's embodiments utilize a large model to generate a high-quality candidate action space, dynamically constraining the exploration scope of the reinforcement learning agent from a generalized policy space to a subset of high-potential policies. This significantly improves the agent's adaptability to different hardware states and drift patterns, substantially reduces the convergence time and trial-and-error costs of online learning, enabling the agent to make accurate decisions with very few experiments (low-shot), while significantly reducing the need for human intervention and the complexity of calibration strategy design, achieving a balance between high efficiency and high-quality calibration.

[0116] According to some embodiments, refer to Figure 3 In step 220, the prompt words are input into the pre-trained large language model to obtain the decision basis vector, which can be implemented through steps S310-S360.

[0117] In step 310, the prompt words are segmented based on the large language model to obtain the segmentation results.

[0118] The word segmenter of the large language model splits the text of the input prompt words into discrete units (Tokens) that the model can process. The token ID sequence obtained from the word segmentation is recorded as the word segmentation result.

[0119] In step 320, the word segmentation results are encoded according to the predefined vocabulary of the large language model to obtain the encoded sequence.

[0120] Based on the predefined vocabulary of the large language model, each token in the word segmentation result is mapped to a unique integer ID in the vocabulary to obtain an encoded sequence.

[0121] In step 330, the encoded sequence is input into the embedding layer of the large language model to obtain the word embedding vector and its positional encoding of the encoded sequence, which is used as the input sequence.

[0122] The resulting encoded sequence is fed into the embedding layer. The embedding layer is essentially a lookup table that converts each ID into a word embedding vector (e.g., with dimensions of 1024 or 2048).

[0123] Meanwhile, in order to avoid losing the order information of each token and to facilitate identification in subsequent processes, positional encoding is added to the word embedding vector to form an input sequence containing order information.

[0124] In step 340, the input sequence is fed into the Transformer layer of the large language model to obtain a sequence of feature vectors.

[0125] The input sequence is fed into the Transformer layer of the large language model. After processing by the attention mechanism, the vector representation in the input sequence is dynamically adjusted according to the context information in the prompt words, resulting in the final hidden state vector of each position in the input sequence after deep processing, which is denoted as the feature vector sequence.

[0126] In step 350, the feature vector sequence is input into the output layer of the large language model to obtain the score vector.

[0127] Based on the feature vector sequence, prediction is made and projected onto a vector of a predefined vocabulary size dimension (e.g., 50,000). The result of the projection is the Logits vector (i.e., the score vector).

[0128] This application's embodiments utilize a parameter-fixed LLM to infer the aforementioned prompt words and restrict it to outputting only one token, thereby obtaining the original Logit vector on the complete vocabulary. .

[0129] In step 360, the score vector is constrained to the candidate action list using masking technology to obtain the decision basis vector.

[0130] After obtaining the Logit vector, a masking technique is used to constrain the Logit vector to only those actions in the candidate action list. The decision-making basis vector is generated by using the tokens corresponding to each predefined candidate action. .

[0131] According to the example implementation, the score corresponding to the non-candidate action token in the Logit vector is set to negative infinity using the masking technique, thereby ensuring that subsequent decisions will only be generated from the predefined K candidate actions.

[0132] This application addresses the structural limitations of frameworks such as directed acyclic graphs (DAGs) in handling complex couplings (especially circular dependencies) between parameters by leveraging the powerful contextual understanding capabilities of large models for implicit modeling. By encoding physical rules and diagnostic knowledge in structured prompts, it enables more flexible and accurate handling of complex dependencies between parameters based on deep causal reasoning rather than relying solely on pre-defined static flowcharts.

[0133] According to some embodiments, the method further includes steps S211-S218.

[0134] In step S211, prior knowledge related to the task objective is extracted based on historical interaction data.

[0135] In step S212, prior knowledge is encoded based on the large language model to obtain prior knowledge encoding.

[0136] In step S213, structured numerical features are extracted based on the latest observation report in the historical interaction data.

[0137] In step S214, historical observation features are extracted based on historical interaction data.

[0138] In step S215, the historical observation features are aggregated over time to obtain the aggregated observation features.

[0139] In step S216, based on historical interaction data, the text description of the state corresponding to the current time step is obtained, and the text description embedding vector is obtained.

[0140] In step S217, the fused features of the current time step are obtained based on prior knowledge encoding, structured numerical features, observed feature aggregation results and / or text description embedding vectors.

[0141] In step S218, the fused features are concatenated into the decision basis vector to obtain a new decision basis vector.

[0142] Specifically, the decision-making basis vector output by the large language model Based on this, one or more of the following elements can be selected for representation and fusion to obtain fused features, which can then be concatenated to the decision basis vector. In the middle, a decision-making basis vector with fusion features will be added. As input to the decision head, the new decision basis vector By integrating more dimensions of information from historical interaction data, more accurate action selection can be achieved.

[0143] According to the example embodiment, the extraction of fused features can be combined with the decision basis vector. The inference is executed in parallel.

[0144] The fusion features include one or more of the following elements: prior knowledge encoding extracted from historical text reports and processed by LLM (such as Logits vectors); structured numerical features extracted from the latest observation reports; temporal aggregation results of historical observation features (such as through RNN encoding); and textual description embedding vectors of the current state.

[0145] According to some embodiments, refer to Figure 4 In step S6, the decision head is subjected to online reinforcement learning based on the experience replay pool, which can be implemented through steps S410-S420.

[0146] In step S410, training samples are obtained by sampling from the experience replay pool using a preset sampling strategy.

[0147] An experience replay mechanism employing a pre-defined sampling strategy samples from an experience replay pool to obtain training samples. The settings of the experience replay mechanism can determine the emphasis of the decision head on recent calibration experiences, thereby better tracking and adapting to real-time changes in different environments.

[0148] In step S420, the decision head is trained using a preset strategy optimization algorithm and a preset reward function based on the training samples, and the decision head is updated based on the training results.

[0149] Based on the training samples, the decision head is optimized using a pre-defined strategy algorithm. parameters Optimization is then performed. The core idea of ​​this type of algorithm is to use the reward signals obtained from the interaction between the agent and the environment to update the policy in order to maximize long-term returns.

[0150] The reward function aims to guide the agent to achieve efficient calibration by combining sparse end targets with dense process costs. The reward function can be selected based on the specific circumstances.

[0151] According to the example embodiment, the single-step reward obtained by the agent at each time step t is defined. : .

[0152] Here, -1 represents a fixed number of steps penalty, encouraging the agent to complete the task as quickly as possible. It is the step cost. It is a terminal reward.

[0153] It is a dense penalty, corresponding to the resources consumed in performing the action in step t: .

[0154] in, and These represent the number of experiments and the time consumed in step t, respectively.

[0155] It is a sparse reward that is non-zero only in the last step of the round (let's call it T): .

[0156] in, To evaluate the calibration results, , As a penalty for failure, It is usually in the range of 10 to 100.

[0157] The reinforcement learning algorithm will use this single-step reward sequence. To evaluate the agent's performance throughout the round and drive policy optimization.

[0158] The preset strategy optimization algorithm can also be selected according to the actual situation, for example: 1) Policy Gradient Methods: such as REINFORCE or its variants with a learnable baseline.

[0159] 2) Actor-Critic Methods: such as the synchronous dominant actor-critic algorithm (A2C) or the asynchronous dominant actor-critic algorithm (A3C).

[0160] 3) Proximal Policy Optimization (PPO) algorithm: By introducing a clipping mechanism into the objective function, the step size of policy updates is limited, thus ensuring the stability of the learning process.

[0161] 4) Maximum Entropy Reinforcement Learning Algorithms: such as the Soft Actor-Critic (SAC) algorithm, which encourages agents to explore more fully by adding a policy entropy regularization term to the optimization objective.

[0162] 5) Advantage-Weighted Regression (AWR) Algorithm: This approach is a simpler and more stable alternative in engineering practice. It cleverly transforms the policy optimization problem into a weighted supervised learning (classification) problem. Its core idea is as follows: First, the advantage function A(s,a) for each state-action pair is calculated using historical experience data. This value measures how much the reward from performing action a in state s exceeds the average expected reward. Then, an exponential weight w(s,a) = exp(A(s,a) / β) is calculated based on the advantage value, where β is a temperature hyperparameter. Finally, the decision head is trained using this weight to minimize the weighted cross-entropy loss.

[0163] In one specific embodiment, the Proximal Policy Optimization (PPO) algorithm is selected as the preset policy optimization algorithm.

[0164] The Proximity Policy Optimization (PPO) algorithm takes the following form: .

[0165] in, The core shear loss term of PPO is calculated using the following formula: .

[0166] In the formula, Indicates the strategy ratio, Based on rewards The generalized advantage estimate (GAE) is calculated. It utilizes a single-step reward sequence. Calculations show that the state is efficiently evaluated. Take action The degree of superiority or inferiority relative to the average performance.

[0167] in, KL divergence penalty term, constraining the decision head strategy The original proposed distribution of LLM Maintain consistency.

[0168] in, Constrained decision basis vector output by LLM The result obtained using Softmax is: .

[0169] in, : KL penalty coefficient, used to balance the weight between policy optimization and prior knowledge preservation.

[0170] Specifically, the online training process refers to Figure 5 Based on the policy optimizer, the loss function and gradient are calculated using the sampled training samples. The parameters of the decision head are updated based on the calculation results. The updated decision head is used for the decision loop of the new round of calibration task, generating new trajectory information (experience) and storing it in the experience replay pool. The decision head is repeatedly subjected to online reinforcement learning using the data in the experience replay pool.

[0171] This application embodiment achieves accurate action decisions by fusing prior suggestions from a large model with hardware feedback signals in real time through online reinforcement learning. This ensures that while maintaining high decision-making efficiency, it can quickly adapt to real-time state changes of quantum hardware.

[0172] According to some embodiments, the sampling strategy includes: Experience replay strategies based on time decay weights, clustering-based sampling strategies, and / or priority experience replay strategies based on fixed-capacity first-in-first-out queues.

[0173] It is understood that the embodiments of this application store historical data on the interaction between the agent and the environment by establishing an experience replay pool. During the training process, the decision head randomly samples batches of historical experience from the experience replay pool for learning. This method can break the temporal correlation between data, improve data utilization efficiency, and enhance the stability of training.

[0174] Different sampling strategies can focus on recent calibration experience in different dimensions.

[0175] In this embodiment, the experience replay strategy based on time decay weights ensures that the agent's learning focus is always on the latest experience that best reflects the current dynamics of the environment, while not completely abandoning earlier experiences, thus better adapting to non-stationary quantum computing environments. The clustering-based sampling strategy effectively ensures that each batch of training samples covers different data patterns, enhancing data diversity and better responding to dynamic environmental changes. The priority experience replay strategy based on a fixed-capacity first-in-first-out queue breaks uniform sampling, allowing the decision head to learn the latest experience more frequently, significantly improving learning efficiency.

[0176] According to the example embodiment, an experience replay strategy based on time decay weight is selected as the sampling strategy, and a time-weighted experience replay pool is adopted. It stores the complete episode trajectory. .

[0177] When updating the strategy, from The probability of sampling the trajectory throughout the entire calibration round follows an exponential decay: ; in, It is the forgetting factor. n is the latest round index.

[0178] This mechanism enables the agent to optimize its calibration strategy in real time based on the latest hardware interaction data and to prioritize recent state changes, thereby possessing efficient online tracking and real-time adaptive capabilities, effectively compensating for parameter drift caused by environmental changes, and maintaining the long-term performance stability of the quantum computer.

[0179] The calibration method for quantum computers based on large models and reinforcement learning provided in this application adopts an architecture of "freezing the large model to provide general priors and using a lightweight decision head for online adaptation". This design enables rapid cross-device calibration knowledge transfer by utilizing a pre-trained large model, while completing the "last mile" personalized adaptation through online learning for specific devices, significantly reducing the cycle and cost of deploying and maintaining calibration schemes for new devices.

[0180] This application provides a general automated decision-making framework, whose application is not limited to a specific calibration process, and can be adapted to: 1) Calibration tasks at different levels: such as single-qubit parameter (frequency, readout cavity parameters) calibration, two-qubit coupling strength and quantum gate parameter calibration, etc.

[0181] 2) Different triggering mechanisms: It can be perfectly adapted to event-driven calibration triggered by abnormal performance indicators (such as a decrease in fidelity) or periodic maintenance calibration triggered by timers.

[0182] Furthermore, the deployment of the calibration method for quantum computers based on large models and reinforcement learning provided in this application is referenced. Figure 6 .

[0183] The following describes an apparatus embodiment of this application, which can be used to perform the method embodiment of this application. For details not disclosed in the apparatus embodiment of this application, please refer to the method embodiment of this application.

[0184] Figure 7 A block diagram of a calibration apparatus for a quantum computer based on large model and reinforcement learning, according to an exemplary embodiment, is shown.

[0185] Figure 7 The apparatus shown can perform the calibration method for a quantum computer based on large models and reinforcement learning according to the embodiments of this application.

[0186] like Figure 7 As shown, a calibration device for a quantum computer based on large models and reinforcement learning may include: See Figure 7 Referring to the preceding description, the prior model unit 710 is used for round tasks and their corresponding task objectives. When the preset judgment indicators do not meet the preset termination conditions, the historical interaction data is input into the pre-trained large language model to obtain the decision basis vector.

[0187] The real-time decision unit 720 is used to input the decision basis vector into the decision head to obtain the target action at the current time step; wherein, the action space of the decision head includes multiple experimental actions and completed actions.

[0188] The calibration observation unit 730 is used to execute the target action when the target action is the experimental action and obtain the observation report of the current time step.

[0189] The data recording unit 740 is used to record the trajectory information of the current time step, add the target action and observation report to the historical interaction data, and jump to step S1. The trajectory information includes the state representation of the current time step, the target action of the current time step, the single-step reward information, and the state representation of the next time step. The single-step reward information is calculated based on a preset reward function, and the state representation of the next time step includes the historical interaction data updated after the current time step.

[0190] The process control unit 750 is used to execute a preset standard verification process when the target action is not completed, obtain the verification result, and save the trajectory information of the calibration round task to the experience playback pool.

[0191] Reinforcement learning unit 760 is used to perform online reinforcement learning on the decision head based on the experience replay pool.

[0192] The device performs functions similar to those described above; other functions are described in the preceding descriptions and will not be repeated here.

[0193] This application discloses an electronic device, including: a processor; and a memory storing a computer program, which, when executed by the processor, causes the processor to execute the above-described instruction generation method.

[0194] For example, refer to Figure 8 , Figure 8 The illustrated electronic device 800 includes a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, for example, via a bus 802. Alternatively, the electronic device 800 may also include a transceiver 804. It should be noted that in practical applications, the transceiver 804 is not limited to one type, and the structure of this electronic device 800 does not constitute a limitation on the embodiments of this application.

[0195] Processor 801 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in this application. Processor 801 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0196] Bus 802 may include a pathway for transmitting information between the aforementioned components. Bus 802 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 802 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0197] The memory 803 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other storage medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0198] The memory 803 stores application code that executes the scheme of this application, and its execution is controlled by the processor 801. The processor 801 executes the application code stored in the memory 803 to implement the content shown in the foregoing method embodiments.

[0199] Figure 8 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0200] This application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, causes the processor to execute an instruction generation method.

[0201] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0202] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A calibration method for a quantum computer based on a large model and reinforcement learning, characterized in that, include: S1: Based on the pre-generated calibration round task and its corresponding task objective, if the preset judgment index does not meet the preset termination condition, the historical interaction data is input into the pre-trained large language model to obtain the decision basis vector. S2: Input the decision basis vector into the decision head to obtain the target action at the current time step, wherein the action space of the decision head includes multiple preset calibration actions, and the categories of the preset calibration actions include experimental actions and completion actions; S3: If the target action is an experimental action, execute the target action and obtain the observation report of the current time step; S4: Record the trajectory information of the current time step, add the target action and the observation report to the historical interaction data, and jump to step S1. The trajectory information includes the state representation of the current time step, the target action of the current time step, the single-step reward information, and the state representation of the next time step. The single-step reward information is calculated based on a preset reward function, and the state representation of the next time step includes the historical interaction data updated after the current time step. S5: If the target action is completed, execute the preset standard verification process, obtain the verification result, and save the trajectory information of the calibration round task to the experience playback pool. S6: Based on the experience replay pool, perform online reinforcement learning on the decision head.

2. The method of claim 1, wherein, The method further includes: If the verification result is successful, the calibration round task is completed; If the verification result is a failure, the calibration round is aborted, and the relevant qubits are locked; and Generate an abort report and / or trigger an alarm based on the aborted calibration round task; and / or The method further includes: If the preset judgment index meets the preset termination condition, the calibration round task is terminated, and the relevant qubits are locked; and A termination report and / or an alarm are generated based on the terminated calibration round task.

3. The method of claim 1, wherein, Step S1 includes: If the preset judgment indicators do not meet the preset termination conditions, a prompt word is generated based on the historical interaction data, the task objective, and the candidate action list, and the state representation of the current time step is obtained based on the prompt word. The prompt words are input into a pre-trained large language model to obtain a decision basis vector.

4. The method of claim 3, wherein, The prompt words are input into a pre-trained large language model to obtain a decision basis vector, including: Based on the large language model, the prompt words are segmented to obtain the segmentation results; The word segmentation results are encoded based on the predefined vocabulary of the large language model to obtain an encoded sequence; The encoded sequence is input into the embedding layer of the large language model to obtain the word embedding vector and its position encoding, which is used as the input sequence. The input sequence is fed into the Transformer layer of the large language model to obtain a feature vector sequence; The feature vector sequence is input into the output layer of the large language model to obtain the score vector; By using masking techniques, the score vector is constrained to the candidate action list to obtain the decision basis vector.

5. The method of claim 1, wherein, The method further includes: Based on the historical interaction data, extract prior knowledge related to the task objective; Based on the large language model, the prior knowledge is encoded to obtain the prior knowledge encoding; Based on the latest observation report in the historical interaction data, structured numerical features are extracted; Based on the historical interaction data, extract historical observation features; The historical observation features are aggregated over time to obtain the aggregated observation features. Based on the historical interaction data, obtain the text description of the state corresponding to the current time step, and obtain the text description embedding vector; The fusion features of the current time step are obtained based on the prior knowledge encoding, the structured numerical features, the observed feature aggregation results, and / or the text description embedding vector. The fused features are concatenated into the decision basis vector to obtain a new decision basis vector.

6. The method of claim 1, wherein, Step S6 includes: Training samples are obtained by sampling from the experience replay pool using a preset sampling strategy; Based on the training samples, the decision head is trained using a preset strategy optimization algorithm and the preset reward function, and the decision head is updated based on the training results.

7. The method according to claim 6, characterized in that, The sampling strategy includes: Experience replay strategies based on time decay weights, clustering-based sampling strategies, and / or priority experience replay strategies based on fixed-capacity first-in-first-out queues.

8. A calibration device for a quantum computer based on large models and reinforcement learning, characterized in that, include: The prior model unit is used for round tasks and their corresponding task objectives. When the preset judgment indicators do not meet the preset termination conditions, the historical interaction data is input into the pre-trained large language model to obtain the decision basis vector. A real-time decision unit is used to input the decision basis vector into the decision head to obtain the target action at the current time step; wherein, the action space of the decision head includes multiple preset calibration actions, and the categories of the preset calibration actions include experimental actions and completion actions; A calibration observation unit is used to execute the target action when the target action is an experimental action, and to obtain an observation report at the current time step; A data recording unit is used to record the trajectory information of the current time step, add the target action and the observation report to the historical interaction data, and jump to step S1. The trajectory information includes the state representation of the current time step, the target action of the current time step, single-step reward information, and the state representation of the next time step. The single-step reward information is calculated based on a preset reward function, and the state representation of the next time step includes the historical interaction data updated after the current time step. The process control unit is used to execute a preset standard verification process when the target action is not completed, obtain the verification result, and save the trajectory information of the calibration round task to the experience playback pool. A reinforcement learning unit is used to perform online reinforcement learning on the decision head based on the experience replay pool.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-7.

Images