Controlling a magnetic confinement apparatus using a control system trained via reinforcement learning

CN123035598APending Publication Date: 2026-11-03GDM HOLDING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611013877.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2025-07-08
Filing Date
2026-07-08
Publication Date
2026-11-03

AI Technical Summary

Technical Problem

Existing magnetic confinement device control systems are complex and inflexible in design, making it difficult to efficiently adapt to changes in plasma state, especially under high temperature and high pressure conditions. Conventional magnetic controllers require a great deal of engineering effort and expertise to adjust control strategies.

Method used

The control system trained using reinforcement learning technology, combined with linear magnetic controllers and neural magnetic controllers, can autonomously learn the optimal control strategy and achieve multi-input multi-output closed-loop control. It is applicable to various magnetically constrained devices, reducing design workload and improving flexibility.

Benefits of technology

It achieves efficient magnetic confinement of plasma under high temperature and high pressure conditions, reduces design complexity and computational cost, improves the versatility and flexibility of the control system, and enables seamless switching between linear and nonlinear control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123035598A_ABST
    Figure CN123035598A_ABST
Patent Text Reader

Abstract

The present disclosure relates to controlling a magnetic confinement apparatus using a control system trained via reinforcement learning. Corresponding methods, systems, and devices are disclosed, including computer programs encoded on computer storage media. Examples of the control system include a linear magnetic controller, a neural magnetic controller, and a monitoring module augmented with a user interface, the control system for magnetically confining a plasma within a magnetic confinement apparatus and monitoring the plasma during magnetic confinement. The linear magnetic controller and the neural magnetic controller are configured to operate under respective linear and nonlinear magnetic control regimes of the plasma. The monitoring module is configured to monitor a triggering condition of the plasma, the triggering condition initiating a switch between the linear magnetic controller and the neural magnetic controller, to transmit monitoring information to the user interface, and to receive user input from the user interface that modifies control of the plasma. Implementations of the control system can be applied to controlled thermonuclear fusion, advanced plasma research, nuclear safety testing, and other applications.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 840,488, filed July 8, 2025. The disclosure of the prior application is considered part of the disclosure of this application and is incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to methods, systems, and apparatus for controlling magnetically confined devices using a control system trained via reinforcement learning, including computer programs encoded on a computer storage medium. Background Technology

[0004] This disclosure relates to control systems for magnetic confinement devices, such as toroidal magnetic confinement devices like tokamas and stellarators. A magnetic confinement device is a system, such as a nuclear fusion system, configured to use a magnetic field to confine a high-temperature plasma, for example, to generate electricity via thermonuclear fusion of that plasma.

[0005] This disclosure also relates to using machine learning models to process data. Machine learning models receive input and generate outputs, such as predicted outputs, based on the received input. Some machine learning models are parametric models and generate outputs based on the received input and the values ​​of the model's parameters. Some machine learning models are deep models that employ multiple layers to generate outputs for received inputs. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers, each of which applies a nonlinear transformation to the received input to generate an output. Summary of the Invention

[0006] This specification describes a control system implemented as a computer program on one or more computers at one or more locations, which can generate control sequences for a magnetic confinement device to magnetically confine plasma therein, wherein the control system has been trained on a model of the magnetic confinement device via reinforcement learning techniques.

[0007] Generally, the magnetic confinement device includes: a chamber configured to support a vacuum for a plasma; a set of control coils configured to generate a magnetic field confining the plasma within the chamber; a power supply for the control coils; and a set of sensors configured to measure the plasma when it is confined by the magnetic field within the chamber. To perform magnetic control of the plasma, the control system is configured to: receive a physical measurement trajectory of the plasma collected by the sensors; process the physical measurement trajectory, conditioned on a target state trajectory of the plasma, to generate a control sequence for the control coils; and transmit the control sequence to the magnetic confinement device to generate a magnetic field confining the plasma within the chamber. The target state trajectory may specify a sequence of magnetic configurations of the plasma that varies with the control sequence, which the control system uses to guide the plasma toward, for example, a series of diverting configurations that forcefully confine the plasma within a specific region of the chamber.

[0008] The control systems described in this paper include examples of linear magnetic controllers. Linear magnetic controllers are analytically tractable. Furthermore, because linear magnetic controllers can be analyzed using standard linear algebraic and linear control theory methods, they can provide valuable information about the equilibrium dynamics of plasmas, such as the stability conditions and sensitivities of plasmas under certain linear observation and control strategies.

[0009] Examples of control systems described in this paper include linear magnetic controllers, neural magnetic controllers, and monitoring modules with added user interfaces. Using such a configuration, the control system can generate control sequences using optimal (or near-optimal) control strategies that maintain the plasma in (or near) equilibrium, adaptively correct for unexpected deviations from equilibrium, and switch control between the linear and neural magnetic controllers if the plasma meets triggering conditions—e.g., indicating that statistical health indicators meet thresholds. This control system can be implemented to magnetically confine plasma to an equilibrium state undergoing sustained thermonuclear fusion, corresponding to plasma temperatures ranging from approximately ten million to one hundred million degrees Celsius. Here, the magnetic confinement device is configured to generate electrical energy via thermonuclear fusion of the plasma while simultaneously confining it, a process known as magnetic confinement fusion (“MCF”). For example, the magnetic confinement device could be a state-of-the-art magnetic confinement fusion reactor with a net energy output—i.e., a Q value greater than 1.

[0010] To learn the optimal control strategy, the linear magnetic controller and the neural magnetic controller have each been trained, either jointly or independently, on a model of the magnetically confined device using reinforcement learning techniques. For example, the reinforcement learning techniques could be actor-commentator reinforcement learning, REINFORCE learning, or Dyna-Q reinforcement learning. In the described example, the model of the magnetically confined device includes a state evolution model of the plasma magnetically coupled to the set of control coils and a diagnostic model of the set of sensors, wherein the state evolution model is derived from a magnetohydrodynamic model of the plasma from a power source model magnetically coupled to the set of control coils. This magnetohydrodynamic model can be a balanced magnetohydrodynamic model, such as the Grad-Shafranov equations, describing the magnetic configuration of the plasma under a given plasma current and control current in each control coil. The power source model can be a circuit model, such as a lumped-element circuit model, describing the evolution of the control current, the electrical properties of each control coil, and the inductive coupling of the set of control coils to the plasma.

[0011] This paper details examples of magnetically confined devices configured as tokamaks, where the control system has been trained on a robust model of the tokamak to learn an optimal strategy for tokamak control. In these examples, the tokamak includes: (i) a toroidal chamber comprising a shell and a set of plasma-oriented components (“PFC”) positioned therein, such as a first wall and a divertor; (ii) a set of axisymmetric control coils, including a central solenoid, poloidal field, and toroidal field control coils; and (iii) a set of sensors, including magnetic field, magnetic flux, voltage, and current sensors. The tokamak model can accurately represent each of these aspects in simulation, thus providing a high-fidelity platform for tokamak reconstruction and control using reinforcement learning. (See “Magnetic control of tokamak plasmas through deep reinforcement learning” by Degrave, J., Felici, F., Buchli, J., et al.) Nature 602, 414–419 (2022) and Tracey, Brendan D. et al., “Towardspractical reinforcement learning for tokamak magnetic control” Fusion Engineering and Design 200 (2024): 114161 provides additional examples of reinforcement learning for tokamak control. Both of these papers are incorporated herein by reference in full for all purposes.

[0012] The subject matter described in this specification can be implemented in specific embodiments to achieve one or more of the following advantages.

[0013] This specification provides a control system for a magnetically confined device, which is trained on a model of the device using reinforcement learning techniques. The model simulates plasma evolution, magnetic equilibrium, power supply, sensing instruments, and plasma and sensor noise, thereby providing a high-fidelity reinforcement learning environment for the magnetic reconstruction and control of the plasma.

[0014] Furthermore, because the control system utilizes reinforcement learning, it can be configured as a closed-loop (i.e., feedback) control system for any magnetic confinement device. That is, the control system can autonomously learn optimal (or near-optimal) control strategies to efficiently command the unique set of control coils of the magnetic confinement device, thus significantly reducing design effort compared to conventional magnetic controllers. For example, the control system can be implemented as a single, computationally inexpensive magnetic control system for the magnetic confinement device, replacing the complex nested control architecture of conventional magnetic controllers. By specifying the control objective at a high level, this approach offers unprecedented flexibility and versatility, shifting the focus to what the magnetic confinement device should accomplish, rather than how it can accomplish it.

[0015] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the description, drawings, and claims.

[0016] By incorporating via reference

[0017] All publications, patents, and patent applications mentioned in this specification are incorporated herein by reference, as if each individual publication, patent, or patent application were individually and separately indicated to the extent of their inclusion herein. In the event of any publication, patent, or patent application incorporated by reference that contradicts the disclosure contained herein, this specification is intended to supersede and / or give precedence to any such contradictory material. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating an example of a magnetic confinement system, including a magnetic confinement device and a control system for controlling the magnetic confinement device.

[0019] Figures 2A to 2B This is a cross-sectional view depicting an example of a magnetic confinement device configured as a tokamak.

[0020] Figure 3A This is a schematic diagram illustrating an example of a control system that includes a linear magnetic controller, a neural magnetic controller, and a monitoring module with an added user interface.

[0021] Figure 3B This is a schematic diagram illustrating an example of a state observer used to perform a state observation protocol within a control loop.

[0022] Figure 3C This is a schematic diagram illustrating an example of a monitoring module used to execute a monitoring protocol within a control loop.

[0023] Figure 4A This is a flowchart of an example process for controlling a magnetically restrained device via a control system that includes a linear magnetic controller, a neural magnetic controller, and a monitoring module with an added user interface.

[0024] Figure 4B This is a flowchart of an example process for executing a control protocol via a linear magnetic controller that includes a state observer and a state feedback controller.

[0025] Figure 4C This is a flowchart of an example process for executing a state observation protocol via a state observer.

[0026] Figure 4D This is a flowchart of an example process for executing a monitoring protocol via a monitoring module with an added user interface.

[0027] Figure 4E This is a flowchart of an example process for executing a switching protocol via a neuromagnetic controller.

[0028] Figure 5A This is a schematic diagram illustrating an example of a training system configured to train a magnetic controller on a model of a magnetically confined device via reinforcement learning techniques.

[0029] Figure 5B This is a flowchart of an example process for training a magnetic controller on a model of a magnetically confined device using reinforcement learning techniques.

[0030] Similar reference numerals and names in the various figures indicate similar elements. Detailed Implementation

[0031] I. Introduction

[0032] Controlled thermonuclear fusion, the fundamental process behind fusion reactors and other thermonuclear systems, is a promising solution for sustainable energy. Fusion reactors—such as magnetically confined fusion reactors—can generate electricity using the products of fusion reactions occurring in a hot plasma and produce very little radioactive waste. Neutron fusion reactors convert neutrons emitted from the plasma into heat, which can then be used in a thermodynamic cycle, for example, for rotating steam turbines, like conventional fission reactors. Neutronless fusion reactors have the potential for even greater efficiency because they can generate electricity directly from charged particles emitted from the plasma, such as alpha particles. That said, one of the most challenging problems in achieving controlled thermonuclear fusion is confining the high-temperature, high-pressure plasma within a suitable chamber so that energy can be safely extracted from the plasma. Typically, due to the extreme temperatures (e.g., tens to hundreds of millions of degrees Celsius), the plasma must not be in direct contact with any surface of the chamber shell but must be suspended in a vacuum within the chamber, and the inherent instability of the plasma further complicates this problem.

[0033] However, since plasma is an ionized gas, it can conduct electricity and generate magnetic fields, and in turn, can be manipulated by magnetic fields. Magnetic confinement devices—such as toroidal magnetic confinement devices like tokamaks and stellarators—utilize time-varying arrangements of magnetic fields to shape, confine, and control plasma in various magnetic potentials. A tokamak is a toroidal magnetic confinement device in which the plasma is confined into a toroidal axisymmetric structure (e.g., a donut shape), with a helical magnetic field consisting of a toroidal magnetic field generated by axisymmetric control coils, and a poloidal magnetic field generated by toroidal plasma currents induced within the plasma. A stellarator, on the other hand, is a toroidal magnetic confinement device in which the helical magnetic field is entirely generated by non-axisymmetric control coils, and the plasma contains little or no internal plasma current.

[0034] Examples of tokamaks currently deployed or under development include the International Thermonuclear Experimental Reactor (“ITER”), the Joint European Ring (“JET”), the Experimental Advanced Superconducting Tokamak (“EAST”), the Variable Configuration Tokamak (“TCV”), and SPARC, which is being developed by the Federal Fusion Systems Corporation (“CFS”). Examples of other types of magnetic confinement devices currently deployed or under development include spherical tokamaks, such as the megaampere spherical tokamak (“MAST”); stellarators, such as the Wendelstein 7-X and Helias designs (e.g., “HELIUS” and “HELIOS”); spherical marks, such as the MS-stellar and spherical mark plasma experiments (“SPP”); field inversion configurations (“FRC”), such as the Princeton field inversion configuration (“PFRC”) and the variable field FRC (“VF-10”); magnetically targeted fusion (“MTF”) devices, such as Z-pinch devices (e.g., the Z-machine at Sandia National Laboratories) and LTX (“LTX-β”) devices; and reverse field pinch (“RFP”) devices, such as the RFX-Mod and Pegasus.

[0035] Conventional magnetic controllers typically employ a set of independent single-input single-output (“SISO”) proportional-integral-derivative (“PID”) controllers to address the high-dimensional, high-frequency, nonlinear problems of plasma confinement. These controllers regulate various configuration parameters of the plasma. This set of PID controllers is designed to avoid mutual interference and is often further enhanced by an external control loop that enables real-time estimation of plasma equilibrium. While these magnetic controllers are successful in some cases, they involve considerable engineering effort and expertise whenever the target plasma configuration changes. Furthermore, these magnetic controllers are typically designed for each magnetic confinement device and its unique set of controls, such as the unique number, arrangement, and magnetic field strength of a particular set of control coils, which can be a daunting task as successive generations of magnetic confinement devices come online.

[0036] To overcome some or all of the challenges described above, this specification provides a control system for a magnetic confinement device, which is trained via reinforcement learning on a model of the magnetic confinement device. The control system may be a multiple-input multiple-output (“MIMO”) control system, comprising a linear magnetic controller, a neural magnetic controller, and a monitoring module that operate collaboratively with each other in the control loop. The control system can operate under both linear and nonlinear magnetic control mechanisms (e.g., for equilibrium and nonequilibrium magnetic configurations of plasma), seamlessly switching control when transitioning between linear and nonlinear magnetic control mechanisms, and monitoring plasma instabilities, system dynamics, and other triggering conditions for initiating switching. The implementation of the control system may also be configured with a user interface, for example, by adding a monitoring module to facilitate real-time user monitoring and control of the magnetic confinement device.

[0037] Because the control system is trained via reinforcement learning, it can be configured as a synchronous linear and nonlinear feedback controller for any magnetically confined device. That is, the control system can autonomously learn optimal (or near-optimal) linear and nonlinear control strategies to efficiently command the unique set of control coils of the magnetically confined device, thus significantly reducing design effort and improving functionality compared to conventional magnetic controllers. For example, this control system can be implemented as a single, computationally inexpensive magnetic control system for the magnetically confined device, replacing the complex nested control architecture of conventional magnetic controllers, while also facilitating nonlinear magnetic control that conventional controllers cannot perform. Because the control objective is specified at a high level, the implementation of this control system can be unprecedentedly flexible and versatile, shifting the focus to what the magnetically confined device should accomplish, rather than how it can accomplish it.

[0038] These and other features related to the magnetic confinement device, control system, linear magnetic controller, neural magnetic controller, monitoring module, user interface, and reinforcement learning techniques presented herein are described in more detail below.

[0039] II. Examples of Magnetic Confinement Systems

[0040] Figure 1 This is a schematic diagram depicting an example of a magnetic confinement system 10, such as a magnetic confinement fusion system. The magnetic confinement system 10 includes a magnetic confinement device 200 and a control system 100 for controlling the magnetic confinement device 200. For example, the control system 100 may be an integrated magnetic control system of the magnetic confinement device 200, or a magnetic control module of the integrated control system of the magnetic confinement device 200.

[0041] At a high level, the magnetic confinement device 200 includes a chamber 210, a set of control coils 220, a power supply 225 for the control coils 220, and a set of sensors 230. The magnetic confinement device 200 is configured to generate plasma 300 within the chamber 210, magnetically confine the plasma 300 within the chamber 210 using the set of control coils 220 powered by the power supply 225, and measure the plasma 300 magnetically confined within the chamber 210 using the set of sensors 230. In the described example, the magnetic confinement device 200 is configured as a toroidal magnetic confinement device, and the plasma 300 has a toroidal configuration. For example, the magnetic confinement device 200 may be a tokamak, stellarator, field inversion configuration (“FRC”), reverse field pinch (“RFP”), spherical mak device, or compact toroidal (“CT”) device. An example of a magnetic confinement device 200 configured as a tokamak is referenced in Section III. Figure 2A-2B Described.

[0042] In some implementations, the magnetic confinement device 200 is a magnetically confined fusion reactor configured to generate electricity via thermonuclear fusion of plasma 300. That is, the magnetic confinement device 200 is configured to convert the products of a fusion reaction occurring within the magnetically confined plasma 300 into electrical energy; this is referred to as magnetic confinement fusion (“MCF”). For neutron thermonuclear fusion of plasma 300, the reaction product is neutrons, which the magnetic confinement device 200 converts into heat for indirect generation of electricity via a thermodynamic cycle, such as a Rankine cycle. For neutronless thermonuclear fusion of plasma 300, the reaction product is charged particles (e.g., alpha particles), which the magnetic confinement device 200 directly converts into electrical energy. The magnetic confinement device 200 can establish and maintain the conditions involved in fusion reactions within the plasma 300 via heating, current driving, and particle fueling, for example, including combinations of neutral beam injection (“NBI”), electron cyclotron resonance heating (“ECRH”), ion cyclotron resonance heating (“ICRH”), low hybrid current driving (“LHCD”), gas injection, and / or projectile injection into the plasma 300.

[0043] The chamber 210, also referred to as a "container," is configured to support a vacuum 300 of plasma generated therein. This vacuum is a low-pressure environment, for example, an extremely low-pressure environment. The chamber 210 may have a maximum pressure of approximately 10. -5 10 -6 10 -7 10 -8 10 -9 The background vacuum pressure is Torr or lower, or within any two of these values. Chamber 210 can have a maximum of about 10. -2 10 -3 10 -410 -5 The operating vacuum pressure is Torr or lower, or within any two of these values. The background vacuum pressure refers to the pressure within chamber 210 after residual gases (e.g., air and / or residual fuel gases) have been evacuated from chamber 210. The background vacuum pressure of chamber 210 can be established using one or more vacuum pumps, such as cryogenic pumps, turbopumps, and / or diffusion pumps. After vacuum is established, fuel gases can be introduced into chamber 210, for example, using one or more gas injectors, to generate plasma 300 via its decomposition. Examples of fuel gases include helium-3 and hydrogen isotopes (e.g., deuterium and tritium). The operating vacuum pressure refers to the pressure of residual gases within chamber 210 surrounding plasma 300 during its magnetic confinement.

[0044] A set of control coils 220, also referred to as a "set of magnetic actuators," is arranged around, within, or in combination with the chamber 210. This set of control coils 220 may include at least approximately 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, or more control coils 220, or a range between any two of these values. (Indexed as...) c Each control coil 220.c includes a corresponding winding having a pair of terminals. The control coil 220.c is configured to operate under a corresponding voltage across the winding terminals ( V c —This is called the "control voltage" —and a corresponding current is conducted around its windings. I c This is referred to as the "control current". In some implementations, the set of control coils 220 is a set of high-temperature superconducting ("HTS") control coils. For example, each control coil 220 in the set of control coils 220 may be composed of a high-temperature superconductor such as rare-earth barium copper oxide ("REBCO"), bismuth strontium calcium copper oxide (e.g., Bi-2223 or Bi-2212), or yttrium barium copper oxide ("YBCO").

[0045] The set of control coils 220 is configured to generate a magnetic field (B) that confines the plasma 300 within the chamber 210 according to a control sequence (U) 145.1-N generated by the control system 100. Given U = The control sequence 145.1-N includes the corresponding control action for the group of control coils 220 at each of the multiple time steps. u n 145.n. Given as The control action 145.n includes the target voltage configuration vector for the group of control coils 220. This specifies the target value of the corresponding control voltage across each control coil 220.c. n Index each time step in the sequence. It is the total number of time steps in the control sequence 145.1-N. It is the control rate of the control sequence, and T This is the total duration of the control sequence. The control sequence 145.1-N can have a control rate of at least approximately 1 kHz, 2 kHz, 3 kHz, 4 kHz, 5 kHz, 6 kHz, 7 kHz, 8 kHz, 9 kHz, 10 kHz, 20 kHz, 30 kHz, 40 kHz, 50 kHz, 60 kHz, 70 kHz, 80 kHz, 90 kHz, 100 kHz or higher, or within any two of these values. The control sequence 145.1-N may have a total duration of at least approximately 10 seconds (“s”), 20 s, 30 s, 40 s, 50 s, 60 s, 70 s, 80 s, 90 s, 100 s, 200 s, 400 s, 500 s or more, or within any two of these values. T ).

[0046] Given The magnetic field comprises the following magnetic superposition: (i) the plasma current in plasma 300 (if present). I p The generated plasma magnetic field (b) p (ii) by the corresponding control current in each control coil 220.c I c The corresponding coil magnetic field (b) generated c For example, the control system 100 can sense plasma currents by generating a control sequence that causes periodic pulses of corresponding control currents in a subset of control coils 220 (e.g., a set of central solenoid coils 220.S). When a control action 145.n is received from the control system 100, the power supply 225 is configured to apply a corresponding control voltage with a corresponding target value specified by the control action 145.n to each control coil 220.c in the set of control coils 220.

[0047] The set of sensors 230 is arranged around, within, or in combination with the chamber 210. The set of sensors 230 may include at least approximately 25, 50, 75, 100, 150, 200, 250, 300, 350, 400, 450, 500, or more sensors 230, or within any two of these values. The set of sensors 230 is a set of real-time sensors. The set of sensors 230 is configured to generate a physical measurement trajectory (Y) 122.y.1-N of plasma 300 during control sequence 145.1-N. The physical measurement trajectory 122.y.1-N includes the corresponding physical measurements of plasma 300 collected by the group of sensors 230 at each time step in the control sequence 145.1-N. y n ) 122.yn.

[0048] The sensor group 230 includes: (i) A set of magnetic field probes, 230 M, used to measure the index. m The corresponding diagnostic field position (r) of each magnetic field probe at 230 μm m The amplitude and direction of the magnetic field B at point ), wherein the set of magnetic field probes 230.M may include at least approximately 25, 50, 75, 100, 150, 200, 250 or more magnetic field probes 230.M, or within any two of these values; (ii) A set of 230.F flux loops is used to measure the index. f The corresponding diagnostic flux position (r) for each flux loop 230.Ff f The flux of the magnetic field at point () The set of flux loops 230.Ff may include at least about 25, 50, 75, 100, 150, 200, 250 or more flux loops 230.F, or within any two of these values; (iii) A set of voltage sensors 230.V for measuring the corresponding control voltage across each control coil 220.c. V c The group of voltage sensors 230.V includes a corresponding voltage sensor 230.Vc for each control coil 220.c; and (iv) A set of current sensors 230.A for measuring the corresponding control current in each control coil 220.c I c The group of current sensors 230.A includes a corresponding current sensor 230.Ac for each control coil 220.c.

[0049] Given The physical measurements of plasma 300 122.yn include: (i) The physical field configuration vector of the magnetic field collected by the magnetic field probe 230.M. ), where the physical field configuration vector specifies the location of each magnetic field probe 230.Mm in its corresponding diagnostic field (r m The physical values ​​of the amplitude and direction of the collected magnetic field; (ii) The physical flux configuration vector of the magnetic field flux collected by the magnetic ring 230.F. ), where the physical flux configuration vector specifies for each flux loop 230.Ff the flux loop 230.Ff at its corresponding diagnostic flux position (r f The corresponding physical value of the flux of the magnetic field collected at point (). (iii) The physical voltage configuration vector of the control coil 220 collected by the group of voltage sensors 230.V. V n The physical voltage configuration vector specifies the physical value of the corresponding control voltage across the control coil 220.c collected by the corresponding voltage sensor 230.Vc for each control coil 220.c; and (iv) The physical current configuration vector of the control coil 220 collected by the group of current sensors 230.A I n The physical current configuration vector specifies the corresponding physical value of the corresponding control current in the control coil 220.c collected by the corresponding current sensor 230.Ac for each control coil 220.c.

[0050] The physical measurement trajectory 122.y.1-N characterizes the physical state trajectory 112.1-N of plasma 300 established during the control sequence 145.1-N. Given X = { x 1 ,x 2 ,…,x N The physical state trajectory 112.1-N includes the corresponding physical state of plasma 300 at each time step in the control sequence 145.1-N. x n ) 122.yn, where the corresponding physical measurement 122.yn of plasma 300 characterizes the physical state 122.yn at that time step. Given as The physical state 112.n of plasma 300 includes: (i) the physical plasma configuration vector of plasma 300 ( (ii) the physical current configuration vector of the control coil 220. The physical plasma configuration vector is a set of configuration parameters that define the magnetic configuration of plasma 300. C Each configuration parameter in ) specifies the corresponding physical value of the configuration parameters of plasma 300.

[0051] In general, this set of configuration parameters ( C This includes plasma shape parameters, plasma control parameters, and / or magnetic configuration parameters that specify the geometric and physical properties of the magnetic configuration of plasma 300. For example, this set of configuration parameters may include one or more of the following: the (e.g., circumferential) plasma current of plasma 300. I p ), plasma safety factor 300 ( q ), the position (r0) of the center 305 of plasma 300, and the elongation ratio of plasma 300 ( ), the triangularity of plasma 300 ( ), plasma 300 (e.g., small) radius ( r 0), or the position of the limiting point of plasma 300 within chamber 210 (r lim The elongation ratio, triangularity, and radius of plasma 300 are plasma shape parameters that specify the geometry of the poloidal cross section of plasma 300, and affect its stability, confinement, and overall performance.

[0052] In some implementations, for example, for the divertor configuration of plasma 300, this set of configuration parameters also includes: the profile of the boundary line 310.lcfs defining plasma 300, the total number of x-points 315 within chamber 210, and the corresponding position (r) of each x-point 315 within chamber 210. x This set of configuration parameters can also include the corresponding path for each of the corresponding pair of divertor branches 325.1 and 325.2 for each x-point 315. Note that, to specify paths, lines, and profiles, the plasma configuration vector can include a set of discrete points along a path, line, or profile. For example, as... Figure 2A As shown, discrete points delineate the polar cross sections of each of the divertor branches 325.1 and 325.2 at the boundary line 310.lcfs and x point 315.

[0053] The control system 100 is communicatively coupled to the magnetic confinement device 200, for example, via a wired communication channel (e.g., cable and / or fiber optic communication channel) and / or a wireless communication channel (e.g., Wi-Fi, Bluetooth, 5G, infrared, and / or microwave communication channel). The control system 100 is configured to magnetically control the plasma 300 by actuating the set of control coils 220, for example, to establish a target magnetic configuration of the plasma 300 having a target plasma current, target position, and target shape within the boundary 310.1cfs. For example, sustained thermonuclear fusion can proceed while the plasma 300 is in an equilibrium magnetic configuration. Several aspects of the plasma 300 and the magnetic confinement device 200 itself can also be studied in the equilibrium state, such as the stability of the plasma 300, the heat and particle emissions of the plasma 300, the degradation of plasma-facing components, and the degradation of the set of control coils 220 and / or the set of sensors 230; this information may be useful for research and development.

[0054] Control system 100 is a closed-loop control system that performs closed-loop (i.e., feedback) control of plasma 300. Control system 100 typically assumes control of plasma 300 when switching from an open-loop (i.e., feedforward) control system that has already driven plasma 300 to its initial state. x 0) 112.0, for example, the initial equilibrium state. The open-loop control system can execute a fixed initiation sequence before the control system 100 takes control to maintain the plasma 300 in the initial state 112.0. For example, the initiation sequence may include: (i) Plasma ignition To convert fuel gas into plasma 300; (ii) Plasma current ramp-up To increase the plasma current of plasma 300 to an operational level (e.g., approximately one million amperes or more); and (iii) Plasma heating This is to heat the plasma to fusion-related temperatures (e.g., approximately ten million to one hundred million degrees Celsius).

[0055] At a higher level, the control system 100 is configured to: receive from the magnetic confinement device 200 the physical measurement trajectory 122.y.1-N of the plasma 300 collected by the set of sensors 230; process the physical measurement trajectory 122.y.1-N, while conditioned on the target state trajectory 122.x.1-N of the plasma 300, to generate a control sequence 145.1-N for the set of control coils 220; and transmit the control sequence 145.1-N to the magnetic confinement device 200 to generate a magnetic field that confines the plasma 300 within the chamber 210.

[0056] Given The target state trajectory 122.x.1-N includes the corresponding target state of plasma 300 at each time step in the control sequence 145.1-N. 122.xn. Given as The target state 122.xn of plasma 300 includes: (i) the target plasma configuration vector of plasma 300 ( (ii) the target value of each configuration parameter in a set of configuration parameters for the specified plasma 300; and (ii) the target current configuration vector of the set of control coils 220. This specifies the corresponding target value of the corresponding control current in each control coil 220.c. The target state trajectory 122.x.1-N serves as a predefined state sequence of the plasma 300 towards which the control system 100 guides the plasma 300. That is, the target state trajectory 122.x.1-N represents the expected state trajectory of the plasma 300, thereby providing the control system 100 with a reference to be adjusted to the physical measurement trajectory 122.y.1-N.

[0057] At each time step in the control sequence 145.1-N, the control system 100 is configured to execute a control loop to generate a corresponding control action 145.n at that time step. Generally, the control system 100 is configured to: receive system input 102.n, including: (i) the target state 122.xn of the plasma 300 at that time step, and (ii) the physical measurement 122.yn of the plasma 300 at that time step; process the system input 102.n at that time step to generate the control action 145.n at that time step; and transmit the control action 145.n to the magnetic confinement device 200 at that time step to generate a magnetic field for confining the plasma 300. The control system 100 then proceeds to the next time step (… n + 1) and the next time step ( n + 2) Repeat the control loop, etc. An example of control system 100 is referenced in Section IV. Figures 3A to 4E It was described in more detail.

[0058] III. Examples of Magnetic Confinement Devices

[0059] Figures 2A to 2B This is a cross-sectional view depicting an example of a magnetic confinement device 200 configured as a tokamak. Figure 2A This is a vertical pole section view of the magnetic confinement device 200. Figure 2B This is a horizontal annular cross-sectional view of the magnetic confinement device 200.

[0060] Here, chamber 210 is defined with respect to radial coordinates ( r ), azimuth coordinates ( ) and vertical coordinates ( z The annular chamber is defined in cylindrical coordinates. The chamber 210 has a central void 212 (or "plasma-free central region"), a rotational central axis 211 extending through the central void 212, and a magnetic axis 305 surrounding the central void 212, defined, for example, by the center 305 of the plasma 300. In other words, when in... When rotated in the direction, chamber 210 has circumferential axisymmetry about the central axis 211, and the central axis 211 is along... z directional extension.

[0061] The chamber 210 may have a large radius of at least about 1 meter (“m”), 2 m, 3 m, 4 m, 5 m, 6 m, 7 m, 8 m, 9 m, 10 m or greater, or within any two of these values. R 0). The large radius of chamber 210 refers to the radial distance from the central axis 211 to the center 305 of plasma 300. Chamber 210 may have a small radius of at least about 0.50 m, 0.75 m, 1.00 m, 1.25 m, 1.50 m, 1.75 m, 2.00 m or greater, or within any two of these values. r 0). The small radius of chamber 210 refers to the radius of the poloidal section of plasma 300, measured from the center 305 of plasma 300 to the boundary line 310.lcfs defining plasma 300. Chamber 210 may have an aspect ratio of at least about 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0 or greater, or within any two of these values. A Given as A = R 0 / r The aspect ratio of chamber 210 is the ratio of the larger radius to the larger radius of chamber 210.

[0062] The chamber 210 includes a housing 213 and a set of plasma-facing components (“PFC”) located within the housing 213, the set of plasma-facing components including a first wall 214 and a divertor 216.

[0063] The outer shell 213 is the outermost layer of the chamber 210, directly facing the outside of the chamber 210. The outer shell 213 has a D-shaped poloidal cross section. In this example, the set of magnetic field probes 230.M and the set of flux loops 230.F are positioned within the chamber 210, between the first wall 214 of the chamber 210 and the outer shell 213. In other examples, one or more of the magnetic field probes 230.M and / or one or more of the flux loops 230.M may be positioned around the chamber 210, for example, between the outer shell 213 and the set of control coils 220.

[0064] The first wall 214 has a D-shaped poloidal cross section consistent with the outer shell 213. The first wall 214 is the innermost layer of the chamber 210, directly facing the plasma 300. The first wall 214 defines the boundary between the plasma 300 and the outer shell 213, thereby preventing direct contact between them. During operation, the first wall 214 can withstand the strong heat flux generated by the plasma 300, for example, during transient events such as rupture or edge local mode (“ELM”). The first wall 214 can be designed to manage this heat flux and withstand steady-state thermal loads. For example, the first wall 214 may comprise an array of tiles composed of and / or coated with one or more high-temperature resistant materials, such as tungsten, beryllium, carbon-based composites (e.g., graphite or carbon fiber composites), advanced alloys (e.g., steel alloys), or combinations thereof.

[0065] Divertor 216 is located at the bottom of housing 213. Divertor 216 includes three divertor target plates 218.1, 218.2, and 218.3, each configured to absorb excess heat flux discharged from plasma 300. For example, each of divertor target plates 218.1-3 can be configured to absorb at least about 10 megawatts per square meter ("MW / m²"). 2 11 MW / m 2 12 MW / m 2 13MW / m 2 14 MW / m 2 15 MW / m 2 16 MW / m 2 17 MW / m 2 18 MW / m 2 19 MW / m 2 20 MW / m 2 Or higher, or within a range of any two of these values, corresponding to the heat flux. Divertor 216 can also be configured to absorb the flux of plasma particles and neutral atoms (or "neutral particles") emitted from plasma 300.

[0066] In implementations of the magnetic confinement device 200 involving neutron thermonuclear fusion in plasma 300, chamber 210 may further include a cladding, such as a solid-breeding cladding or a liquid-breeding cladding. In these cases, the cladding is positioned between the first wall 214 and the outer shell 213, for example, lining the inner surface of the outer shell 213. The cladding is configured to absorb neutrons produced by the neutron fusion reaction in plasma 300 and convert their kinetic energy into heat for generating electrical energy. In implementations using deuterium-tritium (“DT”) fuel, tritium is consumed by plasma 300 but is not abundant in nature. In some implementations, the cladding includes lithium, which undergoes a nuclear reaction with neutrons to produce tritium, thereby ensuring a sustainable fuel cycle. The cladding also protects other components of the magnetic confinement device 200—such as a set of control coils 220 and / or a set of sensors 230—from neutron and gamma radiation, thereby reducing damage and extending the lifespan of the magnetic confinement device 200.

[0067] In cylindrical coordinates, given B = B P + B T The magnetic field includes the poloidal component (B). P ) and circumferential component (B T Given as The poloidal component of a magnetic field is called the "polonic magnetic field," which includes the radial component of the magnetic field. B r ) and the perpendicular component of the magnetic field ( B z Given as The circumferential component of the magnetic field is called the "circumferential magnetic field," which includes the azimuth component of the magnetic field. The magnetic field may have an amplitude of at least approximately 5 Tesla (“T”), 6 T, 7 T, 8 T, 9 T, 10 T, 11 T, 12 T, 13 T, 14 T, 15 T, 16 T, 17 T, 18 T, 19 T, 20 T or greater at magnetic axis 305, or within any two of these values.

[0068] The flux of the magnetic field can be specified with respect to a flux surface 310, which is defined by a loop in a poloidal plane surrounding the magnetic axis 305, the poloidal plane being orthogonal to the magnetic axis 305, wherein the rotating poloidal loop... Rotation in direction. The flux through the polar magnetic field of the band extending from the magnetic axis 305 to the flux surface 310 ( This is called "polonic magnetic flux". The flux of the circumferential magnetic field through the poloidal cross section of the flux surface 310—e.g., defined by a surface bounded by a poloidal ring—is called "polonic magnetic flux". This is referred to as "toroidal magnetic flux". Therefore, a set of flux surfaces 310 with constant pole magnetic flux—for example, a discrete or continuous set of flux surfaces 310—provides pole magnetic flux expressed in radial and vertical coordinates. and circumferential flux expressed in terms of poloidal flux. .

[0069] The closed flux surfaces 310.cfs are a subset of flux surfaces 310 that are closed and confined by the chamber 210 to the plasma 300. The open flux surfaces 310.ofs are a subset of flux surfaces 310 that extend outside the chamber 210. The boundary line 310.lcfs, also known as the “last closed flux surface” (or “LCFS”), is the outermost of the closed flux surfaces 310.cfs that confine the plasma 300, defining the boundary that separates the closed flux surfaces 310.cfs from the open flux surfaces 310.ofs. Here, the magnetic configuration of the plasma 300 is a divertor configuration, such as a single-zero divertor configuration. The magnetic field includes an X point 315 located on the boundary line 310.lcfs, which is a specific point where the magnetic field lines intersect and change direction, forming a characteristic “X” shape. The region outside the boundary 310.lcfs is called the scraping layer (“SOL”), in which plasma 300 interacts with the first wall 214 and divertor 216. Point x 315 includes a pair of divertor branches 320.1 and 320.2, which terminate at corresponding impact points 325.1 and 325.2 on the divertor target plates 218.1, 218.2 and 218.3 of divertor 216.

[0070] The control coil set 220 includes a central solenoid coil 220.S, a poloidal field coil 220.P, and a circumferential field coil 220.T. This set of control coils 220 is circumferentially axisymmetric about the central axis 211 of the chamber 210 and has a geometric configuration similar to that of the International Thermonuclear Experimental Reactor (“ITER”). While the circumferentially axisymmetric configuration is described in detail herein, the geometric configuration of this set of control coils 220 may differ for different configurations of the magnetic confinement device 200—such as a stellarator configuration and other configurations described below.

[0071] The group of central solenoid coils 220.S comprises seven central solenoid coils 220.S.1 to 220.S.7. More generally, the group of central solenoid coils 220.S may include at least approximately 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or more central solenoid coils 220.S, or within any two of these values. The group of central solenoid coils 220.S is stacked on top of each other along the central axis 211 within the central gap 212. The group of central solenoid coils 220.S forms a central solenoid 222 surrounding and extending along the central axis 211. The central solenoid 222 is configured to induce a circumferential plasma current flowing parallel to the magnetic axis 305 within the plasma 300. I p The circumferential plasma current generates (mostly) a poloidal magnetic field, for example, causing... .

[0072] The group of poloidal field coils 220.P comprises seven poloidal field coils 220.P.1 to 220.P.7. More generally, the group of poloidal field coils 220.P may include at least approximately 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or more poloidal field coils 220.P, or within any two of these values. Each poloidal field coil 220.P is positioned in a corresponding circumferential plane orthogonal to the central axis 211 and surrounds the central axis 211 within the circumferential plane. The group of poloidal field coils 220.P is configured to manipulate the poloidal magnetic field generated by the circumferential plasma current for performing shape, position and balance control of the plasma 300.

[0073] The set of toroidal field coils 220.T comprises eight toroidal field coils 220.T.1 to 220.T.8. More generally, the set of toroidal field coils 220.T may include at least about 5, 10, 15, 20, 25, 30 or more toroidal field coils 220.T, or within any two of these values. Each toroidal field coil 220.T is positioned to have a corresponding poloidal plane orthogonal to the magnetic axis 305 and to surround the magnetic axis 305 within that poloidal plane. Typically, the set of toroidal field coils 220.T is a subset of the set of control coils 220 positioned closest to the outer shell 213 of the chamber 210. The set of toroidal field coils 220.T is configured to generate and manipulate a toroidal magnetic field, for example, such that... This is used to perform stability, radial confinement, and energy confinement control of plasma 300.

[0074] Generally, the magnetic superposition of the poloidal and circumferential magnetic fields establishes a helical configuration of the magnetic field within the boundary 310.lcfs, which in turn creates a set of nested closed flux surfaces 310.cfs, allowing for stable magnetic confinement of the plasma 300. The circumferential axisymmetry of the plasma 300 provides several other advantages, such as confining plasma particle trajectories via conservation of angular momentum. However, the circumferential plasma current induced within the plasma 300 can interact with a set of plasma-facing components of the chamber 210, thereby triggering instabilities in the plasma 300 that violate its circumferential axisymmetry, such as magnetohydrodynamic instabilities, kinetic instabilities, and / or positional instabilities. As described herein, the control system 100 can be configured to detect and mitigate such instabilities via active control, monitoring, and constraint avoidance.

[0075] However, the magnetic confinement device 200 can have a configuration with a different topology and / or geometry than a tokamak. For example, an alternative to a tokamak configuration of the magnetic confinement device 200 is a stellarator configuration, in which the set of control coils 220 is non-axisymmetric. Here, the set of control coils 220 directly generates the helical configuration of the magnetic field involved in the stable magnetic confinement of the plasma 300, for example, where the magnetic field is continuous.

[0076] One advantage of the stellarator configuration compared to the tokamak configuration is that inducing plasma current within plasma 300 is unnecessary, thereby reducing the possibility of instability in plasma 300 and the need for active control. Furthermore, the control system 100 can continuously command a set of control coils 220 of the stellarator configuration, whereas the control system 100 typically applies pulses to the central solenoid 222 of the tokamak configuration to induce circumferential plasma current. However, achieving confinement of the plasma particle trajectories in the stellarator configuration imposes limitations on the geometry and / or electrical configuration of the set of control coils 220, for example, by tolerance limits during manufacturing. Therefore, a tokamak configuration of the magnetic confinement device 200 may be advantageous for achieving a simpler design for particle confinement, where the control system 100 counteracts the initiation of instability that may occur due to the induction of circumferential plasma current.

[0077] IV. Examples of Control Systems

[0078] Figure 3A This is a schematic diagram illustrating an example of a control system 100 for a magnetic restraint device 200. The control system 100 includes a linear magnetic controller 110L, a neuromagnetic controller 110N, and a monitoring module 140 with an added user interface 150. The control system 100 is an example of a system implemented as a computer program on one or more computers at one or more locations, implementing the systems, components, and techniques described below.

[0079] Generally, the control system 100 is a multiple-input multiple-output (“MIMO”) control system configured to: receive vector-based inputs including physical measurements 122.yn and a target state 122.xn; and process the vector-based inputs to generate vector-based outputs including control actions 145.n. For example, the control system 100 can be configured as a centralized control and monitoring system that takes into account all interactions between the inputs and outputs of the magnetic confinement device 200. However, designing a MIMO control system is challenging for achieving desired levels of controllability and observability, especially for plasma magnetic control. To mitigate this, the control system 100 has been autonomously trained via reinforcement learning to achieve optimal (or near-optimal) controllability and observability in both linear and nonlinear magnetic control mechanisms.

[0080] More specifically, the linear magnetic controller 110L is governed by a first set of control parameters that control its linear control strategy. The neural magnetic controller 110N is parameterized by a second set of control parameters that govern its nonlinear neural control strategy. The linear 110L and neural 110N magnetic controllers are parameterized using a set of network parameters. Each has been trained, for example jointly or independently, on a model 500 of the magnetic confinement device 200 via reinforcement learning techniques, such that the set of control parameters is optimized for controlling the magnetic confinement device 200. The model 500 of the magnetic confinement device 200 can describe the dynamics of the magnetic confinement device 200 under both equilibrium (e.g., linear) and non-equilibrium (e.g., nonlinear) plasma conditions. Examples of reinforcement learning techniques used to train the linear 110L and neural 110N magnetic controllers on the model 500 of the magnetic confinement device 200 are referenced in Section VI. Figures 5A to 5B Described in more detail

[0081] The control system 100 implements a hybrid control framework to responsively and adaptively switch between linear and nonlinear control mechanisms, wherein each magnetic controller 110L and 110N is optimized separately. For example, the linear magnetic controller 110L can be trained around the equilibrium state 112.0 of the plasma 300, where the magnetic control paradigm is approximately linear, and the neural magnetic controller 110N can be trained to bring the physical state 112.n of the plasma 300 back to magnetic equilibrium, for example, if instability causes the plasma 300 to deviate from the equilibrium state 112.0. The control system 100 uses a monitoring module 140 to monitor such triggering conditions and the switching control between the linear 110L and neural 110N magnetic controllers, for example, simultaneously outputting monitoring information to the user interface 150.

[0082] A. Linear magnetic controller

[0083] To perform state feedback control of the plasma 300, the linear magnetic controller 110L includes a state observer 120 and a state feedback controller 130. The state observer 120 is governed by a set of observer parameters that control its linear state observation strategy. To parameterize, and the state feedback controller 130 is governed by a set of controller parameters that control its linear state control strategy. The state observer 120 can be any linear state observer, such as a Kalman filter, a Lenberger observer, or a linear neural observer. Similarly, the state feedback controller 130 can be any linear state feedback controller, such as a proportional-integral-derivative (“PID”) controller, a quadratic regulator, an H-infinity controller, a full-state feedback controller, a linear model predictive controller, or a linear neural controller.

[0084] In some implementations, the linear magnetic controller 110L is a linear neural network, for example, including a state observer 120 and a state feedback controller 130 as corresponding subnetworks of this linear neural network. In this case, each neural network layer of the linear magnetic controller 110L is a linear neural network layer with applied linear transformations. That is, there are no nonlinear activation functions like ReLU, sigmoid, or tanh in the linear magnetic controller 110L, which makes the linear magnetic controller 110L analytically tractable. For example, gradient descent on the linear magnetic controller 110L can learn low-rank or minimum norm solutions under certain conditions. Furthermore, since the linear magnetic controller 110L can be analyzed using standard linear algebra and linear control theory methods, it provides valuable information about the equilibrium dynamics of plasma 300, such as the stability conditions and sensitivity of plasma 300 under certain linear observation and control strategies.

[0085] Generally, the state feedback controller 130 is designed to directly control the physical state trajectory 112.1-N of the plasma 300 using control sequences 145.1-N, specifically guiding the physical state trajectory 112.1-N to the target state trajectory 122.x.1-N. The state observer 120 observes the inputs and outputs of the magnetic confinement device 200—e.g., physical measurements 122.yn and control actions 145.n—at each time step to generate a predicted state 124.xn of the plasma 300 that estimates the physical state 112.n of the plasma 300. The state feedback controller 130 uses the predicted state 124.xn instead of a direct measurement of the physical state 112.n during the control sequence 145.1-N.

[0086] 1. State Observer

[0087] Figure 3BThis is a schematic diagram depicting an example of a state observer 120 used to perform a state observation protocol within a control loop. In this example, the state observer 120 is a linear time-invariant (“LTI”) Kalman filter, for example, essentially an optimally tuned Lenberger observer. Here, a set of observer parameters for the state observer 120... a = { A,B,C,K} Includes state transition matrix A Control Action Matrix B and diagnostic matrix C Together, these specify the state-space model (“SSM”) of the magnetic confinement device 200 under a given noise model. This set of observer parameters may also include a target gain matrix conditioned on the target state 122.xn. K (as described in more detail below).

[0088] For the SSM, state observer 120 assumes that, within the linear operating range of the magnetic confinement device 200, the physical states 112.n and measurements 122.yn of the plasma 300 satisfy the following SSM equations:

[0089] in This refers to the uncertainty in plasma noise modeling in Plasma 300. This refers to the sensor noise modeling uncertainty in the sensor group 230, and N It is a Gaussian distribution. Q It is the covariance matrix of plasma noise. R It is the covariance matrix of sensor noise, which is usually derived from real-world data from the magnetic confinement device 200.

[0090] At a high level, state observer 120 is configured to: receive physical measurements 122.yn of plasma 300 at the current time step; and process the physical measurements 122.yn of plasma 300 at the current time step according to the set of observer parameters to generate a predicted state 124.xn of plasma 300 at the current time step. To this end, state observer 120 performs a prediction step at each time step, followed by an update step.

[0091] State observer 120 performs a prediction (“prior”) step based on information available to state observer 120 at a previous time step. Specifically, state observer 120 receives prior observer input 172.n, including: (i) the predicted state 124.x.(n-1) of plasma 300 at a previous time step, and (ii) the control action 145.(n-1) at a previous time step. State observer 120 processes the prior observer input 172.n at the current time step according to this set of observer parameters to generate a prior observer output 174.n, which includes: (i) a prior predicted state 124.x.n' of plasma 300 that does not conditioned on the physical measurement 122.yn of plasma 300 at the current time step, and (ii) a predicted measurement 124.yn of plasma 300 that estimates the physical measurement 122.yn of plasma 300 at the current time step.

[0092] To this end, state observer 120 first performs state prediction. State observer 120 generates a priori predicted state of plasma 300 ( ) 124.xn and the prior state covariance matrix at the current time step ( ) 126.x.n', as follows:

[0093] in P n-1 The state covariance matrix 126.x.(n-1) at the previous time step is the state covariance matrix 126.x.(n-1). The prior predicted state 124.x.n' of plasma 300 comprises the sum of the following: (i) the matrix-vector product between the state transition matrix and the predicted state 124.x.(n-1) of plasma 300 at the previous time step; and (ii) the matrix-vector product between the control action matrix and the control action 145.(n-1) at the previous time step. The prior state covariance matrix 126.x.n' characterizes the uncertainty in the prior predicted state 124.x.n' of plasma 300. The prior state covariance matrix 126.x.n' comprises the sum of the following: (i) the matrix product between the state transition matrix, the state covariance matrix 126.x.(n-1) at the previous time step, and the transpose of the state transition matrix; and (ii) the plasma noise covariance matrix.

[0094] Note the first time step in control sequence 145.1-N. n At point 1, the state observer 120 is provided with the initial predicted state of the plasma 300. The initial state covariance matrix of 124.x.0 and the initial state of plasma 300 characterizing the uncertainty in the initial state 124.x.0. P 0) 126.x.0. For example, the initially predicted state 124.x.0 can be initialized to the equilibrium state 112.0 established after the initiation sequence. Furthermore, the initial state covariance matrix can measure the variance in the initiating sequence. P 0 = cov.

[0095] Then, state observer 120 performs measurement prediction. State observer 120 calculates the predicted measurement of plasma 300 at the current time step. 124.yn and measurement covariance matrix ( S n ) 126.yn, as follows:

[0096] The predicted measurement 124.yn of plasma 300 comprises a matrix-vector product between the diagnostic matrix and the prior predicted state 124.x.n' of plasma 300 at the current time step. The measurement covariance matrix 126.yn characterizes the uncertainty in the predicted measurement 124.yn of plasma 300. The measurement covariance matrix 126.yn comprises the sum of the following: (i) the matrix product between the diagnostic matrix, the prior state covariance matrix 126.x.n' at the current time step, and the transpose of the diagnostic matrix; and (ii) the sensor noise covariance matrix.

[0097] Then, the state observer 120 performs an update (“posterior”) step based on the information available to the state observer 120 at the current time step. The state observer 120 receives a posterior observer input 174.n, which includes: (i) a prior observer input 172.n at the current time step, and (ii) a physical measurement 122.yn of the plasma 300 at the current time step. In some implementations, the posterior observer input 176.n further includes a target state 122.xn of the plasma 300 at the current time step. In this case, the target state 122.xn serves as a conditional input to the state observer 120. The state observer 120 processes the posterior observer input 176.n at the current time step according to a set of observer parameters to generate the predicted state 124.xn of the plasma 300 at the current time step.

[0098] After receiving the physical measurement 122.yn from plasma 300, state observer 120 calculates the measurement residual at the current time step. 125.yn and the observer (e.g., Kalman) gain matrix ( Ln ):

[0099] The measurement residual 125.yn (or innovation) measures the deviation between (i) the measurement 122.yn at the current time step and (ii) the predicted measurement 124.yn at the current time step. The observer gain matrix characterizes the effect of the physical measurement 122.yn on the predicted state 124.xn. The observer gain matrix is ​​the matrix product of the prior state covariance matrix 126.x.n' at the current time step, the transpose of the diagnostic matrix, and the inverse of the measurement covariance matrix 126.yn at the current time step.

[0100] To complete the update step, state observer 120 calculates the predicted state 124.xn and state covariance matrix of plasma 300 at the current time step, as follows:

[0101] The predicted state 124.xn includes the incremental state at the current time step. The incremental state is the prior predicted state 124.x.n' of plasma 300. The incremental state consists of the matrix-vector product between the observer gain matrix and the measurement residual 125.yn at the current time step. The state covariance matrix 126.xn characterizes the uncertainty in the predicted state 124.xn of plasma 300. The state covariance matrix 126.xn consists of the prior state covariance matrix 126.x.n' at the current time step minus the matrix product of the observer gain matrix, diagnostic matrix, and prior state covariance matrix 126.x.n' at the current time step.

[0102] In order to conditionally use the target state 122.xn, the state observer 120 utilizes the prior state residual at the current time step. Use '125.x.n' to modify the incremental state at the current time step. Given... The prior state residual 125.n' measures the deviation between the target state 122.xn of plasma 300 at the current time step and the prior predicted state 124.x.n'.

[0103] As an example, state observer 120 can enhance incremental states to This makes the incremental state include the matrix-vector product between the target gain matrix and the prior state residual 125.x.n' at the current time step. This makes the state observer 120 sensitive to the target state 122.xn when performing update steps.

[0104] As another example, the state observer 120 can modulate the observer gain matrix to... ,in This is the observer gain function, which increases with the magnitude of the prior state residual 125.x.n'. This makes the state observer 120 more sensitive to the physical measurement 122.yn when the linear magnetic controller 110L is far from the target state 122.xn. When the linear magnetic controller 110L is close to the target state 122.xn, which typically corresponds to a high-sensitivity region (e.g., for control tasks), the state observer 120 can perform smaller-amplitude updates to avoid errors caused by noise.

[0105] As a concrete example, the observer gain function It can be the Mahalanobis distance between the target state 122.xn of plasma 300 and the prior predicted state 124.x.n'. The Mahalanobis distance is a monotonically increasing function. The square root of the inner product between the Mahalanobis distance and the inverse of the prior state residuals 125.x.n' and the prior state covariance matrix 126.x.n' corresponds to the following:

[0106] For example, the observer gain function can be a logistic function, linear ramp function, quadratic function, exponential function, or other monotonically increasing function of the Mahalanobis distance between the target state 122.xn and the prior predicted state 124.x.n'.

[0107] 2. Status Feedback Controller

[0108] The state feedback controller 130 implements a linear state control strategy parameterized by its set of controller parameters. Given... This set of controller parameters includes one or more controller memory matrices. and one or more controller gain matrices .here, N a It is the total number of controller memory matrices, and N c +1 represents the total number of controller gain matrices. For example, this set of controller parameters can include 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or more controller memory matrices, or a range between any two of these values.

[0109] The status feedback controller 130 is configured to receive controller input ( c n142.n, which includes: (i) the target state 122.xn of plasma 300 at the current time step, and (ii) the predicted state 124.xn of plasma 300 at the current time step. The state feedback controller 130 is configured to process the controller input 142.n at the current time step according to its set of controller parameters to generate the control action 145.n at the current time step. In some implementations, the controller input 142.n is the state residual at the current time step. 125.xn. Given as The state residual 125.xn measures the deviation between the target state 122.xn of plasma 300 at the current time step and the predicted state 124.xn.

[0110] Generally, the state feedback controller 130 generates control actions 145.n based on a linear state control strategy, as follows:

[0111] As shown in equation (3), each controller memory matrix ( M k The gain matrix is ​​associated with the corresponding time step preceding the current time step, for example, with a first-order, second-order, third-order, or higher-order recursion. Similarly, each controller gain matrix ( K j It is associated with the current time step or the corresponding time step before the current time step, such as corresponding to zero-order, first-order, second-order, third-order or higher-order recursions.

[0112] As an example, if the state feedback controller 130 is a PID controller, then the first-order controller memory matrix is ​​the identity matrix. M k = I The rest are zero. The zero-order controller gain matrix is: K 0 = K P + K I + K D The first-order controller gain matrix is K 1 = – K P – 2 K D The second-order controller gain matrix is K 2 = K D The rest are zero. Here, K P It is a proportional gain matrix. K I It is the integral gain matrix, andK D It is the differential gain matrix, which corresponds to the proportional, integral, and derivative terms of the PID controller, respectively.

[0113] Using the recursive relationship in equation (3), the state feedback controller 130 calculates the control action 145.n at the current time step by recursively updating the control actions 145 at one or more previous time steps, while taking into account the current time step and the controller inputs 142 at one or more previous time steps. In terms of computational cost, recursive updates are generally cheaper than direct calculations, especially during real-time control of the magnetically constrained device 200. For example, if the state feedback controller 130 has an integral term, direct calculation involves summing over all past controller inputs 142.(n-1) to 142.1.

[0114] The state feedback controller 130 can perform recursive updates as follows. For each controller memory matrix associated with a corresponding previous time step, the state feedback controller 130 calculates the corresponding matrix-vector product between: (i) the controller memory matrix, and (ii) the control action 145.k at the corresponding previous time step. For each controller gain matrix associated with the current time step or a corresponding previous time step, the state feedback controller 130 calculates the corresponding matrix-vector product between: (i) the controller gain matrix, and (ii) the controller input 142.n at the corresponding previous time step. The state feedback controller 130 then calculates the control action 145.n at the current time step as the sum of the corresponding matrix-vector products of each controller memory matrix and controller gain matrix at the current time step.

[0115] In some implementations, the control system 100 is configured to perform operations on variables in the control loop. z Transformation. For the SSM in equation (1), the transfer function of the controlled object of the magnetic confinement device 200 ( G ) is given as G ( z )= C [ zI – A]- 1 B ,in z yes z Domain variables. Then the open-loop transfer function of the magnetically confined system 10 ( H OL ) is given as:

[0116] in KThis is the controller transfer function of the state feedback controller 130. The open-loop transfer function represents the relationship between the physical measurement 122.yn and the control action 145.n before the application of state feedback control. For achieving the state residual 125.xn as... State feedback control, closed-loop transfer function of magnetic confinement system 10 ( H CL ) is given as:

[0117] Furthermore, the sensitivity function of the magnetic confinement system 10 ( S ) can be defined as S ( z ) = [ I + H OL (z)] -1 This makes the closed-loop transfer function H CL ( z ) = T ( z ) is defined as the complementary sensitivity function ( T The sensitivity function represents the ability of the linear magnetic controller 110L to suppress disturbances, and the complementary sensitivity function represents the ability of the linear magnetic controller 110L to track the target state 122.x.

[0118] The closed-loop poles of the magnetically confined system 10 are determined by the characteristic equation det[ S ( z )] = 0 in z The solution is defined in the domain variables. That is, the solution that makes the determinant of the sensitivity function equal to zero. z These values ​​are typically the roots of the characteristic polynomial. In some implementations, this set of controller parameters... The magnetically constrained system 10 is configured such that each closed-loop pole lies within the unit circle, for example, such that the state feedback controller 130 satisfies the Routh-Hurwitz stability criterion. For instance, during training of the linear magnetic controller 110L, the Routh-Hurwitz stability criterion can be set as a constraint on the state feedback controller 130.

[0119] The property of the LTI representation of the magnetic confinement system 10 is that any of its transfer functions—including the controlled object transfer function, the controller transfer function, the open-loop transfer function, the closed-loop transfer function, the sensitivity function, and / or the complementary sensitivity function—can be passed through... z The domain diagram was analyzed, providing insights related to the stability, controllability, observability, system response, and / or mode coupling in the magnetically confined system 10. For example, zDomain graphs can be pole-zero plots, root locus plots, Bode plots, Nyquist plots, Nichols plots, singular value plots, or the response of a transfer function to an input function (e.g., a unit impulse function, a unit step function, or a sine function). z Domain graph, or any appropriate transfer function z Domain diagram.

[0120] B. Monitoring module

[0121] Figure 3C This is an example schematic block diagram depicting a monitoring module 140 for executing a monitoring protocol within a control loop. Generally, the monitoring module 140 operates on system data received and generated by the linear magnetic controller 110L to indirectly acquire the behavior of the plasma 300, the linear magnetic controller 110L, and / or the interactions between them, for example, to determine whether one or both are behaving abnormally. The monitoring module 140 may operate with a purely observational logging capability or with a real-time capability affecting the dynamics of the magnetically confined system 10.

[0122] More specifically, at each time step in the control sequence 145.1-N, the monitoring module 140 is configured to receive monitoring input 162.n at the current time step from the linear magnetic controller 110L:

[0123] The monitoring input 162.n includes: (i) the state prediction block 128.xn at the current time step, and / or (ii) the measurement prediction block 128.yn at the current time step. The state prediction block 128.xn and the measurement prediction block 128.yn characterize the statistical features of the physical state trajectory 112.1-N and the physical measurement trajectory 122.y.1-N of the plasma 300, respectively. The state prediction block 128.xn includes the target state 122.xn at the current time step, the predicted state 124.xn, the state residual 125.xn, and the state covariance matrix 126.xn. Similarly, the measurement prediction block 128.yn includes the physical measurement 122.yn at the current time step, the predicted measurement 124.yn, the measurement residual 125.yn, and the measurement covariance matrix 126.yn.

[0124] Monitoring module 140 is configured to process monitoring input 162.n to generate an event signal 165.n at the current time step. Event signal 165.n indicates whether the plasma 300 meets a trigger condition at the current time step. For example, event signal 165.n may include a corresponding binary value for each trigger condition monitored by monitoring module 140, wherein the binary value has a high (“H”) value (e.g., 1) if the trigger condition is met at the current time step, and a low (“L”) value (e.g., 0) if the trigger condition is not met at the current time step.

[0125] To this end, monitoring module 140 performs statistical analysis on state prediction block 128.xn and / or measurement prediction block 128.yn to generate monitoring output at the current time step. m n ) 164.n. In the example described herein, the statistical analysis includes one or more of the following: anomaly analysis, instability analysis, or nonlinear analysis, used to detect anomalies, instabilities, and nonlinearities in plasma 300, respectively. Given as The monitoring output 164.n includes: (i) the state health vector at the current time step. 132.xn, and / or (ii) the measured health vector at the current time step. 132.yn. The state health vector 132.xn includes multiple statistical health indices 35.1.xn to 35.hxn obtained after performing statistical analysis on the state prediction block 128.xn. Similarly, the measurement health vector 132.yn includes multiple statistical health indices 35.1.yn to 35.hyn obtained after performing statistical analysis on the measurement prediction block 128.yn. Here, i Index each statistical health indicator 35.in in the health vector 132.n, and h It is the total number of statistical health indicators 35.n in health vector 132.n. For example, health vector 132.n can each include 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25 or more statistical health indicators 35.n.

[0126] Statistical health indicators 35.n are quantitative measures derived from system data that help assess the health, stability, and / or performance of plasma 300 over time. Each statistical health indicator 35.in has one or more corresponding thresholds that indicate the detection of unexpected deviations in plasma 300, such as the maximum and / or minimum thresholds of statistical health indicator 35.i. Monitoring module 140 determines from monitoring output 164.n whether each statistical health indicator 35.in in the state health vector 132.xn and / or measurement health vector 132.yn meets its corresponding threshold. If a statistical health indicator 35.in meets its threshold, monitoring module 140 generates an event signal 165.n indicating that plasma 300 meets a trigger condition at the current time step; for example, a high binary value if statistical health indicator 35.in meets its threshold, and a low value if it does not.

[0127] For the sake of brevity, this article will refer to the given... The general prediction block 128.n describes examples of statistical analysis and statistical health indicators 35.1.n to 35.hn, which includes samples. 122.n, Prediction The residuals between sample 124.n, sample 122.n, and prediction 124.n 125.n, and the covariance matrix characterizing the uncertainty in predicting 124.n. 126.n. Monitoring module 140 can perform statistical analysis on state prediction block 128.xn and measurement prediction block 128.yn in parallel. Although monitoring module 140 can also perform joint statistical analysis on state prediction block 128.xn and measurement prediction block 128.yn, state observer 120 structurally decouples the prediction step and the update step, thus allowing a simpler, separate approach. However, it should be noted that physical measurement 122.yn is observed by control system 100, while target state 122.xn is defined by control system 100. Therefore, statistical analysis on state prediction block 128.xn and measurement prediction block 128.yn may have different interpretations, but monitoring module 140 can perform one or both to monitor the health of magnetic confinement system 10.

[0128] 1. Anomaly Analysis

[0129] Triggering conditions can indicate that an anomaly has been detected in plasma 300. Generally, this anomaly is a transient, unexpected deviation from the nominal operating conditions of plasma 300, which is detected by monitoring module 140 at the current time step. For example, the anomaly could be an early warning of instability in plasma 300 or a sudden change in the behavior of plasma 300.

[0130] As an example, the monitoring module 140 can calculate the Mahalanobis distance at the current time step. D n 35.1.n, as follows:

[0131] The Mahalanobis distance 35.1.n measures the distance between the following two things with respect to the covariance matrix 126.n at the current time step: (i) the sample 122.n at the current time step, and (ii) the prediction 124.n at the current time step. The Mahalanobis distance 35.1.n corresponds to the square root of the inner product between the residual 125.n and the inverse of the covariance matrix 126.n. The Mahalanobis distance 35.1.n normalizes (e.g., whitens) the sample 122.n to have zero mean and unit covariance. A large value of the Mahalanobis distance 35.1.n indicates that the sample 122.n and the prediction 124.n are statistically far apart, which is a strong signal of anomalies in Plasma 300.

[0132] The monitoring module 140 determines whether the Mahalanobis distance 35.1.n at the current time step is greater than a threshold, for example, a threshold in the range of approximately 2 to 3. If so, the monitoring module 140 generates an event signal 165.n indicating that an anomaly has been detected in the plasma 300 at the current time step.

[0133] As another example, the monitoring module 140 can calculate the anomaly likelihood at the current time step ( P n 35.2.n, as follows:

[0134] in N It follows a Gaussian distribution. The anomaly likelihood 35.2.n is the complement of the likelihood of sample 122.n at the current time step, given the prediction 124.n and the covariance matrix 126.n at the current time step. As the name suggests, a large value of the anomaly likelihood 35.2.n indicates that sample 122.n is unlikely, which is another strong signal of anomalies in plasma 300.

[0135] The monitoring module 140 determines whether the anomaly likelihood 35.2.n at the current time step is greater than a threshold, for example, a threshold in the range of approximately 0.95 to 0.99. If so, the monitoring module 140 generates an event signal 165.n indicating that an anomaly has been detected in the plasma 300 at the current time step.

[0136] 2. Instability Analysis

[0137] Triggering conditions can indicate that instability has been detected in plasma 300. Generally, instability is a transient phenomenon detected by monitoring module 140 at multiple time steps. For example, instability can indicate that plasma 300 is experiencing unexpected disturbances, such as turbulence or chaotic disturbances.

[0138] As an example, monitoring module 140 can calculate the local Lyapunov exponent at the current time step. 35.3.n, as follows:

[0139] Where log is a logarithm, such as the natural logarithm. The local Lyapunov exponent 35.3.n comprises the logarithm of the ratio of (i) the Mahalanobis distance 35.1.n at the current time step, and (ii) the Mahalanobis distance 35.1.(n-1) at the previous time step. Negative, zero, or positive values ​​of the local Lyapunov exponent 35.3.n indicate that plasma 300 is locally stable, locally critically stable, or locally unstable, respectively. Local instability may be transient, but a large positive value of the local Lyapunov exponent 35.3.n can indicate the onset of rapid instability in plasma 300.

[0140] The monitoring module 140 determines whether the local Lyapunov index 35.3.n at the current time step is greater than a threshold, for example, a threshold in the range of approximately 2 to 10. If so, the monitoring module 140 generates an event signal 165.n indicating that local instability has been detected in plasma 300 at the current time step.

[0141] As another example, the monitoring module 140 can calculate the global Lyapunov exponent at the current time step. 35.4.n, as follows:

[0142] The global Lyapunov exponent 35.4.n comprises the mean of the local Lyapunov exponents 35.3.n to 35.3.1 at the current time step and at each previous time step. The global Lyapunov exponent 35.4.n is proportional to the logarithm of the ratio of (i) the Mahalanobis distance 35.1.n at the current time step and (ii) the Mahalanobis distance 35.1.1 at the first time step. Negative, zero, or positive values ​​of the global Lyapunov exponent 35.4.n indicate whether plasma 300 is globally stable, globally critically stable, or globally unstable, respectively.

[0143] Monitoring module 140 determines whether the global Lyapunov exponent 35.4.n at the current time step is greater than a threshold, for example, a threshold in the range of approximately 0 to 0.1. If so, monitoring module 140 generates an event signal 165.n indicating that a global instability has been detected in plasma 300 at the current time step.

[0144] As yet another example, the monitoring module 140 can calculate the Kolmogorov-Sinai entropy at the current time step. H n 35.5.n, as follows:

[0145] It measures the information loss rate of plasma 300. Here, It is a modified linear unit that returns strictly positive values ​​of its independent variable. The Kolmogorov-Sinai entropy 35.5.n comprises the mean of the positive local Lyapunov indices 35.3.n to 35.3.1 at the current time step and every previous time step. Large values ​​of the Kolmogorov-Sinai entropy 35.5.n indicate that plasma 300 is chaotically unstable, which may be due to the cancellation between positive and negative values, preventing it from being captured by the global Lyapunov indices 35.4.n.

[0146] Monitoring module 140 determines whether the Kolmogorov-Sinai entropy 35.5.n at the current time step is greater than a threshold, for example, a threshold in the range of approximately 0.1 to 0.5. If so, monitoring module 140 generates an event signal 165.n indicating that chaotic instability has been detected in plasma 300 at the current time step.

[0147] 3. Nonlinear Analysis

[0148] Triggering conditions can indicate that nonlinearity has been detected in plasma 300. Generally, nonlinearity is a deterministic pattern of plasma 300 detected by monitoring module 140 at multiple time steps. For example, nonlinearity may manifest as correlation, structure dependence, or emergent behavior of plasma 300.

[0149] Monitoring module 140 identifies nonlinearity by performing a set of statistical tests on the residuals 125.n-1 obtained by state observer 120. Assuming state observer 120 is accurate within the linear operating mechanism of magnetic confinement device 200, residuals 125.n-1 should be Gaussian distributed, zero-mean, and uncorrelated. Therefore, to test for nonlinearity, monitoring module 140 can perform the chi-square test, Wald test, and Ljung-Box test to determine whether each of these properties holds under the corresponding null hypothesis. If state observer 120 fails each test, there is a high probability of nonlinearity in plasma 300, as this phenomenon cannot be explained by the linear state observation strategy. More generally, this set of statistical tests may include one or more of the following: chi-square test, variance test, Wald test, mean test, Ljung-Box test, Box-Pierce test, autocorrelation test, Shapiro-Wilk test, Anderson-Darling test, Kolmogorov-Sinai test, cumulative sum test, runs test, or any other statistical test.

[0150] To prepare, the monitoring module 140 can perform a whitening transformation on the residual 125.n at the current time step to generate a normalized version of the residual 125.n, i.e. .here, It is the inverse square root of the covariance matrix 126.n at the current time step, for example, corresponding to the inverse of the standard deviation matrix at the current time step. Then, the monitoring module 140 targets one or more lags ( p For each of the given values, calculate the corresponding lag-p sample autocovariance matrix at the current time step. ,as follows:

[0151] The sample covariance matrix at the current time step The autocovariance matrix is ​​defined as the lag-0 sample autocovariance matrix. The lag-p sample autocovariance matrix is ​​the result of each normalized residual (125.n) and the lag... p The mean of the outer product between the corresponding normalized residuals 125.(np) at each time step. Then, the monitoring module 140 can calculate the corresponding lag-p sample autocorrelation matrix at the current time step for each lag. ,as follows:

[0152] It includes the matrix product between the corresponding lag-p sample autocovariance matrix and the inverse of the sample covariance matrix at the current time step.

[0153] To perform a chi-square (or variance) test, the monitoring module 140 calculates the chi-square test statistic at the current time step. 35.6.n, as follows:

[0154] Where tr represents the trace, and It is the Euclidean norm. The chi-square test statistic 35.6.n includes the trace of the sample covariance matrix at the current time step. This is equivalent to the mean of the Euclidean norms of the normalized residuals 125.n to 125.1 at the current time step and at each previous time step, or the mean of the squared Mahalanobis distances 35.1.n to 35.1.1 at the current time step and at each previous time step. A large value of the chi-square test statistic 35.6.n indicates that plasma 300 has variance that cannot be explained by state observer 120.

[0155] Monitoring module 140 determines whether the chi-square test statistic 35.6.n at the current time step meets a threshold, for example, rejecting the null hypothesis about the chi-square test statistic 35.6.n. If so, monitoring module 140 generates an event signal 165.n indicating that unexplained variance has been detected in plasma 300 at the current time step.

[0156] To perform a Wald (or mean) test, monitoring module 140 calculates the Wald test statistic at the current time step. W n 35.7.n, as follows:

[0157] in The mean-normalized residuals comprise the mean of the normalized residuals from the current time step to each previous time step, ranging from 125.n to 125.1. The Wald test statistic 35.7.n comprises the inner product between the mean-normalized residuals at the current time step and the sample covariance matrix. A large value of the Wald test statistic 35.7.n indicates that plasma 300 has a bias that cannot be explained by state observer 120.

[0158] The monitoring module 140 determines whether the Wald test statistic 35.7.n at the current time step meets a threshold, for example, by rejecting the null hypothesis about the Wald test statistic 35.7.n. If so, the monitoring module 140 generates an event signal 165.n indicating that an unexplained bias has been detected in the plasma 300 at the current time step.

[0159] To perform the Ljung-Box (or autocorrelation) test, the monitoring module 140 calculates the Ljung-Box test statistic at the current time step. 35.8.n, as follows:

[0160] in It is the Frobenius norm. The Ljung-Box test statistic 35.8.n comprises the weighted sum of the corresponding Frobenius norms of the autocorrelation matrix for each sample, from lag -1 to lag -p. A large value of the Ljung-Box test statistic 35.8.n indicates that plasma 300 has a correlation that cannot be explained by state observer 120.

[0161] Monitoring module 140 determines whether the Ljung-Box test statistic 35.8.n at the current time step meets a threshold, for example, by rejecting the null hypothesis regarding the Ljung-Box test statistic 35.8.n. If so, monitoring module 140 generates an event signal 165.n indicating that an unexplained correlation has been detected in plasma 300 at the current time step.

[0162] In some implementations, monitoring module 140 determines whether the chi-square test statistic 35.6.n, Wald test statistic 35.7.n, and Ljung-Box test statistic 35.8.n each meet their respective thresholds, for example, rejecting the corresponding null hypothesis for each test statistic 35.6.n, 35.7.n, and 35.8.n. If so, monitoring module 140 generates an event signal 165.n indicating that a nonlinearity has been detected in plasma 300 at the current time step.

[0163] c. User Interface

[0164] User interface 150 includes: a user input device 152 through which a user provides user input 156.n to control system 100; and a display device 154 for displaying monitoring information of magnetic restraint system 10 to the user. For example, user input device 152 may include a touchscreen display, touchpad, or keyboard and pointing device (e.g., mouse or trackball). Display device 154 may be a light-emitting diode (“LED”) display, an organic LED (“OLED”) display, or a liquid crystal display (“LCD”) monitor. In some implementations, user input device 152 and display device 154 are the same device, such as a touchscreen display.

[0165] Monitoring module 140 is configured to transmit system output 104.n to user interface 150, which includes monitoring input 162.n, monitoring output 164.n, and event signal 165.n at each time step in control sequence 145.1-N. In some implementations, system output 104.n further includes control action 145.n at that time step. This allows the user to view some or all of the visual representations of the inputs and outputs of control system 100 during active control of plasma 300. For example, system output 104.n at each time step can be displayed on display device 154 as a system performance graph, such as a trend graph, histogram, or bar graph.

[0166] In some implementations, the system output 104.n further includes the transfer function of the magnetic confinement system 10. z Domain diagram. For example, the transfer function can be a controller transfer function, an open-loop transfer function, a closed-loop transfer function, a sensitivity function, or a complementary sensitivity function. z Domain graphs can be pole-zero plots, root locus plots, Bode plots, Nyquist plots, Nichols plots, singular value plots, or the response of a transfer function to an input function (e.g., a unit impulse function, a unit step function, or a sine function). z Domain graph, or any appropriate transfer function z Domain diagram.

[0167] In some implementations, the monitoring module 140 is configured to adjust the linear magnetic controller 110L based on the monitoring output 164.n. This allows the monitoring module 140 to actively suppress the initiation of unexpected deviations in the plasma 300. For example, the monitoring module 140 can utilize the monitoring output 164.n to enhance the control action 145.n, as follows:

[0168] in It is a set of one or more monitoring gain matrices, and N b This is the total number of monitoring gain matrices. For the examples described herein, it is generally preferred to have... In the case of This is because a larger value for the monitoring output 164.n characterizes the onset of unexpected deviations in plasma 300. This enhancement effectively changes the linear state control strategy of the state feedback controller 130 to a nonlinear state control strategy, for example, driving both the controller input 142.n and the monitoring output 164.n to zero.

[0169] In some implementations, monitoring module 140 is configured to receive user input 156.n from user interface 150 at one or more time steps in the control sequence 145.1-N. For example, user input 156.n is provided as a user input gesture to user input device 152 of user interface 150. User input 156.n may specify user adjustment of linear magnetic controller 110L. Monitoring module 140 then adjusts linear magnetic controller 110L according to user input 156.n. For example, user input 156.n may specify a subset of a set of controller parameters of state feedback controller 130. The corresponding set value for each controller parameter, i.e. The monitoring module 140 then initializes each controller parameter in the subset of controller parameters using the corresponding setpoint specified by the user input 156.n. This manually alters the linear state control strategy of the state feedback controller 130, the effects of which can be encoded at each subsequent time step in the monitoring input 162.n, monitoring output 164.n, event signal 165.n, control action 145.n, and / or transfer function. z In a domain diagram, for example, it is displayed on a display device 154 of the user interface 150.

[0170] D. Neuromagnetic controller

[0171] The control system 100 is configured to read the event signal 165.n at each time step to determine, for example, whether the plasma 300 meets the trigger condition at the current time step by determining whether the event signal 165.n contains at least one high binary value. If the plasma 300 does not meet the trigger condition at the current time step, the control system 100 proceeds to the next time step. If the plasma 300 meets the trigger condition at the current time step, the control system 100 records the current time step as the initiation time step. And initiate a handover protocol to be used at every time step following that initiation time step. The order was executed.

[0172] Generally, the switching protocol involves the neural magnetic controller 110N taking over control of the magnetic restraint device 200 at each time step after the initiation time step, until the triggering condition is no longer met at a specific time step in the control sequence 145.1-N. That is, during the switching protocol, the state observer 120 can continue to generate the predicted state 124.xn and the predicted measurement 124.yn, and the monitoring module 140 can continue to process the state prediction block 128.xn and the measurement prediction block 128.yn as a response to update event signal 165.n.

[0173] During the protocol switching, the control system 100 is configured to perform the following operations: The control system 100 forwards network inputs to the neuromagnetic controller 110N. 144.n, which includes: (i) the target state 122.xn of plasma 300 at the current time step, and (ii) the physical measurement 122.yn of plasma 300 at the current time step. Here, network input 144.n can also be referred to as the “observation” of the physical state 112.n of plasma 300 conditioned on the target state 122.xn. Control system 100 uses neuromagnetic controller 110N to process network input 144.n at the current time step to generate network output 146.n, which defines a strategy at the current time step for selecting control actions for the set of control coils 220. The operation of the neuromagnetic controller 110N can be concisely described as being determined by this set of network parameters of the neuromagnetic controller 110N ( ) Parameterized nonlinear parameter functions .

[0174] In some implementations, the strategy is defined as a probability distribution of possible control actions of the group of control coils 220. The network output 144.n includes a set of distribution parameters for this probability distribution. Given a functional form of a probability distribution, the set of distribution parameters specifies that probability distribution. Therefore, the neural magnetic controller 110N provides a mapping from observations of the plasma 300 to a nonlinear neural control strategy for the plasma 300. , making In some implementations, this probability distribution is a Gaussian distribution. Furthermore, this set of distribution parameters includes mean control actions. and control action covariance matrix For example, making In other implementations, the probability distribution can be a beta distribution, a softmax distribution, an exponential family distribution, a Laplace distribution, etc.

[0175] The neural magnetic controller 110N can have any suitable neural network architecture that enables it to perform the function described herein, namely, processing network input 144 to generate network output 146 for that network input 144. The neural magnetic controller 110N can include any suitable number (e.g., 5 layers, 25 layers, or 100 layers) and any suitable type of neural network layers (e.g., fully connected layers, recurrent layers, convolutional layers, self-attention layers, etc.) connected in any suitable configuration (e.g., as a linear sequence of layers, residual configuration, etc.). In some implementations, the neural magnetic controller 110N is a feedforward neural network including one or more feedforward layers. Feedforward layers are computationally fast because they primarily involve two operations: (i) matrix multiplication and (ii) activation functions. To further improve speed, in some implementations, the neural magnetic controller 110N includes ten or fewer neural network layers, such as nine or fewer, eight or fewer, seven or fewer, six or fewer, or five or fewer.

[0176] The control system 100 generates neural control actions for a set of control coils 220 at the current time step based on the network output 146.n at the current time step. 145.n. For example, to implement a stochastic control strategy, the control system 100 can generate a probability distribution based on the set of distribution parameters at the current time step, and then sample neural control actions 145.n from that probability distribution, i.e. As another example, to implement a deterministic control strategy, control system 100 can determine the expected value of the probability distribution based on the set of distribution parameters at the current time step, and then perform neural control action 145. .n is returned as the expected value of this probability distribution, i.e. For example, in the case of a Gaussian distribution, it makes Deterministic control strategies may be advantageous for real-world deployments (e.g., inference) on the magnetically confined device 200, while stochastic control strategies may be advantageous for training on the model 500 of the magnetically confined device 200.

[0177] The control system 100 initializes the control action 145.n at the current time step to the neural control action 145.n at the current time step, that is... The control system 100 transmits the control action to the magnetic restraint device 200. Therefore, during the protocol switching, the magnetic restraint device 200 operates based on neural control action 145.n, and the state feedback controller 130 is bypassed. However, the state observer 120 can still use the physical measurement 122.yn and neural control action 145 as described above. The neural control strategy generates a priori predicted state 124.x.n', the predicted state 124.xn, and the predicted measurement 124.yn. These can then be processed by the monitoring module 140 to recalculate the event signal 165.n at the current time step in the switching protocol.

[0178] The control system 100 determines whether the event signal 165.n still indicates that the plasma 300 meets the trigger condition at the current time step, for example, by determining whether the event signal 165.n still includes at least one high binary value at the current time step. If the event signal 165.n still indicates that the plasma 300 meets the trigger condition at the current time step, the control system 100 repeats the above operation at the next time step in the switching protocol. If the event signal 165.n no longer indicates that the plasma 300 meets the trigger condition at the current time step, the control system 100 records the current time step as the termination time step. and at the termination time step The switching protocol is terminated at this point. Then, the linear magnetic controller 110L takes over control of the magnetic restraint device 200 again. The control system 100 can perform this process once or multiple times during the control sequence 145.1-N, switching control between the linear magnetic controller 110L and the neuromagnetic controller 110N in response to the event signal 165.n at each time step.

[0179] V. Examples of Control, Monitoring, and Switching Protocols

[0180] Figure 4A This is a flowchart of an example process 1000 for controlling a magnetic confinement device 200. For convenience, process 1000 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification (e.g., Figures 3A to 3C The control system 100) can execute process 1000.

[0181] The control system 100 receives the target state trajectory 122.x.1-n (1100) of the plasma 300 at each of the multiple time steps, which includes the corresponding target state 122.xn.

[0182] In some implementations, the control system 100 takes over control of the magnetic confinement device 200 from the open-loop control system (1150). For example, the open-loop control system may be configured to execute an initiation sequence on the magnetic confinement device 200 to establish the initial state 112.0 of the plasma 300.

[0183] The control system 100 generates a control sequence 145.1-N under the condition of the target state trajectory 122.x.1-N of the plasma 300. The control sequence includes a corresponding control action 145.n (1200) for a set of control coils 220 at each time step.

[0184] At each time step in control sequence 145.1-N, control system 100 performs the following operations: The control system 100 executes the control protocol via the linear magnetic controller 110L to generate the control action 145.n (1300.n) at the current time step.

[0185] The control system 100 executes a monitoring protocol via the monitoring module 140 and the user interface 150 to generate an event signal 165.n (1400.n) at the current time step.

[0186] The control system 100 determines whether the event signal 165.n at the current time step indicates that the plasma 300 meets the trigger condition (1500.n) at the current time step.

[0187] If the event signal 165.n at the current time step does not indicate that the plasma 300 meets the trigger condition at the current time step, the control system 100 is interrupted to operation 1300.n. If the event signal 165.n at the current time step indicates that the plasma 300 meets the trigger condition at the current time step, the control system 100 is interrupted to operation 1600.n.

[0188] The control system 100 records the current time step as the initiation time step (1600.n).

[0189] The control system 100 initiates a switching protocol via the neuromagnetic controller 110N to initialize the control action 145.n at each time step after the initiation time step to the corresponding neural control action 145.n (1700.n) at that time step.

[0190] Figure 4B This is a flowchart of an example process 1300.n for executing a control protocol to generate corresponding control actions 145.n at each time step in the control sequence 145.1-N. For convenience, process 1300.n will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification (e.g., Figures 3A to 3C The control system 100) can execute process 1300. n.

[0191] The control system 100 receives the target state 122.xn (1310.n) of the plasma 300 at the current time step.

[0192] The control system 100 receives from the magnetic confinement device 200 physical measurements 122.yn (1320.n) of the plasma 300, which characterize the physical state 112.n of the plasma 300 at the current time step.

[0193] The control system 100 forwards the physical measurement 122.yn of the plasma 300 at the current time step to the state observer 120 (1330.n).

[0194] The control system 100 uses the state observer 120 to process the physical measurements 122.yn of the plasma 300 at the current time step to generate the predicted state 124.xn (1340.n) of the plasma 300, which estimates the physical state 112.n of the plasma 300 at the current time step.

[0195] The control system 100 forwards the controller input 142.n to the state feedback controller 130, which includes: (i) the target state 122.xn of the plasma 300 at the current time step, and (ii) the predicted state 124.xn (1350.n) of the plasma 300 at the current time step.

[0196] The control system 100 uses a state feedback controller 130 to process the controller input 142.n to generate control actions 145.n (1360.n) for a set of control coils 220 at the current time step.

[0197] The control system 100 transmits the control action 145.n at the current time step to the magnetic confinement device 200 to generate a magnetic field (1370.n) that confines the plasma 300 within the chamber 210.

[0198] Figure 4C This is a flowchart of an example process 1340.n for executing a state observation protocol to generate the corresponding predicted state 124.xn of plasma 300 at each time step in the control sequence 145.1-N. For convenience, process 1300.n will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification (e.g., Figure 3C The state observer 120) is an executable process 1340.n.

[0199] The state observer 120 receives prior observer input 172.n, including: (i) the predicted state 124.xn of the plasma 300 at the previous time step, and (ii) the control action 145.n (1342.n) at the previous time step.

[0200] The state observer 120 processes the prior observer input 172.n at the current time step according to its set of observer parameters to generate a prior observer output 174.n, which includes: (i) estimating the prior predicted state 124.x.n' of the plasma 300 without conditioned on the physical measurement 122.yn of the plasma 300 at the current time step, and (ii) estimating the predicted measurement 124.yn (1344.n) of the plasma 300 with respect to the physical measurement 122.yn of the plasma 300 at the current time step.

[0201] State observer 120 receives a posteriori observer input 176.n, which includes: (i) a priori observer input 172.n at the current time step, and (ii) physical measurements 122.yn (1346.n) of plasma 300 at the current time step. In some implementations of process 1340.n, the posteriori observer input 176.n at the current time step further includes the target state 122.xn of plasma 300 at the current time step.

[0202] The state observer 120 processes the posterior observer input 176.n according to its set of observer parameters to generate the predicted state 124.xn (1348.n) of the plasma 300 at the current time step.

[0203] Figure 4D This is a flowchart of an example process 1400.n for executing a monitoring protocol to generate a corresponding event signal 165.n of plasma 300 at each time step in the control sequence 145.1-N. For convenience, process 1400.n will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification (e.g., Figure 3C The monitoring module 140) can execute process 1400.n.

[0204] The monitoring module 140 receives monitoring input 162.n, which includes: (i) a state prediction block 128.xn at the current time step, and / or (ii) a measurement prediction block 128.yn (1410.n) at the current time step. The state prediction block 128.xn includes the target state 122.xn at the current time step, the predicted state 124.xn, the state residual 125.xn, and the state covariance matrix 126.xn. The measurement prediction block 128.yn includes the physical measurement 122.yn at the current time step, the predicted measurement 124.yn, the measurement residual 125.yn, and the measurement covariance matrix 126.yn.

[0205] The monitoring module 140 performs statistical analysis on the state prediction block 128.xn and / or measurement prediction block 128.yn at the current time step to generate monitoring output 164.n, which includes: (i) a state health vector 132.xn at the current time step, and / or (ii) a measurement health vector 132.yn (1420.n) at the current time step. The state health vector 132.xn includes multiple statistical health indicators 35.1.xn to 35.hxn of the state prediction block 128.xn at the current time step. The measurement health vector 132.yn includes multiple statistical health indicators 35.1.yn to 35.hyn of the measurement prediction block 128.yn at the current time step.

[0206] The monitoring module 140 generates an event signal 165.n (1430.n) indicating the state health vector 132.xn and / or 134.xn, and measures whether one or more of the statistical health indicators 35.n in the health vector meet a threshold at the current time step.

[0207] The monitoring module 140 transmits system output 104.n to the user interface 150. This system output includes the monitoring input 162.n, the monitoring output 164.n, and the event signal 165.n (1440.n) at the current time step. In some implementations of process 1400.n, the system output 104.n at the current time step further includes the control action 145.n at the current time step.

[0208] In some implementations of process 1400.n, monitoring module 140 uses the monitoring output 164.n at the current time step to add control action 145.n (1450.n) at the current time step.

[0209] In some implementations of process 1400.n, monitoring module 140 receives user input 156.n (1460.n) from user interface 150 specifying user adjustment of linear magnetic controller 110L. Monitoring module 140 then adjusts linear magnetic controller 110L (1470.n) according to user input 156.n. For example, user input 156.n can specify a corresponding setpoint for each of one or more control parameters in a set of control parameters for linear magnetic controller 110L, and monitoring module 140 can initialize each of the one or more control parameters using the corresponding setpoint specified by user input 156.n.

[0210] Figure 4E It is used to execute the switching protocol to generate corresponding neural control actions 145 for a set of control coils 220 at each time step after the initiation time step. A flowchart of example process 1700.n. For convenience, process 1700.n will be described as being executed by a system of one or more computers located at one or more locations. For example, a system properly programmed according to this specification (e.g., Figures 3A to 3C The control system 100) can execute process 1700.n.

[0211] The control system 100 forwards input 144.n to the neural magnetic controller 110N network, which includes: (i) the target state 122.xn of the plasma 300 at the current time step, and (ii) the physical measurement 122.yn (1710.n) of the plasma 300 at the current time step.

[0212] The control system 100 uses a neuromagnetic controller 110N to process the network input 144.n at the current time step to generate a network output 146.n, which defines a strategy (1720.n) at the current time step for selecting the control action for the group of control coils 220.

[0213] The control system 100 generates neural control actions 145.n (1730.n) for a set of control coils 220 at the current time step based on the network output 146.n at the current time step.

[0214] The control system 100 initializes the control action 145.n at the current time step to the neural control action 145 at the current time step. .n (1740.n).

[0215] The control system 100 determines whether the event signal 165.n still indicates that the plasma 300 meets the trigger condition (1750.n) at the current time step.

[0216] If event signal 165.n still indicates that plasma 300 meets the triggering condition at the current time step, then control system 100 is interrupted to operation 1710.n.

[0217] If event signal 165.n no longer indicates that plasma 300 meets the triggering condition at the current time step, control system 100 is interrupted to operation 1760.n.

[0218] The control system 100 records the current time step as the termination time step (1760.n).

[0219] The control system 100 terminates the switching protocol (1770.n) at the termination time step. Then, the linear magnetic controller 110L can take over control of the magnetic confinement device 200 again when the switching protocol is terminated.

[0220] VI. Examples of Magnetic Confinement Training Systems

[0221] Figure 5A This is a schematic diagram illustrating an example of a training system 20 configured to train either a linear magnetic controller 110L or a neural magnetic controller 110N of a control system 100 on a model 500 of a magnetically restrained device 200 via reinforcement learning techniques. That is, the magnetic controller 110 mentioned in this example of training system 20 can be either a linear magnetic controller 110L or a neural magnetic controller 110N, as distinguished where appropriate. Training system 20 is an example of a system implemented as a computer program on one or more computers at one or more locations, implementing the systems, components, and techniques described below.

[0222] Generally, the reinforcement learning technique implemented by the training system 20 can be any type of reinforcement learning technique. For example, the reinforcement learning technique can be an actor-commentator reinforcement learning technique, a Q-learning technique, a SARSA learning technique, a deep Q-network learning technique, a Dyna-Q reinforcement learning technique, a REINFORCE learning technique, a neighbor-policy optimization learning technique, or a natural policy gradient learning technique. In the described example, the reinforcement learning technique is an actor-commentator reinforcement learning technique, such as a distributed actor-commentator reinforcement learning technique, for example, utilizing a synchronous method to train the magnetic controller 110. Additional examples of actor-commentator reinforcement learning techniques are described in "Asynchronous Methods for Deep Reinforcement Learning" by Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirzav, Alex Gravesv, Timothy Lillicrap, Alex Graves, David Silver, and Koray Kavukcuoglu. International Conference on Machine Learning , pp. 1928-1937, PmLR (2016).

[0223] Training system 20 includes a set of actors 22 and learners 24. Training system 20 may include at least about 2, 4, 8, 16, 32, 64, 128, 256, 512 or more actors 22, or within any two of these values. aEach actor 22a in the indexed group of actors 22 performs a corresponding series of rounds, where each round includes a corresponding instance of the magnetic controller 110 interacting with the simulation 50. The simulation 50, also referred to as the "simulation environment," includes (or defines) a model 500 of the magnetic constraint device 200 and a reward function 540 of the magnetic controller 110. Here, the group of actors 22 executes a shared model of the magnetic controller 110. That is, a set of control parameters of the magnetic controller 110 is shared among each instance of the magnetic controller 110.

[0224] When the termination condition is met in simulation 50 or when a fixed simulation time has elapsed in the round, actor 22.a terminates the round. Upon completion of the round, training system 20 records the simulation trajectory (D) of that round. a 612.a is forwarded to learner 24, where the simulation trajectory 612.a represents the corresponding instance of magnetic controller 110 that interacts with simulation 50 during the round. Learner 24 can then use simulation trajectory 612.a to train magnetic controller 110, for example, as the group of actors 22 execute their respective series of rounds in parallel with each other, to continuously update the global group control parameters of magnetic controller 110, for example, until magnetic controller 110 learns an optimal (or near-optimal) control policy.

[0225] A. Model

[0226] Generally, model 500 of the magnetic confinement device 200 is a nonlinear model of the magnetic confinement device 200. In this example, model 500 is a first-order time nonlinear model, including: (i) a state transition model of the plasma 300 magnetically coupled to a set of control coils 220 ( f ) 515, and (ii) a diagnostic model for a set of sensors 230 ( g 530. Therefore, the model 500 of the magnetic confinement device 200 can be represented by the numerical code of simulation 50 as follows:

[0227] The state transition model 515 and the diagnostic model 530 are each nonlinear in their respective independent variables.

[0228] The state transition model 515 is configured to generate a virtual state of plasma 300 at the next virtual time step in response to the model input. 112 (n+1), the model input includes: (i) the virtual state of plasma 300 at the current virtual time step. 112 .n, and (ii) the control action for a set of control coils 220 at the current virtual time step ( un 145.n. In simulation 50, virtual state 112 .n is used as the physical state 112.n, that is The task of the magnetic controller 110 is to guide the virtual state 112 during the analog control loop. The particle .n moves toward the target state 122.xn. Here, simulation 50 also uses plasma noise to model the uncertainties in plasma 300. To enhance the state transition model 515. In this case, simulation 50 is configured to sample the plasma noise at the current virtual time step and use the plasma noise sampled at the current virtual time step to adjust the virtual state 112 of plasma 300 at the next virtual time step. Incrementing by (n+1), for example, this elevates the state transition model 515 from a deterministic model to a stochastic model.

[0229] Diagnostic model 530 is configured to respond to the virtual state of plasma 300 at the current virtual time step. 112 The .n parameter is used to generate a virtual measurement of the plasma at the current virtual time step of 300. 122 .yn. Virtual measurement of plasma 300 at each virtual time step 122 .yn (given as) ) characterizes the corresponding virtual state 112 of plasma 300 at this virtual time step. In simulation 50, virtual measurement 122. .yn is used for physical measurement 122.yn, that is Furthermore, the magnetic controller 110 processes virtual measurements 122 during the analog control loop. Instead of the physical measurement 122.yn, the simulation 50 also uses sensor noise to model the uncertainties in a set of sensors 230. To enhance the diagnostic model 530. In this case, simulation 50 is configured to sample the sensor noise at the current virtual time step and use the sensor noise sampled at the current virtual time step to make the virtual measurement 122 of the plasma 300 at the current virtual time step. The .yn increments, for example, thus upgrading the diagnostic model 530 from a deterministic model to a stochastic model.

[0230] In this example, the state transition model 515 is derived from the magnetohydrodynamic model 510 of the plasma 300, which is magnetically coupled to the power supply model 520 of the set of control coils 220. For example, the magnetohydrodynamic model 510 and the power supply model 520 are magnetically coupled to each other via corresponding boundary conditions and mutual inductance. The magnetohydrodynamic model 510 can be a resistive or ideal magnetohydrodynamic model of the plasma 300, including non-equilibrium or equilibrium models (e.g., quasi-static resistive or static ideal magnetohydrodynamic models).

[0231] As a specific example, the magnetohydrodynamic model 510 can be, for example, an equilibrium magnetohydrodynamic model described by the Grad-Shafranov equations under free or fixed boundary conditions, which are generated by the force balance between the pressure gradient within plasma 300 and the Lorentz force induced on plasma 300 by the magnetic field. In this case, within the boundary 310.lcfs, the equilibrium magnetohydrodynamic model 510 of plasma 300 is given as:

[0232] in It is the Shafranov operator (also known as the polar Laplace operator). It is the permeability of free space. It is the circumferential current density of plasma 300. p It is the pressure distribution of plasma 300, and This is the flux function of the circumferential magnetic field within the boundary 310.lcfs. Simulation 50 can specify the pressure distribution and / or the circumferential flux function as a given function of the poloidal magnetic flux, for example, a function with a power-law dependence of the poloidal magnetic flux or a function derived from a physics-based model. With the pressure distribution and circumferential flux function specified, Simulation 50 can implement numerical code in radial (...) coordinates of the cylindrical coordinate system. r ) and vertical ( z The coordinates are used to solve for the poloidal magnetic flux described in equation (12a). Examples of such numerical codes for reconstructing magnetohydrodynamic equilibrium include, but are not limited to, EFIT, LIUQE, CHEASE, HELENA, JSOLVER, ESC, FREEGS, TokaMac, VMEC, PIES, STELLOPT, PEST, CORSICA, and OPAL.

[0233] Generally speaking, for the virtual state 112 of plasma 300 .n (given as) The equilibrium magnetohydrodynamic model 510 responds to the virtual current configuration vector. To generate poloidal magnetic flux This virtual current configuration vector specifies the corresponding virtual value of the corresponding control current in each control coil 220.c. Then, simulation 50 is performed based on the poloidal magnetic flux. Determine the virtual plasma configuration vector This virtual plasma configuration vector specifies the corresponding virtual value for each configuration parameter of plasma 300. For example, simulation 50 can be configured according to... The virtual value of the plasma current at plasma 300 is obtained, based on... The virtual value of the safety factor of plasma 300 was obtained, based on The virtual value of the center 305 of plasma 300 is obtained, and so on.

[0234] In this example, power supply model 520 is a circuit model of a set of control coils 220 inductively coupled to plasma 300, such as a lumped element circuit model. Power supply model 520 describes a response to a target voltage configuration vector specified by control action 145.n. The evolution of the virtual current configuration vector. For example, it includes the back electromotive force of the plasma current from plasma 300 on a set of control coils 220. The power supply model 520 can be provided in discrete time as follows:

[0235] in M It is an inductor matrix between a set of control coils 220. R It is a resistor matrix that controls the 220 coils. It is a set of mutual inductance vectors between control coil 220 and plasma 300, and This is the incremental time (or length) for each virtual time step. Note that the power supply model 520 may also include inductive coupling with a passive conductor located within and / or around chamber 210, the passive conductor having a current that evolves under the influence of the target voltage configuration vector. The state transition model 515 is then a numerical code of simulation 50 that realizes the solution (e.g., inverse solution) of the nonlinear coupling equations (12a)-(12b). Examples of such numerical codes for time-evolving magnetohydrodynamic equilibrium include, but are not limited to, FGE, DINA, CREATE-NL, CREATE-L, CREATE-D, TSC, CORSICA, and JOREK.

[0236] B. actor

[0237] Each round, executed by actor 22.a, involves a corresponding instance of magnetic controller 110 interacting with simulation 50. This round has multiple virtual time steps. n = 1, 2…N ,in N This is the total number of virtual time steps in this round, corresponding to a fixed simulation time of 50. Each virtual time step can have the same length as a physical time step. In other words, the control rate of the magnetic controller 110 on model 500 can be the same as the control rate of the magnetic controller 110 on the magnetic restraint device 200 in the real world. If at the virtual time step... n < N If the termination condition in simulation 50 is met, then actor 22.a terminates the round at that virtual time step. Although model 500 of the magnetic confinement device 200 is generally accurate, there are dynamics of physical state 112.n that may be affected by virtual state 112. The area that is poorly represented by .n, or the simulation 50 outside the operating limits of the magnetic confinement device 200.

[0238] To initiate a round, actor 22.a is configured to receive target state trajectories 122.x.1-N, which include the corresponding target state 112.xn of plasma 300 at each virtual time step in the round. Then, at each virtual time step in the round, actor 22.a is configured to perform the following operation to execute the analog control loop of magnetic controller 110: Actor 22.a, based on the virtual state 112 of plasma 300 at the current virtual time step. The .n method determines whether the physical feasibility constraints of the magnetic confinement device 200 are violated at the current virtual time step. Here, the physical feasibility constraint means the virtual state 112 of the plasma 300. It would be feasible to define the physical state 112.n of plasma 300 as .n, for example, based on constraints on the magnetic configuration of plasma 300 and / or the operating conditions of a set of control coils 220. For example, for each configuration parameter in a set of configuration parameters of plasma 300, actor 22.a can determine whether the corresponding virtual value of the configuration parameter at the current virtual time step satisfies the corresponding threshold. As another example, for each control coil 220.c in a set of control coils 220, actor 22.a can determine whether the corresponding virtual value of the corresponding control current in control coil 220.c at the current virtual time step satisfies the threshold.

[0239] If the physical feasibility constraint of the magnetic constraint device 200 is violated at the current virtual time step, actor 22.a will terminate simulation 50 at that current virtual time step. If the physical feasibility constraint of the magnetic constraint device 200 is not violated at the current virtual time step, actor 22.a will continue to use the magnetic controller 110 to generate control action 145.n at the current virtual time step.

[0240] Actor 22.a receives simulation output from simulation 50, which includes: (i) a virtual state 112 characterizing the plasma 300 at the current virtual time step. Virtual measurement of 300 plasmas of .n 122 .yn, and (ii) based on the target state 122.xn and virtual state 112 of plasma 300 at the current virtual time step. The state tracking error between .n is a reward of 542.n for the magnetic controller 110. Actor 22.a forwards system input 102.n to the magnetic controller 110, the system input including: (i) the target state 122.xn of plasma 300 at the current virtual time step, and (ii) the virtual measurement 122 of plasma 300 at the current virtual time step. Actor 22.a uses magnetic controller 110 to process system input 102.n to generate control action 145.n for a set of control coils 220 at the current virtual time step. Then, actor 22.a transmits the control action 145.n at the current virtual time step to simulation 50 to simulate the magnetic field that confines plasma 300 within chamber 210 according to the control action; Actor 22.a then proceeds to the next virtual time step in the round ( n At + 1), the analog control loop of the magnetic controller 110 is executed, and in the next virtual time step of the round ( n The analog control loop is executed at +2), and so on.

[0241] In each virtual time step of the round ( n At point ), simulation 50 is configured to perform the following operations to simulate the evolution of plasma 300: Simulation 50 receives model input from model 500, which includes: (i) the virtual state 112 of plasma 300 at the previous virtual time step. (i) and (ii) the control actions 145(n-1) of a set of control coils 220 at the previous virtual time step. Simulation 50 processes the model input at the current virtual time step using the state transition model 515 of plasma 300 to generate the virtual state 112 of plasma 300 at the current virtual time step. The simulation 50 uses a set of sensors 230 to process the virtual state of the plasma 300 at the current virtual time step 112. .n, to generate a virtual measurement 122 of plasma 300 at the current virtual time step. .yn. Simulation 50 returns the model output of model 500, which includes: (i) the virtual state 112 of plasma 300 at the current virtual time step. .n, and (ii) the virtual measurement of plasma 300 at the current virtual time step 122 .yn.

[0242] Simulation 50 receives the target state 122.xn and virtual state 112 at the current virtual time step. State tracking error between .n Given as The state tracking error measures the target state 122.xn and the virtual state 112 of plasma 300 at the current virtual time step. The deviation between .n. Simulation 50 uses reward function 540 to process the state tracking error at the current virtual time step to generate a reward 542.n for the magnetic controller 110 at the current virtual time step. Therefore, given as The reward 542.n is a function of the state tracking error at the current virtual time step. Then, simulation 50 returns the simulation output at the current virtual time step, which includes: (i) the virtual measurement 122 of plasma 300 at the current virtual time step. .yn, and (ii) the reward 542.n for the magnetic controller 110 at the current virtual time step.

[0243] In general, the state tracking error is the amount that the magnetic controller 110 seeks to minimize at each virtual time step. For example, the reward 542.n could be the target state 122.xn of plasma 300 and the virtual state 112. Mahalanobis distance between .n D n The Mahalanobis distance is a monotonically decreasing function of the state tracking error and the plasma noise covariance matrix. Q The square root of the inner product of the inverse matrix of is shown below:

[0244] For example, reward 542.n can be a negative logic function, a negative linear ramp function, a negative quadratic function, a negative exponential function, or the target state 122.xn and virtual state 112 of plasma 300. Other monotonically decreasing functions of the Mahalanobis distance between n.

[0245] Simulation 50 then proceeds to the next virtual time step in the round ( n The simulated evolution of plasma 300 is performed at +1), and in the next virtual time step of the round ( n The simulation evolution is performed at +2), and so on.

[0246] In the final virtual time step n = NAfter completing the round, actor 22.a forwards the simulated trajectory 612.a for that round to learner 24. Simulated trajectory 612.a may include some or all of the simulated data generated at each virtual time step in that round. In this example, given as The simulated trajectory 612.a includes: (i) Given as Virtual measurement trajectory of plasma 300 122 .y.1-N, which includes the corresponding virtual measurements of plasma 300 at each virtual time step in the round. 122 .yn; (ii) Given as The reward trajectory (R) 542.1-N of the magnetic controller 110 includes the corresponding reward for the magnetic controller 110 at each virtual time step in the round. r n 542.n; (iii) Given as The target state trajectory of plasma 300 122.x.1-N, which includes the corresponding target state of plasma 300 at each virtual time step in the round. 122.xn; (iv) Given as Virtual state trajectory of plasma 300 112 .1-N, which includes the corresponding virtual state of plasma 300 at each virtual time step in the round. 112 .n; and (v) is given as The control sequence (U) 145.1-N generated by the magnetic controller 110 includes corresponding control actions for a set of control coils 220 at each virtual time step of the round. u n )145.n.

[0247] Then, learner 24 can use analog trajectory 612.a to train magnetic controller 110, while actor 22.a repeats this process for each round of its corresponding series of rounds in which magnetic controller 110 interacts with analog 50.

[0248] C. learning device

[0249] The learner 24 includes a commentator neural network 610 and a replay buffer 620, wherein the replay buffer 620 stores each simulated trajectory 612a forwarded to the learner 24 for training the magnetic controller 110. The learner 24 is configured to periodically sample the simulated trajectory 612 of each round from the replay buffer 620, for example, by random sampling or sampling according to an algorithm, and jointly train the magnetic controller 110 and the commentator neural network 610 on that simulated trajectory 612 of that round.

[0250] Specifically, for each virtual time step in a round ( n Learner 24 is configured to perform the following operations: Learner 24 samples the corresponding commentator network input 622.n from the simulated trajectory 612 at the virtual time step, which includes: (i) a virtual measurement 122 of the plasma 300 at the virtual time step. .yn, and (ii) the reward 542.n for the magnetic controller 110 at the virtual time step. In some implementations, the commentator network input 622.n further includes one or more of the following: the target state 122.xn of the plasma 300, the virtual state 112 of the plasma 300, and the virtual state 112 of the plasma 300. The control action at the virtual time step is 145.n. The learner 24 uses a commentator neural network 610 based on its set of commentator network parameters. The commentator network input 622.n for the virtual time step is processed to generate the commentator network output 624.n for that virtual time step. The commentator network output 624.n includes the value function. This value function estimates the expected return for a virtual time step in a round ( R n For example, the expected reward for a virtual time step could be the sum of expected discounts on future rewards:

[0251] It includes each subsequent virtual time step in the round ( k > n The corresponding reward is the sum of expected discounts of 542.(n+1)-N, where It is the discount factor of the total discount, and the value of this discount factor is in Within the range.

[0252] Learner 24 calculates the time difference (“TD”) error for virtual time steps in a round. Given as The time difference error includes the following errors: (i) the time difference objective for the virtual time step. , and (ii) the value function for virtual time steps Then, learner 24 calculates the advantage function. This includes the time difference error for virtual time steps within a round. For example, the advantage function for virtual time steps could be a one-step advantage function. As another example, the advantage function for a virtual time step can be a multi-step (e.g., generalized) advantage function. It includes the sum of decays of the corresponding time difference errors for the virtual time step and each subsequent virtual time step, where It is the attenuation factor of the total attenuation, and the value of this attenuation factor is in Within the range.

[0253] Learner 24 generates commentator objective function 630.crtc, which includes the squared magnitude of the corresponding time difference error for each virtual time step in the round:

[0254] Learner 24 relative to a set of commentator network parameters of commentator neural network 610 The learner optimizes (e.g., minimizes) the commentator objective function 630.crtc. For example, learner 24 can compute the gradient of the commentator objective function 630.crtc with respect to a set of commentator network parameters of the commentator neural network 610, for example, using backpropagation, and use this gradient to iteratively update the set of commentator network parameters in multiple training iterations. Learner 24 can use a gradient descent method with a specific learning rate and / or weight decay (e.g., stochastic), such as implicit update, momentum method, Adam, RMSProp, AdaGrad, etc., to update the set of commentator network parameters of the commentator neural network 610 with the optimized value of the commentator objective function 630.crtc. Learner 24 can perform any appropriate number of training iterations to optimize the commentator objective function 630.crtc. For example, learner 24 can perform at least about 100, 1,000, 10,000, 100,000 or more training iterations, or any range between any two of these values.

[0255] Learner 24 generates actor objective function 630.actr, which includes the corresponding advantage function for each virtual time step in the round. For example, for a linear magnetic controller 110L implementing a deterministic control policy, the actor objective function 630.actr can be given as:

[0256] Here, the actor objective function 630.actr comprises the sum of the corresponding advantage functions for each virtual time step in the round. As another example, for a neural magnetic controller 110N implementing a stochastic control policy, the actor objective function 630.actr can be given as:

[0257] Thus, the actor objective function 630.actr comprises the sum of the corresponding advantage functions for each virtual time step in the round, multiplied by the log-likelihood of the corresponding control action 145.n sampled from the probability distribution at the virtual time step. Specifically, the log-likelihood comprises the probability distribution evaluated for the control action 145.n at the virtual time step. The logarithm of the probability distribution, where the probability distribution is determined by a set of distribution parameters at the virtual time step. Parameterization.

[0258] Learner 24 optimizes (e.g., maximizes) the actor objective function 630.actr with respect to a set of control parameters of magnetic controller 110. For example, learner 24 can compute the gradient of the actor objective function 630.actr with respect to a set of control parameters of magnetic controller 110 using backpropagation, and iteratively update the set of control parameters using this gradient over multiple training iterations. Learner 24 can use gradient descent methods with specific learning rates and / or weight decay (e.g., stochastic) such as implicit updates, momentum methods, Adam, RMSProp, AdaGrad, etc., to update the set of control parameters of magnetic controller 110 with the optimized value of actor objective function 630.actr. Learner 24 can perform any appropriate number of training iterations to optimize actor objective function 630.actr. For example, learner 24 can perform at least about 100, 1,000, 10,000, 100,000 or more training iterations, or any range between any two of these values.

[0259] In some implementations, learner 24 optimizes (e.g., maximizes) the actor objective function 630.actr with respect to a set of control parameters of magnetic controller 110, while being subject to one or more constraints. For example, for linear magnetic controller 110L, learner 24 can optimize the actor objective function 630.actr while being subject to the constraint that state feedback controller 130 satisfies the Routh-Hurwitz stability criterion. Learner 24 can implement the Routh-Hurwitz stability criterion by specifying a set of controller parameters of state feedback controller 130. The magnetic confinement system 10 is configured such that each closed-loop pole lies within a unit circle, for example, where each closed-loop pole is a function of the sensitivity of the magnetic confinement system 10. The corresponding roots of the determinant, for example, related to the definition of the sensitivity function. See Equations (4a)-(4b).

[0260] Generally, the commentator neural network 610 can have any suitable neural network architecture that enables it to perform the described function—that is, to process the commentator network input 622 to generate the commentator network output 624. Specifically, the commentator neural network 610 can include any suitable type of neural network layers (e.g., fully connected layers, recurrent layers, convolutional layers, self-attention layers, etc.) connected in any suitable number (e.g., 5 layers, 25 layers, or 100 layers) and in any suitable configuration (e.g., as a linear sequence of layers, residual configuration, etc.). For example, the commentator neural network 610 can be a feedforward neural network, a recurrent neural network, a convolutional neural network, an attention neural network, or a combination thereof. Furthermore, since the commentator neural network 610 is implemented only during training, it can include a large number of neural network layers, such as 10 or more, 15 or more, 20 or more, 25 or more, 50 or more, or 100 or more. Therefore, the magnetic controller 110 typically has fewer neural network layers and / or fewer network parameters than the commentator neural network 610.

[0261] Note that due to the virtual state 112 of model 500 for evolving plasma 300 Due to the computational requirements of .n, the data acquisition rate of the training system 20 may be slower compared to typical reinforcement learning environments (e.g., those used for computer games). To overcome the inadequacy of simulated data, the training system 20 can implement maximum a posteriori policy optimization (“MPO”) techniques—for example, as described in Abdolmaleki et al.'s "Maximum a Posteriori Policy Optimization". arXiv:1806.06920 The technique described in (2018) – or variations thereof. MPO supports a distributed architecture that can collect data across multiple parallel streams. Specifically, the distributed architecture allows a set of global parameters to be defined for the magnetic controller 110 and the commentator neural network 610, for example, in central memory. Multiple parallel streams (e.g., independent threads, GPUs, TPUs, CPUs, etc.) can each execute a corresponding local instance of training system 20 using the corresponding set of local parameters of the magnetic controller 110 and the commentator neural network 610. Each stream can then update the set of global parameters using the corresponding set of local parameters generated by the corresponding local instance of training system 20. This approach can significantly accelerate the training process of the magnetic controller 110.

[0262] Figure 5B This is a flowchart of an example process 1800 for training a magnetic controller 110 on a model 500 of a magnetic confinement device 200 via reinforcement learning techniques. For convenience, process 1800 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification (e.g., Figure 5A The training system 20) can execute process 1800.

[0263] The training system 20 receives the target state trajectory 122.x.1-n (1810) of the plasma 300 at each of the multiple virtual time steps, which includes the corresponding target state 122.xn.

[0264] The training system 20 executes simulation 50 (1820) within a virtual time step, which includes model 500 of magnetic constraint device 200 and reward function 540 of magnetic controller 110.

[0265] At each virtual time step, training system 20 performs operations (1830.n) to (1860.n): Training system 20 receives simulation output from simulation 50, which includes: (i) a virtual state 112 characterizing the plasma 300 at the current virtual time step. Virtual measurement of 300 plasmas of .n 122 .yn, and (ii) based on the target state 122.xn and virtual state 112 of plasma 300 at the current virtual time step. The state tracking error between .n is 542.n (1830.n) for magnetic controller 110.

[0266] The training system 20 forwards system input 102.n to the magnetic controller 110. This system input includes: (i) the target state 122.xm of the plasma 300 at the current virtual time step, and (ii) the virtual measurement 122 of the plasma 300 at the current virtual time step. .yn (1840.n).

[0267] The training system 20 uses the magnetic controller 110 to process the system input 102.n at the current virtual time step to generate control actions 145.n (1850.n) for a set of control coils 220 at the current virtual time step.

[0268] Then, the training system 20 transmits the control action 145.n at the current virtual time step to the simulation 50 to simulate the magnetic field (1860.n) that confines the plasma 300 within the chamber 210 according to the control action.

[0269] The training system 20 returns a simulation trajectory 612 (1880) including the corresponding simulation output generated by the simulation 50 at each virtual time step.

[0270] The training system 20 trains the magnetic controller 110 (1880) on the simulated trajectory 612 using reinforcement learning techniques.

[0271] Scope of this disclosure

[0272] This specification uses the term "configured" in conjunction with system and computer program components. For a system of one or more computers to be configured to perform a specific operation or action, this means that the system has software, firmware, hardware, or a combination thereof installed thereon that causes the system to perform those operations or actions in operation. For one or more computer programs configured to perform a specific operation or action, this means that the one or more programs include instructions that, when executed by a data processing device, cause that device to perform that operation or action.

[0273] Embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their equivalents), or in combinations thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination thereof. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals (e.g., machine-generated electrical, optical, or electromagnetic signals) generated to encode information for transmission to a suitable receiver device for execution by the data processing device.

[0274] The term "data processing device" refers to data processing hardware and includes all kinds of devices, apparatuses, and machines for processing data, such as programmable processors, computers, or multiple processors or computers. The device may also be or further include a dedicated logic circuit system, such as a field-programmable gate array ("FPGA") or an application-specific integrated circuit ("ASIC"). In addition to hardware, the device may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0275] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages); and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not need to, correspond to a file in a file system. A program may be stored as a part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a single computer or on multiple computers located at a site or distributed across multiple sites and interconnected via a data communication network.

[0276] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines may be installed and run on the same one or more computers.

[0277] The processes and logic flows described in this specification can be executed by one or more programmable computers, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by a dedicated logic circuit system (e.g., an FPGA or ASIC) or by a combination of a dedicated logic circuit system and one or more programmable computers.

[0278] A computer suitable for executing computer programs may be based on a general-purpose or special-purpose microprocessor or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory or both. The basic elements of a computer are the central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into a special-purpose logic circuit system. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to or from them or both. However, a computer need not have such devices. Furthermore, a computer may be embedded in another device (e.g., a mobile phone, a personal digital assistant (“PDA”), a mobile audio or video player, a game console, a global positioning system (“GPS”) receiver, or a portable storage device, such as a universal serial bus (“USB”) flash drive, to name a few).

[0279] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0280] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device (e.g., a cathode ray tube (“CRT”) or liquid crystal display (“LCD”) monitor for displaying information to the user) and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's device in response to a request received from a web browser. Similarly, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and receiving response messages from the user in response.

[0281] Data processing devices used to implement machine learning models may also include, for example, dedicated hardware accelerator units for handling the general and computationally intensive portions of machine learning training or production (i.e., inference, workloads).

[0282] Machine learning models can be implemented and deployed using machine learning frameworks such as TensorFlow or JAX.

[0283] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer having a graphical user interface, web browser, or app that a user can interact with through an implementation of the subject matter described in this specification), or any combination of one or more such back-end components, middleware components, or front-end components. Components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), such as the Internet.

[0284] A computing system may include clients and servers. Clients and servers are typically remote to each other and interact via a communication network. The client-server relationship is established by computer programs executed on respective computers that establish a client-server relationship between them. In some embodiments, the server transmits data (e.g., HTML pages) to a user device, for example, for the purpose of displaying data to a user interacting with the device acting as a client and receiving user input from that user. Data generated at the user device, such as the result of user interaction, may be received at the server from the device.

[0285] Other embodiments

[0286] In addition to the embodiments described in the appended claims and the embodiments described above, the following numbered embodiments are also innovative: 1. A method for controlling a magnetic confinement device executed by one or more computers, the magnetic confinement device including a chamber and a plurality of control coils configured to generate a magnetic field confining a plasma within the chamber according to a control sequence, the control sequence including a corresponding control action at each of a plurality of time steps, the method comprising: receiving a target state trajectory including a corresponding target state of the plasma at each of the plurality of time steps; and at each of the plurality of time steps: receiving from the magnetic confinement device a physical state of the plasma characterizing the physical state of the plasma at the current time step. Measurement; forwarding system inputs to a linear magnetic controller, the system inputs including: (i) the target state of the plasma at the current time step, and (ii) the physical measurements of the plasma at the current time step; processing the system inputs at the current time step using the linear magnetic controller to generate the control actions for the plurality of control coils at the current time step, wherein the linear magnetic controller has been trained on a model of the magnetic confinement device via reinforcement learning techniques; and transmitting the control actions at the current time step to the magnetic confinement device to generate the magnetic field confining the plasma within the chamber based on the control actions.

[0287] 2. The method according to Embodiment 1, wherein the linear magnetic controller includes a state observer and a state feedback controller, the state observer and the state feedback controller having been jointly trained on the model of the magnetic confinement device via the reinforcement learning technique.

[0288] 3. The method according to Embodiment 2, wherein the state observer is configured as a Kalman filter, a Lombard observer, or a linear neural observer.

[0289] 4. The method according to any one of embodiments 1 to 3, wherein the state feedback controller is configured as a proportional-integral-derivative controller, a quadratic regulator, an H-infinity controller, a full-state feedback controller, a linear model predictive controller, or a linear neural controller.

[0290] 5. The method according to any one of Embodiments 2 to 4, wherein: the state observer is configured to, at each of the plurality of time steps: receive the physical measurement of the plasma at the current time step; and process the physical measurement of the plasma at the current time step according to a set of observer parameters to generate a predicted state of the plasma that estimates the physical state of the plasma at the current time step; the state feedback controller is configured to, at each of the plurality of time steps: receive a controller input, the controller input including: (i) the target state of the plasma at the current time step, and (ii) the predicted state of the plasma at the current time step; and process the controller input at the current time step according to a set of controller parameters to generate the control action at the current time step; and the set of observer parameters and the set of controller parameters have been jointly trained on the model of the magnetic confinement device via the reinforcement learning technique.

[0291] 6. The method according to Embodiment 5, wherein processing the physical measurements of the plasma at the current time step according to the set of observer parameters to generate a predicted state of the plasma at the current time step includes, at each of the plurality of time steps: receiving a prior observer input, the a prior observer input including: (i) the predicted state of the plasma at a previous time step, and (ii) the control action at the previous time step; processing the a prior observer input at the current time step according to the set of observer parameters to generate a prior observer output, the a prior observer output including: (i) not based on the current time step The physical measurement of the plasma is a priori predicted state of the plasma that conditionally estimates the physical state of the plasma at the current time step, and (ii) a predicted measurement of the plasma that estimates the physical measurement of the plasma at the current time step; receiving a posteriori observer input, the posteriori observer input including: (i) the prior observer output at the current time step, and (ii) the physical measurement of the plasma at the current time step; and processing the posteriori observer input at the current time step according to the set of observer parameters to generate the predicted state of the plasma at the current time step.

[0292] 7. The method according to Embodiment 6, wherein the set of observer parameters includes a state transition matrix, a control action matrix, and a diagnostic matrix, and wherein processing the prior observer input at the current time step according to the set of observer parameters to generate the prior observer output at the current time step includes, at each of the plurality of time steps: calculating a first matrix-vector product between the state transition matrix and the predicted state of the plasma at the previous time step; calculating a second matrix-vector product between the control action matrix and the control action at the previous time step; returning the sum of the first matrix-vector product and the second matrix-vector product at the current time step as the prior predicted state of the plasma at the current time step; and returning a third matrix-vector product between the diagnostic matrix and the prior predicted state of the plasma at the current time step as the predicted measurement of the plasma at the current time step.

[0293] 8. The method according to any one of Embodiments 6 to 7, wherein processing the posterior observer input at the current time step according to the set of observer parameters to generate the predicted state of the plasma at the current time step includes, at each of the plurality of time steps: generating an observer gain matrix at the current time step according to the set of observer parameters; generating a measurement residual, the measurement residual including the deviation between the physical measurement of the plasma at the current time step and the predicted measurement; generating an incremental state, the incremental state including a fourth matrix-vector product between the observer gain matrix at the current time step and the measurement residual; and incrementing the prior predicted state of the plasma at the current time step with the incremental state at the current time step to generate the predicted state of the plasma at the current time step.

[0294] 9. The method according to Embodiment 8, wherein the set of observer parameters further includes a target gain matrix, and wherein the incremental state at the current time step further includes a matrix-vector product of the following: (i) the target gain matrix, and (ii) a priori state residual, the priori state residual including the deviation between the target state of the plasma at the current time step and the prior predicted state.

[0295] 10. The method according to any one of Embodiments 8 to 9 when it is also subordinate to Embodiment 7, wherein generating the observer gain matrix at the current time step based on the set of observer parameters includes, at each of the plurality of time steps: receiving the state covariance matrix at the previous time step; calculating a first matrix product between the state transition matrix, the state covariance matrix at the previous time step, and the transpose of the state transition matrix; generating a prior state covariance matrix including the first matrix product at the current time step and the plasma noise covariance matrix; calculating a second matrix product between the diagnostic matrix, the prior state covariance matrix at the current time step, and the transpose of the diagnostic matrix; generating a prior state covariance matrix including the first matrix product at the current time step and the plasma noise covariance matrix; calculating a second matrix product between the diagnostic matrix, the prior state covariance matrix at the current time step, and the transpose of the diagnostic matrix; generating a prior state covariance matrix including the first matrix product at the current time step and the plasma noise covariance matrix; and generating a prior state covariance matrix including the first matrix product at the current time step and the plasma noise covariance matrix. The measurement covariance matrix is ​​obtained by multiplying the second matrix product at the previous time step and the sensor noise covariance matrix; the third matrix product is obtained by multiplying the prior state covariance matrix, the transpose of the diagnostic matrix, and the inverse of the measurement covariance matrix at the current time step as the observer gain matrix at the current time step; the fourth matrix product is obtained by multiplying the observer gain matrix, the transpose of the diagnostic matrix, and the prior state covariance matrix at the current time step; and the prior state covariance matrix at the current time step is decreased by the fourth matrix product at the current time step to generate the state covariance matrix at the current time step.

[0296] 11. The method according to any one of embodiments 5 to 10, wherein at each of the plurality of time steps, the controller input at the current time step is a state residual, the state residual including the deviation between the target state and the predicted state of the plasma at the current time step.

[0297] 12. The method according to any one of Embodiments 5 to 11, wherein: the set of controller parameters includes: one or more controller memory matrices, each controller memory matrix associated with a corresponding time step preceding the current time step; and one or more controller gain matrices, each controller gain matrix associated with the current time step or a corresponding time step preceding the current time step, and the controller input at the current time step is processed according to the set of controller parameters to generate the control action at the current time step, including at each of the plurality of time steps: for each controller memory matrix, a corresponding matrix-vector product is calculated between (i) the controller memory matrix and (ii) the control action at the corresponding time step preceding the current time step; for each controller gain matrix, a corresponding matrix-vector product is calculated between (i) the controller gain matrix and (ii) the controller input at the current time step or the corresponding time step preceding the current time step; and the sum of the corresponding matrix-vector products including each controller memory matrix and controller gain matrix at the current time step is generated as the control action at the current time step.

[0298] 13. The method according to any one of Embodiments 6 to 12, further comprising, at each of the plurality of time steps: receiving a monitor input, the monitor input comprising: (i) a state prediction block, the state prediction block comprising: (a) the target state of the plasma at the current time step; (b) the predicted state of the plasma at the current time step; (c) a state residual, the state residual comprising the deviation between the target state and the predicted state of the plasma at the current time step; and (d) a state covariance matrix, the state covariance matrix characterizing the uncertainty of the predicted state of the plasma at the current time step; And / or (ii) a measurement prediction block, the measurement prediction block comprising: (a) the physical measurement of the plasma at the current time step; (b) the predicted measurement of the plasma at the current time step; (c) a measurement residual, the measurement residual comprising the deviation between the physical measurement of the plasma at the current time step and the predicted measurement; and (d) a measurement covariance matrix, the measurement covariance matrix characterizing the uncertainty of the predicted measurement of the plasma at the current time step; and processing the monitor input at the current time step to generate an event signal indicating whether the plasma satisfies the triggering condition at the current time step.

[0299] 14. The method according to Example 13, wherein processing the monitor input at the current time step to generate the event signal indicating whether the plasma meets a trigger condition at the current time step includes, at each of the plurality of time steps: performing statistical analysis on the state prediction block and / or the measurement prediction block at the current time step to generate a monitor output, the monitor output including: (i) a state health vector including a plurality of statistical health indicators of the state prediction block at the current time step; and / or (ii) a measurement health vector including a plurality of statistical health indicators of the measurement prediction block at the current time step; and generating the event signal indicating whether one or more of the plurality of statistical health indicators in the state health vector and / or the measurement health vector meet a corresponding threshold at the current time step.

[0300] 15. The method according to embodiment 14 further includes, at each of the plurality of time steps, using the monitor output at the current time step to enhance the control action at the current time step, so as to generate the magnetic field constraining the plasma according to the control action.

[0301] 16. The method according to any one of Examples 14 to 15, wherein at each of the plurality of time steps: the statistical analysis includes one or more of the following: anomaly analysis, instability analysis, or nonlinear analysis; and the plurality of statistical health indicators include one or more of the following: Mahalanobis distance, anomaly probability, local Lyapunov index, global Lyapunov index, Kolmogorov-Sinai entropy, or a set of test statistics.

[0302] 17. The method according to Example 16, wherein at each of the plurality of time steps, the set of test statistics includes chi-square test statistics, Wald test statistics, and Ljung-Box test statistics.

[0303] 18. The method according to any one of embodiments 14 to 17, further comprising, at each of the plurality of time steps: transmitting a system output to a user interface, the system output including the monitor input at the current time step, the monitor output at the current time step, and the event signal at the current time step.

[0304] 19. The method according to embodiment 18, wherein at each of the plurality of time steps, the system output at the current time step further includes the control action at the current time step.

[0305] 20. The method according to any one of embodiments 18 to 19, further comprising, at each of the plurality of time steps: receiving user input from the user interface, the user input specifying a corresponding setpoint value of the controller parameter at the current time step for each controller parameter in a subset of the set of controller parameters of the state feedback controller; and initializing each controller parameter in the subset of controller parameters using the corresponding setpoint value at the current time step.

[0306] 21. The method according to embodiment 20, wherein at each of the plurality of time steps, the system output at the current time step further includes the transfer function of the linear magnetic controller at the current time step. z Domain diagram.

[0307] 22. The method according to embodiment 21, wherein... z Domain graphs include one or more of the following: Bode graphs, Nyquist graphs, Nichols graphs, root locus graphs, and the transfer function's response to the input function. z Domain diagram, sensitivity function of the transfer function z Domain diagram, or complementary sensitivity function of the transfer function. z Domain diagram.

[0308] 23. The method according to any one of embodiments 13 to 22, further comprising, at each of the plurality of time steps: determining whether the event signal at the current time step indicates that the plasma satisfies the trigger condition at the current time step; and if the event signal at the current time step indicates that the plasma satisfies the trigger condition at the current time step: recording the current time step as an initiation time step; and in response to the trigger condition, initiating a switching protocol to be executed at each time step after the initiation time step.

[0309] 24. The method according to embodiment 23, wherein executing the switching protocol in response to the triggering condition includes, at each time step after the initiation time step: determining whether the event signal at the current time step still indicates that the plasma satisfies the triggering condition at the current time step; and if the event signal at the current time step no longer indicates that the plasma satisfies the triggering condition at the current time step: recording the current time step as a termination time step, and terminating the switching protocol at the termination time step.

[0310] 25. The method according to any one of Embodiments 23 to 24, wherein executing the switching protocol in response to the triggering condition comprises, at each time step after the initiation time step: forwarding a network input including the system input at the current time step to a neuromagnetic controller; processing the network input at the current time step using the neuromagnetic controller to generate a network output, the network output defining a strategy for selecting control actions for the plurality of control coils at the current time step; generating a neural control action for the plurality of control coils at the current time step from the network output at the current time step; and initializing the control action at the current time step to the neural control action at the current time step before transmitting the control action at the current time step to the magnetic confinement device to generate the magnetic field confining the plasma within the chamber according to the control action.

[0311] 26. The method according to embodiment 25, wherein at each time step after the initiation time step, the strategy is a probability distribution of possible control actions for the plurality of control coils at the current time step, and the network output at the current time step includes a set of distribution parameters of the probability distribution at the current time step.

[0312] 27. The method according to embodiment 26, wherein at each time step after the initiation time step, generating the neural control action at the current time step based on the network output at the current time step comprises: determining an expected value of the probability distribution at the current time step based on the set of distribution parameters at the current time step; and returning the expected value of the probability distribution at the current time step as the neural control action at the current time step.

[0313] 28. The method according to embodiment 26, wherein at each time step after the initiation time step, generating the neural control action at the current time step based on the network output at the current time step comprises: generating the probability distribution at the current time step based on the set of distribution parameters at the current time step; and sampling the neural control action at the current time step from the probability distribution at the current time step.

[0314] 29. The method according to any one of embodiments 26 to 28, wherein at each time step after the initiation time step, the probability distribution is a Gaussian distribution, and the set of distribution parameters at the current time step includes: (i) the average control action at the current time step, and (ii) the control action covariance matrix at the current time step.

[0315] 30. The method according to any one of embodiments 25 to 29, wherein the neuromagnetic controller is a feedforward neural network.

[0316] 31. The method according to any one of Examples 25 to 30, wherein the neuromagnetic controller comprises ten or fewer neural network layers.

[0317] 32. The method according to any one of embodiments 25 to 31, wherein the neuromagnetic controller has been trained on the model of the magnetically confined device via the reinforcement learning technique.

[0318] 33. The method according to any one of Embodiments 1 to 32, wherein at each of a plurality of time steps: the target state of the plasma at the current time step includes: a target plasma configuration vector, the target plasma configuration vector specifying a corresponding target value of the configuration parameter at the current time step for each of a plurality of configuration parameters of the plasma; and a target current configuration vector, the target current configuration vector being used to specify a corresponding target value of a corresponding control current in the control coil at the current time step for each of the plurality of control coils; and the physical state of the plasma at the current time step includes: a physical plasma configuration vector, the physical plasma configuration vector specifying a corresponding physical value of the configuration parameter at the current time step for each of the plurality of configuration parameters of the plasma; and a physical current configuration vector, the physical current configuration vector being used to specify a corresponding physical value of the corresponding control current in the control coil at the current time step for each of the plurality of control coils.

[0319] 34. The method according to embodiment 33, wherein the plurality of configuration parameters includes one or more of the following: the plasma current of the plasma, the safety factor of the plasma, the position of the center of the plasma, the elongation ratio of the plasma, the triangularity of the plasma, the radius of the plasma, or the position of the limit point of the plasma in the chamber.

[0320] 35. The method according to embodiment 34, wherein the plurality of configuration parameters further includes: the outline of the boundary line defining the plasma, the total number of x points in the cavity, and the corresponding position of the x point in the cavity for each x point.

[0321] 36. The method according to any one of embodiments 33 to 35, wherein at each of the plurality of time steps, the control action at the current time step includes a target voltage configuration vector, the target voltage configuration vector being used to specify a corresponding target value of a corresponding control voltage across the control coil at the current time step for each of the plurality of control coils.

[0322] 37. The method according to embodiment 36, wherein at each of the plurality of time steps, the physical measurement of the plasma at the current time step includes: a physical field configuration vector specifying, for each of the plurality of diagnostic field positions in the chamber, corresponding physical values ​​of the amplitude and direction of the magnetic field at the current time step; a physical flux configuration vector specifying, for each of the plurality of diagnostic flux positions in the chamber, corresponding physical values ​​of the magnetic flux of the magnetic field at the current time step; a physical voltage configuration vector specifying, for each of the plurality of control coils, a corresponding physical value of the corresponding control voltage across the control coil at the current time step; and a physical current configuration vector specifying, for each of the plurality of control coils, a corresponding physical value of the corresponding control current in the control coil at the current time step.

[0323] 38. The method according to any one of Embodiments 1 to 37, wherein the magnetic confinement device is a toroidal magnetic confinement device, and the chamber is a toroidal chamber having a central void, a rotational central axis extending through the central void, and a magnetic axis surrounding the central void.

[0324] 39. The method according to embodiment 38, wherein the annular magnetic confinement device is a tokamak, the magnetic shaft is located in a plane perpendicular to the central axis, and the plurality of control coils are axially symmetrical about the central axis.

[0325] 40. The method according to embodiment 39, wherein the plurality of control coils comprises: a plurality of central solenoid coils stacked on top of each other within the central gap along the central axis, the plurality of central solenoid coils forming a central solenoid configured to induce a circumferential plasma current within the plasma for generating a poloidal component of the magnetic field; a plurality of poloidal field coils, each poloidal field coil orbiting the central axis in a corresponding circumferential plane orthogonal to it, the plurality of poloidal field coils configured to manipulate the poloidal component of the magnetic field generated by the circumferential plasma current; and a plurality of circumferential field coils, each circumferential field coil orbiting the magnetic axis in a corresponding circumferential plane orthogonal to it, the plurality of circumferential field coils configured to generate a circumferential component of the magnetic field.

[0326] 41. The method according to any one of Examples 1 to 40, wherein the plurality of control coils are a plurality of high-temperature superconducting control coils.

[0327] 42. The method according to any one of Examples 1 to 41, wherein the magnetic confinement device generates electrical energy via thermonuclear fusion of the plasma.

[0328] 43. The method according to any one of Embodiments 1 to 42, wherein the plurality of time steps are a plurality of physical time steps, and wherein training the linear magnetic controller on the model of the magnetic confinement device via the reinforcement learning technique comprises: receiving a target state trajectory, the target state trajectory including a corresponding target state of the plasma at each of the plurality of virtual time steps; performing a simulation within the plurality of virtual time steps, the simulation including the model of the magnetic confinement device and a reward function; at each of the plurality of virtual time steps: receiving a simulation output from the simulation, the simulation output including: (i) a virtual measurement of the plasma characterizing the virtual state of the plasma at the current virtual time step, and (ii) a state based on the target state of the plasma and the virtual state at the current virtual time step. The system inputs are: (i) the target state of the plasma at the current virtual time step, and (ii) the virtual measurement of the plasma at the current virtual time step; (iii) the system inputs at the current virtual time step are processed by the linear magnetic controller to generate control actions for the plurality of control coils at the current virtual time step; (iv) the control actions at the current virtual time step are transmitted to the simulation to simulate the magnetic field confining the plasma within the chamber according to the control actions; (v) the simulation trajectory is returned, the simulation trajectory including the corresponding simulation output generated by the simulation at each of the plurality of virtual time steps; and (vi) the linear magnetic controller is trained on the simulation trajectory via the reinforcement learning technique.

[0329] 44. A method for controlling a magnetic confinement device executed by one or more computers, the magnetic confinement device comprising a chamber and a plurality of control coils configured to generate a magnetic field confining a plasma within the chamber according to a control sequence, the control sequence including a corresponding control action at each of a plurality of time steps, the method comprising: Receive the target state trajectory, which includes the corresponding target state of the plasma at each of multiple time steps; and At each of the plurality of time steps: The physical measurements of the plasma, characterizing the physical state of the plasma at the current time step, are received from the magnetic confinement device. Inputs are forwarded to the linear magnetic controller system, the system inputs including: (i) the target state of the plasma at the current time step, and (ii) the physical measurements of the plasma at the current time step; The linear magnetic controller is used to process the system input at the current time step to generate the control action for the plurality of control coils at the current time step; and The control action at the current time step is transmitted to the magnetic confinement device to generate the magnetic field that confines the plasma within the chamber.

[0330] 45. The method according to embodiment 44 further includes training the linear magnetic controller via reinforcement learning techniques.

[0331] 46. ​​The method according to embodiment 44 further includes training the linear magnetic controller on a model of the magnetic confinement device via reinforcement learning techniques.

[0332] 47. The method according to embodiment 46, wherein the plurality of time steps are a plurality of physical time steps, and wherein training the linear magnetic controller on the model of the magnetic confinement device via the reinforcement learning technique comprises: receiving a target state trajectory, the target state trajectory including a corresponding target state of the plasma at each of the plurality of virtual time steps; performing a simulation within the plurality of virtual time steps, the simulation including the model of the magnetic confinement device and a reward function; at each of the plurality of virtual time steps: receiving a simulation output from the simulation, the simulation output including: (i) a virtual measurement of the plasma characterizing the virtual state of the plasma at the current virtual time step, and (ii) a state tracking based on the target state and the virtual state of the plasma at the current virtual time step. The error is rewarded; system inputs are forwarded to the linear magnetic controller, the system inputs including: (i) the target state of the plasma at the current virtual time step, and (ii) the virtual measurement of the plasma at the current virtual time step; the system inputs at the current virtual time step are processed using the linear magnetic controller to generate control actions for the plurality of control coils at the current virtual time step; and the control actions at the current virtual time step are transmitted to the simulation to simulate the magnetic field confining the plasma in the chamber according to the control actions; a simulation trajectory is returned, the simulation trajectory including the corresponding simulation output generated by the simulation at each of the plurality of virtual time steps; and the linear magnetic controller is trained on the simulation trajectory via the reinforcement learning technique.

[0333] 48. The method according to embodiment 43 or 47, wherein performing the simulation including the model of the magnetic confinement device and the reward function within the plurality of virtual time steps comprises, at each of the plurality of virtual time steps: receiving model input, the model input including: (i) the virtual state of the plasma at the previous virtual time step, and (ii) the control action at the previous virtual time step; processing the model input at the current virtual time step using the model of the magnetic confinement device to generate a model output, the model output including: (i) the virtual state of the plasma at the current virtual time step, and (ii) the virtual measurement of the plasma at the current virtual time step; receiving the state tracking error between the target state and the virtual state of the plasma at the current virtual time step; processing the state tracking error at the current virtual time step using the reward function to generate the reward at the current virtual time step; and returning the simulation output at the current virtual time step, the simulation output including: (i) the virtual measurement of the plasma at the current virtual time step, and (ii) the reward at the current virtual time step.

[0334] 49. The method according to embodiment 48, wherein the model of the magnetic confinement device comprises: a state transition model of the plasma magnetically coupled to the plurality of control coils, the state transition model being configured to: receive the model input at the current virtual time step; and process the model input at the current virtual time step to generate the virtual state of the plasma at the current virtual time step; and a diagnostic model of a plurality of sensors configured to measure the plasma magnetically coupled to the plurality of control coils, the diagnostic model being configured to: receive the virtual state of the plasma at the current virtual time step; and process the virtual state of the plasma at the current virtual time step to generate the virtual measurement of the plasma at the current virtual time step.

[0335] 50. The method according to Example 49, wherein the state transition model is derived from the magnetohydrodynamic model of the plasma from the power supply model magnetically coupled to the plurality of control coils.

[0336] 51. The method according to Example 50, wherein the magnetohydrodynamic model is a equilibrium magnetohydrodynamic model of the plasma.

[0337] 52. The method according to any one of embodiments 49 to 51, wherein performing the simulation including the model of the magnetic confinement device and the reward function within the plurality of virtual time steps includes, at each of the plurality of virtual time steps: determining, based on the virtual state of the plasma at the current virtual time step, whether the physical feasibility constraint of the magnetic confinement device is violated at the current virtual time step; and if the physical feasibility constraint of the magnetic confinement device is violated at the current virtual time step, terminating the simulation at the current virtual time step.

[0338] 53. The method according to embodiment 52, wherein at each of the plurality of virtual time steps: the virtual state of the plasma at the current virtual time step includes: a virtual plasma configuration vector, the virtual plasma configuration vector specifying a corresponding virtual value of the configuration parameter at the current virtual time step for each of the plurality of configuration parameters of the plasma; and a virtual current configuration vector, the virtual current configuration vector specifying a virtual value of the corresponding control current in the control coil at the current virtual time step for each of the plurality of control coils; and determining whether the physical feasibility constraint of the magnetic confinement device is violated at the current virtual time step based on the virtual state of the plasma includes one or more of the following: determining whether the corresponding virtual value of the configuration parameter at the current virtual time step satisfies a corresponding threshold for each of the plurality of configuration parameters of the plasma; or determining whether the corresponding virtual value of the corresponding control current in the control coil at the current virtual time step satisfies a corresponding threshold for each of the plurality of control coils.

[0339] 54. The method according to any one of embodiments 43 or 47 to 53, wherein the reinforcement learning technique is an actor-commentator reinforcement learning technique, and wherein training the linear magnetic controller on the simulated trajectory via the actor-commentator reinforcement learning technique includes jointly training the linear magnetic controller and the commentator neural network on the simulated trajectory.

[0340] 55. The method according to embodiment 54, jointly training the linear magnetic controller and the commentator neural network on the simulated trajectory includes, for each of the plurality of virtual time steps: forwarding a commentator network input including the simulated output at the virtual time step to the commentator neural network; and processing the commentator network input at the virtual time step using the commentator neural network to generate a commentator network output including a value function that estimates a cumulative metric of the corresponding reward received by the linear magnetic controller at each subsequent virtual time step.

[0341] 56. The method according to embodiment 55, wherein for each of the plurality of virtual time steps, the commentator network input for the virtual time step further includes one or more of the following: the target state of the plasma at the virtual time step, the virtual state of the plasma at the virtual time step, or the control action at the virtual time step.

[0342] 57. The method according to any one of embodiments 55 to 56, wherein jointly training the linear magnetic controller and the commentator neural network on the simulated trajectory further comprises: for each of the plurality of virtual time steps: calculating a time difference error, the time difference error comprising errors between (i) a time difference objective for the virtual time step, and (ii) the value function for the virtual time step; generating an advantage function, the advantage function comprising the time difference error for the virtual time step; generating a commentator objective function, the commentator objective function comprising a corresponding squared magnitude of the corresponding time difference error for each of the plurality of virtual time steps; generating an actor objective function, the actor objective function comprising a corresponding advantage function for each of the plurality of virtual time steps; optimizing the commentator objective function relative to a set of commentator network parameters of the commentator neural network; and optimizing the actor objective function relative to a set of control parameters of the linear magnetic controller.

[0343] 58. The method according to any one of Examples 54 to 57, wherein the actor-commentator reinforcement learning technique is a maximum a posteriori policy optimization (“MPO”) technique.

[0344] 59. The method according to any one of embodiments 54 to 58, wherein the actor-commentator reinforcement learning technique is a distributed actor-commentator reinforcement learning technique.

[0345] 60. A method executed by one or more computers for training a linear magnetic controller to control a magnetic confinement device, the magnetic confinement device including a chamber and a plurality of control coils configured to generate a magnetic field confining a plasma within the chamber according to a control sequence, the control sequence including a corresponding control action at each of a plurality of physical time steps, the method comprising: receiving a target state trajectory including a corresponding target state of the plasma at each of a plurality of virtual time steps; performing a simulation within the plurality of virtual time steps, the simulation including a model of the magnetic confinement device and a reward function; at each of the plurality of virtual time steps: receiving a simulation output from the simulation, the simulation output including: (i) a virtual measurement of the plasma characterizing a virtual state of the plasma at the current virtual time step, and (ii) a virtual measurement of the plasma based on the current virtual time step. The system inputs are: (i) the target state of the plasma at the current virtual time step and (ii) the virtual measurement of the plasma at the current virtual time step; (ii) the system inputs are forwarded to the linear magnetic controller to generate control actions for the plurality of control coils at the current virtual time step; (iii) the control actions are transmitted to the simulation to simulate the magnetic field confining the plasma within the chamber according to the control actions; (iv) the simulation trajectory is returned, which includes the corresponding simulation output generated by the simulation at each of the plurality of virtual time steps; and (v) the linear magnetic controller is trained on the simulation trajectory via the reinforcement learning technique.

[0346] 61. The method according to embodiment 60, wherein the linear magnetic controller includes a state observer and a state feedback controller, and wherein training the linear magnetic controller on the simulated trajectory via the reinforcement learning technique includes jointly training the state observer and the state feedback controller on the simulated trajectory via the reinforcement learning technique.

[0347] 62. The method according to any one of embodiments 60 to 61, further comprising: after training the linear magnetic controller via the reinforcement learning technique to control the magnetic confinement device on the model of the magnetic confinement device; generating the control sequence using the linear magnetic controller, the control sequence including the corresponding control action for the plurality of control coils at each of the plurality of physical time steps; and transmitting the control sequence to the magnetic confinement device to generate the magnetic field confining the plasma within the chamber according to the control sequence.

[0348] 63. The method according to embodiment 60 further includes, after training the linear magnetic controller, controlling the magnetic confinement device according to the control sequence, the control including: receiving a target state trajectory, the target state trajectory including a corresponding target state of the plasma at each of a plurality of physical time steps; and at each of the plurality of physical time steps: receiving from the magnetic confinement device a physical measurement of the plasma characterizing the physical state of the plasma at the current physical time step; forwarding system input to the linear magnetic controller, the system input including: (i) the target state of the plasma at the current physical time step, and (ii) the physical measurement of the plasma at the current physical time step; processing the system input at the current physical time step using the linear magnetic controller to generate the control action for the plurality of control coils at the current physical time step; and transmitting the control action at the current physical time step to the magnetic confinement device to generate the magnetic field confining the plasma within the chamber according to the control action.

[0349] 64. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the method according to any one of embodiments 1 to 63.

[0350] 65. A non-transitory computer-readable storage medium storing one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the method according to any one of Embodiments 1 to 63.

[0351] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, in some cases one or more features from that combination may be removed from the claimed combination, and the claimed combination may involve sub-combinations or variations thereof.

[0352] Similarly, although operations are depicted in the accompanying drawings and described in a specific order in the claims, this should not be construed as requiring such operations to be performed in the specific order shown or in a sequential order, or requiring all shown operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0353] Specific embodiments of this subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired result. As an example, the processes depicted in the drawings do not necessarily require a specific order or sequence shown to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method for controlling a magnetic confinement device executed by one or more computers, the magnetic confinement device comprising a chamber and a plurality of control coils configured to generate a magnetic field confining a plasma within the chamber according to a control sequence, the control sequence comprising a corresponding control action at each of a plurality of time steps, the method comprising: Receive the target state trajectory, the target state trajectory including the corresponding target state of the plasma at each of the plurality of time steps; as well as At each of the plurality of time steps: The physical measurements of the plasma, characterizing the physical state of the plasma at the time step, are received from the magnetic confinement device. Inputs are forwarded to the linear magnetic controller system, the system inputs including: (i) the target state of the plasma at the time step, and (ii) the physical measurements of the plasma at the time step; The linear magnetic controller is used to process the system input at the time step to generate the control action for the plurality of control coils at that time step. The linear magnetic controller has been trained on a model of the magnetic confinement device using reinforcement learning techniques; and The control action at the time step is transmitted to the magnetic confinement device to generate the magnetic field for confining the plasma within the chamber.

2. The method of claim 1, wherein the linear magnetic controller comprises a state observer and a state feedback controller, the state observer and the state feedback controller having been jointly trained on the model of the magnetically confined device via the reinforcement learning technique.

3. The method of claim 2, wherein the state observer is configured as a Kalman filter, a Lombard observer, or a linear neural observer.

4. The method according to claim 2, wherein: The state observer is configured to be at each of the plurality of time steps: Receive the physical measurement of the plasma at the time step; as well as The physical measurements of the plasma at the time step are processed based on a set of observer parameters to generate a predicted state of the plasma that estimates the physical state of the plasma at the time step. The state feedback controller is configured to, at each of the plurality of time steps: Receive controller input, the controller input including: (i) the target state of the plasma at the time step, and (ii) the predicted state of the plasma at the time step; as well as Based on a set of controller parameters, the controller input at the time step is processed to generate the control action at the time step; and The set of observer parameters and the set of controller parameters have been jointly trained on the model of the magnetic confinement device using the reinforcement learning technique.

5. The method of claim 4, wherein processing the physical measurements of the plasma at the time step according to the set of observer parameters to generate the predicted state of the plasma at the time step includes at each of the plurality of time steps: Receive prior observer input, the prior observer input including: (i) the predicted state of the plasma at the previous time step, and (ii) the control action at the previous time step; Based on the set of observer parameters, the prior observer input at the time step is processed to generate a prior observer output, the prior observer output including: (i) a priori predicted state of the plasma that estimates the physical state of the plasma at the time step without conditioned on the physical measurement of the plasma at the time step, and (ii) a predicted measurement of the plasma that estimates the physical measurement of the plasma at the time step; Receive posterior observer input, the posterior observer input including: (i) the prior observer output at the time step, and (ii) the physical measurement of the plasma at the time step; and Based on the set of observer parameters, the posterior observer input at the time step is processed to generate the predicted state of the plasma at the time step.

6. The method of claim 5, wherein the set of observer parameters includes a state transition matrix, a control action matrix, and a diagnostic matrix, and wherein processing the prior observer input at the time step according to the set of observer parameters to generate the prior observer output at the time step includes at each of the plurality of time steps: Calculate the first matrix-vector product between the state transition matrix and the predicted state of the plasma at the previous time step; Calculate the second matrix-vector product between the control action matrix and the control action at the previous time step; Return the sum of the first matrix-vector product and the second matrix-vector product at the time step as the prior predicted state of the plasma at the time step; as well as Return the third matrix-vector product between the diagnostic matrix and the prior predicted state of the plasma at the time step as the predicted measurement of the plasma at the time step.

7. The method of claim 6, wherein processing the posterior observer input at the time step according to the set of observer parameters to generate the predicted state of the plasma at the time step includes at each of the plurality of time steps: Based on the set of observer parameters, generate the observer gain matrix at the time step; Generate measurement residuals, which include the deviation between the physical measurement of the plasma at the time step and the predicted measurement; Generate an incremental state, the incremental state comprising a fourth matrix-vector product between the observer gain matrix and the measurement residual at the time step; and The prior predicted state of the plasma at the time step is incremented by the incremental state at the time step to generate the predicted state of the plasma at the time step.

8. The method according to claim 4, wherein: The set of controller parameters includes: One or more controller memory matrices, each controller memory matrix being associated with a corresponding time step preceding the stated time step; and One or more controller gain matrices, each controller gain matrix being associated with the time step or a corresponding time step preceding the time step, and Processing the controller input at the time step according to the set of controller parameters to generate the control action at the time step includes at each of the plurality of time steps: For each controller memory matrix, calculate the matrix-vector product between the following two: (i) the controller memory matrix, and (ii) the control action at the corresponding time step prior to the time step; For each controller gain matrix, compute the matrix-vector product of the following two: (i) the controller gain matrix, and (ii) the controller input at the time step or at the corresponding time step prior to the time step; and Generate the sum of the corresponding matrix-vector products of each controller memory matrix and controller gain matrix at the time step, as the control action at the time step.

9. The method of claim 5, further comprising at each of the plurality of time steps: Receive monitor input, the monitor input including: (i) A state prediction block, the state prediction block comprising: (a) The target state of the plasma at the time step; (b) The predicted state of the plasma at the time step; (c) State residual, the state residual comprising the deviation between the target state and the predicted state of the plasma at the time step; and (d) A state covariance matrix, which characterizes the uncertainty of the predicted state of the plasma at the time step; and / or (ii) A measurement prediction block, the measurement prediction block comprising: (a) The physical measurement of the plasma at the time step; (b) The predicted measurement of the plasma at the time step; (c) Measuring residuals, said measurement residuals including the deviation between the physical measurement of the plasma at the said time step and the predicted measurement; and (d) Measure the covariance matrix, which characterizes the uncertainty of the predicted measurement of the plasma at the time step; and The monitor input at the time step is processed to generate an event signal indicating whether the plasma meets the triggering conditions at the time step.

10. The method of claim 9, wherein processing the monitor input at the time step to generate the event signal indicating whether the plasma satisfies a trigger condition at the time step comprises at each of the plurality of time steps: Perform statistical analysis on the state prediction block and / or the measurement prediction block at the time step to generate a monitor output, the monitor output including: (i) A state health vector, which includes multiple statistical health indicators of the state prediction block at the time step; and / or (ii) Measure a health vector, the measured health vector including multiple statistical health indicators of the measured prediction block at the time step; as well as Generate an event signal indicating whether one or more of the statistical health indicators in the state health vector and / or the measured health vector meet a corresponding threshold at the time step.

11. The method of claim 10, further comprising at each of the plurality of time steps: The monitor output at the time step is used to enhance the control action at the time step, so as to generate the magnetic field that confines the plasma according to the control action.

12. The method of claim 10, wherein at each of the plurality of time steps: The statistical analysis includes one or more of the following: anomaly analysis, instability analysis, or nonlinear analysis; and The plurality of statistical health indicators include one or more of the following: Mahalanobis distance, anomaly probability, local Lyapunov index, global Lyapunov index, Kolmogorov-Sinai entropy, or a set of test statistics.

13. The method of claim 9, further comprising at each of the plurality of time steps: Determine whether the event signal at the time step indicates that the plasma satisfies the trigger condition at the time step; and If the event signal at the time step indicates that the plasma satisfies the triggering condition at the time step, then: Record the time step as the initiation time step; and In response to the triggering condition, a switching protocol is initiated to be executed at each time step following the initiation time step.

14. The method of claim 13, wherein executing the switching protocol in response to the triggering condition comprises at each time step after the initiation time step: Determine whether the event signal at the time step still indicates that the plasma satisfies the triggering condition at that time step; and If the event signal at the time step no longer indicates that the plasma meets the triggering condition at the time step, then: The time step is recorded as the termination time step, and the switching protocol is terminated at the termination time step.

15. The method of claim 13, wherein executing the switching protocol in response to the triggering condition comprises at each time step after the initiation time step: Forward network input, including the system input at the time step, to the neuromagnetic controller; The neural magnetic controller is used to process the network input at the time step to generate a network output that defines a strategy for selecting the control action for the plurality of control coils at the time step. The neural control action for the plurality of control coils is generated from the network output at the time step; as well as Before transmitting the control action at the time step to the magnetic confinement device, the control action at the time step is initialized to the neural control action at the time step to generate the magnetic field that confines the plasma within the chamber based on the control action.

16. The method of claim 1, wherein at each of the plurality of time steps: The target state of the plasma at the time step includes: A target plasma configuration vector, wherein the target plasma configuration vector specifies a target value for each of a plurality of configuration parameters of the plasma at the time step; as well as A target current configuration vector is used to specify a target value for the corresponding control current in the control coil at the time step for each of the plurality of control coils. and The physical state of the plasma at the specified time step includes: A physical plasma configuration vector, wherein the physical plasma configuration vector specifies the corresponding physical value of the configuration parameter at the time step for each of the plurality of configuration parameters of the plasma; as well as A physical current configuration vector is used to specify the corresponding physical value of the corresponding control current in the control coil at the time step for each of the plurality of control coils.

17. The method of claim 1, wherein the magnetic confinement device is a toroidal magnetic confinement device, and the chamber is a toroidal chamber having a central void, a rotational central axis extending through the central void, and a magnetic axis surrounding the central void.

18. The method of claim 1, wherein the plurality of time steps are a plurality of physical time steps, and wherein training the linear magnetic controller on the model of the magnetic confinement device via the reinforcement learning technique comprises: Receive a target state trajectory, the target state trajectory including the corresponding target state of the plasma at each of a plurality of virtual time steps; Simulations are performed within the plurality of virtual time steps, the simulations including the model and reward function of the magnetic confinement device; At each of the plurality of virtual time steps: The simulation output is received from the simulation, the simulation output including: (i) a virtual measurement of the plasma characterizing the virtual state of the plasma at the virtual time step, and (ii) a reward based on the state tracking error between the target state and the virtual state of the plasma at the virtual time step; The linear magnetic controller forwards inputs to the system, the system inputs including: (i) the target state of the plasma at the virtual time step, and (ii) the virtual measurement of the plasma at the virtual time step; The linear magnetic controller is used to process the system input at the virtual time step to generate control actions for the plurality of control coils at the virtual time step; and The control action at the virtual time step is transmitted to the simulation to simulate the magnetic field used to confine the plasma within the chamber; Returning the simulation trajectory, the simulation trajectory including the corresponding simulation output generated by the simulation at each of the plurality of virtual time steps; and The linear magnetic controller is trained on the simulated trajectory using the reinforcement learning technique.

19. A system comprising: One or more computers; as well as One or more storage devices are communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for controlling a magnetic confinement device, the magnetic confinement device including a chamber and a plurality of control coils configured to generate a magnetic field confining a plasma within the chamber according to a control sequence, the control sequence including a corresponding control action at each of a plurality of time steps, the operations including: Receive a target state trajectory, the target state trajectory including the corresponding target state of the plasma at each of the plurality of time steps; and At each of the plurality of time steps: The physical measurements of the plasma, characterizing the physical state of the plasma at a time step, are received from the magnetic confinement device. Inputs are forwarded to the linear magnetic controller system, the system inputs including: (i) the target state of the plasma at the time step, and (ii) the physical measurements of the plasma at the time step; The linear magnetic controller is used to process the system input at the time step to generate the control action for the plurality of control coils at that time step. The linear magnetic controller has been trained on a model of the magnetic confinement device using reinforcement learning techniques; and The control action at the time step is transmitted to the magnetic confinement device to generate the magnetic field for confining the plasma within the chamber.

20. A non-transitory computer storage medium storing one or more instructions, said instructions, when executed by one or more computers, causing said one or more computers to perform operations for controlling a magnetic confinement device, said magnetic confinement device including a chamber and a plurality of control coils configured to generate a magnetic field confining a plasma within the chamber according to a control sequence, said control sequence including a corresponding control action at each of a plurality of time steps, said operations including: Receive the target state trajectory, the target state trajectory including the corresponding target state of the plasma at each of the plurality of time steps; as well as At each of the plurality of time steps: The physical measurements of the plasma, characterizing the physical state of the plasma at a time step, are received from the magnetic confinement device. Inputs are forwarded to the linear magnetic controller system, the system inputs including: (i) the target state of the plasma at the time step, and (ii) the physical measurements of the plasma at the time step; The linear magnetic controller is used to process the system input at the time step to generate the control action for the plurality of control coils at that time step. The linear magnetic controller has been trained on a model of the magnetic confinement device using reinforcement learning techniques; and The control action at the time step is transmitted to the magnetic confinement device to generate the magnetic field for confining the plasma within the chamber.