Information processing device and information processing method

Abstract

This information processing device comprises: an encoder that converts a predetermined physical quantity into a latent variable in a latent space; a prediction unit that performs time-series prediction on the latent variable; and a control unit that obtains a target amount of control on the basis of a prediction result of the prediction unit, and executes control toward the target amount so as to pass through a route that connects the shortest distance between the latent variable and the target amount in the latent space.

Description

Information processing device and information processing method

[0001] The present invention relates to a technique for controlling plasma in a nuclear fusion reactor.

[0002] In the field of machine learning, representation learning such as variational autoencoder (VAE) (Non-Patent Document 1) is used to convert a multidimensional input space into a lower-dimensional latent space and extract features from the input space. Most reinforcement learning techniques use a large amount of multidimensional training data for training, but this can lead to problems such as poor sample efficiency and overfitting.

[0003] On the other hand, embedding the input space into a lower-dimensional latent space is expected to improve generalization performance and learning efficiency (Non-Patent Document 2).It has also been proposed that the shortest distance to the control target in the latent space corresponds to the optimal behavior in the actual input space (Non-Patent Documents 3 and 4).

[0004] In the field of nuclear fusion, there are examples where plasma position control has been successfully achieved by identifying the physical quantities to be controlled using VAE based on experimental data and then using a preprogram based on the identified quantities (Non-Patent Document 5). There are also examples where plasma position and shape have been successfully controlled by offline learning of experimental data using reinforcement learning (Non-Patent Document 6). However, with these conventional technologies, it is unclear whether they can be generalized across devices where plasma physical quantities, such as current density, magnetic field, density, and pressure, vary greatly, or where reactor structures exist around the plasma are different.

[0005] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. Timothée Lesort, Natalia Diaz-Rodriguez, Jean-Franois Goudou, and David Filliat. State representation learning for control: An overview. Neural Networks, 108:379-392, 2018. Masahiro Suzuki and Yutaka Matsuo. Rethinking action representation in reinforcement learning based on deep metric learning in latent representation space. Proceedings of the Second Workshop of the Japanese Society for Artificial Intelligence, 2021(AGI-018):05, 2021. Alex X Lee, Anusha Nagabandi, Pieter Abbeel, and Sergey Levine. Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model. Advances in Neural Information Processing Systems, 33:741-752, 2020.Y Wei, JP Levesque, CJ Hansen, ME Mauel, and GA Navratil. A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (vae) neural network. Nuclear Fusion, 61(12):126063, 2021.Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de Las Casas, et al.Magnetic control of tokamak plasmas through deep reinforcement learning. Nature, 602(7897):414-419, 2022.

[0006] The present invention has been made in consideration of the above points, and aims to provide a technology that makes it possible to predict the state of plasma in nuclear fusion and control the operation or design of a nuclear fusion reactor based on the prediction results.

[0007] According to the disclosed technology, there is provided an information processing device including: an encoder that converts a predetermined physical quantity into a latent variable in a latent space; a prediction unit that performs time series prediction on the latent variable; and a control unit that calculates a target quantity for control based on a prediction result by the prediction unit, and executes control toward the target quantity so as to take a path in the latent space that connects the latent variable and the target quantity by the shortest distance.

[0008] The disclosed technology provides a technology that makes it possible to predict the state of plasma during nuclear fusion and control the operation or design of a nuclear fusion reactor based on the prediction results.

[0009] FIG. 1 is a diagram illustrating an example of the overall configuration of a system according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating the operation of the information processing device 100. FIG. 3 is a diagram illustrating a map obtained when a change in plasma position is converted into a two-dimensional latent space. FIG. 4 is a diagram illustrating an example of control in the latent space performed by the information processing device 100. FIG. 5 is a diagram illustrating an example of control in the latent space performed by the information processing device 100. FIG. 6 is a diagram illustrating an example of control in the latent space performed by the information processing device 100. FIG. 7 is a diagram illustrating an example of the functional configuration of the information processing device 100. FIG. 8 is a diagram specifically illustrating the operation of the information processing device 100. FIG. 9 is a diagram illustrating an example of the hardware configuration of the information processing device 100.

[0010] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment. Note that in the present embodiment, the physical quantity of interest is the physical quantity of plasma, but the technology according to the present embodiment can also be applied to physical quantities other than the physical quantity of plasma.

[0011] (System Configuration and Operational Overview) In this embodiment, the nuclear fusion device is a tokamak-type nuclear fusion reactor, which will be referred to as a tokamak device hereinafter.

[0012] An example of the overall configuration of a system according to this embodiment is shown in Fig. 1. As shown in Fig. 1, this system includes an information processing device 100, a tokamak device 200, and a plasma measurement device 300. The tokamak device 200 is a device that confines ultra-high temperature plasma to perform nuclear fusion.

[0013] The plasma measuring device 300 is a device that measures the physical quantities of plasma in the tokamak device 200. The information processing device 100 is a device that predicts and controls plasma based on the physical quantities of plasma obtained by the plasma measuring device 300.

[0014] The operation of the information processing device 100 will be described with reference to the flowchart of Fig. 2. An overview will be given here, and more specific configurations and operations will be described later.

[0015] In S1 (step 1), the information processing device 100 converts the plasma physical quantities obtained from the plasma measuring device 200 into latent variables using a generative model. Note that the "latent variables" may also be called "latent expressions" or "latent states."

[0016] In S2, the information processing device 100 performs time series prediction on the latent variables acquired in S1, and predicts the trajectory of time development of the latent variables in the latent space.

[0017] In S3, the information processing device 100 calculates a target quantity in the latent space based on the latent variables calculated in the past and the latent variables predicted in S3.

[0018] In S4, the information processing device 100 controls the target quantity calculated in S3 so that it follows a path that connects the current latent variable and the target quantity along the shortest distance in the latent space. Following Non-Patent Document 3, this point-to-point vector is defined as the behavior in the latent space (latent behavior).

[0019] (Example) A more specific example of operation will be described below as an example. Here, predictive control of the position of core plasma in the tokamak device 200 will be considered.

[0020] First, the information processing device 100 converts the plasma measurement values ​​in the tokamak device 200 and the plasma physical quantities calculated based on them (e.g., the position and shape of the plasma, the gradient of the confinement magnetic field, the normalized plasma pressure, etc.) from input space to latent space using VAE.

[0021] Figure 3 shows a map of the plasma position change converted into a two-dimensional potential space. The gray area in the map represents the amount of plasma position change. Furthermore, since the larger the amount of position change, the more difficult it is to feedback control the plasma, the marginal boundary of the stable operation region of the core is shown by a black dotted line.

[0022] The solid line indicates the trajectory of time evolution in the latent space. In other words, by tracing the latent variables at each time, the state of the plasma discharge can be expressed as a trajectory in the latent space. Considering a fusion reactor as a power generation reactor that requires steady-state operation, it is necessary to prevent this trajectory from crossing the stability boundary for stable operation.

[0023] 4 to 7 ((a) to (d)) are diagrams showing examples of control in the latent space executed by the information processing device 100.

[0024] Assume that development data up to time t has been obtained. Then, as shown in FIG. 4 , the information processing device 100 performs time series prediction of the latent variables using LSTM (Long Short-Term Memory) or the like, and calculates a control target value (goal) in the latent space as shown in FIG. 5 . In the example of FIG. 4 , as a result of the time series prediction, the state trajectory in the latent space exceeds the stability boundary. Therefore, as shown in FIG. 5 , the control target value is set so as not to exceed the stability boundary.

[0025] Next, the information processing device 100 searches for latent actions from the current latent state to the target amount, as shown in FIG. 6, and calculates and executes the optimal action among them, as shown in FIG.

[0026] In identifying the optimal action in the latent space, the information processing device 100 determines the position in the latent space at that time (z ｔ ) to the target quantity, the vector connecting the shortest path is calculated. The actual action corresponding to the action in the latent space is executed on the plasma through a control actuator such as a magnetic coil.

[0027] 8 shows an example of the functional configuration of an information processing device 100 that executes the above-described processes. As shown in FIG. 8, the information processing device 100 includes an encoder 110, a storage unit 120, a prediction unit 130, and a control unit 140.

[0028] The encoder 110 is a neural network model (which may also be called a representation) such as a VAE, and converts the physical quantity of plasma into a latent representation. The storage unit 120 is a functional unit that stores data related to processing, and may also be called a memory.

[0029] The prediction unit 130 is, for example, an LSTM, and performs time-series prediction of latent expressions as shown in Fig. 4. The control unit 140 calculates latent behaviors and executes control as shown in Figs.

[0030] <Device Operation Example> The operation of the information processing device 100 will be described in more detail with reference to Fig. 9. In Fig. 9, the storage unit 120 is represented by M, the prediction unit 130 is represented by LSTM, and the control unit 140 is represented by C.

[0031] First, the overall operation shown in Fig. 9 will be described. In the example of Fig. 9, the measured value of the plasma or the value calculated based on the measured value is input to a generative model (expressor f) such as a VAE. Ｌ ) is used to convert the input space into a latent space. In this case, a distance-trained model (representer f) is used to create a latent space where data with similar characteristics are close and dissimilar data are far away. Ｌ ) is constructed through offline learning.

[0032] At time t, state s ｔ is the expression f Ｌ By this, the potential state z ｔ At this time, the control target value z obtained at time t-1 is ｇ ｔ-１ Also new z ｇ ｔ The information stored in the memory unit 120 (M) that stores past latent states is combined with the current information to perform time series prediction using LSTM, and predict the trajectory of the latent state. The control unit 140 (C) also uses r as a reward. ｔ the latent state z ｔ and the target quantity z ｇ ｔ Distance e from ｔ The control target value z ｇ ｔ and potential action a ｔ The control unit 140(C) calculates the latent behavior a ｔ Using this, plasma control is performed in real space through control actuators such as magnetic coils. The above operations are repeated at each time.

[0033] The operation of the information processing device 100 will be described in more detail with reference to Fig. 9. First, based on the measurement signal obtained by the measurement using the plasma measuring device 300, the equilibrium quantities (plasma position / shape, internal inductance, poloidal beta, etc.) are calculated as plasma physical quantities. This is done in the state s ｔ Equivalent to s ｔ may be calculated outside the information processing device 100 or may be calculated by the information processing device 100. Here, the externally calculated s ｔis input to the information processing device 100.

[0034] The state s corresponding to the above macroscopic plasma physical quantity ｔ is the input to the encoder 110 (e.g., VAE), and the encoder 110 ｔ is transformed from the input space to the latent space, and z ｔ At this time, arbitrariness remains in the embedding into the latent space. Next, the information processing device 100 performs control on the latent space. The processing contents will be described in more detail below.

[0035] First, offline learning is performed using metric learning to convert the input space into an appropriate latent space, and a representation f is generated, which is a neural network used as the encoder 110. Ｌ Construct an expressor f Ｌ The construction of the information processing device 100 may be performed by the information processing device 100 or may be performed outside the information processing device 100.

[0036] expressor f Ｌ In constructing the representation f, measurement values ​​of plasma experiments and calculation results of physical simulations that simulate them are used as training data. More specifically, for example, calculation results of plasma equilibrium, transport time evolution, and magnetohydrodynamic stability analysis codes for each equilibrium can be used as training data. As mentioned above, the representation f Ｌ In offline learning, distance learning is performed to create a latent space in which data with similar characteristics are close and dissimilar data are far apart.

[0037] When controlling plasma in a nuclear fusion reactor, the information processing device 100 uses a representation f Ｌ Using this, the state s of the control object (here, plasma) in the real space at time t is ｔ Z ｔ = f Ｌ (s ｔ ) and convert it into a latent representation in the latent space.

[0038] At time t, the offline constructed prediction unit 130 (LSTM in the example of FIG. 9) predicts the latent state z ｔ＋ｎ(n=1, 2, ...) in the latent space. ｔ The stability of the operation of the tokamak device 200 is determined based on the trajectory of the trajectory, and if it is predicted that the stability boundary will be exceeded, this is avoided by taking potential actions.

[0039] More specifically, the control unit 140 calculates the control target z ｇ and the current latent state z ｔ The error e(z ｇ , z ｔ The amount of change in the latent state that minimizes the ｔ = -α(∂e(z ｔ , z ｇ ) / ∂s ｔ ) and action a ｔ That is, as mentioned above, the latent state z ｔ From the target quantity z ｇ The system controls the object so that it follows the path that connects the shortest distance in the latent space.

[0040] As a result, a new state s ｔ＋１ This state s ｔ＋１ expressor f Ｌ and convert it into the latent space by ｔ＋１ = f Ｌ (s ｔ＋１ The information processing device 100 repeats this series of operations until the error e becomes sufficiently small. ｎ＋１ -t ｎ is determined by the time resolution of the measurement system and the requirements of the control actuator.

[0041] The technology according to this embodiment can be used for input space as one-dimensional quantities, two-dimensional quantities, or three-dimensional quantities that take into account the distribution of physical quantities, in addition to zero-dimensional quantities (spatial average values ​​and integral values). If the input space is changed from plasma measurement to plasma physical simulation, and the control actuator is changed as an actual action to specifying a search range for a numerical simulation, the technology according to this embodiment can also be used as a predictive controller for nuclear fusion reactor design.

[0042] The design of a fusion reactor as a power reactor is a multi-objective optimization problem for many parameters, such as the reactor's fusion output, device size, and construction costs, under physical and engineering constraints, and optimization is difficult. In the example, we dealt with plasma position displacement, but by replacing this with a design parameter, it can be applied to the optimization of fusion reactor design in a high-dimensional parameter space.

[0043] (Hardware Configuration Example) The information processing device 100 described in this embodiment can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0044] That is, the information processing device 100 can be realized by using hardware resources such as a CPU and memory built into a computer to execute a program corresponding to the processing performed by the information processing device 100. The program can be recorded on a computer-readable recording medium (such as a portable memory) and can be saved or distributed. The program can also be provided via a network such as the Internet or email.

[0045] Fig. 10 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 10 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.

[0046] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0047] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the information processing device 100 in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0048] (Summary, Effects, etc. of the Embodiment) As described above, the key point of the technology described in this embodiment is that the information processing device 100 uses the encoder 110 (a generative model such as a VAE) to convert the plasma physical quantity from the input space to the latent space. At this time, an appropriate latent space is obtained by using the encoder 110 that has performed distance learning such that input variables with similar attributes are close to each other in the latent space and dissimilar attributes are far from each other. Another key point is that the latent variables in the latent space are used as the control target quantities, rather than the plasma physical quantities.

[0049] The technology according to this embodiment is expected to provide the following effects in controlling fusion plasma.

[0050] By using the encoder 110 constructed through offline learning to appropriately embed plasma physical quantities from the input space into the latent space, plasma control can be treated as an optimization problem for a target quantity in a latent space with a dimension lower than the input dimension.

[0051] - Since the differential value of the latent variable is used to calculate the latent behavior, backpropagation calculation can be performed.

[0052] -Physical interpretation can be made from the correspondence between the latent space and the input space.

[0053] ・The actuator to be controlled can be identified from the distribution in the latent space.

[0054] - By calculating the probability distribution of latent variables, gaps in the input space can be filled.

[0055] -Due to the high scalability of deep learning, it is possible to consider architectures that combine it with other representation learning methods.

[0056] - For instabilities such as magnetohydrodynamic instability, adding linear growth rates to the input space makes it possible to control multiple phenomena.

[0057] In addition, the following effects can be expected for fusion reactor design.

[0058] - The design period can be shortened by automating the optimization process.

[0059] -By creating a latent space that can reproduce the input space, it can be used as a database for reactor design.

[0060] The following additional notes are provided regarding the above-described embodiments.

[0061] <Additional Notes> (Additional Item 1) An information processing device comprising: an encoder that converts a predetermined physical quantity into a latent variable in a latent space; a prediction unit that performs time series prediction on the latent variable; and a control unit that calculates a target quantity for control based on a prediction result by the prediction unit, and executes control toward the target quantity in the latent space so as to take a path connecting the latent variable and the target quantity by the shortest distance. (Additional Item 2) The information processing device according to Additional Item 1, wherein the encoder is a neural network model constructed by distance learning such that input variables with similar attributes are close to each other in the latent space and input variables with dissimilar attributes are farther apart. (Additional Item 3) The information processing device according to Additional Item 1, wherein the predetermined physical quantity is a physical quantity of plasma in a tokamak fusion reactor. (Addendum 4) An information processing method executed by an information processing device, comprising: a step of converting a predetermined physical quantity into a latent variable in a latent space; a step of performing time series prediction on the latent variable; and a step of calculating a target quantity for control based on a prediction result of the time series prediction, and executing control toward the target quantity in the latent space so as to take a path that connects the latent variable and the target quantity by the shortest distance.

[0062] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0063] REFERENCE SIGNS LIST 100 Information processing device 110 Encoder 120 Storage unit 130 Prediction unit 140 Control unit 200 Tokamak device 300 Plasma measurement device 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims

1. An information processing device comprising: an encoder that converts a predetermined physical quantity into a latent variable in a latent space; a prediction unit that performs time series prediction on the latent variable; and a control unit that calculates a target quantity for control based on the prediction result by the prediction unit and executes control toward the target quantity so as to take a path in the latent space that connects the latent variable and the target quantity by the shortest distance.

2. The information processing device according to claim 1, wherein the encoder is a neural network model constructed by distance learning in which input variables with similar attributes are closer to each other in the latent space, and input variables with dissimilar attributes are farther apart.

3. The information processing device according to claim 1, wherein the predetermined physical quantity is a physical quantity of plasma in a tokamak type nuclear fusion reactor.

4. An information processing method executed by an information processing device, comprising: a step of converting a predetermined physical quantity into a latent variable in a latent space; a step of performing time series prediction on the latent variable; and a step of calculating a target quantity for control based on the prediction result of the time series prediction, and executing control toward the target quantity in the latent space so as to take a path connecting the latent variable and the target quantity by the shortest distance.

Images