Information processing device, information processing method, and program

Abstract

To provide an information processor which can predict the state of an environment precisely and accurately.SOLUTION: A structure conversion unit 30 converts the structure of observation data obtained by observing the state in a time space into observation data of the structure of lattice data. A potential time space mapping unit 40 maps from a first actual time space to a potential time space for the observation data converted into the lattice data and the prediction data. A non-linear conversion unit 50 performs a non-linear conversion on the observation data and the prediction data on which mapping was performed, in the potential time space. A high-resolution analysis data acquisition unit 60 maps from the potential time space to a second actual time space for the observation data and the prediction data on which the non-linear conversion was performed, and acquires high-resolution analysis data.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] This disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] For example, in fields such as weather forecasting or oceanographic forecasting, it is necessary to accurately and efficiently predict environmental conditions through simulations and other means. Similarly, it is necessary to efficiently predict the environmental conditions within a device. Methods to improve the accuracy of predictions include increasing resolution (super-resolution) and data assimilation. In relation to this technology, Non-Patent Document 1 discloses the technology of super-resolution data assimilation (SRDA). Furthermore, Non-Patent Documents 2 to 5 disclose technologies related to this disclosure. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Sebastien Barthelemy, et al, “Super-resolution data assimilation”, Ocean Dynamics 72, 661-678 (2022), https: / / doi.org / 10.1007 / s10236-022-01523-x [Non-Patent Document 2] Bin Li, et al, “3D-ReConstnet: A Single-View 3D-Object Point Cloud Reconstruction Network,” IEEE Access, 2020, https: / / ieeexplore.ieee.org / documedo / 9086481?source=authoralert [Non-Patent Document 3] Ashish Vaswani, et al, “Attention is all you need,” Advances in neural information processing systems, 2017, https: / / arxiv.org / abs / 1706.03762 [Non-Patent Document 4] Zhicheng Geng, et al, “RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution”, Computer Vision and Pattern Recognition, 27 Mar 2022, https: / / arxiv.org / abs / 2203.14186 [Non-Patent Document 5] Wenzhe Shi, et al, “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network,” Computer Vision and Pattern Recognition, 2016, https: / / arxiv.org / abs / 1609.05158 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The technology described in Non-Patent Document 1 simply combines a super-resolution method with a data assimilation method. Therefore, Non-Patent Document 1 performs data assimilation using ensemble computation. In ensemble computation, it is necessary to simulate various similar situations. Consequently, the technology described in Non-Patent Document 1 may incur increased computational costs if accurate predictions are attempted. Therefore, the technology described in Non-Patent Document 1 may not be able to perform accurate and efficient predictions.

[0005] This disclosure aims to provide an information processing device, an information processing method, and a program that can accurately and efficiently predict the state of the environment. [Means for solving the problem]

[0006] The information processing device according to this disclosure includes: a structure conversion unit that converts the structure of observation data, which is data obtained by observing a state in spacetime, into observation data of the structure of grid data, which represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime; a latent spacetime mapping unit that maps the observation data converted into grid data and the grid data in spacetime obtained by simulation, which includes at least the time of the observation data and prediction data for a time period earlier than said time, from a first real spacetime to a latent spacetime with fewer elements than the first real spacetime; a nonlinear transformation unit that performs a nonlinear transformation on the observation data and prediction data that have been mapped in the latent spacetime; and a high-resolution analysis data acquisition unit that obtains high-resolution analysis data, which is grid data in spacetime with higher resolution in spacetime than the prediction data, by mapping the observation data and prediction data that have undergone the nonlinear transformation from the latent spacetime to a second real spacetime with more elements than the latent spacetime and higher resolution than the first real spacetime, and the latent spacetime mapping unit and the nonlinear transformation unit perform data assimilation between the observation data and the prediction data.

[0007] The information processing method according to this disclosure converts the structure of observation data, which is data obtained by observing a state in spacetime, into observation data of the structure of grid data, which represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime; maps the observation data converted into grid data and the grid data in spacetime obtained by simulation, which includes at least the time of the observation data and time prior to that time, from a first real spacetime to a latent spacetime with fewer elements than the first real spacetime; performs a nonlinear transformation on the mapped observation data and the predicted data in the latent spacetime; maps the observed data and the predicted data that have undergone the nonlinear transformation from the latent spacetime to a second real spacetime with more elements than the latent spacetime and higher resolution than the first real spacetime, thereby obtaining high-resolution analysis data, which is grid data in spacetime with higher resolution in spacetime than the predicted data; maps from the first real spacetime to the latent spacetime; and performs a nonlinear transformation on the observation data and the predicted data in the latent spacetime, thereby performing data assimilation between the observation data and the predicted data.

[0008] The program according to the present disclosure includes a process of converting the structure of observation data, which is data obtained by observing the state in space-time, into the structure of lattice data indicating numerical values defined on lattice points arranged at predetermined intervals in space-time, a process of mapping the observation data converted into the lattice data and lattice data in space-time obtained by simulation, which is prediction data at least including the time of the observation data and the time including the past from the time, from a first real space-time to a latent space-time having a smaller number of elements than the first real space-time, a process of performing a non-linear transformation on the observation data and the prediction data mapped in the latent space-time, and a process of obtaining high-resolution analysis data, which is lattice data in space-time and has a higher resolution in space-time than the prediction data, by mapping the observation data and the prediction data subjected to the non-linear transformation from the latent space-time to a second real space-time having a larger number of elements than the latent space-time and a higher resolution than the first real space-time, and causing a computer to execute the processes. By the process of mapping from the first real space-time to the latent space-time and the process of performing a non-linear transformation on the observation data and the prediction data in the latent space-time, data assimilation between the observation data and the prediction data is performed.

Effect of the Invention

[0009] According to the present disclosure, it is possible to provide an information processing apparatus, an information processing method, and a program capable of accurately and efficiently predicting the state of the environment.

Brief Description of the Drawings

[0010] [Figure 1] It is a diagram for explaining the resolution of calculation and lattice data. [Figure 2] It is a diagram for explaining the resolution of calculation and lattice data. [Figure 3] It is a diagram for explaining a super-resolution simulation method. [Figure 4] It is a diagram showing the configuration of the information processing apparatus according to the present embodiment. [Figure 5]This is a flowchart showing an information processing method performed by the information processing device according to this embodiment. [Figure 6] This is a diagram showing the configuration of the information processing device according to Embodiment 1. [Figure 7] This is a diagram illustrating the technology involved in the comparative example. [Figure 8] This is a diagram illustrating super-resolution and data assimilation according to Embodiment 1. [Figure 9] This figure compares the experimental results according to Embodiment 1 with those according to the comparative example. [Figure 10] This diagram illustrates a method for learning the components according to Embodiment 1 using the variational Bayesian method. [Figure 11] This block diagram schematically shows examples of hardware configurations for computing processing units that can implement the devices and systems according to each embodiment. [Modes for carrying out the invention]

[0011] (Summary of the embodiment) Prior to describing the embodiments, an overview of these embodiments will be provided. While these embodiments will be described below, they do not limit the invention as defined in the claims. Furthermore, not all combinations of features described within the embodiments are necessarily essential to the solution of the invention. Also, the indices (letters, etc.) used in the following description are not necessarily common throughout this specification.

[0012] For example, in the field of weather forecasting (weather forecasting simulation), microweather forecasting is sometimes applied. Microweather refers to the weather near the ground up to an altitude of about 100m, which is greatly affected by artificial structures and human activities. Microweather forecasting provides simulation results with a resolution 100 to 1000 times higher than general weather forecasting. Microweather forecasting is mainly provided for urban areas, but its application is not limited to cities. Due to its ultra-high resolution, microweather forecasting can incorporate airflow passing over buildings and heat exhaust from buildings, which are not considered in normal weather forecasting. In other words, microweather forecasting can simulate atmospheric flow that is closer to reality. In the near future, microweather forecasting is expected to acquire observational data from sensors and cameras placed in urban areas, drones, smartphones, etc., and make predictions using this observational data. In order to make such predictions accurately, it is necessary to increase the resolution of the calculations.

[0013] Figures 1 and 2 are diagrams illustrating the computation resolution and grid data. Figure 1 shows the 3D computation mesh G1. The 3D computation mesh G1 is represented by a grid corresponding to the 3D space defined by the 3D coordinate space of the X, Y, and Z axes. The smaller the spacing of this grid, the higher the resolution of the computation in the spatial direction. Conversely, the larger the spacing of the grid, the lower the resolution of the computation in the spatial direction.

[0014] Figure 2 shows the 4D computation mesh G2. The 4D computation mesh G2 is constructed such that the 3D computation mesh G1 is arranged in the time direction (indicated by the T-axis), that is, in a time series. Here, the shorter the time interval (sampling period; the interval between T1 and T2 in Figure 2), the higher the resolution of the computation in the time direction. Conversely, the longer the time interval, the lower the resolution of the computation in the time direction. In this embodiment, "spacetime" is described as a 4D spacetime defined by 3D space and 1D time, but the dimensions of spacetime are not limited to 4 dimensions.

[0015] Here, the data representing numerical values ​​of physical quantities in the 4D computational mesh G2 can be considered as grid data in 4D spacetime. Grid data represents numerical values ​​of physical quantities (such as velocity) defined on grid points arranged at predetermined intervals in spacetime. In other words, each point in 3D space (3D computational mesh G1) has a numerical value (physical quantity, etc.) that represents the state of that point, and as shown in the 4D computational mesh G2, the numerical value of each point changes in the 1D time direction. This change in the time direction of each point in 3D space is represented by the grid data. In this case, the grid data can be represented by a 4D data array (numerical array) that represents the numerical values ​​of those physical quantities. The number of elements in the data array is called the number of elements. A data array may be provided for each physical quantity. Grid data can also be described as structured data. Furthermore, when handling grid data in a neural network, as described later, the grid data may represent physical quantities that humans can understand, or it may represent numerical values ​​that humans cannot understand. In other words, grid data has a structure that represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime.

[0016] On the other hand, increasing the computational resolution to perform microclimate prediction simulations increases the amount of data to be handled, resulting in significant computational costs. Therefore, it is conceivable to obtain high-resolution (HR) calculation results by applying super-resolution (SR) to low-resolution (LR) simulation results.

[0017] Figure 3 illustrates the super-resolution simulation method. The super-resolution simulation system performs super-resolution on the low-resolution prediction results obtained from low-resolution simulations. This results in high-resolution prediction results. The super-resolution system is trained beforehand using deep learning (neural network) with high-resolution results obtained from high-resolution simulations. In other words, the super-resolution system is trained by supervised learning using a large amount of high-resolution results as training data. With this configuration, high-resolution prediction results can be obtained by performing low-resolution simulations during operation, thus reducing computational costs.

[0018] Furthermore, to improve the accuracy of prediction simulations, data assimilation between observed data and prediction results can be considered. One data assimilation method, the ensemble Kalman filter, simulates many similar situations using ensemble calculations and estimates the error from the variability of the prediction results. Based on the magnitude of this error, the degree to which the prediction results should be brought closer to the observed data is determined. By performing data assimilation, the prediction results can be brought closer to the observed data, thereby improving the accuracy of the prediction results. Here, as mentioned above, the technology in Non-Patent Document 1 simply combines the super-resolution method and the data assimilation method. In other words, in Non-Patent Document 1, super-resolution and data assimilation are performed independently. In such a method, data assimilation must be performed using ensemble calculations. Therefore, the computational cost increases.

[0019] In contrast, in this embodiment, as described below, super-resolution and data assimilation are performed simultaneously using time-series data of the grid data. This eliminates the need for ensemble calculations. Therefore, this embodiment makes it possible to perform predictions accurately and efficiently.

[0020] Figure 4 shows the configuration of the information processing device 10 according to this embodiment. The information processing device 10 is, for example, a computer. The information processing device 10 includes a simulation unit 20, an observation data acquisition unit 22, a prediction data acquisition unit 24, a structure transformation unit 30, a latent spatiotemporal mapping unit 40, a nonlinear transformation unit 50, a high-resolution analysis data acquisition unit 60, and a low-resolution analysis data calculation unit 70. These components can be realized by a hardware configuration described later. The functions of these components will be described later.

[0021] Figure 5 is a flowchart showing the information processing method performed by the information processing device 10 according to this embodiment. The simulation unit 20 performs a simulation of the state of the environment (step S20). Specifically, the simulation unit 20 performs a low-resolution simulation as described above. More specifically, the simulation unit 20 performs a low-resolution simulation in the time and spatial directions.

[0022] The observation data acquisition unit 22 acquires one or more types of observation data (step S22). The observation data is data obtained by observing the state in space and time. The observation data can be acquired, for example, from sensors and cameras placed in the environment, drones, smartphones, etc. Here, the structure of the observation data does not have to be that of grid data. Also, if the structure of the observation data is that of grid data, its resolution is arbitrary and can be low resolution or high resolution. Details of the observation data will be described later.

[0023] The prediction data acquisition unit 24 acquires prediction data (step S24). Specifically, the prediction data acquisition unit 24 acquires prediction data, which is the simulation result from the simulation unit 20. Here, the prediction data is time-series data showing the temporal change in state. The prediction data is also spatiotemporal grid data obtained by the simulation. The prediction data includes prediction data for time (time series) that includes at least the time of the observed data and time prior to that time. Here, "time of the observed data" includes the latest time (reference time) in all observed data. Details of the prediction data will be described later.

[0024] The structure transformation unit 30 transforms the structure of the observed data (step S30). Specifically, the structure transformation unit 30 transforms the structure of the observed data into observed data of the structure of grid data. More specifically, the structure transformation unit 30 transforms the structure of the observed data, which is data obtained by observing a state in spacetime, into observed data of the structure of grid data, which represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime. Details of the function of the structure transformation unit 30 will be described later.

[0025] The latent spatiotemporal mapping unit 40 maps the observed data and predicted data to latent spatiotemporal space (step S40). Specifically, the latent spatiotemporal mapping unit 40 maps the observed data, which has been converted to grid data in the S30 process, and the predicted data obtained in the S24 process, from the first real spatiotemporal space to latent spatiotemporal space. As will be described later, the latent spatiotemporal mapping unit 40 (process S40) performs data assimilation between the observed data and the predicted data.

[0026] Here, latent spacetime is a spacetime with fewer elements than the first real spacetime. In other words, latent spacetime is a spacetime with lower resolution than the first real spacetime. Therefore, the number of elements in the data array in latent spacetime is less than the number of elements in the data array in the first real spacetime. Furthermore, the data in latent spacetime can consist of numerical arrays that compress the temporal and spatial information of the observed and predicted data. It can also be said that latent spacetime is a latent space that includes the concept of time (time series). In addition, dimensions are not distinguished in latent spacetime. That is, in latent spacetime, time and space are not distinguished, and the dimensions of 3D space are also not distinguished. Furthermore, the first real spacetime is the spacetime corresponding to the environment in which the predicted data is obtained.

[0027] Furthermore, the latent spatiotemporal mapping unit 40 can obtain data in which the observed data (observed data converted to grid data) has been mapped to the latent spatiotemporal space, and data in which the predicted data has been mapped (projected) to the latent spatiotemporal space, through the mapping to the latent spatiotemporal space described above. The latent spatiotemporal mapping unit 40 may also map the predicted data to the latent spatiotemporal space using the observed data converted to grid data. Therefore, "data in which the predicted data has been mapped to the latent spatiotemporal space" may also include data obtained by mapping the observed data to the latent spatiotemporal space. On the other hand, when the latent spatiotemporal mapping unit 40 maps observed data alone to the latent spatiotemporal space.

[0028] Data assimilation between observed and predicted data occurs by mapping the predicted data using the observed data converted to grid data. In other words, when the observed data converted to grid data and the predicted data are mapped to latent spacetime, they can be mixed (fused). To put it another way, in latent spacetime, the observed data converted to grid data is incorporated into the predicted data. A more detailed description of the latent spacetime mapping unit 40 will be given later.

[0029] The nonlinear transformation unit 50 performs a nonlinear transformation in the latent spacetime (step S50). Specifically, the nonlinear transformation unit 50 performs a nonlinear transformation on the mapped observed data and predicted data in the latent spacetime. More specifically, the nonlinear transformation unit 50 may merge the observed data and predicted data mapped to the latent spacetime to obtain data in the latent spacetime (latent spacetime data; fused data). Therefore, the nonlinear transformation unit 50 (processing in S50) performs data assimilation between the observed data and predicted data. The nonlinear transformation unit 50 may also repeat the nonlinear transformation to make the distribution of numerical values ​​of the latent spacetime data mapped to the latent spacetime discontinuous. Furthermore, the nonlinear transformation unit 50 may make the distribution of numerical values ​​of the latent spacetime data mapped to the latent spacetime more complex or simpler through the nonlinear transformation. In this case, the nonlinear transformation unit 50 may perform the nonlinear transformation in such a way that appropriate super-resolution is achieved. Note that the nonlinear transformation unit 50 does not need to change the number of elements when changing the distribution of numerical values. Note that super-resolution can be achieved by increasing the number of elements. More detailed information about the functions of the nonlinear transformation unit 50 will be provided later.

[0030] The high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data (step S60). Specifically, the high-resolution analysis data acquisition unit 60 maps the observed and predicted data, which have undergone nonlinear transformation, from latent spacetime to a second real spacetime. As a result, the high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data. In other words, the high-resolution analysis data is analysis data that has been super-resolved in both the temporal and spatial directions.

[0031] Here, the second real spatiotemporal space is a real spatiotemporal space with a larger number of elements than the latent spatiotemporal space and a higher resolution than the first real spatiotemporal space. Therefore, the second real spatiotemporal space can be said to be a high-resolution space (HR space). Furthermore, the high-resolution analysis data (HR analysis data) is grid data in spatiotemporal space. Furthermore, the high-resolution analysis data is data with a higher resolution in spatiotemporal space than the prediction data. Furthermore, the high-resolution analysis data may be time-series data in a time period (time series) that includes the past and future of the time of the observed data. That is, if the input prediction data exists in the future relative to the time of the observed data (reference time), the high-resolution analysis data is analysis data that includes similar future times. In other words, the prediction data may be prediction data in a time period that includes the time of the observed data and the time period that includes the past and future of that time. In this case, the high-resolution analysis data acquisition unit 60 may acquire high-resolution analysis data in a time period that includes the past and future of the time of the observed data. More detailed functions of the high-resolution analysis data acquisition unit 60 will be described later.

[0032] The low-resolution analysis data calculation unit 70 calculates low-resolution analysis data (step S70). Specifically, the low-resolution analysis data calculation unit 70 calculates low-resolution analysis data using high-resolution analysis data. Here, low-resolution analysis data (LR analysis data) is analysis data with lower spatiotemporal resolution than high-resolution analysis data. More specifically, the low-resolution analysis data calculation unit 70 calculates low-resolution analysis data by applying arithmetic operations (mathematical methods) such as algebraic interpolation to the high-resolution analysis data. More detailed functions of the low-resolution analysis data calculation unit 70 will be described later.

[0033] The simulation unit 20 takes the low-resolution analysis data as input and performs the simulation at the next timing (S20). This repeats the processing flow shown in Figure 5.

[0034] As described above, the information processing device 10 according to this embodiment is configured to acquire high-resolution analysis data by performing data assimilation and super-resolution on spatiotemporal observation data and spatiotemporal low-resolution prediction data. Furthermore, the information processing device 10 is configured to perform processing in latent spatiotemporal space when acquiring high-resolution analysis data. Therefore, it becomes possible to predict the state of the environment accurately and efficiently.

[0035] Furthermore, the information processing device 10 according to this embodiment is configured to calculate low-resolution analysis data. This makes it possible to continuously perform low-resolution simulations. Therefore, it becomes possible to perform simulations efficiently.

[0036] Furthermore, the nonlinear transformation unit 50 may perform super-resolution in the time direction by deforming the data array on the data mapped to latent spacetime. In this case, the high-resolution analysis data acquisition unit 60 may acquire high-resolution analysis data by independently performing super-resolution in the spatial direction for each time point in the time direction on the data that has been super-resolved in the time direction in latent spacetime. This makes it possible to process even more efficiently. More details will be described later.

[0037] Furthermore, the structural transformation unit 30, the latent spatiotemporal mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be implemented by a trained model learned by a machine learning algorithm. In this case, the structural transformation unit 30, the latent spatiotemporal mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be implemented by a trained model learned by supervised learning using data with higher spatiotemporal resolution than the predicted data as training data. Alternatively, the structural transformation unit 30, the latent spatiotemporal mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be implemented by a trained model learned by unsupervised learning in a manner that reduces the loss function. This makes it possible to efficiently acquire high-resolution analysis data using observation data that is not grid data and low-resolution predicted data. More details will be described later.

[0038] (Embodiment 1) The embodiments will be described below with reference to the drawings. For clarity of explanation, the following descriptions and drawings have been omitted and simplified as appropriate. In addition, the same elements are denoted by the same reference numerals in each drawing, and redundant explanations have been omitted where necessary.

[0039] <Information Processing Device> Figure 6 shows the configuration of the information processing device 100 according to Embodiment 1. The information processing device 100 is, for example, a computer. The information processing device 100 includes a learning processing unit 110, a simulation unit 120, an observation data acquisition unit 122, a prediction data acquisition unit 124, a structure transformation unit 130, a latent spatiotemporal mapping unit 140, a nonlinear transformation unit 150, a high-resolution analysis data acquisition unit 160, and a low-resolution analysis data calculation unit 170. These components can be realized by the hardware configuration described later.

[0040] The learning processing unit 110 performs machine learning, such as neural networking, on the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160. As a result, the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 become trained models and can realize their respective functions.

[0041] Here, when performing machine learning, the learning processing unit 110 may learn the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 together, rather than learning them separately. Specifically, the learning processing unit 110 learns the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a single, integrated manner using an end-to-end deep learning method. In other words, the learning processing unit 110 performs machine learning by treating each function of the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as layers of a neural network. To put it another way, the learning processing unit 110 performs machine learning by treating the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as a single neural network. The pre-training may be performed separately for each of the following components: the structural transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160.

[0042] The learning processing unit 110 may perform supervised learning or unsupervised learning. When performing supervised learning, the learning processing unit 110 may, for example, use high-precision and high-resolution time-series weather data as training data. Details of the processing of the learning processing unit 110 will be described later. Note that the structure transformation unit 130, latent spatiotemporal mapping unit 140, nonlinear transformation unit 150, and high-resolution analysis data acquisition unit 160 are not limited to being implemented as trained models learned by machine learning by the learning processing unit 110.

[0043] The simulation unit 120 corresponds to the simulation unit 20 described above. The simulation unit 120 performs low-resolution simulations in the time and spatial directions. The simulation unit 120 generates low-resolution prediction data (simulation data) using the input initial state.

[0044] Low-resolution prediction data (LR prediction simulation data) is digital data defined on grid points in spacetime. In other words, prediction data is grid data defined in spacetime. To put it another way, prediction data has a data array defined by a grid structure. That is, prediction data, which is grid data in spacetime, has a data array for each grid point at predetermined intervals in the spatial and temporal directions. In the 4D calculation mesh G2 shown in Figure 2, the grid points are arranged in the temporal direction, and in the 3D calculation mesh G1 of the 4D calculation mesh G2, the grid points are arranged in the three spatial directions. The grid data of the prediction data has numerical values ​​at each of these grid points. The spacing between grid points can be equal or unequal. Note that since the simulation unit 120 performs low-resolution simulations, the grid data of the obtained prediction data has, on average, longer temporal and spatial spacing compared to high-resolution grid data.

[0045] For example, prediction data is a data array A in 4-dimensional spacetime. hijk Assume that the following is defined: The subscript h corresponds to the time direction (t direction), the subscript i corresponds to the X-axis direction in 3D space, the subscript j corresponds to the Y-axis direction in 3D space, and the subscript k corresponds to the Z-axis direction in 3D space. In this case, the data array A hijk The subscripts h,i,j,k take integer values. Each of these sets of integer values ​​(h,i,j,k) corresponds to one of the grid points of the 4D computational mesh G2. Thus, the numerical value A on the grid point is determined. hijk This allows for a unique specification. Thus, the prediction data is a data array A with integer subscripts for each dimension in a four-dimensional space. hijk It is defined as follows.

[0046] Here, data array A hijk In this case, let's assume there are H elements in the time direction, that is, the subscript h can take H values. Similarly, let's assume there are I elements in the X direction, that is, the subscript i can take I values. Also, let's assume there are J elements in the Y direction, that is, the subscript j can take J values. Also, let's assume there are K elements in the Z direction, that is, the subscript k can take K values. In this case, data array A hijk The number of elements is H*I*J*K. If H=I=J=K=10, then the data array A hijk The number of elements is 10 4 This is the result.

[0047] Predictive data includes past time data in the time series. It may also include future time data in the time series. Furthermore, as mentioned above, predictive data is time data that includes the time (reference time) of the observation data described later.

[0048] The prediction data includes data showing the physical variables (physical quantities) necessary to predict the state of the atmosphere and other environmental conditions based on physical equations. In the case of weather prediction simulations, the physical equations mentioned above are, for example, the Navier-Stokes equations or thermodynamic equations for fluids. The physical quantities are, for example, air velocity, pressure, temperature, water vapor mixing ratio, and cloud droplet number density. For each of these physical quantities, the aforementioned four-dimensional data array may be provided. The data array can be said to represent the "field" in physics for each physical quantity.

[0049] Furthermore, it should be noted that in this embodiment, the prediction data (LR simulation results) are obtained from a single scenario. In other words, the prediction data is a single simulation result obtained when a simulation (physical simulation) is performed from a single initial state. In contrast, in the data assimilation described in Non-Patent Document 1, simulations are performed for various similar situations in order to perform ensemble calculations. In other words, Non-Patent Document 1 performs prediction simulations using multiple scenarios.

[0050] The prediction data acquisition unit 124 corresponds to the above-described prediction data acquisition unit 24. The prediction data acquisition unit 124 acquires the above-described prediction data (low-resolution physical simulation prediction results) from the simulation unit 120. The prediction data is represented by the following equation (1). [Number] ···(1)

[0051] Here, x t L is a vector field consisting of all physical quantities of the prediction data. That is, x is a vector field indicating a set of values (vectors) at each grid point in the three-dimensional space at a certain time t. And equation (1) indicates a set of vector fields of all physical quantities from time t = 0 to time t = n. In other words, equation (1) indicates all of the data arrays (numerical arrays) in the four-dimensional spacetime regarding the prediction data. Also, the subscript L indicates low resolution in the spatial direction. Also, t indicates the time stamp of the prediction data. t indicates a time stamp with a long time interval. That is, t indicates low resolution in the time direction (that is, a long time interval). That is, equation (1) represents data with a small number of grids (a small number of elements; low resolution) in the four-dimensional spacetime.

[0052] The observation data acquisition unit 122 corresponds to the above-described observation data acquisition unit 22. Similar to the observation data acquisition unit 22, the observation data acquisition unit 122 acquires observation data from sensors, cameras, drones, smartphones, etc. arranged in the environment. The observation data is represented by the following equation (2). [Number] ···(2)

[0053] Equation (2) represents a set of digital data consisting of numerical values ​​(observed values) o that indicate a certain state at time τ. Here, τ represents the timestamp of the observed data. The time intervals of τ do not have to be equal. Also, o may or may not represent a physical quantity. In other words, o may represent a numerical value of a non-physical quantity.

[0054] Furthermore, the observed data may be grid data. Alternatively, the observed data does not need to be grid data. In other words, the observed data may be unstructured data (non-grid data). In unstructured observed data, observed values ​​are associated with temporal and spatial information, but there is no regularity in the intervals of that temporal and spatial information. In other words, unstructured observed data does not have a spatial mesh structure. Therefore, the observed data may represent observed values ​​in random times and spaces.

[0055] Furthermore, observational data may include a variety of miscellaneous and different types of data. Observational data may be image data, sound data, point data, or log data. Observational data may also be a set of values ​​for physical quantities representing atmospheric conditions, or digital data from which those physical quantities can be estimated. For example, observational data may show AMeDAS observation values. In this case, the observation values ​​may show temperature, humidity, or wind speed at various locations. Observational data may also show the radiance of an object (such as a building). This allows for the estimation of the temperature at the location and time observed for that object. Observational data may also show the acceleration log of an airborne aircraft such as a drone. This allows for the estimation of wind speed or turbulent dissipation rate at the location and time observed for that aircraft. Observational data may also show images of the sky. This allows for the estimation of cloud cover or precipitation at the location and time the image was taken. Observational data may also show sales of frozen desserts (ice cream, sherbet, ice pops, shaved ice, etc.) at a certain location (such as a convenience store). This allows for the estimation of the local temperature in that area. In other words, the higher the sales of frozen desserts, the higher the temperature in that region can be estimated to be.

[0056] Furthermore, when a sufficiently accurate simulation is performed, the simulation results can be said to accurately represent the actual situation. Therefore, the results of a sufficiently accurate simulation can be considered as observational data, representing reality well. Depending on the settings, it is possible to conduct "experiments that closely resemble reality," such as microclimate simulations, which incorporate the shape of buildings and the waste heat emitted from them. Such simulations have sufficiently small errors and can therefore be considered as observational data, representing reality well.

[0057] Furthermore, the spatiotemporal resolution in which the observational data is observed may be high or low. If the observational data represents satellite data or radar data, it is obtained by line (1D) observation or surface (2D) observation. If the observational data is obtained by Doppler LIDAR (Light Detection and Ranging), it is obtained by 3D observation. In these cases, the spatial resolution (and the time interval of observation) can be used to determine whether the resolution of the observational data is low or high.

[0058] Furthermore, when observational data corresponds to AMeDAS data, it is obtained through point (0-dimensional) observations. Even in this case, the resolution can be defined as follows: Even with point observations, there is a representative scale that indicates the scale of the state represented by the observed value. Observational data with a large representative scale can be considered low-resolution observational data. Observational data with a small representative scale can be considered high-resolution observational data. For example, typical weather models observe observational values ​​with coarse spatial resolution, such as those representing a horizontal range of 1km to 10km. Therefore, strict conditions are imposed on the observations, such as being in a location without nearby obstructions, on grassland, and not exposed to direct sunlight, thus avoiding the influence of artificial heat emissions. Such observational data can be considered low-resolution observational data. On the other hand, micrometeorological models can observe observational values ​​with fine spatial resolution, such as those representing a horizontal range of 1m to 5m, which are affected by artificial heat emissions. Micrometeorological models can perform simulations considering the impact of such observational data on the atmosphere. Such observational data can be considered high-resolution observational data.

[0059] It should be noted that high-resolution observational data and low-resolution predictional data differ significantly in spatial and temporal resolution. Therefore, when performing data assimilation, it is necessary to average the high-resolution observational data in the spatiotemporal direction to match the low-resolution predictional data. Consequently, it is usually difficult to directly perform data assimilation on high-resolution observational data and low-resolution predictional data. In contrast, this embodiment allows data assimilation to be performed on arbitrary observational data and low-resolution predictional data.

[0060] The structure transformation unit 130 corresponds to the structure transformation unit 30 described above. The structure transformation unit 130 functions as a structuring device. The structure transformation unit 130 may be implemented by an existing structuring device. The structure transformation unit 130 transforms observation data, which is unstructured data (non-grid data), into observation data with the structure of grid data. In other words, the structure transformation unit 130 converts observation data into grid data. The structure transformation unit 130 transforms observation data into data on each grid point of grid data in high-resolution spatiotemporal space. In other words, the structure transformation unit 130 transforms observed values ​​into physical quantities defined on the grid. The grid data obtained by transforming the observation data may correspond to grid data with a larger number of elements (i.e., higher resolution) than the grid data of the prediction data.

[0061] If s() is the function representing the structuring device realized by the structural transformation unit 130, then the function of the structural transformation unit 130 is expressed by the following equation (3). The left side of equation (3) corresponds to the output data of the structural transformation unit 130. Note that the observation data input to the structural transformation unit 130 (structuring device) may be grid data. Also, the observation data input to the structuring device may represent physical quantities or numerical values ​​other than physical quantities.

number

[0062] Here, o T Ho is a vector field consisting of all observed values ​​of the observed data. That is, o is a vector field representing the set of values ​​(vectors) at each grid point in 3D space at a given time T. The left side of equation (3) represents the set of vector fields of all observed values ​​(physical quantities, etc.) from time T=0 to time T=N. In other words, the left side of equation (3) represents all of the data arrays (numerical arrays) in 4D spacetime corresponding to the observed data (physical quantities, etc.). The subscript H indicates high resolution in the spatial direction. T represents the timestamp of the observed data. T represents a timestamp with a short time interval. That is, T indicates high resolution in the time direction (i.e., a short time interval). In short, the left side of equation (3) represents data with a large number of grids (a large number of elements; high resolution) in 4D spacetime.

[0063] The structuring unit of the structure transformation unit 130 may be implemented by a trained model learned by machine learning, such as a neural network. In this case, the structure transformation unit 130 may be implemented by a linear projection operator, a fully connected layer, or a graph convolutional network (the same applies to the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160, which will be described later). In this case, the structure transformation unit 130 is trained by the learning processing unit 110. For example, the structuring unit may be implemented by the technology shown in Non-Patent Literature 2. As described above, the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 are trained end-to-end using an end-to-end deep learning method. The structure transformation unit 130 (structuring unit) is then trained to transform the observed data into appropriate grid data for performing super-resolution data assimilation according to this embodiment.

[0064] Furthermore, the structurer does not necessarily have to be implemented using a pre-trained model learned by machine learning. In this case, the structure transformation unit 130 may read the time and position information of the observed data and project the observed data onto the grid point closest to the read time and position information among the grid points arranged at predetermined intervals in spacetime. In this case, the structure transformation unit 130 may also substitute missing values ​​for grid points to which the observed data is not projected.

[0065] Even if the observed data input to the structural transformation unit 130 represents non-physical quantities, the structural transformation unit 130 can output grid data showing numerical values ​​of physical quantities, etc., corresponding to the observed data. For example, if the observed data input to the structural transformation unit 130 represents the sales of frozen desserts at each location at each time, the structural transformation unit 130 can output grid data showing the temperature at each location at each time. The structural transformation unit 130 may also accept various multiple sets of observed data as input. In this case, the structural transformation unit 130 can output grid data showing numerical values ​​of physical quantities, etc., corresponding to the multiple sets of observed data. For example, if the observed data input to the structural transformation unit 130 represents the acceleration of drones and the sales of frozen desserts at each location at each time, the structural transformation unit 130 can output grid data showing the wind speed and temperature at each location at each time. Furthermore, the structural transformation unit 130 is not limited to outputting grid data showing human-understandable physical quantities corresponding to the input observed data. The structural transformation unit 130 may also output grid data corresponding to numerical arrays that humans cannot understand (i.e., only neural networks can understand). In other words, the structure transformation unit 130 outputs grid data (numerical array) that represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime. Furthermore, if the observed data is image data (camera data), the structure transformation unit 130 (or another component) may perform preprocessing using object recognition or pixel segmentation. A pre-trained neural network or the like may be used for this preprocessing. The structure transformation unit 130 may then perform the structure transformation process described above on the data that has undergone such preprocessing.

[0066] As described above, the structural transformation unit 130 takes observational data, which is either non-grid data or grid data, and represents observed values ​​of physical or non-physical quantities, as input. The structural transformation unit 130 then outputs grid data of a four-dimensional spatiotemporal data array (numerical array) relating to physical quantities (temperature or wind speed, etc.) that represent the state of the environment (e.g., atmospheric conditions). With this configuration, the structural transformation unit 130 can perform a nonlinear transformation on observational data of non-physical quantities or non-grid data that cannot be handled by existing data assimilation methods, and convert it into observational data of numerical arrays that can be assimilated.

[0067] The latent spatiotemporal mapping unit 140 corresponds to the latent spatiotemporal mapping unit 40 described above. The latent spatiotemporal mapping unit 140 functions as an encoder. The latent spatiotemporal mapping unit 140 may be implemented by an existing encoder. The latent spatiotemporal mapping unit 140 maps the observed data and predicted data in the first real spacetime to the latent spacetime. For each time step, the latent spatiotemporal mapping unit 140 converts the observed data and predicted data structured on a four-dimensional spacetime into data in the latent spacetime. In other words, the latent spatiotemporal mapping unit 140 independently calculates the output for each time step for each input step.

[0068] The function representing the encoder implemented in the latent spatiotemporal mapping unit 140 is e x (),e o If (), then the function of the latent spatiotemporal mapping unit 140 is expressed by the following equations (4) and (5). Here, e x () is a function for mapping the predicted data x to the latent spacetime. o The function () maps the observed data o to latent spacetime. This maps the observed data o and the predicted data x to latent spacetime at each time point t.

number

number

[0069] As a result, the latent spatiotemporal mapping unit 140 obtains latent spatiotemporal data represented by the following equation (6). Equation (6) shows the set of latent spatiotemporal data at each time from time t=0 to t=n.

number

[0070] Here, equation (4) is the observed data converted to grid data o t Using predictive data x t L By mapping this to latent spacetime, the mapping data p t This shows that the mapping data p is obtained. t This represents a numerical array in latent spatiotemporal space corresponding to the low-resolution prediction data. Here, the latent spatiotemporal mapping unit 140 may perform preprocessing and main processing on the observed data. For example, as preprocessing, the latent spatiotemporal mapping unit 140 processes the input observed data o T H Observational data with a structure that matches the spatiotemporal lattice structure of the prediction data. t It may be converted to o. t This is the observation data converted into grid data. T This is obtained by coarsening the time or spatial step to match the spatiotemporal grid structure of the low-resolution prediction data. Then, for example, in this process, the latent spatiotemporal mapping unit 140 transforms the o t Simultaneous data assimilation, which reflects the low-resolution prediction data, creates a mapping data p corresponding to the prediction data. t You may obtain it.

[0071] Furthermore, equation (5) is the observed data converted to grid data o t By mapping this to latent spacetime, the mapping data q t This indicates that the mapping data q can be obtained. t This represents a numerical array in latent spacetime corresponding to the observed data. When the latent spacetime mapping unit 40 maps the observed data individually to latent spacetime, it maps the observed data to latent spacetime.

[0072] Here, as can be seen from the subscript t in equations (4) to (6), in the processing of the latent spatiotemporal mapping unit 140, the temporal resolution (time step t) within the latent spatiotemporal mapping unit 140 is matched to the resolution of the predicted data. On the other hand, the spatial resolution is smaller than the resolution of the predicted data. In other words, as described above, the number of elements in the numerical array of the mapping data in latent spatiotemporal is less than the number of elements in the numerical array of the predicted data. Therefore, the latent spatiotemporal mapping unit 140 reduces the number of elements in the numerical arrays of the observed data and predicted data (dimensionality reduction in phase space).

[0073] Furthermore, equations (4) to (6) show that the observed data and the predicted data are merged in latent spacetime. In other words, equation (6) shows the data (merged data) that is the result of the merging of the observed data and the predicted data in latent spacetime. Therefore, it can be said that data assimilation is performed on the observed data and the predicted data by the latent spacetime mapping unit 140.

[0074] The latent spatiotemporal mapping unit 140 (encoder) may be implemented by a trained model learned through machine learning, such as a neural network. For example, the latent spatiotemporal mapping unit 140 may reduce the number of elements while performing a nonlinear transformation using convolution and pooling. The neural network may also be a neural network that reflects physical symmetry. A convolutional neural network may be used to reflect spatial translational symmetry. A group convolutional neural network may be used to reflect spatial rotational symmetry. A vision transformer or graph convolutional neural network may be used to reflect relabeling symmetry. This makes it possible to perform transformations that take physical symmetry into account, unlike existing data assimilation methods. This is also true for the decoder of the high-resolution analysis data acquisition unit 160, which will be described later.

[0075] As described above, the latent spatiotemporal mapping unit 140 takes as input low-resolution prediction data, which is a set of data arrays in four-dimensional spatiotemporal space, and observation data that has been transformed into grid data. The latent spatiotemporal mapping unit 140 then outputs mapping data on latent spatiotemporal space with a small number of elements. With this configuration, the latent spatiotemporal mapping unit 140 can improve the processing efficiency of the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160. In other words, since the number of elements in the numerical array is reduced, the amount of data to be processed is reduced. Therefore, the amount of computing resources required is suppressed.

[0076] The nonlinear transformation unit 150 corresponds to the nonlinear transformation unit 50 described above. The nonlinear transformation unit 150 functions as a time series converter. The nonlinear transformation unit 150 may be implemented by an existing time series converter. The nonlinear transformation unit 150 performs a nonlinear transformation on the mapped observed data and predicted data in the latent spacetime. More specifically, the nonlinear transformation unit 150 may perform a nonlinear transformation on the time series to generate data in which the mapped observed data and predicted data in the latent spacetime are fused. The nonlinear transformation unit 150 according to Embodiment 1 may then perform super-resolution in the time direction on the fused data. Here, as described above, the number of elements in the latent spacetime is reduced compared to the real spacetime. Therefore, the amount of data to be processed is suppressed, and super-resolution in the time direction can be performed efficiently.

[0077] If F() is the function representing the time series converter implemented by the nonlinear conversion unit 150, then the function of the nonlinear conversion unit 150 is expressed by the following equation (7). The left side of equation (7) corresponds to the output data of the nonlinear conversion unit 150. As mentioned above, the subscript T indicates high resolution in the time direction. Also, r T This represents the data (fused data) obtained at time T, where the observed data and predicted data mapped to latent spacetime are merged through data assimilation. The left-hand side of equation (7) represents the set of fused data from time T=0 to time T=N. In other words, the left-hand side of equation (7) represents the data array (numerical array) in 4-dimensional spacetime corresponding to the fused data.

number

[0078] As described above, equation (7) shows that the observed data and the predicted data are merged in latent spacetime. In other words, the left side of equation (7) represents the data (fused data) that has been merged in latent spacetime from the observed data and the predicted data. Therefore, it can be said that data assimilation is performed on the observed data and the predicted data by the nonlinear transformation unit 150.

[0079] The time series converter of the nonlinear transformation unit 150 may be implemented by a trained model learned through machine learning, such as a neural network. In this case, the nonlinear transformation unit 150 may be implemented by a neural network utilizing an attention mechanism. Alternatively, the time series converter of the nonlinear transformation unit 150 may perform the nonlinear transformation using a technique called a Transformer, as described in Non-Patent Document 3, for example.

[0080] Furthermore, the nonlinear transformation unit 150 may implicitly calculate the prediction error using spatiotemporal information. In other words, the nonlinear transformation unit 150 may calculate the error within the neural network from the time evolution of the spatial pattern. To put it another way, the nonlinear transformation unit 150 may calculate the prediction error by matching spatiotemporal patterns and determining the magnitude of the error relative to the ground truth data. The nonlinear transformation unit 150 may be implemented by a neural network that performs data assimilation between the predicted data and the observed data using the prediction error as a weight. This allows for efficient computation of data assimilation between the predicted data and the observed data without the need for an ensemble.

[0081] Furthermore, the time series transformer of the nonlinear transformation unit 150 may repeatedly perform linear transformations (e.g., affine transformations) and nonlinear transformations such as ReLU (Rectified Linear Unit). In the process of performing this processing, the nonlinear transformation unit 150 may transform the data array to make the time step finer, for example, using the technology described in Non-Patent Literature 4. The nonlinear transformation unit 150 may divide the multiple elements in the latent spacetime into two parts: time-direction elements and spatial-direction elements, and increase the number of time-direction elements. For example, suppose there are H' time-direction elements, I' elements in the X-axis direction, J' elements in the Y-axis direction, and K' elements in the Z-axis direction. In this case, the nonlinear transformation unit 150 transforms the 4-dimensional array H'×I'×J'×K' into a 2-dimensional array H'×M, where M is the number of spatial elements, and M = I'×J'×K'. The nonlinear transformation unit 150 may then transform this array into an array of 2H'×M / 2. As a result, the number of elements in the time direction doubles, and the time step becomes half. Therefore, super-resolution in the time direction is performed on the data mapped to latent spacetime. In this way, the nonlinear transformation unit 150 may perform super-resolution in the time direction by deforming the data array of the data mapped to latent spacetime.

[0082] As described above, the nonlinear transformation unit 150 takes a set of numerical arrays (data arrays) corresponding to observed data and predicted data in latent spacetime as input. The nonlinear transformation unit 150 may also output time-series data in latent spacetime that has been super-resolved in the time direction. The time resolution of this time-series data in latent spacetime that has been super-resolved in the time direction is higher than the time resolution of the predicted data. As a result, the nonlinear transformation unit 150 performs super-resolution in the time direction in latent spacetime with a small number of elements (i.e., low-dimensional in phase space), which reduces the amount of computation and therefore the processing cost. Consequently, computational efficiency can be increased.

[0083] Furthermore, the time series converter in the nonlinear transformation unit 150 performs super-resolution in the time direction. As mentioned above, since the number of elements in the numerical array is small in latent spacetime, the amount of memory required is saved, making it possible to handle long time series. This makes it possible to perform super-resolution in the time direction efficiently. The processing of the nonlinear transformation unit 150 allows for data assimilation between observed data and predicted data while performing super-resolution in the time direction. Therefore, it is possible to perform data assimilation and super-resolution in the time direction efficiently.

[0084] The high-resolution analysis data acquisition unit 160 corresponds to the high-resolution analysis data acquisition unit 60 described above. The high-resolution analysis data acquisition unit 160 has the function of a decoder. The high-resolution analysis data acquisition unit 160 may be implemented by an existing decoder. The high-resolution analysis data acquisition unit 160 maps the observed and predicted data, which have undergone nonlinear transformation, from latent spacetime to a second real spacetime. For each time step, the high-resolution analysis data acquisition unit 160 maps the data obtained by the processing of the nonlinear transformation unit 150 from latent spacetime to the second spacetime. As a result, the high-resolution analysis data acquisition unit 160 performs super-resolution in the spatial direction on the data obtained by the processing of the nonlinear transformation unit 150. As a result, the high-resolution analysis data acquisition unit 160 converts the data in latent spacetime into high-resolution analysis data for each time step. In other words, in the high-resolution analysis data acquisition unit 160, the output for each time step is calculated independently for each input at each time step.

[0085] In this way, the high-resolution analysis data acquisition unit 160 acquires high-resolution analysis data Da1. As described above, high-resolution analysis data Da1 is data that has higher spatiotemporal resolution than the prediction data. Furthermore, depending on the time range of the input prediction data, high-resolution analysis data Da1 may be analysis data in a time (time series) that includes the past and future of the time of the observation data. Therefore, high-resolution analysis data Da1 can be analysis data that has been extrapolated in the time direction (time extrapolation) with respect to the time of the observation data (reference time).

[0086] If d() is the function representing the decoder implemented in the high-resolution analysis data acquisition unit 160, then the function of the high-resolution analysis data acquisition unit 160 is expressed by the following equation (8). d() is the fused data r with respect to time T. T This is a function for mapping from latent spacetime to a second real spacetime. As a result, at each time T, the fused data, which is the result of data assimilation between observed data and predicted data, is mapped to a high-resolution second real spacetime. This allows for spatial super-resolution of the fused data, which has already undergone super-resolution in the time direction by the nonlinear transformation unit 150.

number

[0087] As a result, the high-resolution analysis data acquisition unit 160 obtains second real spatiotemporal data represented by the following equation (9). Equation (9) represents the set of high-resolution analysis data y at each time point from time T=0 to T=N. Equation (9) corresponds to the high-resolution analysis data Da1, which is the output data of the high-resolution analysis data acquisition unit 160. The high-resolution analysis data shown in equation (9) is time-series data on a high-resolution grid. The high-resolution analysis data Da1 may include past, present, and future data.

number

[0088] Here, y T Hy is a vector field consisting of all physical quantities in the high-resolution analysis data. That is, y is a vector field representing the set of values ​​(vectors) at each grid point in 3D space at a given time T. Equation (9) represents the set of vector fields of all physical quantities from time T=0 to time T=N. In other words, equation (9) represents all of the data arrays (numerical arrays) in 4D spacetime relating to the high-resolution analysis data. The subscript H indicates high resolution in the spatial direction. T represents the timestamp of the high-resolution analysis data. T represents a timestamp with a short time interval. That is, T indicates high resolution in the time direction (i.e., a short time interval). In short, equation (9) represents data with a large number of grids (a large number of elements; high resolution) in 4D spacetime.

[0089] The decoder of the high-resolution analysis data acquisition unit 160 may be implemented by a trained model learned through machine learning, such as a neural network. For example, the neural network related to the high-resolution analysis data acquisition unit 160 may be a neural network that reflects physical symmetry. This makes it possible to perform transformations that take physical symmetry into account, unlike existing data assimilation methods. Furthermore, the high-resolution analysis data acquisition unit 160 may generate high-resolution analysis data by extrapolating in the time direction (time extrapolation) with respect to the time of the observed data (reference time) using a neural network. That is, the neural network of the high-resolution analysis data acquisition unit 160 can be trained to output high-resolution analysis data that is time extrapolated with respect to the reference time, using time series data with higher resolution and higher accuracy in spatiotemporal dimensions than the predicted data as training data.

[0090] Furthermore, the decoder of the high-resolution analysis data acquisition unit 160 may repeatedly perform linear transformations and nonlinear transformations (e.g., ReLU) to increase the spatial resolution. In the process of performing this processing, the high-resolution analysis data acquisition unit 160 may transform the data array (numerical array) to increase the spatial resolution, for example, by using a technique called Pixel Shuffle as shown in Non-Patent Document 5. In this case, for example, suppose the fused data is a numerical array with n elements in the time direction and m elements in the spatial direction. In this case, the high-resolution analysis data acquisition unit 160 may transform the n × m array into an n / 2 × 2m array. As a result, the number of elements in the spatial direction doubles (m → 2m), and the spatial resolution doubles.

[0091] As described above, the high-resolution analysis data acquisition unit 160 takes a numerical array (data array), which is time-series data in latent spatiotemporal space corresponding to the fused data, as input. The high-resolution analysis data acquisition unit 160 then outputs a 4-dimensional numerical array, which is time-series data on a high-resolution grid in a second real spatiotemporal space with a large number of elements. With this configuration, the high-resolution analysis data acquisition unit 160 can efficiently perform spatial super-resolution in real spatiotemporal space on fused data that has undergone data assimilation and super-resolution in the temporal direction.

[0092] In other words, the high-resolution analysis data acquisition unit 160 processes the numerical arrays (fused data) in the latent spatiotemporal space for each time point independently. This allows the high-resolution analysis data acquisition unit 160 to perform spatial super-resolution without referring to temporal information. Therefore, the processing in the high-resolution analysis data acquisition unit 160 saves necessary computational resources (memory, etc.), enabling efficient super-resolution of three-dimensional space. Consequently, it becomes possible to efficiently obtain high-resolution analysis data with super-resolution applied in both the temporal and spatial directions. Furthermore, if the prediction data is prediction data for time including the time of the observation data and time before and in the future of that time, the high-resolution analysis data acquisition unit 160 can acquire high-resolution analysis data for time including the time before and in the future of the observation data. This allows for the provision of high-resolution analysis data while simultaneously providing future prediction information with time extrapolation. Consequently, it becomes possible to provide services with higher added value.

[0093] Furthermore, by obtaining high-resolution analysis data with super-resolution applied in both the temporal and spatial directions, low-resolution prediction simulations can be supplemented as follows. In practical terms, the resolution of prediction simulations such as weather and oceanographic forecasts is often insufficient in both the temporal and spatial directions. For example, in aquaculture, when the water temperature rises, the activity level of fish increases, so if the fish are overfed, they may die due to lack of oxygen. Therefore, aquaculture operators sometimes want to know the timing of water temperature rise in order to adjust the amount of feed. However, in the case of current ocean forecasts, the resolution in the planar direction is relatively low, at about 10 km. With prediction simulations of this resolution, there is a risk that the water temperature near the aquaculture operator's fish farm may not be able to be predicted accurately. In order to accurately predict the water temperature of each aquaculture operator's fish farm, a fine resolution (high resolution) of at least 1 km, and preferably around 100 m, is required. In contrast, this embodiment allows for the acquisition of high-resolution analysis data with super-resolution applied in both the temporal and spatial directions, enabling the provision of accurate information to businesses seeking pinpoint weather and oceanographic forecasts.

[0094] The low-resolution analysis data calculation unit 170 corresponds to the low-resolution analysis data calculation unit 70 described above. The low-resolution analysis data calculation unit 170 calculates low-resolution analysis data Da2 using high-resolution analysis data Da1. The resolution of the low-resolution analysis data Da2 may correspond to the resolution of the prediction data. The low-resolution analysis data Da2 is grid data having a data array with a lower resolution than the high-resolution analysis data Da1. The low-resolution analysis data Da2 may also be snapshot data at a certain time.

[0095] Specifically, the low-resolution analysis data calculation unit 170 calculates low-resolution analysis data Da2 by inputting the high-resolution analysis data y into a predetermined function f(y). Function f may, for example, be a function representing algebraic interpolation. Function f may be a function that performs algebraic interpolation operations such as linear interpolation. Function f may also be a function defined by linear interpolation, bicubic interpolation, or Lanczos interpolation. For example, the low-resolution analysis data calculation unit 170 can calculate low-resolution analysis data Da2 from high-resolution analysis data Da1 by resizing (reducing the resolution of) the high-resolution analysis data. Specifically, the low-resolution analysis data calculation unit 170 expands or shrinks the grid data by locally interpolating the values ​​corresponding to each grid point of the grid data using a polynomial. This calculates the low-resolution analysis data Da2.

[0096] As described above, the low-resolution analysis data Da2 is input to the simulation unit 120 and used to perform a predictive simulation for the next timing. Alternatively, the low-resolution analysis data Da2 may be used to numerically solve physical equations and obtain future predictions. This allows for the time evolution of the state in the predictive simulation. In this case, the low-resolution analysis data Da2 may represent the state at the current time.

[0097] As described above, the information processing device 100 according to Embodiment 1 is configured to perform data assimilation and super-resolution in a latent spatiotemporal space with a reduced number of elements. This reduces the number of elements, i.e., the amount of data, to be processed, making it possible to perform data assimilation and super-resolution efficiently. Therefore, it becomes possible to efficiently assimilate observational data representing non-physical quantities, and observational data that is irregular in the time and space directions, with predictive data. Furthermore, as described above, when the learning processing unit 110 learns the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in an end-to-end learning manner, a large amount of computational resources are required. Therefore, by reducing the amount of data to be processed by processing in latent spatiotemporal space as described above, it becomes possible to perform such learning efficiently.

[0098] Furthermore, the information processing device 100 according to Embodiment 1 is configured to perform super-resolution in the time direction in the nonlinear transformation unit 150 and super-resolution in the spatial direction in the high-resolution analysis data acquisition unit 160. This makes it possible to efficiently perform super-resolution in both the time and spatial directions. In other words, performing super-resolution in both the time and spatial directions simultaneously requires a large amount of memory and computation time. In particular, during the learning phase, it is necessary to store the computation graph and gradient values ​​in order to backpropagate the error. Therefore, a huge amount of memory and computation time is required. In contrast, in the information processing device 100 according to Embodiment 1, the nonlinear transformation unit 150 efficiently references the time dimension in order to perform super-resolution in the time direction on a latent spacetime with a small number of elements. On the other hand, the high-resolution analysis data acquisition unit 160 processes snapshots of each time point, so it does not need to consider the time dimension. Furthermore, since it processes snapshots of each time point, these snapshots can be processed simultaneously. Therefore, it is possible to reduce the required computational resources.

[0099] <Comparison with the comparative example> Next, we will compare the comparative example described in Non-Patent Document 1 with the technology described in Embodiment 1. As mentioned above, the technology described in Non-Patent Document 1 simply combines the super-resolution technique and the data assimilation technique. In other words, in Non-Patent Document 1, super-resolution and data assimilation are performed independently.

[0100] Figure 7 is a diagram illustrating the technology of the comparative example. Figure 7 illustrates the technology of Non-Patent Literature 1 as a comparative example. In the comparative example, as shown by the white dots, physical simulations are performed for various situations using ensemble calculations. Then, as shown by the black dots, we assume that there is observational data at time t1. In this case, as shown by the white triangular dots, super-resolution is performed using the results of the physical simulation at time t1, and high-resolution prediction is made. This super-resolution is performed independently for each of the ensemble calculation results, i.e., each situation, at time t1. Then, at time t1, data assimilation is performed between the high-resolution ensemble prediction results and the observational data, and as shown by the black triangular dots, high-resolution prediction data with the observed data assimilated is obtained as the final output. Thus, the comparative example utilizes ensemble calculations. Also, in the comparative example, super-resolution at a given moment is performed independently of data assimilation at the same moment. And data assimilation is performed in a high-resolution space.

[0101] Figure 8 is a diagram illustrating super-resolution and data assimilation according to Embodiment 1. Embodiment 1 utilizes a single scenario. That is, a physical simulation is performed from a single initial state, and a single simulation result is obtained in the time series, as shown by the white dots. In Figure 8, the physical simulation results (predicted data), which are time series data, are obtained corresponding to times t1, t2, t3, and t4. Then, instead of performing super-resolution at a certain time, super-resolution and data assimilation are performed simultaneously in a latent spacetime with a reduced number of elements, using the observed data (shown by the black dots) and the predicted data (time series data) as input. As a result, high-resolution data, which is time series data, is obtained, as shown by the black triangular dots. Furthermore, as shown in Figure 8, the high-resolution data has higher resolution in the time direction compared to the physical simulation. Thus, unlike the comparative example, Embodiment 1 does not perform ensemble calculations in the simulation. Also, unlike the comparative example, Embodiment 1 performs super-resolution and data assimilation simultaneously in a latent spacetime. Also, unlike the comparative example, Embodiment 1 utilizes time series information to perform super-resolution and data assimilation.

[0102] Figure 9 compares the experimental results according to Embodiment 1 with those according to the comparative example. Figure 9 shows the time series of the mean absolute error (MAE) of the vorticity field ω. Graph A shows the case where neither data assimilation nor super-resolution is performed. Graph B shows the case of the comparative example. Graph C shows the case of Embodiment 1. As shown in Figure 9, the error is minimized in the case of Embodiment 1. Therefore, high-precision prediction can be achieved with the technology according to Embodiment 1.

[0103] Furthermore, in the comparative example, the calculation time per experiment was 320 seconds. In contrast, in the experimental results for Embodiment 1, the calculation time per experiment was 61 seconds. Thus, the technology according to Embodiment 1 achieves a significant reduction in calculation time compared to the comparative example.

[0104] <Learning Method> Next, the learning method of the learning processing unit 110 of the information processing device 100 according to Embodiment 1 will be described. As described above, the learning processing unit 110 learns the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in an end-to-end learning manner. In other words, the learning processing unit 110 may learn the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as a single neural network with these as each layer. To put it another way, the learning processing unit 110 takes observed data and predicted data as input and learns the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 together so that appropriate high-resolution analysis data is output.

[0105] First, let's explain the first learning method. The first learning method is supervised learning. The training data (ground truth data) is, for example, high-precision and high-resolution data. In the case of a weather forecasting system, the training data is, for example, high-precision and high-resolution weather data. In other words, the training data is high-precision and high-resolution time-series data of atmospheric physical variables. The training data is, for example, time-series data (4-dimensional numerical arrays) of velocity fields, temperature fields, density fields, etc. Alternatively, the results of ultra-high-resolution micrometeorological simulations may also be used as training data. The learning processing unit 110 takes the above-mentioned forecast data and observation data as input and, using the above-mentioned training data, updates the neural network parameters (weights, etc.) in the direction of the gradient of the error between the training data and the final output (high-resolution analysis data) using the backpropagation method. By repeating this process, the learning processing unit 110 learns the neural network that constitutes the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160.

[0106] Next, we will explain the second learning method. The second learning method is unsupervised learning. In the supervised learning described above, high-precision and high-resolution time-series data of atmospheric physical variables is required as training data. However, obtaining such data can be difficult. In contrast, unsupervised learning does not require such training data. Unsupervised learning may be performed using, for example, variational Bayesian methods. Unsupervised learning may also be performed using adversarial learning. Below, we will explain the learning method using variational Bayesian methods.

[0107] Variational Bayesian methods are a type of approximation method that approximates the true probability distribution p with a simpler probability distribution q. The parameters of q are then estimated using methods such as minimizing the KL divergence. Variational Bayesian methods use the true physical variables of the environment, such as the atmosphere, as hidden states, and are probabilistic models that provide observed or low-resolution predicted input values ​​based on these hidden states.

[0108] As one example of implementing variational Bayesian methods, we introduce a lower bound on the log-likelihood ln(p(o|x)) and maximize this lower bound. The log-likelihood ln(p(o|x)) can be transformed using Jensen's inequality as shown in equation (10) below. Here, o corresponds to the observed data and x corresponds to the low-resolution prediction data. During the transformation of equation (10), a hidden variable y is introduced. This hidden variable corresponds to the high-resolution analysis data Da1 shown in equation (9). As a result, we can derive a loss function that makes it possible to estimate the hidden variable y from the observed data o and the low-resolution prediction x.

number

[0109] In equation (10), the observed value o corresponds to low-resolution or high-resolution observed data. o corresponds to equation (2) (or the left-hand side of equation (3)) described above. The hidden variable y corresponds to the high-resolution analysis data Da1. Note that the observed value o may also be high-precision predicted data. The variable x corresponds to the input low-resolution predicted data (equation (1)). This probabilistic model consists of a recognition model and a generative model. The recognition model (the part that calculates the hidden variable) corresponds to the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 (encoder / decoder model) in the information processing device 100. The generative model will be described later.

[0110] The fourth equation in equation (10) (the rightmost side; the final form of the transformation) corresponds to the variational lower bound (VLB), which is the lower bound of the log-likelihood. The first term of the variational lower bound is E q [p(o│x,y)] represents the reconstruction error and corresponds to the log-likelihood of the observed data o. The second term of the variational lower bound, KL(q(y|x,o)|p(y|x)), represents the KL divergence. KL divergence is an index that corresponds to the distance between distribution q and distribution p.

[0111] The learning processing unit 110 updates the model parameters (neural network parameters) using backpropagation and gradient descent to maximize (increase) the variational lower bound shown in equation (10) 4, and performs learning. This variational lower bound corresponds to the training error. Maximizing the variational lower bound corresponds to minimizing the loss function in machine learning. In other words, increasing the variational lower bound corresponds to decreasing the loss function in machine learning. At this time, learning proceeds so that the reconstruction error and the KL divergence are balanced. This balance between the reconstruction error and the KL divergence corresponds to the fusion of observed data and predicted data in data assimilation.

[0112] Furthermore, it has been newly discovered that maximizing the variational lower bound has an aspect of generalizing minimum variance estimation. That is, by averaging the observed values ​​o and predicted data x with their respective errors as weights (performing a weighted average), high-resolution analysis data y can be estimated. In other words, since observed values ​​are generally more accurate than predicted data, the analysis data can be estimated by averaging the observed values ​​with a larger weight than the predicted data.

[0113] Figure 10 is a diagram illustrating a method for learning the components according to Embodiment 1 using the variational Bayes method. When learning using the variational Bayes method, the learning processing unit 110 includes a sampler 112 and an observation data generation unit 114. The sampler 112 and the observation data generation unit 114 can be considered a generative model that generates observed values ​​(pseudo-observation data Da3). This generative model allows for the reconstruction error E q [p(o|x,y)] can be calculated. Specifically, the observation data generation unit 114 calculates p(o|x,y). p(o|x,y) represents the distribution of pseudo-observation data o when the prediction data x and high-resolution analysis data y are determined.

[0114] On the other hand, as described above, the structural transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 can be considered as a recognition model. The KL divergence can be calculated using this recognition model. Specifically, the recognition model calculates q(y|x,o) in the KL divergence. q(y|x,o) represents the distribution of the high-resolution analysis data y when the predicted data x and observed data o are inputs. Note that p(y|x) is the prior distribution, which can be obtained by making appropriate assumptions or by pre-training.

[0115] Sampler 112 samples a probability distribution corresponding to the high-resolution analysis data generated through training. This means that Sampler 112 replaces the neural network with a probability distribution (probability model). In other words, since neural networks typically produce deterministic outputs, it is difficult for them to produce random outputs such as probability distributions. Therefore, Sampler 112 combines random numbers sampled from a Gaussian distribution with the output from the neural network (high-resolution analysis data) to simulate a probability distribution (re-parametrization trick). This makes it possible to estimate the error of the high-resolution analysis data. Specifically, the probability distribution is simulated as shown in equation (11) below. μ and σ are deterministic variables given by the neural network, and ε is a random number sampled from a Gaussian distribution.

number

[0116] Specifically, the sampler 112 receives high-resolution analysis data Da1 (corresponding to y in equation (10)) from the high-resolution analysis data acquisition unit 160. The sampler 112 samples random numbers from a Gaussian distribution and adds noise to the high-resolution analysis data. This adds randomness to the high-resolution analysis data provided by the neural network, allowing data that can be considered as values ​​sampled from a probability distribution to be obtained. Therefore, the high-resolution analysis data can be represented as a probability distribution. In other words, the sampler 112 obtains sampled data when the high-resolution analysis data is represented as a probability distribution. The sampler 112 then outputs the sampled data of the high-resolution analysis data to the observation data generation unit 114. Note that when representing a more complex probability distribution, that is, when the noise is generated from a distribution more complex than a Gaussian distribution, a mixture distribution or normalizing flows may be used. In this case, a complex distribution can be obtained by applying a nonlinear transformation to a random variable that follows a simple probability distribution such as a Gaussian distribution.

[0117] The observation data generation unit 114 generates pseudo-observation data Da3 using sampled data from the high-resolution analysis data generated by the sampler 112. In other words, the observation data generation unit 114 converts the sampled data from the high-resolution analysis data into pseudo-observation data Da3. The observation data generation unit 114 can generate pseudo-observation data Da3 that has no data loss in the spatiotemporal direction. The observation data generation unit 114 may be implemented by a neural network that has been trained in advance by machine learning. The observation data generation unit 114 may be implemented by a neural network that reflects physical symmetry. Unsupervised learning can be realized by generating pseudo-observation data Da3.

[0118] Specifically, the observation data generation unit 114 may generate pseudo-observation data Da3 by performing the reverse processing of the processing performed by the structuring unit (structural transformation unit 130) described above. In other words, the observation data generation unit 114 may generate pseudo-observation data Da3 using a technique substantially similar to that of the structuring unit. More specifically, the observation data generation unit 114 picks up data of grid points at arbitrary time and position from the sampling data of high-resolution analysis data, which is grid data, and repeatedly performs linear and nonlinear transformations on that data. As a result, the observation data generation unit 114 obtains pseudo-observation data Da3 in a format substantially similar to that of observation data o obtained by the observation data acquisition unit 122. Therefore, pseudo-observation data Da3 may be non-grid data. Also, pseudo-observation data Da3 may represent numerical values ​​of non-physical quantities.

[0119] As described above, the learning processing unit 110, which applies the variational Bayes method, generates pseudo-observation data Da3 from the high-resolution analysis data Da1. Here, in the variational lower bound expressed by equation 4 of equation (10), the inferred high-resolution analysis data y is a hidden state. Note that this hidden state is not the final output in the framework of the variational Bayes method. In the variational Bayes method, the pseudo-observation data Da3 is the final output during the learning phase. And since the high-resolution analysis data y is a hidden state, there is no need to prepare ground truth data corresponding to the high-resolution analysis data y during the learning phase. Therefore, it becomes unnecessary to prepare high-precision and high-resolution meteorological data required in supervised learning.

[0120] (Example hardware configuration) The following describes an example of hardware resource configurations for realizing the devices and systems according to each of the above embodiments using a single computing device (information processing device, computer). However, each embodiment of the device (information processing device) may be realized using at least two computing devices, either physically or functionally. Furthermore, each embodiment of the device may be realized as a dedicated device or as a general-purpose information processing device.

[0121] Figure 11 is a schematic block diagram showing an example of the hardware configuration of a computing device that can realize the device and system according to each embodiment. The computing device 1000 has a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF 1007 (IF: Interface). Therefore, it can be said that the device according to each embodiment has a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF 1007. The computing device 1000 may be connectable to an input device 1005 and an output device 1006. The computing device 1000 may also be equipped with an input device 1005 and an output device 1006. Furthermore, the computing device 1000 can send and receive information with other computing devices and communication devices via the communication IF 1007.

[0122] The non-volatile recording medium 1004 is a computer-readable medium, such as a Compact Disc or a Digital Versatile Disc. Alternatively, the non-volatile recording medium 1004 may be a USB (Universal Serial Bus) memory, a Solid State Drive, or the like. The non-volatile recording medium 1004 can hold the program without power supply, making it portable. Note that the non-volatile recording medium 1004 is not limited to the media described above. Furthermore, instead of the non-volatile recording medium 1004, the program may be supplied via the communication interface 1007 and a communication network.

[0123] The volatile memory device 1002 is computer-readable and can temporarily store data. The volatile memory device 1002 is a type of memory such as DRAM (dynamic random access memory) or SRAM (static random access memory).

[0124] In other words, when the CPU 1001 executes a software program (computer program; hereinafter simply referred to as "program") stored on disk 1003, it copies it to the volatile storage device 1002 and performs arithmetic processing. The CPU 1001 reads the data necessary for program execution from the volatile storage device 1002. If display is required, the CPU 1001 displays the output result on the output device 1006. When a program is input from an external source, the CPU 1001 obtains the program from the input device 1005. The CPU 1001 interprets and executes the program corresponding to the function (processing) of each component shown in Figures 4, 6, and 10 described above. The CPU 1001 executes the processing described in each embodiment described above. In other words, the function of each component shown in Figures 4, 6, and 10 described above can be realized by the CPU 1001 executing a program stored on disk 1003 or the volatile storage device 1002.

[0125] In other words, each embodiment can be considered to be achievable by the program described above. Furthermore, each embodiment can also be considered to be achievable by a computer-readable non-volatile recording medium on which the program described above is recorded.

[0126] (modified version) It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, in the flowchart described above, the order of each process (step) can be changed as appropriate. Also, one or more of the multiple processes (steps) may be omitted. For example, in the flowchart of Figure 5, the process of S22 may be executed before the process of S20. Also, the process of S24 may be executed before the process of S22. Also, the process of S70 may be omitted.

[0127] Furthermore, while the above-described embodiment explained the case of weather forecasting, this embodiment is not limited to weather forecasting. This embodiment can be applied to any prediction simulation that utilizes grid data. For example, this embodiment can be applied to ocean forecasting. This embodiment can also be applied to astrophysical simulations.

[0128] Furthermore, in this embodiment, the dimensions of "spacetime" are not limited to four dimensions, which consist of three dimensions of space and one dimension of time. The dimensions of "spacetime" may be three dimensions, which consist of two dimensions of space and one dimension of time. Alternatively, the dimensions of "spacetime" may be larger than four dimensions, such as 10 dimensions.

[0129] In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically or otherwise propagating signals. [Explanation of Symbols]

[0130] 10 Information Processing Devices 20 Simulation Department 22 Observation Data Acquisition Unit 24 Predictive Data Acquisition Unit 30 Structural transformation section 40 Latent spacetime mapping section 50 Nonlinear Transformation Unit 60. High-resolution analysis data acquisition unit 70 Low-resolution analysis data calculation unit 100 Information Processing Devices 110 Learning Processing Unit 112 Sampler 114 Observation Data Generation Unit 120 Simulation Department 122 Observation Data Acquisition Unit 124 Prediction Data Acquisition Unit 130 Structural transformation section 140 Latent spacetime mapping section 150 Nonlinear Transformation Unit 160 High-resolution analysis data acquisition unit 170 Low-resolution analysis data calculation unit

Claims

1. A structure conversion unit converts the structure of observational data, which is data obtained by observing a state in spacetime, into observational data of the structure of grid data, which represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime. A latent spatiotemporal mapping unit performs a mapping from a first real spatiotemporal space to a latent spatiotemporal space with fewer elements than the first real spatiotemporal space, using the observed data converted into the aforementioned grid data and the spatiotemporal grid data obtained by the simulation, which includes at least the time of the observed data and the time prior to that time. In the aforementioned latent spacetime, a nonlinear transformation unit performs a nonlinear transformation on the mapped observed data and the predicted data, A high-resolution analysis data acquisition unit obtains high-resolution analysis data, which is grid data in spacetime and has higher resolution in spacetime than the prediction data, by mapping the observation data and prediction data to which the nonlinear transformation has been applied from the latent spacetime to a second real spacetime having a larger number of elements than the latent spacetime and higher resolution than the first real spacetime. It has, The latent spatiotemporal mapping unit and the nonlinear transformation unit perform data assimilation between the observed data and the predicted data. Information processing device.

2. A low-resolution analysis data calculation unit calculates low-resolution analysis data that has a lower spatiotemporal resolution than the high-resolution analysis data using the aforementioned high-resolution analysis data, A simulation unit that performs the simulation using the low-resolution analysis data, The information processing apparatus according to claim 1, further comprising:

3. The nonlinear transformation unit performs super-resolution in the time direction by deforming the data array with respect to the data mapped to the latent spacetime. The information processing apparatus according to claim 1.

4. The high-resolution analysis data acquisition unit acquires the high-resolution analysis data by performing super-resolution in the spatial direction independently for each time point in the temporal direction on the data that has been super-resolved in the temporal direction in the latent spacetime. The information processing apparatus according to claim 3.

5. The aforementioned prediction data is prediction data for the time of the observation data and for time periods including the time prior to and in the future of that time. The aforementioned high-resolution analysis data acquisition unit acquires high-resolution analysis data in time, including past and future time of the observed data. The information processing apparatus according to claim 1.

6. The structural transformation unit, the latent spatiotemporal mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model learned by a machine learning algorithm. The information processing apparatus according to claim 1.

7. The structural transformation unit, the latent spatiotemporal mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model that has been trained by supervised learning using data with higher spatiotemporal resolution than the predicted data as training data. The information processing apparatus according to claim 6.

8. The structural transformation unit, the latent spatiotemporal mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are implemented by a trained model that has been trained by unsupervised learning in such a way as to reduce the loss function. The information processing apparatus according to claim 6.

9. The structure of observational data, which is data obtained by observing a state in spacetime, is converted into observational data of grid data, which represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime. The observed data converted into the aforementioned grid data and the spatiotemporal grid data obtained by the simulation, which includes at least the time of the observed data and the time prior to that time, are mapped from the first real spatiotemporal space to a latent spatiotemporal space with fewer elements than the first real spatiotemporal space. In the aforementioned latent spacetime, a nonlinear transformation is performed on the mapped observed data and the predicted data. By mapping the observed data and predicted data to which the aforementioned nonlinear transformation has been applied, from the latent spacetime to a second real spacetime which has a larger number of elements than the latent spacetime and a higher resolution than the first real spacetime, high-resolution analysis data is obtained which is lattice data in spacetime and has a higher resolution in spacetime than the predicted data. Data assimilation between the observed data and the predicted data is performed by mapping from the first real spacetime to the latent spacetime, and by performing a nonlinear transformation on the observed data and the predicted data in the latent spacetime. Information processing methods.

10. A process that converts the structure of observational data, which is data obtained by observing a state in spacetime, into observational data of the structure of grid data, which represents numerical values ​​defined on grid points arranged at predetermined intervals in spacetime, and A process of mapping the observed data converted into the aforementioned grid data and the spatiotemporal grid data obtained by the simulation, which includes at least the time of the observed data and the time prior to that time, from a first real spatiotemporal space to a latent spatiotemporal space with fewer elements than the first real spatiotemporal space, The process involves performing a nonlinear transformation on the observed data and predicted data that have been mapped in the aforementioned latent spacetime, The process involves mapping the observed data and predicted data, to which the aforementioned nonlinear transformation has been applied, from the latent spacetime to a second real spacetime having a larger number of elements than the latent spacetime and higher resolution than the first real spacetime, thereby obtaining high-resolution analysis data that is lattice data in spacetime and has higher resolution in spacetime than the predicted data. Have the computer run it, Data assimilation between the observed data and the predicted data is performed by a process of mapping from the first real spacetime to the latent spacetime, and a process of performing a nonlinear transformation on the observed data and the predicted data in the latent spacetime. program.

Images