Information processing device, information processing method, and program

MYPI2025001806A0Undetermined Publication Date: 2024-03-29INSTITUTE OF SCIENCE TOKYO
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025001806
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-09-29
Filing Date
2023-09-29
Publication Date
2024-03-29

AI Technical Summary

Technical Problem

Current methods for predicting environmental states, such as weather forecasting, face challenges in achieving accurate and efficient predictions due to increased calculation costs when using super-resolution data assimilation techniques, which often require ensemble calculations.

Method used

The proposed solution involves an information processing device and method that converts observation data into grid data, maps it to a latent spatiotemporal space with fewer elements, performs nonlinear transformation, and then maps it back to a higher resolution space for data assimilation, eliminating the need for ensemble calculations.

Benefits of technology

This approach allows for accurate and efficient prediction of environmental states by reducing calculation costs and improving prediction accuracy through simultaneous super-resolution and data assimilation in a latent space-time framework.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025001806_DRAWING_001
    Figure PI2025001806_DRAWING_001
  • Figure PI2025001806_DRAWING_002
    Figure PI2025001806_DRAWING_002
  • Figure PI2025001806_DRAWING_003
    Figure PI2025001806_DRAWING_003
Patent Text Reader

Abstract

An information processing apparatus that can perform prediction of a state of an environment with high accuracy and efficiently is provided. A structure transforming unit (30) transforms a structure of observation data obtained by observing a state in space-time into observation data having a structure of grid data. A latent space-time mapping unit (40) maps the observation data transformed into the grid data and prediction data from first real space-time to latent space-time. A non-linear transformation unit (50) performs non-linear transformation in the latent space-time on the observation data and the prediction data having been mapped. A high-resolution analytical data acquisition unit (60) maps the observation data and the prediction data subjected to the non-linear transformation from the latent space-time to second real space-time, to acquire high-resolution analytical data. Fig. 4
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] For example, in the fields of weather forecasting or oceanographic forecasting, there is a demand for accurate and efficient prediction of environmental conditions through simulations and the like. Similarly, there is a demand for efficient prediction of environmental conditions within a device. Techniques for improving prediction accuracy include increasing resolution (super-resolution) and data assimilation. In relation to this technology, Non-Patent Document 1 discloses a technology called super-resolution data assimilation (SRDA). Furthermore, Non-Patent Documents 2 to 5 disclose technologies related to the present disclosure.

[0003] Sebastien Barthelemy, et al、「Super-resolution data assimilation」、Ocean Dynamics 72, 661-678 (2022)、https: / / doi.org / 10.1007 / s10236-022-01523-xBin Li, et al、「3D-ReConstnet: A Single-View 3D-Object Point Cloud Reconstruction Network」、IEEE Access、2020、https: / / ieeexplore.ieee.org / documedo / 9086481?source=authoralertAshish Vaswani, et al、「Attention is all you need」、Advances in neural information processing systems、2017、https: / / arxiv.org / abs / 1706.03762Zhicheng Geng, et al、「RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution」、Computer Vision and Pattern Recognition、27 Mar 2022、https: / / arxiv.org / abs / 2203.14186Wenzhe Shi, et al、「Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network」、Computer Vision and Pattern Recognition、2016、https: / / arxiv.org / abs / 1609.05158

[0004] The technology disclosed in Non-Patent Document 1 simply combines a super-resolution technique and a data assimilation technique. Therefore, in Non-Patent Document 1, data assimilation is performed using ensemble calculations. Here, ensemble calculations require simulating various similar situations. Therefore, when attempting to perform accurate predictions with the technology disclosed in Non-Patent Document 1, there is a risk that the calculation costs will increase. Therefore, there is a risk that the technology disclosed in Non-Patent Document 1 will not be able to perform accurate and efficient predictions.

[0005] An object of the present disclosure is to provide an information processing device, an information processing method, and a program that are capable of accurately and efficiently predicting the state of an environment.

[0006] The information processing device according to the present disclosure includes a structure conversion unit that converts the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure that indicates numerical values ​​defined on lattice points arranged at predetermined intervals in space-time; a latent space-time mapping unit that maps the observation data converted into lattice data and prediction data, which is space-time lattice data obtained by simulation and includes at least the time of the observation data and a time earlier than the time of the observation data, from a first real-time space to a latent space having a smaller number of elements than the first real-time space; a nonlinear transformation unit that performs a nonlinear transformation on the mapped observation data and the prediction data in the latent space; and a high-resolution analysis data acquisition unit that maps the nonlinearly transformed observation data and the prediction data from the latent space to a second real-time space having a larger number of elements than the latent space and having a higher resolution than the first real-time space, thereby acquiring high-resolution analysis data that is space-time lattice data and has a higher resolution in space-time than the prediction data, and data assimilation of the observation data and the prediction data is performed by the latent space mapping unit and the nonlinear transformation unit.

[0007] An information processing method according to the present disclosure converts the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values ​​defined on lattice points arranged at predetermined intervals in space-time, maps the observation data converted into lattice data and prediction data, which is lattice data in space-time obtained by simulation and which is at least for a time before the observation data, from a first real-time space to a latent-time space having fewer elements than the first real-time space, performs a nonlinear transformation on the mapped observation data and the prediction data in the latent-time space, and maps the nonlinearly transformed observation data and the prediction data from the latent-time space to a second real-time space having a larger number of elements than the latent-time space and a higher resolution than the first real-time space, thereby obtaining high-resolution analysis data which is lattice data in space-time and has a higher resolution in space-time than the prediction data, performs mapping from the first real-time space to the latent-time space, and performs a nonlinear transformation on the observation data and the prediction data in the latent-time space, thereby performing data assimilation between the observation data and the prediction data.

[0008] The program according to the present disclosure includes a process for converting a structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values ​​defined on lattice points arranged at predetermined intervals in space-time; a process for mapping the observation data converted into the lattice data and prediction data, which is lattice data in space-time obtained by simulation and includes at least the time of the observation data and a time before the time of the observation data, from a first real-time space to a latent-time space having fewer elements than the first real-time space; and a process for mapping the mapped observation data and prediction data in the latent-time space. and a process of mapping the observation data and the prediction data that have been subjected to the nonlinear transformation from the latent space-time to a second real space-time that has a larger number of elements than the latent space-time and a higher resolution than the first real space-time, thereby obtaining high-resolution analysis data that is lattice data in space-time and has a higher resolution in space-time than the prediction data. Data assimilation of the observation data and the prediction data is performed by the process of mapping from the first real space-time to the latent space-time and the process of performing nonlinear transformation on the observation data and the prediction data in the latent space-time.

[0009] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a program that are capable of accurately and efficiently predicting the state of an environment.

[0010] FIG. 1 is a diagram for explaining calculation resolution and grid data. FIG. 2 is a diagram for explaining calculation resolution and grid data. FIG. 3 is a diagram for explaining a super-resolution simulation method. FIG. 4 is a diagram for explaining a configuration of an information processing device according to the present embodiment. FIG. 5 is a flowchart showing an information processing method executed by the information processing device according to the present embodiment. FIG. 6 is a diagram for explaining a configuration of an information processing device according to a first embodiment. FIG. 7 is a diagram for explaining a technology according to a comparative example. FIG. 8 is a diagram for explaining super-resolution and data assimilation according to the first embodiment. FIG. 9 is a diagram comparing experimental results according to the first embodiment with experimental results according to a comparative example. FIG. 10 is a diagram for explaining a method of learning components according to the first embodiment using variational Bayesian method. FIG. 11 is a block diagram schematically showing an example of the hardware configuration of a calculation processing device capable of realizing an apparatus and system according to each embodiment.

[0011] (Outline of the embodiment) Before describing the embodiment, an outline of the embodiment will be described. Note that the embodiment will be described below, but the following embodiment does not limit the invention according to the claims. Furthermore, not all combinations of features described in the embodiment are necessarily essential to the solution of the invention. Furthermore, in the following description, indexes (such as English letters) used are not necessarily common throughout this specification.

[0012] For example, in the field of weather forecasting (weather forecast simulation), micrometeorological forecasting is sometimes applied. Micrometeorological forecasting refers to the weather near the ground up to an altitude of approximately 100 meters, which is significantly affected by man-made structures and human activity. Micrometeorological forecasting provides simulation results with a resolution approximately 100 to 1,000 times higher than that of conventional weather forecasting. Micrometeorological forecasting is primarily provided for urban areas, but its application is not limited to cities. Due to its ultra-high resolution, micrometeorological forecasting can incorporate flow past buildings and heat exhaust from buildings, which are not considered in conventional weather forecasting. In other words, micrometeorological forecasting can simulate atmospheric flow more realistically. In the near future, micrometeorological forecasting is expected to use observation data acquired from sensors, cameras, drones, smartphones, etc. placed in urban environments. To make such predictions accurately, high computational resolution is required.

[0013] 1 and 2 are diagrams for explaining calculation resolution and grid data. FIG. 1 shows a three-dimensional calculation mesh G1. The three-dimensional calculation mesh G1 is represented by a grid corresponding to a three-dimensional space defined by a three-dimensional coordinate space of the X-axis, Y-axis, and Z-axis. The shorter the grid spacing, the higher the resolution of the calculation in the spatial direction. Conversely, the longer the grid spacing, the lower the resolution of the calculation in the spatial direction.

[0014] FIG. 2 shows a four-dimensional computational mesh G2. The four-dimensional computational mesh G2 is configured by arranging the three-dimensional computational mesh G1 in the time direction (shown by the T axis), that is, in a time series. Here, the time interval (sampling period; T in FIG. 2) is 1 and T 2 The shorter the time interval between the time points, the higher the resolution of the calculation in the time direction. Conversely, the longer the time interval, the lower the resolution of the calculation in the time direction. Note that in this embodiment, "space-time" is described as a four-dimensional space-time defined by three-dimensional space and one-dimensional time, but the dimensions of space-time are not limited to four dimensions.

[0015] Here, data indicating the numerical values ​​of physical quantities, etc. in the four-dimensional computational mesh G2 can be considered as grid data in four-dimensional space-time. The grid data indicates the numerical values ​​of physical quantities (e.g., velocity) defined on grid points arranged at predetermined intervals in space-time. That is, at each point in three-dimensional space (the three-dimensional computational mesh G1), a numerical value (e.g., physical quantity) indicating the state of that point exists, and as shown in the four-dimensional computational mesh G2, the numerical value of each point changes in one dimension, time. This change in time at each point in three-dimensional space is indicated by grid data. In this case, the grid data can be represented as a four-dimensional data array (numerical array) indicating the numerical values ​​of the physical quantities, etc. The number of elements in the data array is referred to as the number of elements. Furthermore, a data array can be provided for each physical quantity. Furthermore, the grid data can also be referred to as structured data. Furthermore, when grid data is handled in a neural network, as described below, the grid data may indicate physical quantities that humans can understand, or may indicate numerical values ​​that humans cannot understand. In other words, the grid data has a structure indicating numerical values ​​defined on grid points arranged at predetermined intervals in space-time.

[0016] On the other hand, if a micrometeorological forecast simulation is performed with a higher calculation resolution, the amount of data to be handled increases, resulting in a significant calculation cost. Therefore, it is conceivable to obtain high-resolution (HR) calculation results by performing super-resolution (SR) on low-resolution (LR) simulation results.

[0017] FIG. 3 is a diagram illustrating a super-resolution simulation method. The super-resolution simulation system uses a super-resolution device to perform super-resolution on low-resolution prediction results obtained by performing a low-resolution simulation. This results in high-resolution prediction results. The super-resolution device learns through deep learning (neural network) using high-resolution results obtained in advance from a high-resolution simulation. In other words, the super-resolution device learns by performing supervised learning using a large amount of high-resolution results as training data in advance. With this configuration, high-resolution prediction results can be obtained by performing a low-resolution simulation during operation, thereby reducing computational costs.

[0018] Furthermore, to improve the accuracy of forecast simulations, it is possible to perform data assimilation between observation data and forecast results. The ensemble Kalman filter, one data assimilation method, simulates a large number of similar situations using ensemble calculations and estimates the error from the variance in the forecast results. Then, based on the magnitude of this error, the degree to which the forecast results should approximate the observed data is determined. By performing data assimilation, the forecast results can be brought closer to the observed data, thereby improving the accuracy of the forecast results. Here, as mentioned above, the technology in Non-Patent Document 1 simply combines a super-resolution technique and a data assimilation technique. In other words, Non-Patent Document 1 performs super-resolution and data assimilation independently. Such a method requires data assimilation using ensemble calculations, which increases computational costs.

[0019] In contrast, in this embodiment, as will be described below, super-resolution and data assimilation are performed simultaneously using time-series data of grid data. This eliminates the need for ensemble calculations. Therefore, this embodiment makes it possible to perform accurate and efficient predictions.

[0020] 4 is a diagram showing the configuration of an information processing device 10 according to this embodiment. The information processing device 10 is, for example, a computer. The information processing device 10 includes a simulation unit 20, an observation data acquisition unit 22, a prediction data acquisition unit 24, a structure transformation unit 30, a latent space-time mapping unit 40, a nonlinear transformation unit 50, a high-resolution analysis data acquisition unit 60, and a low-resolution analysis data calculation unit 70. These components can be realized by a hardware configuration described below. The functions of these components will be described later.

[0021] 5 is a flowchart showing an information processing method executed by the information processing device 10 according to this embodiment. The simulation unit 20 simulates the state of the environment (step S20). Specifically, the simulation unit 20 performs the low-resolution simulation described above. More specifically, the simulation unit 20 performs the low-resolution simulation in the time direction and the space direction.

[0022] The observation data acquisition unit 22 acquires one or more types of observation data (step S22). The observation data is data obtained by observing a state in space and time. The observation data can be acquired, for example, from sensors and cameras placed in the environment, drones, smartphones, etc. Here, the structure of the observation data does not need to be a lattice data structure. Furthermore, if the structure of the observation data is a lattice data structure, its resolution is arbitrary and may be low resolution or high resolution. Details of the observation data will be described later.

[0023] The prediction data acquisition unit 24 acquires prediction data (step S24). Specifically, the prediction data acquisition unit 24 acquires prediction data, which is the result of simulation by the simulation unit 20. Here, the prediction data is time-series data indicating changes in state over time. The prediction data is also lattice data in space-time obtained by simulation. The prediction data is prediction data for a time (time series) that includes at least the time of the observation data and a time prior to that time. Here, the "time of the observation data" includes the latest time (reference time) in all the observation data. Details of the prediction data will be described later.

[0024] The structure conversion unit 30 converts the structure of the observation data (step S30). Specifically, the structure conversion unit 30 converts the structure of the observation data into observation data with a lattice data structure. More specifically, the structure conversion unit 30 converts the structure of the observation data, which is data obtained by observing a state in space-time, into observation data with a lattice data structure that indicates numerical values ​​defined on lattice points arranged at predetermined intervals in space-time. The function of the structure conversion unit 30 will be described in detail later.

[0025] The latent space-time mapping unit 40 maps the observation data and the prediction data into latent space-time (step S40). Specifically, the latent space-time mapping unit 40 maps the observation data converted into lattice data in the process of S30 and the prediction data obtained in the process of S24 from the first real space-time into latent space-time. As will be described later, the latent space-time mapping unit 40 (processing of S40) performs data assimilation between the observation data and the prediction data.

[0026] Here, the latent space-time is a space-time with fewer elements than the first real space-time. In other words, the latent space-time is a space-time with lower resolution than the first real space-time. Therefore, the number of elements in the data array in the latent space-time is fewer than the number of elements in the data array in the first real space-time. Data in the latent space-time can be composed of a numerical array that compresses the temporal and spatial information of the observed data and the predicted data. Note that the latent space can also be considered a latent space that includes the concept of time (time series). Furthermore, dimensions are not distinguished in the latent space. In other words, the latent space does not distinguish between time and space, nor does it distinguish between the dimensions of three-dimensional space. Furthermore, the first real space-time is a space-time corresponding to the environment in which the predicted data is obtained.

[0027] Furthermore, by mapping to latent time space as described above, the latent space-time mapping unit 40 can obtain data in which observation data (observation data converted into lattice data) is mapped to latent time space, and data in which prediction data is mapped (projected) to latent time space. The latent space-time mapping unit 40 may use the observation data converted into lattice data to map prediction data to latent time space. Therefore, "data in which prediction data is mapped to latent time space" may also include data obtained by mapping observation data to latent time space. On the other hand, when mapping observation data, the latent space-time mapping unit 40 maps the observation data alone to latent time space.

[0028] Data assimilation between the observation data and the forecast data is performed by mapping the forecast data using the observation data converted to grid data. That is, the observation data converted to grid data and the forecast data can be mixed (fused) when they are mapped to latent space-time. In other words, in latent space-time, the observation data converted to grid data is incorporated into the forecast data. The function of the latent space-time mapping unit 40 will be described in more detail below.

[0029] The nonlinear transformation unit 50 performs a nonlinear transformation in the latent space-time (step S50). Specifically, the nonlinear transformation unit 50 performs a nonlinear transformation on the mapped observation data and predicted data in the latent space-time. More specifically, the nonlinear transformation unit 50 may fuse the observation data and predicted data mapped to the latent space-time to obtain data in the latent space (latent space-time data; fused data). Thus, the nonlinear transformation unit 50 (processing of S50) performs data assimilation between the observation data and the predicted data. The nonlinear transformation unit 50 may also repeat the nonlinear transformation to make the distribution of the values ​​of the latent space-time data mapped to the latent space-time space discontinuous. The nonlinear transformation unit 50 may also use the nonlinear transformation to make the distribution of the values ​​of the latent space-time data mapped to the latent space-time space more complex or simple. In this case, the nonlinear transformation unit 50 may perform the nonlinear transformation so as to appropriately perform super-resolution. Note that the nonlinear transformation unit 50 does not need to change the number of elements when changing the distribution of values. Note that super-resolution can be achieved by increasing the number of elements. The function of the nonlinear conversion section 50 will be described in more detail later.

[0030] The high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data (step S60). Specifically, the high-resolution analysis data acquisition unit 60 maps the observation data and prediction data that have been subjected to nonlinear transformation from the latent time space to a second real time space. In this way, the high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data. In other words, the high-resolution analysis data is analysis data that has been subjected to super-resolution in the time direction and the space direction.

[0031] Here, the second real-time space is a real-time space having a larger number of elements than the latent-time space and a higher resolution than the first real-time space. Therefore, the second real-time space can be considered a high-resolution space (HR space). Furthermore, the high-resolution analysis data (HR analysis data) is lattice data in space-time. Furthermore, the high-resolution analysis data is data with a higher resolution in space-time than the predicted data. Furthermore, the high-resolution analysis data may be time-series data for a time (time series) including the past and future of the time of the observation data. That is, when the input predicted data exists in the future relative to the time of the observation data (reference time), the high-resolution analysis data is analysis data including a similar future time. That is, the predicted data may be predicted data for a time including the time of the observation data and the past and future of that time. In this case, the high-resolution analysis data acquisition unit 60 may acquire high-resolution analysis data for a time including the past and future of the time of the observation data. A more detailed description of the functions of the high-resolution analysis data acquisition unit 60 will be given later.

[0032] The low-resolution analysis data calculation unit 70 calculates low-resolution analysis data (step S70). Specifically, the low-resolution analysis data calculation unit 70 calculates low-resolution analysis data using high-resolution analysis data. Here, the low-resolution analysis data (LR analysis data) is analysis data with lower resolution in time and space than the high-resolution analysis data. More specifically, the low-resolution analysis data calculation unit 70 calculates low-resolution analysis data by performing arithmetic operations (mathematical techniques) such as algebraic interpolation on the high-resolution analysis data. The functions of the low-resolution analysis data calculation unit 70 will be described in more detail below.

[0033] The simulation unit 20 receives the low-resolution analysis data and performs a simulation for the next timing (S20), thereby repeating the processing flow shown in FIG.

[0034] As described above, the information processing device 10 according to the present embodiment is configured to acquire high-resolution analysis data by performing data assimilation and super-resolution on observation data in space-time and low-resolution prediction data in space-time. The information processing device 10 is also configured to perform processing in latent space-time when acquiring the high-resolution analysis data. Therefore, it becomes possible to accurately and efficiently predict the state of the environment.

[0035] Furthermore, the information processing device 10 according to the present embodiment is configured to calculate low-resolution analysis data, which allows for continuous low-resolution simulation, thereby enabling efficient simulation.

[0036] Furthermore, the nonlinear transformation unit 50 may perform super-resolution in the time direction by transforming the data array of the data mapped to the latent time space. In this case, the high-resolution analysis data acquisition unit 60 may acquire high-resolution analysis data by performing super-resolution in the space direction independently for each time point in the time direction on the data that has been super-resolved in the time direction in the latent time space. This enables more efficient processing. Details will be described later.

[0037] Furthermore, the structure transformation unit 30, the latent space-time mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be realized by a trained model trained using a machine learning algorithm. In this case, the structure transformation unit 30, the latent space-time mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be realized by a trained model trained by supervised learning using data with a higher spatiotemporal resolution than the predicted data as training data. Alternatively, the structure transformation unit 30, the latent space-time mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 may be realized by a trained model trained by unsupervised learning so as to reduce the loss function. This enables efficient acquisition of high-resolution analysis data using non-lattice observation data and low-resolution predicted data. Details will be described later.

[0038] (Embodiment 1) Hereinafter, an embodiment will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, in each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0039] <Information Processing Device> Fig. 6 is a diagram showing the configuration of an information processing device 100 according to the first embodiment. The information processing device 100 is, for example, a computer. The information processing device 100 includes a learning processing unit 110, a simulation unit 120, an observation data acquisition unit 122, a prediction data acquisition unit 124, a structure transformation unit 130, a latent space-time mapping unit 140, a nonlinear transformation unit 150, a high-resolution analysis data acquisition unit 160, and a low-resolution analysis data calculation unit 170. These components can be realized by a hardware configuration described below.

[0040] The learning processing unit 110 performs machine learning such as a neural network on the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160. As a result, the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 can realize their respective functions as trained models.

[0041] Here, when performing machine learning, the learning processing unit 110 may learn the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 together, rather than separately. Specifically, the learning processing unit 110 uses an end-to-end deep learning technique to train the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a continuous manner. In other words, the learning processing unit 110 performs machine learning by regarding the functions of the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as layers of a neural network. In other words, the learning processing unit 110 performs machine learning by treating the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as a single neural network. Note that the pre-learning may be performed separately in each of the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160.

[0042] The learning processing unit 110 may perform supervised learning or unsupervised learning. When performing supervised learning, the learning processing unit 110 may perform learning using, for example, highly accurate and high-resolution time-series weather data as teacher data. Details of the processing by the learning processing unit 110 will be described later. Note that the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 are not limited to being realized as a trained model trained by machine learning by the learning processing unit 110.

[0043] The simulation unit 120 corresponds to the simulation unit 20 described above. The simulation unit 120 performs low-resolution simulation in the time and space directions. The simulation unit 120 generates low-resolution prediction data (simulation data) using the input initial state.

[0044] Low-resolution prediction data (LR prediction simulation data) is digital data defined on grid points in space-time. In other words, the prediction data is grid data defined in space-time. In other words, the prediction data has a data array defined by a grid structure. That is, the prediction data, which is grid data in space-time, has a data array for each grid point at a predetermined interval in the spatial and temporal directions. In the four-dimensional computational mesh G2 shown in FIG. 2 , grid points are arranged in the time axis direction, and in the three-dimensional computational mesh G1 of the four-dimensional computational mesh G2, grid points are arranged in the three spatial axes directions. The grid data of the prediction data has a numerical value at each of these grid points. The intervals between the grid points may be equal or unequal. Note that, because the simulation unit 120 performs a low-resolution simulation, the grid data of the obtained prediction data has longer intervals in the time and spatial directions on average compared to high-resolution grid data.

[0045] For example, the predicted data is a data array A in four-dimensional space-time. hijk The subscript h corresponds to the time direction (t direction), the subscript i corresponds to the X-axis direction of the three-dimensional space, the subscript j corresponds to the Y-axis direction of the three-dimensional space, and the subscript k corresponds to the Z-axis direction of the three-dimensional space. hijk The subscripts h, i, j, k are integer values. One of the sets of integer values ​​(h, i, j, k) corresponds to one of the grid points of the four-dimensional calculation mesh G2. hijk In this way, the predicted data is stored in a data array A with integer subscripts in each dimension in the four-dimensional space. hijk is defined as:

[0046] Here, data array A hijk In the data array A, it is assumed that there are H elements in the time direction. In other words, the subscript h takes H values. Similarly, it is assumed that there are I elements in the X-axis direction, that is, the subscript i takes I values. It is also assumed that there are J elements in the Y-axis direction, that is, the subscript j takes J values. It is also assumed that there are K elements in the Z-axis direction, that is, the subscript k takes K values. In this case, the data array A hijkThe number of elements is H*I*J*K. If H=I=J=K=10, then the data array A hijk The number of elements is 10 4 This becomes:

[0047] The predicted data includes data for past times in the time series. The predicted data may also include data for future times in the time series. As described above, the predicted data is time data that includes the time (reference time) of the observation data described below.

[0048] The forecast data includes data indicating physical variables (physical quantities) necessary to predict the state of an environment such as the atmosphere based on physical equations. In the case of a weather forecast simulation, the physical equations are, for example, the Navier-Stokes equations of fluids or thermodynamic equations. The physical quantities are, for example, air velocity, pressure, temperature, water vapor mixing ratio, and cloud particle number density. The four-dimensional data array described above can be provided for each of these physical quantities. The data array can also be said to represent the "field" of each physical quantity in physics.

[0049] It should also be noted that in this embodiment, the predicted data (LR simulation results) are obtained from a single scenario. In other words, the predicted data is a single (single) simulation result obtained when a simulation (physical simulation) is performed from a unique initial state. In contrast, the data assimilation method disclosed in Non-Patent Document 1 performs simulations for various similar situations in order to perform ensemble calculations. In other words, Non-Patent Document 1 performs predictive simulations using multiple scenarios.

[0050] The predicted data acquisition unit 124 corresponds to the above-described predicted data acquisition unit 24. The predicted data acquisition unit 124 acquires the above-described predicted data (predicted results of the low-resolution physical simulation) from the simulation unit 120. The predicted data is expressed by the following equation (1). ...(1)

[0051] Here, x t Lis a vector field consisting of all physical quantities of the predicted data. That is, x is a vector field that indicates a set of values ​​(vectors) at each grid point in three-dimensional space at a certain time t. Equation (1) indicates a set of vector fields of all physical quantities from time t = 0 to time t = n. In other words, Equation (1) indicates all data arrays (numerical arrays) in four-dimensional space-time related to the predicted data. The subscript L indicates low resolution in the spatial direction. t indicates a timestamp with a long time interval. In other words, t indicates low resolution in the time direction (i.e., a long time interval). In other words, Equation (1) represents data with a small number of grids in four-dimensional space-time (a small number of elements; low resolution).

[0052] The observation data acquisition unit 122 corresponds to the above-described observation data acquisition unit 22. Similar to the observation data acquisition unit 22, the observation data acquisition unit 122 acquires observation data from sensors and cameras, drones, smartphones, etc. arranged in the environment. The observation data is expressed by the following equation (2). ...(2)

[0053] Equation (2) represents a set of digital data consisting of a numerical value (observation value) o that indicates a certain state at time τ. Here, τ represents a timestamp of the observation data. The time intervals of τ do not have to be equal. Furthermore, o may or may not represent a physical quantity. In other words, o may represent a numerical value of a non-physical quantity.

[0054] Note that the observation data may be grid data. Alternatively, the observation data does not need to be grid data. In other words, the observation data may be unstructured data (non-grid data). In unstructured observation data, observation values ​​are associated with time information and spatial information, but the intervals in time and space have no regularity. In other words, unstructured observation data does not have a spatial mesh structure. Therefore, the observation data may represent observation values ​​at random times and spaces.

[0055] The observation data may also include a variety of miscellaneous data of different qualities. The observation data may be image data, sound data, point data, or log data. The observation data may also be a collection of values ​​of physical quantities representing atmospheric conditions, or digital data from which these physical quantities can be estimated. For example, the observation data may represent observation values ​​from an AMeDAS (Automated Meteorological Agency) system. In this case, the observation values ​​may represent temperature, humidity, wind speed, etc. at various locations. The observation data may also represent the radiance of an object (e.g., a building). This allows for estimation of the temperature near the object at a location and at the time of observation. The observation data may also represent an acceleration log of an airborne aircraft such as a drone. This allows for estimation of the wind speed or turbulence dissipation rate at the location of the aircraft and at the time of observation. The observation data may also represent images of the sky. This allows for estimation of cloud cover or precipitation at the location and time of the image. The observation data may also represent sales of frozen desserts (e.g., ice cream, sorbet, popsicles, shaved ice, etc.) at a certain location (e.g., a convenience store). This allows for estimation of the local temperature in the area. That is, it can be estimated that the higher the sales of frozen desserts, the higher the temperature in that area.

[0056] Furthermore, when a simulation is performed with sufficient accuracy, the simulation results can be said to adequately represent the actual state. Therefore, the results of a simulation with sufficient accuracy can be considered to be observation results that accurately represent reality and can be regarded as observation data. Depending on the settings, it is possible to perform "realistic experiments" that incorporate the shape of a building and even the exhaust heat emitted from the building, such as in a micrometeorological simulation. Such simulations have sufficiently small errors, so they can be considered to be observation results that accurately represent reality, i.e., observation data.

[0057] Furthermore, the spatial and temporal resolution at which the observation data is observed may be high resolution or low resolution. When the observation data represents satellite data or radar data, it is obtained by line (one-dimensional) observation or surface (two-dimensional) observation. When the observation data is data obtained by a Doppler lidar (LiDAR: Light Detection And Ranging), it is obtained by three-dimensional observation. In these cases, whether the resolution of the observation data is low resolution or high resolution can be determined depending on the spatial resolution (and the time interval of the observation).

[0058] Furthermore, when observation data corresponds to AMeDAS data, it is obtained through point (zero-dimensional) observation. Even in this case, the resolution can be defined as follows. That is, even for point observations, there is a representative scale that indicates the scale at which the observation value represents. Observation data with a large representative scale can be considered low-resolution observation data. Observation data with a small representative scale can be considered high-resolution observation data. For example, typical meteorological models obtain observation values ​​with a coarse spatial resolution, representing a horizontal range of 1 km to 10 km. Therefore, strict conditions are imposed on observations, such as being in a location with no nearby obstructions, on grass, not exposed to direct sunlight, and not being affected by anthropogenic heat. Such observation data can be considered low-resolution observation data. On the other hand, micrometeorological models can obtain observation values ​​with a fine spatial resolution, representing a horizontal range of 1 m to 5 m, which is affected by anthropogenic heat. Micrometeorological models can perform simulations taking into account the impact of such observation data on the atmosphere. Such observation data can be considered high-resolution observation data.

[0059] Note that high-resolution observation data and low-resolution prediction data differ significantly in spatial and temporal resolution. Therefore, when performing data assimilation, it is necessary to average the high-resolution observation data in the spatial and temporal directions to match the low-resolution prediction data. Therefore, it should be noted that it is usually difficult to directly perform data assimilation between high-resolution observation data and low-resolution prediction data. In contrast, in this embodiment, data assimilation can be performed between any observation data and low-resolution prediction data.

[0060] The structure converter 130 corresponds to the structure converter 30 described above. The structure converter 130 functions as a structurizer. The structure converter 130 may be implemented by an existing structurizer. The structure converter 130 converts observation data, which is unstructured data (non-lattice data), into observation data with a lattice data structure. In other words, the structure converter 130 converts the observation data into grid data. The structure converter 130 converts the observation data into data on each lattice point of lattice data in high-resolution space-time. In other words, the structure converter 130 converts the observation values ​​into physical quantities defined on a lattice. Note that the lattice data obtained by converting the observation data may correspond to lattice data with a larger number of elements (i.e., higher resolution) than the lattice data of the prediction data.

[0061] If the function representing the structurizer realized by the structure conversion unit 130 is s(), the function of the structure conversion unit 130 is expressed by the following equation (3). The left side of equation (3) corresponds to the output data of the structure conversion unit 130. Note that the observation data input to the structure conversion unit 130 (structurizer) may be lattice data. Furthermore, the observation data input to the structurizer may indicate a physical quantity or a numerical value other than a physical quantity. ...(3)

[0062] Here, o T His a vector field consisting of all observed values ​​of the observation data. That is, o is a vector field indicating a set of values ​​(vectors) at each grid point in three-dimensional space at a certain time T. The left side of equation (3) indicates a set of vector fields of all observed values ​​(physical quantities, etc.) from time T = 0 to time T = N. In other words, the left side of equation (3) indicates all data arrays (numerical arrays) in four-dimensional space-time corresponding to the observation data (physical quantities, etc.). The subscript H indicates high resolution in the spatial direction. T indicates a timestamp of the observation data. T indicates a timestamp with a short time interval. In other words, T indicates high resolution in the time direction (i.e., a short time interval). In other words, the left side of equation (3) indicates data with a large number of grids in four-dimensional space-time (a large number of elements; high resolution).

[0063] The structurizer of the structure conversion unit 130 may be realized by a trained model trained by machine learning, such as a neural network. In this case, the structure conversion unit 130 may be realized by a linear projection operator, a fully connected layer, or a graph convolutional network (the same applies to the latent space-time mapping unit 140, nonlinear transformation unit 150, and high-resolution analysis data acquisition unit 160, which will be described later). In this case, the structure conversion unit 130 is trained by the learning processing unit 110. For example, the structurizer may be realized by the technology disclosed in Non-Patent Document 2. As described above, the structure conversion unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 are trained in an integrated manner using an end-to-end deep learning technique. The structure conversion unit 130 (structurizer) is then trained to convert the observation data into grid data appropriate for performing the super-resolution data assimilation according to this embodiment.

[0064] Furthermore, the structurizer does not have to be realized by a trained model trained by machine learning. In this case, the structure conversion unit 130 may read time and position information of the observation data and project the observation data onto a lattice point that is closest to the read time and position information among lattice points arranged at a predetermined interval in space-time. In this case, the structure conversion unit 130 may also substitute missing values ​​for lattice points onto which observation data is not projected.

[0065] Even if the observation data input to the structure conversion unit 130 indicates non-physical quantities, the structure conversion unit 130 may output grid data indicating the numerical values ​​of physical quantities, etc. corresponding to the observation data. For example, if the observation data input to the structure conversion unit 130 indicates the sales of frozen desserts at each location at each time, the structure conversion unit 130 may output grid data indicating the temperature at each location at each time. The structure conversion unit 130 may also receive a variety of observation data. In this case, the structure conversion unit 130 may output grid data indicating the numerical values ​​of physical quantities, etc. corresponding to the plurality of observation data. For example, if the observation data input to the structure conversion unit 130 indicates the acceleration of a drone and the sales of frozen desserts at each location at each time, the structure conversion unit 130 may output grid data indicating the wind speed and the temperature at each location at each time. The structure conversion unit 130 is not limited to outputting grid data indicating human-understandable physical quantities corresponding to the input observation data. The structure conversion unit 130 may also output grid data corresponding to a numerical array that is incomprehensible to humans (i.e., that can only be understood by a neural network). In other words, the structure conversion unit 130 outputs lattice data (numerical array) indicating numerical values ​​defined on lattice points arranged at predetermined intervals in space-time. Furthermore, when the observation data is image data (camera data), the structure conversion unit 130 (or another component) may perform preprocessing using object recognition or pixel segmentation. This preprocessing may utilize a pre-trained neural network or the like. The structure conversion unit 130 may then perform the above-described structural conversion process on the data that has undergone such preprocessing.

[0066] As described above, the structure conversion unit 130 receives as input observation data that is non-grid data or grid data and indicates observed values ​​of physical or non-physical quantities. The structure conversion unit 130 then outputs grid data of a four-dimensional space-time data array (numerical array) related to physical quantities (temperature, wind speed, etc.) that indicate the state of the environment (e.g., atmospheric conditions). With this configuration, the structure conversion unit 130 can nonlinearly convert observation data of non-physical quantities or non-grid data that cannot be handled by existing data assimilation methods, and convert it into observation data of a numerical array that can be assimilated.

[0067] The latent space-time mapping unit 140 corresponds to the above-described latent space-time mapping unit 40. The latent space-time mapping unit 140 functions as an encoder. The latent space-time mapping unit 140 may be realized by an existing encoder. The latent space-time mapping unit 140 maps the observed data and predicted data in the first real space-time to latent space-time. For each time, the latent space-time mapping unit 140 converts the observed data and predicted data structured in four-dimensional space-time into data in latent space-time. In other words, in the latent space-time mapping unit 140, an output for each time is calculated independently for an input for each time.

[0068] The function representing the encoder realized by the latent space-time mapping unit 140 is e x (), e o Then, the function of the latent space-time mapping unit 140 is expressed by the following equations (4) and (5). x () is a function for mapping the prediction data x to the latent space-time. o ( ) is a function for mapping the observed data o to the latent space-time. As a result, at each time t, the observed data o and the predicted data x are each mapped to the latent space-time. ...(4) ...(5)

[0069] As a result, the latent space-time mapping unit 140 obtains data of the latent space-time expressed by the following equation (6): Equation (6) represents a set of data of the latent space-time at each time from t=0 to t=n. ...(6)

[0070] Here, equation (4) is the observation data o converted into grid data. t Using the prediction data x t L By mapping the data p t This shows that the mapping data p t indicates a numerical array in the latent space-time corresponding to the low-resolution prediction data. Here, the latent space-time mapping unit 140 may perform pre-processing and main processing on the observation data. For example, as pre-processing, the latent space-time mapping unit 140 may perform the following: T H The observation data o t It may be converted to t is the observation data o converted into grid data T The time or space step of the low-resolution prediction data is coarsened to fit the space-time grid structure of the low-resolution prediction data. t By simultaneous data assimilation, which reflects the low-resolution predicted data, the mapping data p t may be obtained.

[0071] Furthermore, equation (5) is the observation data o t By mapping the latent space-time, the mapping data q t This shows that the mapping data q t denotes a numerical array in latent space-time corresponding to the observed data. When mapping the observed data, the latent space-time mapping unit 40 maps the observed data alone into latent space-time.

[0072] As can be seen from the subscript t in equations (4) to (6), in the processing of the latent space-time mapping unit 140, the resolution in the time direction (time step t) inside the latent space-time mapping unit 140 is matched to the resolution of the predicted data. Meanwhile, the resolution in the space direction is smaller than the resolution of the predicted data. In other words, as described above, the number of elements in the numerical array of the mapped data in latent space-time is smaller than the number of elements in the numerical array of the predicted data. Therefore, the latent space-time mapping unit 140 reduces the number of elements in the numerical arrays of the observed data and the predicted data (dimensionality compression in topological space).

[0073] Furthermore, equations (4) to (6) indicate that the observed data and the predicted data are fused in latent space-time. In other words, equation (6) indicates data (fused data) in which the observed data and the predicted data are fused in latent space-time. Therefore, it can be said that the latent space-time mapping unit 140 performs data assimilation on the observed data and the predicted data.

[0074] The latent space-time mapper 140 (encoder) may be implemented by a trained model trained by machine learning, such as a neural network. For example, the latent space-time mapper 140 may use convolution and pooling to perform nonlinear transformation while reducing the number of elements. The neural network may also be a neural network that reflects physical symmetry. A convolutional neural network may be employed to reflect spatial translational symmetry. A group convolutional neural network may be employed to reflect spatial rotational symmetry. A vision transformer or a graph convolutional neural network may be employed to reflect relabeling symmetry. This enables transformation that takes physical symmetry into account, unlike existing data assimilation methods. The same applies to the decoder of the high-resolution analysis data acquisition unit 160, which will be described later.

[0075] As described above, the latent space-time mapping unit 140 receives as input low-resolution predicted data, which is a set of data arrays in four-dimensional space-time, and observed data converted into lattice data. The latent space-time mapping unit 140 then outputs mapped data in latent space-time with a small number of elements. This configuration of the latent space-time mapping unit 140 can improve the processing efficiency of the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160. In other words, the number of elements in the numerical array is reduced, thereby reducing the amount of data to be processed. This reduces the amount of computational resources required.

[0076] The nonlinear transformation unit 150 corresponds to the nonlinear transformation unit 50 described above. The nonlinear transformation unit 150 functions as a time series transformer. The nonlinear transformation unit 150 may be implemented by an existing time series transformer. The nonlinear transformation unit 150 performs a nonlinear transformation on the mapped observation data and predicted data in the latent time space. More specifically, the nonlinear transformation unit 150 may perform a nonlinear transformation on the time series to generate data in which the observation data and predicted data mapped to the latent time space are fused. The nonlinear transformation unit 150 according to the first embodiment may then perform super-resolution on the fused data in the time direction. Here, as described above, the number of elements in the latent time space is reduced compared to the real time space. Therefore, the amount of data to be processed is reduced, allowing for efficient super-resolution in the time direction.

[0077] If the function representing the time series converter realized by the nonlinear conversion unit 150 is F(), the function of the nonlinear conversion unit 150 is expressed by the following equation (7). The left side of equation (7) corresponds to the output data of the nonlinear conversion unit 150. As mentioned above, the subscript T indicates high resolution in the time direction. Also, r T represents data (fused data) obtained by fusing the observation data and predicted data mapped to the latent space-time at time T through data assimilation. The left side of equation (7) represents the set of fused data from time T = 0 to time T = N. In other words, the left side of equation (7) represents a data array (numerical array) in four-dimensional space-time corresponding to the fused data. ... (7)

[0078] As described above, Equation (7) indicates that the observation data and the prediction data are fused in the latent space-time. In other words, the left side of Equation (7) indicates data (fused data) in which the observation data and the prediction data are fused in the latent space-time. Therefore, it can be said that the nonlinear transformation unit 150 performs data assimilation on the observation data and the prediction data.

[0079] The time series transformer of the nonlinear transformer 150 may be realized by a trained model trained by machine learning, such as a neural network. In this case, the nonlinear transformer 150 may be realized by a neural network that uses an attention mechanism. Furthermore, the time series transformer of the nonlinear transformer 150 may perform nonlinear transformation using, for example, a technology called a Transformer described in Non-Patent Document 3.

[0080] Alternatively, the nonlinear transformation unit 150 may implicitly calculate the prediction error using information in the spatiotemporal direction. That is, the nonlinear transformation unit 150 may calculate the error within a neural network from the temporal change of the spatial pattern. In other words, the nonlinear transformation unit 150 may calculate the prediction error by matching spatiotemporal patterns and determining the magnitude of the error relative to the ground truth data. The nonlinear transformation unit 150 may be realized by a neural network that performs data assimilation between the predicted data and the observed data using the prediction error as a weight. This allows data assimilation between the predicted data and the observed data through efficient calculations that do not require ensembles.

[0081] Furthermore, the time series transformer of the nonlinear transformation unit 150 may repeatedly execute a linear transformation (e.g., an affine transformation) and a nonlinear transformation such as ReLU (Rectified Linear Unit). In the process of executing this process, the nonlinear transformation unit 150 may transform the data array to make the time step finer, for example, using the technique of Non-Patent Document 4. The nonlinear transformation unit 150 may divide multiple elements in the latent space-time into elements in the time direction and elements in the space direction, thereby increasing the number of elements in the time direction. For example, assume that there are H' elements in the time direction, I' elements in the X-axis direction, J' elements in the Y-axis direction, and K' elements in the Z-axis direction. In this case, the nonlinear transformation unit 150 transforms a four-dimensional array H'×I'×J'×K' into a two-dimensional array H'×M, where M is the number of elements in the space direction, and M = I'×J'×K'. The nonlinear transformation unit 150 may then transform this array into a 2H'×M / 2 array. This doubles the number of elements in the time direction, and halves the time step. Therefore, super-resolution is performed in the time direction on the data mapped to the latent space-time. In this way, the nonlinear transformation unit 150 may perform super-resolution in the time direction by transforming the data array of the data mapped to the latent space-time.

[0082] As described above, the nonlinear transformation unit 150 receives as input a set of numerical arrays (data arrays) corresponding to observed data and predicted data in the latent time space. The nonlinear transformation unit 150 may then output time series data in the latent time space that has been super-resolved in the time direction. The time resolution of the time series data in the latent time space that has been super-resolved in the time direction is higher than the time resolution of the predicted data. This allows the nonlinear transformation unit 150 to perform super-resolution in the time direction in a latent time space with a small number of elements (i.e., low-dimensional in the topological space), thereby reducing the amount of calculation and, therefore, the processing cost. This allows for increased computational efficiency.

[0083] Furthermore, super-resolution in the time direction is performed by the time series transformer of the nonlinear transformation unit 150. As described above, since the number of elements in the numerical array is small in the latent time space, the required memory amount is reduced, and long time series can be handled. This makes it possible to efficiently perform super-resolution in the time direction. Furthermore, the processing of the nonlinear transformation unit 150 makes it possible to perform data assimilation between the observed data and the predicted data while performing super-resolution in the time direction. Therefore, it is possible to efficiently perform data assimilation and super-resolution in the time direction.

[0084] The high-resolution analysis data acquisition unit 160 corresponds to the above-described high-resolution analysis data acquisition unit 60. The high-resolution analysis data acquisition unit 160 functions as a decoder. The high-resolution analysis data acquisition unit 160 may be realized by an existing decoder. The high-resolution analysis data acquisition unit 160 maps the observation data and prediction data that have been subjected to nonlinear transformation from the latent time space to a second real time space. The high-resolution analysis data acquisition unit 160 maps, for each time, the data obtained by the processing of the nonlinear transformation unit 150 from the latent time space to the second time space. In this way, the high-resolution analysis data acquisition unit 160 performs super-resolution in the spatial direction on the data obtained by the processing of the nonlinear transformation unit 150. In this way, the high-resolution analysis data acquisition unit 160 converts the data in the latent time space into high-resolution analysis data for each time. In other words, the high-resolution analysis data acquisition unit 160 independently calculates the output for each time for the input for each time.

[0085] In this way, the high-resolution analysis data acquisition unit 160 acquires the high-resolution analysis data Da1. As described above, the high-resolution analysis data Da1 is data with a higher spatial and temporal resolution than the predicted data. Furthermore, the high-resolution analysis data Da1 may be analysis data in a time series that includes the past and future of the time of the observation data, depending on the time range of the input predicted data. Therefore, the high-resolution analysis data Da1 can be analysis data that has been extrapolated in the time direction (time extrapolated) with respect to the time of the observation data (reference time).

[0086] If the function representing the decoder realized by the high-resolution analysis data acquisition unit 160 is d(), the function of the high-resolution analysis data acquisition unit 160 is expressed by the following equation (8): d() is the fusion data r T is a function for mapping from the latent space-time to the second real space-time. As a result, at each time T, fusion data obtained by fusing observation data and prediction data by data assimilation is mapped to the second real space-time with high resolution. As a result, super-resolution in the spatial direction is performed on the fusion data that has been super-resolved in the temporal direction by the nonlinear transformation unit 150. ...(8)

[0087] As a result, the high-resolution analysis data acquisition unit 160 obtains data of the second real-time space expressed by the following equation (9). Equation (9) shows a set of high-resolution analysis data y at each time from time T=0 to T=N. Equation (9) corresponds to the high-resolution analysis data Da1, which is output data from the high-resolution analysis data acquisition unit 160. The high-resolution analysis data shown in equation (9) is time-series data on a high-resolution grid. The high-resolution analysis data Da1 may include past, present, and future data. ... (9)

[0088] Here, y T H is a vector field consisting of all physical quantities of the high-resolution analysis data. That is, y is a vector field indicating a set of values ​​(vectors) at each grid point in three-dimensional space at a certain time T. Equation (9) indicates a set of vector fields of all physical quantities from time T=0 to time T=N. In other words, Equation (9) indicates all data arrays (numerical arrays) in four-dimensional space-time related to the high-resolution analysis data. The subscript H indicates high resolution in the spatial direction. T indicates a timestamp with a short time interval. In other words, T indicates high resolution in the time direction (i.e., a short time interval). In other words, Equation (9) indicates data with a large number of grids in four-dimensional space-time (a large number of elements; high resolution).

[0089] The decoder of the high-resolution analysis data acquisition unit 160 may be realized by a trained model trained by machine learning, such as a neural network. For example, the neural network related to the high-resolution analysis data acquisition unit 160 may be a neural network that reflects physical symmetry. This enables conversion that takes physical symmetry into account, unlike existing data assimilation methods. Furthermore, the high-resolution analysis data acquisition unit 160 may generate high-resolution analysis data that has been extrapolated (time-extrapolated) in the time direction relative to the time of the observation data (reference time) using a neural network. In other words, the neural network of the high-resolution analysis data acquisition unit 160 may be trained to output high-resolution analysis data that has been time-extrapolated relative to the reference time using time-series data with higher spatiotemporal resolution and accuracy than the predicted data as training data.

[0090] Furthermore, the decoder of the high-resolution analysis data acquisition unit 160 may repeatedly perform linear transformation and nonlinear transformation (e.g., ReLU) to increase the spatial resolution. In the process of performing this process, the high-resolution analysis data acquisition unit 160 may transform the data array (numerical array) to increase the spatial resolution, for example, using a technique called Pixel Shuffle, as described in Non-Patent Document 5. In this case, for example, assume that the fusion data is a numerical array in which the number of elements in the time direction is n and the number of elements in the spatial direction is m. In this case, the high-resolution analysis data acquisition unit 160 may transform an n × m array into an n / 2 × 2 m array. This doubles the number of elements in the spatial direction (m → 2 m), thereby doubling the spatial resolution.

[0091] As described above, the high-resolution analysis data acquisition unit 160 receives as input a numerical array (data array) that is time-series data in latent space-time corresponding to the fusion data. The high-resolution analysis data acquisition unit 160 then outputs a four-dimensional numerical array that is time-series data on a high-resolution grid in the second real-time space with a large number of elements. With this configuration, the high-resolution analysis data acquisition unit 160 can efficiently perform super-resolution in the spatial direction in real-time space on the fusion data that has been subjected to data assimilation and super-resolution in the temporal direction.

[0092] That is, the high-resolution analysis data acquisition unit 160 independently processes the numerical array (fused data) in the latent space-time at each time. This allows the processing in the high-resolution analysis data acquisition unit 160 to perform super-resolution in the spatial direction without referencing information in the temporal direction. Therefore, the processing in the high-resolution analysis data acquisition unit 160 saves the necessary computational resources (memory amount, etc.), enabling efficient super-resolution in three-dimensional space. This makes it possible to efficiently obtain high-resolution analysis data that has been super-resolved in the temporal and spatial directions. Furthermore, when the predicted data is predicted data for the time of observation data and a time period that includes both the past and future of the observation data, the high-resolution analysis data acquisition unit 160 can acquire high-resolution analysis data for a time period that includes both the past and future of the observation data. This allows the provision of high-resolution analysis data and future prediction information that has undergone time extrapolation at the same time. This makes it possible to provide services with higher added value.

[0093] Furthermore, obtaining high-resolution analysis data with super-resolution in both the temporal and spatial directions can compensate for low-resolution predictive simulations as follows. In practice, the resolution of predictive simulations for weather and oceanographic forecasts is often insufficient in both the temporal and spatial directions. For example, in aquaculture, rising water temperature increases fish activity, and overfeeding fish can result in death due to lack of oxygen. Therefore, aquaculture operators often need to know the timing of water temperature rises in order to adjust the amount of feed they need. However, current ocean forecasts have a relatively low horizontal resolution of about 10 km. A predictive simulation with this level of resolution may not accurately predict the water temperature near the aquaculture operator's fish pens. To accurately predict the water temperature in each aquaculture operator's fish pens, a fine resolution (high resolution) of at least 1 km, preferably about 100 m, is required. In contrast, in this embodiment, high-resolution analysis data with super-resolution in both the time and space directions can be obtained, making it possible to provide accurate information to businesses that want pinpoint weather and sea condition forecasts.

[0094] The low-resolution analysis data calculation unit 170 corresponds to the above-mentioned low-resolution analysis data calculation unit 70. The low-resolution analysis data calculation unit 170 calculates low-resolution analysis data Da2 using the high-resolution analysis data Da1. The resolution of the low-resolution analysis data Da2 may correspond to the resolution of the predicted data. The low-resolution analysis data Da2 is lattice data having a data array with a lower resolution than the high-resolution analysis data Da1. The low-resolution analysis data Da2 may also be snapshot data at a certain time.

[0095] Specifically, the low-resolution analysis data calculation unit 170 uses a predetermined function f(y) and inputs the high-resolution analysis data y described above into the function f to calculate the low-resolution analysis data Da2. The function f may be, for example, a function representing algebraic interpolation. The function f may also be a function that performs an algebraic interpolation operation such as linear interpolation. The function f may also be a function defined by linear interpolation, bicubic interpolation, or Lanczos interpolation. For example, the low-resolution analysis data calculation unit 170 can calculate the low-resolution analysis data Da2 from the high-resolution analysis data Da1 by resizing (reducing the resolution) the high-resolution analysis data. Specifically, the low-resolution analysis data calculation unit 170 locally interpolates values ​​corresponding to each lattice point of the lattice data using a polynomial to enlarge or reduce the lattice data. This calculates the low-resolution analysis data Da2.

[0096] As described above, the low-resolution analysis data Da2 is input to the simulation unit 120 and used to perform a predictive simulation of the next timing. The low-resolution analysis data Da2 may also be used to numerically solve physical equations to obtain future predictions. This allows the time evolution of the state of the predictive simulation to be performed. In this case, the low-resolution analysis data Da2 may represent the state at the current time.

[0097] As described above, the information processing device 100 according to the first embodiment is configured to perform data assimilation and super-resolution in a latent space-time with a reduced number of elements. This allows processing to be performed with a reduced number of elements, i.e., the amount of data, handled in the calculations, thereby enabling efficient data assimilation and super-resolution. Therefore, observation data indicating non-physical quantities and observation data that is irregular in the time and space directions can be efficiently assimilated into prediction data. Furthermore, as described above, when the learning processing unit 110 uses an end-to-end learning technique to train the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a seamless manner, a large amount of computational resources is required. Therefore, by performing processing in latent space-time as described above, the amount of data to be processed can be reduced, enabling such learning to be performed efficiently.

[0098] Furthermore, the information processing device 100 according to the first embodiment is configured so that the nonlinear transformation unit 150 performs super-resolution in the time direction and the high-resolution analysis data acquisition unit 160 performs super-resolution in the space direction. This enables efficient super-resolution in the time direction and the space direction. That is, simultaneous super-resolution in both the time direction and the space direction requires a large amount of memory and computation time. Particularly during the learning stage, backpropagation of errors requires storage of a computation graph and gradient values. This requires a huge amount of memory and computation time. In contrast, in the information processing device 100 according to the first embodiment, the nonlinear transformation unit 150 efficiently references the time dimension to perform super-resolution in the time direction in a latent time space with a small number of elements. Meanwhile, the high-resolution analysis data acquisition unit 160 processes snapshots at each time point, eliminating the need to consider the time dimension. Furthermore, because snapshots at each time point are processed, these snapshots can be processed simultaneously. This reduces the computational resources required.

[0099] <Comparison with Comparative Example> Next, the comparative example according to Non-Patent Document 1 will be compared with the technology according to the first embodiment. As described above, the technology of Non-Patent Document 1 simply combines a super-resolution technique and a data assimilation technique. In other words, in Non-Patent Document 1, super-resolution and data assimilation are performed independently.

[0100] FIG. 7 is a diagram illustrating a technology according to a comparative example. FIG. 7 illustrates a technology according to Non-Patent Document 1 as a comparative example. In the comparative example, as indicated by the open circle dots, physical simulations are performed for various situations using ensemble calculations. Then, as indicated by the filled circle dots, observation data is assumed to be available at time t1. In this case, as indicated by the open triangular dots, super-resolution is performed using the results of the physical simulations at time t1, and high-resolution predictions are made. This super-resolution is performed independently at time t1 for each of the ensemble calculation results, i.e., each situation. Then, at time t1, data assimilation is performed on the high-resolution ensemble prediction results and the observation data, and high-resolution prediction data into which the observation data has been assimilated is obtained as the final output, as indicated by the filled triangular dots. In this way, the comparative example utilizes ensemble calculations. Furthermore, in the comparative example, super-resolution at a given moment is performed independently of data assimilation at the same moment. Then, data assimilation is performed in high-resolution space.

[0101] FIG. 8 is a diagram illustrating super-resolution and data assimilation according to the first embodiment. In the first embodiment, a single scenario is used. That is, a physical simulation is performed from a unique initial state, and a unique simulation result is obtained in a time series, as indicated by the open dots. In FIG. 8 , physical simulation results (predicted data) are obtained as time-series data corresponding to times t1, t2, t3, and t4. Instead of performing super-resolution at a specific time, observation data indicated by the closed dots and predicted data, which is time-series data, are used as input, and super-resolution and data assimilation are simultaneously performed in a latent time space with a reduced number of elements. This results in high-resolution data, which is time-series data, as indicated by the closed triangular dots. Furthermore, as shown in FIG. 8 , the high-resolution data has a higher resolution in the time direction than the physical simulation. Thus, in the first embodiment, unlike the comparative example, ensemble calculation is not performed in the simulation. Furthermore, in the first embodiment, unlike the comparative example, super-resolution and data assimilation are simultaneously performed in a latent time space. Furthermore, in the first embodiment, unlike the comparative example, time-series information is used to perform super-resolution and data assimilation.

[0102] Fig. 9 is a diagram comparing the experimental results according to the first embodiment with those according to the comparative example. Fig. 9 is a diagram showing the time series of the mean absolute error (MAE) of the vorticity field ω. Graph A shows the case where neither data assimilation nor super-resolution is performed. Graph B shows the case of the comparative example. Graph C shows the case of the first embodiment. As shown in Fig. 9, the error is minimized in the case of the first embodiment. Therefore, the technology according to the first embodiment can achieve highly accurate prediction.

[0103] Furthermore, in the experimental results for the comparative example, the calculation time per experiment was 320 seconds. In contrast, in the experimental results for the first embodiment, the calculation time per experiment was 61 seconds. Thus, the technology according to the first embodiment has achieved a significant reduction in calculation time compared to the comparative example.

[0104] <Learning Method> Next, a learning method performed by the learning processing unit 110 of the information processing device 100 according to the first embodiment will be described. As described above, the learning processing unit 110 uses an end-to-end learning technique to train the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a continuous manner. In other words, the learning processing unit 110 may train the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as a single neural network having these units as layers. In other words, the learning processing unit 110 uses the observation data and the prediction data as inputs and trains the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 collectively so as to output appropriate high-resolution analysis data.

[0105] First, the first learning method will be described. The first learning method is supervised learning. The training data (ground truth data) is, for example, high-precision, high-resolution data. In the case of a weather forecasting system, the training data is, for example, high-precision, high-resolution meteorological data. That is, the training data is high-precision, high-resolution time-series data of atmospheric physical variables. The training data is, for example, time-series data (four-dimensional numerical arrays) of velocity fields, temperature fields, density fields, etc. Alternatively, the training data may be the results of an ultra-high-resolution micrometeorological simulation. The learning processing unit 110 receives the above-described forecast data and observation data as input, and updates the neural network parameters (weights, etc.) in the gradient direction of the error between the training data and the final output (high-resolution analysis data) by backpropagation using the above-described training data. By repeating this process, the learning processing unit 110 trains the neural network comprising the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160.

[0106] Next, a second learning method will be described. The second learning method is unsupervised learning. In the above-described supervised learning, for example, highly accurate and high-resolution time-series data of atmospheric physical variables is required as training data. However, it may be difficult to obtain such data. In contrast, in unsupervised learning, such training data is not required. Unsupervised learning may be performed, for example, by the variational Bayes method. Alternatively, unsupervised learning may be performed by adversarial learning. Below, a learning method using the variational Bayes method will be described.

[0107] The variational Bayes method is a type of approximation method that approximates the true probability distribution p with a simple probability distribution q. The parameters of q are then estimated by minimizing the KL divergence, etc. The variational Bayes method is a probabilistic model that uses the true physical variables of the environmental state, such as the atmosphere, as hidden states and provides observed values ​​or low-resolution predicted values ​​of inputs based on these hidden states.

[0108] As one example of implementing the variational Bayes method, a lower bound for the log-likelihood ln(p(o|x)) is introduced and this lower bound is maximized. The log-likelihood ln(p(o|x)) can be transformed into the following equation (10) using Jensen's inequality. Here, o corresponds to the observed data and x corresponds to the low-resolution predicted data. During the transformation of equation (10), a hidden variable y is introduced. This hidden variable corresponds to the high-resolution analysis data Da1 shown in equation (9). As a result, a loss function can be derived that enables the hidden variable y to be estimated from the observed data o and the low-resolution prediction x. ...(10)

[0109] In equation (10), the observed value o corresponds to low-resolution or high-resolution observed data. o corresponds to the above-mentioned equation (2) (or the left side of equation (3)). The hidden variable y corresponds to high-resolution analysis data Da1. Note that the observed value o may be highly accurate predicted data. The variable x corresponds to the input low-resolution predicted data (equation (1)). This probabilistic model is composed of a recognition model and a generative model. The recognition model (the part that calculates the hidden variables) corresponds to the structural transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 (encoder-decoder model) in the information processing device 100. The generative model will be described later.

[0110] The fourth equation (the rightmost side; the final form of the transformation) in equation (10) corresponds to the variational lower bound (VLB), which is the lower bound of the log-likelihood. The first term E q [p(o|x,y)] denotes the reconstruction error and corresponds to the log-likelihood of the observed data o. The second term of the variational lower bound, KL(q(y|x,o)|p(y|x)), denotes the KL divergence. The KL divergence is an index corresponding to the distance between distribution q and distribution p.

[0111] The learning processing unit 110 performs learning by updating the model parameters (neural network parameters) using backpropagation and gradient descent so as to maximize (increase) the variational lower bound shown in the fourth equation of equation (10). This variational lower bound corresponds to the training error. Maximizing the variational lower bound corresponds to minimizing the loss function in machine learning. In other words, increasing the variational lower bound corresponds to decreasing the loss function in machine learning. At this time, learning progresses so that the reconstruction error and the KL divergence are balanced. This balancing of the reconstruction error and the KL divergence corresponds to fusing observed data and predicted data in data assimilation.

[0112] It has been newly discovered that maximizing the variational lower bound has the aspect of generalizing minimum variance estimation. That is, by averaging the observed value o and the predicted data x using their respective errors as weights (performing a weighted average), high-resolution analysis data y can be estimated. In other words, since observed values ​​are generally more accurate than predicted data, analysis data can be estimated by weighting the observed values ​​more heavily than the predicted data and averaging them.

[0113] 10 is a diagram for explaining a method of learning the components according to the first embodiment using the variational Bayes method. When learning using the variational Bayes method, the learning processing unit 110 has a sampler 112 and an observation data generating unit 114. The sampler 112 and the observation data generating unit 114 can be regarded as a generative model that generates observation values ​​(pseudo observation data Da3). This generative model allows the reconstruction error E q [p(o|x, y)] can be calculated. Specifically, p(o|x, y) is calculated by the observation data generator 114. p(o|x, y) represents the distribution of the pseudo observation data o when the predicted data x and the high-resolution analysis data y are determined.

[0114] On the other hand, as described above, the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 can be considered as a recognition model. This recognition model can calculate the KL divergence. Specifically, the recognition model calculates q(y|x, o) in the KL divergence. q(y|x, o) indicates the distribution of the high-resolution analysis data y when the predicted data x and the observed data o are input. Note that p(y|x) is a prior distribution, which can be obtained by appropriate assumptions or prior learning.

[0115] The sampler 112 samples a probability distribution corresponding to the high-resolution analysis data generated by learning. As a result, the sampler 112 replaces the neural network with a probability distribution (probability model). Since neural networks typically produce deterministic output, it is difficult to produce random output such as a probability distribution. Therefore, the sampler 112 combines random numbers sampled from a Gaussian distribution with the output from the neural network (high-resolution analysis data) to pseudo-represent the probability distribution (re-parametrization trick). This enables error estimation of the high-resolution analysis data. Specifically, the pseudo-representation of the probability distribution is as shown in the following equation (11). μ and σ are deterministic variables provided by the neural network, and ε is a random number sampled from the Gaussian distribution. ...(11)

[0116] Specifically, the sampler 112 receives high-resolution analysis data Da1 (corresponding to y in Equation (10)) from the high-resolution analysis data acquisition unit 160. The sampler 112 samples random numbers from a Gaussian distribution and adds noise to the high-resolution analysis data. This adds randomness to the high-resolution analysis data provided by the neural network, making it possible to acquire data that can be considered as sampled values ​​from a probability distribution. This makes it possible to express the high-resolution analysis data as a probability distribution. In other words, the sampler 112 acquires sampled data when the high-resolution analysis data is expressed as a probability distribution. The sampler 112 then outputs the sampled data of the high-resolution analysis data to the observation data generation unit 114. Note that, when expressing a more complex probability distribution, i.e., when noise is generated from a distribution more complex than a Gaussian distribution, a mixture distribution or normalizing flows may be used. In this case, a complex distribution can be obtained by performing a nonlinear transformation on a random variable that follows a simple probability distribution, such as a Gaussian distribution.

[0117] The observation data generator 114 generates pseudo observation data Da3 using sampling data of the high-resolution analysis data generated by the sampler 112. That is, the observation data generator 114 converts the sampling data of the high-resolution analysis data into pseudo observation data Da3. The observation data generator 114 can generate pseudo observation data Da3 that is free of gaps in the spatiotemporal direction. The observation data generator 114 may be implemented by a neural network that has been trained in advance by machine learning. The observation data generator 114 may be implemented by a neural network that reflects physical symmetry. By generating the pseudo observation data Da3, unsupervised learning can be achieved.

[0118] Specifically, the observation data generator 114 may generate the pseudo observation data Da3 by performing the reverse process of the process performed by the above-mentioned structurizer (structure converter 130). That is, the observation data generator 114 may generate the pseudo observation data Da3 using a technique substantially similar to that of the structurizer. More specifically, the observation data generator 114 selects data for a grid point at an arbitrary time and position from the sampling data of the high-resolution analysis data, which is grid data, and repeatedly performs linear and nonlinear transformations on the data. In this way, the observation data generator 114 acquires pseudo observation data Da3 in a format substantially similar to the format of the observation data o acquired by the observation data acquisition unit 122. Therefore, the pseudo observation data Da3 may be non-grid data. Furthermore, the pseudo observation data Da3 may represent the numerical values ​​of non-physical quantities.

[0119] As described above, the learning processing unit 110 applying the variational Bayes method generates pseudo observation data Da3 from the high-resolution analysis data Da1. Here, in the variational lower bound expressed by the fourth equation of Equation (10), the inferred high-resolution analysis data y is a hidden state. It should be noted that this hidden state is not the final output within the framework of the variational Bayes method. In the variational Bayes method, the pseudo observation data Da3 is the final output in the learning phase. And because the high-resolution analysis data y is a hidden state, there is no need to prepare ground truth data corresponding to the high-resolution analysis data y in the learning phase. Therefore, there is no need to prepare highly accurate and high-resolution weather data, which is necessary for supervised learning.

[0120] (Hardware Configuration Example) An example of the configuration of hardware resources for realizing the devices and systems according to the above-described embodiments using one calculation processing device (information processing device, computer) will be described. However, the device (information processing device) according to each embodiment may be realized physically or functionally using at least two calculation processing devices. Furthermore, the device according to each embodiment may be realized as a dedicated device or a general-purpose information processing device.

[0121] FIG. 11 is a block diagram illustrating an example of the hardware configuration of a computing device capable of realizing the devices and systems according to each embodiment. The computing device 1000 includes a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF (Interface) 1007. Therefore, it can be said that the devices according to each embodiment include the CPU 1001, the volatile storage device 1002, the disk 1003, the non-volatile recording medium 1004, and the communication IF 1007. The computing device 1000 may be connectable to an input device 1005 and an output device 1006. The computing device 1000 may also include the input device 1005 and the output device 1006. The computing device 1000 can also transmit and receive information to and from other computing devices and communication devices via the communication IF 1007.

[0122] The nonvolatile recording medium 1004 is a computer-readable medium, such as a compact disc or a digital versatile disc. The nonvolatile recording medium 1004 may also be a universal serial bus (USB) memory, a solid state drive, or the like. The nonvolatile recording medium 1004 stores the program and allows portability without requiring a power supply. The nonvolatile recording medium 1004 is not limited to the above-mentioned medium. Instead of the nonvolatile recording medium 1004, the program may be supplied via the communication IF 1007 and a communication network.

[0123] The volatile storage device 1002 is computer-readable and can temporarily store data. The volatile storage device 1002 is a memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0124] That is, when executing a software program (computer program: hereinafter simply referred to as "program") stored on disk 1003, CPU 1001 copies the program to volatile storage device 1002 and executes arithmetic processing. CPU 1001 reads data necessary for executing the program from volatile storage device 1002. When display is required, CPU 1001 displays the output result on output device 1006. When a program is input from outside, CPU 1001 acquires the program from input device 1005. CPU 1001 interprets and executes programs corresponding to the functions (processing) of each component shown in FIGS. 4, 6, and 10 described above. CPU 1001 executes the processing described in each of the above-mentioned embodiments. In other words, the functions of each component shown in FIGS. 4, 6, and 10 described above can be realized by CPU 1001 executing a program stored on disk 1003 or volatile storage device 1002.

[0125] That is, each of the embodiments can be realized by the above-described program. Furthermore, each of the embodiments can be realized by a computer-readable non-volatile recording medium on which the above-described program is recorded.

[0126] (Modifications) The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, in the above-described flowchart, the order of each process (step) can be changed as appropriate. Furthermore, one or more of the multiple processes (steps) may be omitted. For example, in the flowchart of FIG. 5, the process of S22 may be performed before the process of S20. Furthermore, the process of S24 may be performed before the process of S22. Furthermore, the process of S70 may be omitted.

[0127] Furthermore, in the above-described embodiment, a case where weather forecasting is performed has been described, but this embodiment is not limited to the case where weather forecasting is performed. This embodiment can be applied to any predictive simulation that uses grid data. For example, this embodiment can also be applied to ocean forecasting. This embodiment can also be applied to astrophysical simulations.

[0128] Furthermore, in this embodiment, the dimensions of "space-time" are not limited to four dimensions consisting of three-dimensional space and one-dimensional time. The dimensions of "space-time" may be three dimensions consisting of two-dimensional space and one-dimensional time. Alternatively, the dimensions of "space-time" may be a dimension greater than four, such as ten dimensions.

[0129] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0130] This application claims priority based on Japanese Patent Application No. 2022-155991, filed September 29, 2022, the disclosure of which is incorporated herein by reference in its entirety.

[0131] REFERENCE SIGNS LIST 10 Information processing device 20 Simulation unit 22 Observation data acquisition unit 24 Prediction data acquisition unit 30 Structure transformation unit 40 Latent space-time mapping unit 50 Nonlinear transformation unit 60 High-resolution analysis data acquisition unit 70 Low-resolution analysis data calculation unit 100 Information processing device 110 Learning processing unit 112 Sampler 114 Observation data generation unit 120 Simulation unit 122 Observation data acquisition unit 124 Prediction data acquisition unit 130 Structure transformation unit 140 Latent space-time mapping unit 150 Nonlinear transformation unit 160 High-resolution analysis data acquisition unit 170 Low-resolution analysis data calculation unit

Claims

1. A structural transformation unit that transforms the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure that indicates numerical values ​​defined on lattice points arranged at predetermined intervals in space-time; a latent space-time mapping unit that maps the observation data converted into lattice data and prediction data, which is space-time lattice data obtained by simulation and covers at least the time of the observation data and a time before the said time, from a first real-time space to a latent space-time having a smaller number of elements than the first real-time space; a nonlinear transformation unit that performs a nonlinear transformation on the mapped observation data and the prediction data in the latent space; and a high-resolution analysis data acquisition unit that maps the nonlinearly transformed observation data and the prediction data from the latent space to a second real-time space that has a larger number of elements than the latent space and a higher resolution than the first real-time space, thereby acquiring high-resolution analysis data that is space-time lattice data and has a higher resolution in space-time than the prediction data, wherein data assimilation of the observation data and the prediction data is performed by the latent space-time mapping unit and the nonlinear transformation unit. Information processing device.

2. The information processing device according to claim 1, further comprising: a low-resolution analysis data calculation unit that uses the high-resolution analysis data to calculate low-resolution analysis data having lower resolution in time and space than the high-resolution analysis data.

3. The information processing device according to claim 1, wherein the nonlinear transformation unit performs super-resolution in the time direction by transforming a data array of the data mapped to the latent space-time.

4. The information processing device according to claim 3, wherein the high-resolution analysis data acquisition unit acquires the high-resolution analysis data by performing super-resolution in the spatial direction independently for each time point in the time direction on data that has been super-resolved in the time direction in the latent space-time.

5. The information processing device described in claim 1, wherein the predicted data is predicted data for a time including the time of the observation data and a time including the past and future of that time, and the high-resolution analysis data acquisition unit acquires high-resolution analysis data for a time including the past and future of the time of the observation data.

6. The information processing device according to claim 1, wherein the structural transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model trained by a machine learning algorithm.

7. The information processing device according to claim 6, wherein the structural transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model trained by supervised learning using data with a higher spatiotemporal resolution than the predicted data as training data.

8. The information processing device according to claim 6, wherein the structural transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are realized by a trained model trained by unsupervised learning so as to reduce a loss function.

9. Converting the structure of observation data, which is data obtained by observing a state in space-time, into observation data with a lattice data structure showing numerical values ​​defined on lattice points arranged at predetermined intervals in space-time; mapping the observation data converted into lattice data and prediction data, which is space-time lattice data obtained by simulation and which is at least the time of the observation data and a time before that time, from a first real-time space to a latent-time space having fewer elements than the first real-time space; performing a nonlinear transformation on the mapped observation data and the prediction data in the latent-time space; mapping the nonlinearly transformed observation data and the prediction data from the latent-time space to a second real-time space having a larger number of elements than the latent-time space and a higher resolution than the first real-time space, thereby obtaining high-resolution analysis data which is space-time lattice data and has a higher resolution in space-time than the prediction data; performing data assimilation between the observation data and the prediction data by mapping from the first real-time space to the latent-time space and performing a nonlinear transformation on the observation data and the prediction data in the latent-time space. Information processing methods.

10. A process of converting the structure of observation data, which is data obtained by observing a state in space-time, into observation data having a lattice data structure indicating numerical values ​​defined on lattice points arranged at predetermined intervals in space-time; a process of mapping the observation data converted into lattice data and prediction data, which is lattice data in space-time obtained by simulation and includes at least the time of the observation data and a time before that time, from a first real-time space to a latent-time space having fewer elements than the first real-time space; a process of performing a nonlinear transformation on the mapped observation data and prediction data in the latent-time space; and a process of mapping the nonlinearly transformed observation data and prediction data from the latent-time space to a second real-time space having a larger number of elements than the latent-time space and a higher resolution than the first real-time space, thereby obtaining high-resolution analysis data, which is lattice data in space-time and has a higher resolution in space-time than the prediction data, a process of mapping from the first real-time space to the latent-time space, and a process of performing a nonlinear transformation on the observation data and the prediction data in the latent-time space, thereby performing data assimilation between the observation data and the prediction data.