Information processing device, information processing method, and program

Abstract

Provided is an information processing device capable of predicting an environmental state with high precision and high efficiency. A structure conversion unit (30) converts the structure of observation data obtained by observing a spatio-temporal state into observation data having a grid data structure. A potential spatio-temporal mapping unit (40) maps the observation data and the prediction data converted into the grid data from the first actual spatio-temporal to a potential spatio-temporal. A nonlinear conversion unit (50) non-linearly converts the mapped observation data and prediction data in a potential space. A high-resolution analysis data acquisition unit (60) maps the observation data and the prediction data, which have been subjected to the nonlinear transformation, from the potential space-time to a second actual space-time, thereby acquiring high-resolution analysis data.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. Background Art

[0002] For example, in fields such as weather forecasting or sea state forecasting, it is required to predict the state of the environment with high accuracy and high efficiency through simulation. Similarly, it is also required to predict the state of the environment within the device with high efficiency. As methods of improving prediction accuracy, there are increasing resolution (super-resolution) and data assimilation. In connection with this technology, non-patent document 1 discloses a technology of super-resolution data assimilation (SRDA). In addition, non-patent documents 2 to 5 disclose technologies related to the present disclosure.

[0003] Prior art literature

[0004] Non-patent literature

[0005] Non-patent literature 1: Sebastien Barthelemy, et al., "Super-resolution data assimilation", Ocean Dynamics 72, 661-678 (2022), https: / / doi.org / 10.1007 / s10236-022-01523-x

[0006] Non-patent literature 2: Bin Li, et al., "3D-ReConstnet: A Single-View 3D-Object PointCloud Reconstruction Network", IEEE Access, 2020, https: / / ieeexplore.ieee.org / documedo / 9086481? source = authoralert

[0007] Non-patent literature 3: Ashish Vaswani, et al., "Attention is all you need", Advances in neural information processing systems, 2017, https: / / arxiv.org / abs / 1706.03762

[0008] Non-patent literature 4: Zhicheng Geng, et al., "RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution", Computer Vision and Pattern Recognition, 27 Mar 2022, https: / / arxiv.org / abs / 2203.14186

[0009] Non-patent document 5: Wenzhe Shi, et al., "Real-Time Single Image and VideoSuper-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network", Computer Vision and Pattern Recognition, 2016, https: / / arxiv.org / abs / 1609.05158 Summary of the invention

[0010] Problems to be solved by the invention

[0011] In the technology involved in non-patent document 1, the super-resolution method and the data assimilation method are simply combined. Therefore, in non-patent document 1, data assimilation is performed by ensemble calculation. Among them, in ensemble calculation, it is necessary to simulate various similar situations. Therefore, in the technology involved in non-patent document 1, if high-precision prediction is to be performed, the calculation cost may increase. Therefore, in the technology involved in non-patent document 1, it may not be possible to perform high-precision and efficient prediction.

[0012] An object of the present disclosure is to provide an information processing device, an information processing method, and a program that can predict an environmental state with high accuracy and efficiency.

[0013] Means of solving problems

[0014] The information processing device involved in the present disclosure comprises: a structure transformation unit, which transforms the structure of observation data obtained as the state in observation spacetime into observation data having a structure of grid data, wherein the grid data represents numerical values ​​defined on grid points arranged at prescribed intervals in spacetime; a potential spacetime mapping unit, which maps the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time from a first actual spacetime to a potential spacetime having a smaller number of elements than the first actual spacetime, wherein the prediction data is grid data in spacetime obtained by simulation; and a nonlinear transformation unit, which maps the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time from a first actual spacetime to a potential spacetime having a smaller number of elements than the first actual spacetime. A nonlinear transformation unit performs nonlinear transformation on the observed data and the predicted data that have been mapped in the latent space-time; and a high-resolution analysis data acquisition unit maps the observed data and the predicted data that have been subjected to the nonlinear transformation from the latent space-time to a second actual space-time that has a larger number of elements than the latent space-time and a higher resolution than the first actual space-time, thereby obtaining high-resolution analysis data having a higher resolution in space-time than the predicted data, wherein the high-resolution analysis data is grid data in space-time, and data assimilation of the observed data and the predicted data is performed by the latent space-time mapping unit and the nonlinear transformation unit.

[0015] In the information processing method involved in the present disclosure, the structure of observation data obtained as data on the state in observation space-time is transformed into observation data with a structure of grid data, wherein the grid data represents numerical values ​​defined on grid points arranged at prescribed intervals in space-time; the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time are mapped from a first actual space-time to a potential space-time having a smaller number of elements than the first actual space-time, wherein the prediction data is grid data on space-time obtained by simulation; in the potential space-time, the observation data to be mapped The observation data and the predicted data are subjected to nonlinear transformation; and the observation data and the predicted data subjected to the nonlinear transformation are mapped from the potential space-time to a second actual space-time having a larger number of elements than the potential space-time and a higher resolution than the first actual space-time, thereby obtaining high-resolution analysis data having a higher resolution in space-time than the predicted data, wherein the high-resolution analysis data is grid data in space-time, and data assimilation of the observation data and the predicted data is performed by mapping from the first actual space to the potential space-time and performing nonlinear transformation on the observation data and the predicted data in the potential space-time.

[0016] The program involved in the present disclosure enables a computer to perform the following processing: transforming the structure of observation data obtained as data on the state in observation space-time into observation data with a structure of grid data, wherein the grid data represents numerical values ​​defined on grid points arranged at prescribed intervals in space-time; mapping the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time from a first actual space-time to a potential space-time with fewer elements than the first actual space-time, wherein the prediction data is grid data on space-time obtained by simulation; in the potential space-time, mapping the observation data transformed into the grid data The observation data and the predicted data are subjected to nonlinear transformation; and the observation data and the predicted data subjected to the nonlinear transformation are mapped from the potential space-time to a second actual space-time having a larger number of elements than the potential space-time and a higher resolution than the first actual space-time, thereby obtaining high-resolution analysis data having a higher resolution in space-time than the predicted data, the high-resolution analysis data being grid data in space-time, and the data assimilation of the observation data and the predicted data is performed by mapping the processing from the first actual space to the potential space-time, and performing nonlinear transformation on the observation data and the predicted data in the potential space-time.

[0017] Effects of the Invention

[0018] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a program that can predict an environmental state with high accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a diagram used to illustrate the resolution and grid data of the calculation.

[0020] Figure 2 is a diagram used to illustrate the resolution and grid data of the calculation.

[0021] Figure 3 A diagram for explaining a super-resolution simulation method.

[0022] Figure 4 It is a schematic diagram showing the structure of the information processing device involved in this embodiment.

[0023] Figure 5 This is a flowchart showing an information processing method executed by the information processing device according to the present embodiment.

[0024] Figure 6 This is a schematic diagram showing the structure of the information processing device involved in the first embodiment.

[0025] Figure 7 It is a diagram for explaining the technique according to the comparative example.

[0026] Figure 8 This is a diagram for explaining super-resolution and data assimilation involved in Implementation Example 1.

[0027] Fig. 9 This is a diagram comparing the experimental results according to Embodiment 1 and the experimental results according to the comparative example.

[0028] Fig.10 This is a diagram for explaining a method of learning the components involved in Implementation Example 1 using the variational Bayes method.

[0029] Fig.11 This is a block diagram schematically showing an example of the hardware configuration of a calculation processing device that can realize the devices and systems involved in each embodiment. DETAILED DESCRIPTION

[0030] (Overview of Embodiments)

[0031] Before describing the embodiment, the outline of the embodiment is described. It should be noted that the embodiment is described below, but the following embodiment does not limit the invention involved in the claims. In addition, the combination of features described in the embodiment is not necessarily required for the solution of the invention. In addition, in the following description, the indexes (English letters, etc.) used are not necessarily common throughout the entire specification.

[0032] For example, in the field of weather forecasting (weather forecast simulation), micrometeorological forecasting is sometimes applied. Micrometeorology refers to the weather near the ground at an altitude of about 100m, which is greatly affected by artificial structures and human activities. In micrometeorological forecasting, high-resolution simulation results of about 100 to 1000 times are provided compared to general weather forecasting. Micrometeorological forecasts are mainly provided for urban areas, but the applicable locations are not limited to cities. Due to the ultra-high resolution, airflow through buildings and exhaust heat from buildings that are not considered in ordinary weather forecasts can be incorporated into micrometeorological forecasts. In other words, micrometeorological forecasts can simulate atmospheric flows that are closer to reality. In micrometeorological forecasts in the near future, it is believed that observation data will be obtained from sensors and cameras, drones, smartphones, etc. configured in urban environments, and predictions will be made using these observation data. In order to make such predictions with high accuracy, it is necessary to increase the resolution of the calculation.

[0033] Figure 1 and Figure 2 is a diagram used to illustrate the resolution and grid data of the calculation. Figure 1The three-dimensional calculation grid G1 is shown. The three-dimensional calculation grid G1 is represented by a grid corresponding to the three-dimensional space defined by the three-dimensional coordinate space of the X-axis, the Y-axis, and the Z-axis. The shorter the grid interval, the higher the resolution of the calculation in the spatial direction. On the contrary, the longer the grid interval, the lower the resolution of the calculation in the spatial direction.

[0034] Figure 2 The four-dimensional calculation grid G2 is shown. The four-dimensional calculation grid G2 is configured so that the three-dimensional calculation grid G1 is arranged in the time direction (represented by the T axis), that is, in the time series. Among them, the time interval (sampling period; in Figure 2 The shorter the time interval (in which the time interval is the interval between T1 and T2), the higher the resolution of the calculation in the time direction. On the contrary, the longer the time interval, the lower the resolution of the calculation in the time direction. It should be noted that in this embodiment, "spacetime" is described as a four-dimensional spacetime defined by three-dimensional space and one-dimensional time, but the spacetime dimension is not limited to four dimensions.

[0035] Among them, data representing numerical values ​​such as physical quantities in the four-dimensional calculation grid G2 can be regarded as grid data in four-dimensional space-time. Grid data represents numerical values ​​such as physical quantities (speed, etc.) defined on grid points arranged at prescribed intervals in space-time. That is, at each point in the three-dimensional space (three-dimensional calculation grid G1), there is a numerical value (physical quantity, etc.) representing the state of the point, as shown in the four-dimensional calculation grid G2, and the numerical value of each point changes in the one-dimensional time direction. The change in the time direction of each point in the three-dimensional space is represented by grid data. In this case, the grid data can be represented by a four-dimensional data array (numerical array) representing the numerical values ​​of its physical quantities, etc. The number of elements in the data array is called the number of elements. In addition, the data array can be set for each physical quantity. In addition, the grid data can also be structured data after being structured. In addition, when processing grid data in the neural network described later, the grid data can represent physical quantities that humans can understand, or it can represent numerical values ​​that humans cannot understand. That is, the grid data has a structure representing numerical values ​​defined on grid points arranged at prescribed intervals in space-time.

[0036] On the other hand, when the resolution of the calculation is increased to simulate micro-meteorological forecasts, the amount of data to be processed increases, so the calculation cost is huge. Therefore, it is believed that by performing super resolution (SR) on the simulation results of low resolution (LR), high resolution (HR) calculation results can be obtained.

[0037] Figure 3This is a diagram for explaining the super-resolution simulation method. The super-resolution simulation system performs super-resolution on the low-resolution prediction results obtained by performing low-resolution simulation by means of a super-resolution instrument. In this way, a high-resolution prediction result can be obtained. The super-resolution instrument uses the high-resolution results obtained by the high-resolution simulation in advance and performs learning by deep learning (neural network). That is, the super-resolution instrument performs learning by performing supervised learning using a large amount of high-resolution results as supervisory data in advance. According to this configuration, when operating, a high-resolution prediction result can be obtained by performing a low-resolution simulation, so that the computational cost can be suppressed.

[0038] In addition, in order to improve the accuracy of the prediction simulation, data assimilation of the observed data and the predicted results can be considered. In the ensemble Kalman filter, which is one of the data assimilation methods, many similar situations are simulated by ensemble calculations, and the error is estimated based on the deviation of the predicted result. Then, based on the size of the error, the degree to which the predicted result is close to the observed data is determined. By performing data assimilation, the predicted result can be made close to the observed data, so the accuracy of the predicted result can be improved. Here, as described above, in the technology of non-patent document 1, the super-resolution method and the data assimilation method are simply combined. That is, in non-patent document 1, super-resolution and data assimilation are performed independently. In this method, data assimilation needs to be performed through ensemble calculations. Therefore, the computational cost increases.

[0039] In contrast, in this embodiment, as described below, the time series data of the grid data is used to perform super-resolution and data assimilation at the same time. As a result, it is not necessary to perform set calculations. Therefore, in this embodiment, predictions can be made with high accuracy and efficiency.

[0040] Figure 4 Schematic diagram of the structure of the information processing device 10 involved in this embodiment. The information processing device 10 is, for example, a computer. The information processing device 10 includes a simulation unit 20, an observation data acquisition unit 22, a prediction data acquisition unit 24, a structure transformation unit 30, a potential space-time mapping unit 40, a nonlinear transformation unit 50, a high-resolution analysis data acquisition unit 60, and a low-resolution analysis data calculation unit 70. These components can be implemented by the hardware configuration described later. The functions of these components will be described later.

[0041] Figure 5 1 is a flowchart showing an information processing method performed by the information processing device 10 according to the present embodiment. The simulation unit 20 performs a simulation of an environmental state (step S20). Specifically, the simulation unit 20 performs a low-resolution simulation as described above. More specifically, the simulation unit 20 performs a low-resolution simulation in the time direction and the space direction.

[0042] The observation data acquisition unit 22 obtains one or more observation data (step S22). Observation data is data obtained by observing the state in time and space. The observation data can be obtained, for example, from sensors and cameras, drones, smart phones, etc. configured in the environment. Among them, the structure of the observation data does not need to be the structure of grid data. In addition, if the structure of the observation data is the structure of grid data, its resolution is arbitrary, and can be low resolution or high resolution. The details of the observation data will be described later.

[0043] The prediction data acquisition unit 24 obtains prediction data (step S24). Specifically, the prediction data acquisition unit 24 obtains prediction data as a simulation result of the simulation unit 20. Among them, the prediction data is time series data representing the change of the state over time. In addition, the prediction data is grid data in time and space obtained by simulation. The prediction data is prediction data in at least the moment of observation data and the past time (time series) earlier than the moment. Among them, the "moment of observation data" includes the latest moment (reference moment) of all observation data. The details of the prediction data will be described later.

[0044] The structure conversion unit 30 converts the structure of the observation data (step S30). Specifically, the structure conversion unit 30 converts the structure of the observation data into observation data of the structure of grid data. More specifically, the structure conversion unit 30 converts the structure of the observation data, which is data obtained by observing the state in time and space, into observation data of the structure of grid data, wherein the grid data represents numerical values ​​defined at grid points arranged at predetermined intervals in time and space. The function of the structure conversion unit 30 will be described in detail later.

[0045] The latent space-time mapping unit 40 maps the observed data and the predicted data to the latent space-time (step S40). Specifically, the latent space-time mapping unit 40 maps the observed data converted into the grid data in the process of S30 and the predicted data obtained in the process of S24 from the first actual space-time to the latent space-time. As described later, the latent space-time mapping unit 40 (process of S40) performs data assimilation of the observed data and the predicted data.

[0046] Among them, the latent spacetime is a spacetime with fewer elements than the first actual spacetime. In other words, the latent spacetime is a spacetime with a lower resolution than the first actual spacetime. Therefore, the number of elements in the data array in the latent spacetime is less than the number of elements in the data array in the first actual spacetime. In addition, the data on the latent spacetime can be composed of a numerical array after compressing the information of time and space for the observed data and the predicted data. It should be noted that the latent spacetime can also be called a latent space that includes the concept of time (time series). In addition, in the latent spacetime, dimensions are not distinguished. That is, in the latent spacetime, time and space are not distinguished, nor are the dimensions of three-dimensional space. In addition, the first actual spacetime is the spacetime corresponding to the environment in which the predicted data is obtained.

[0047] In addition, the latent spacetime mapping unit 40 can obtain data in which the observed data (observation data transformed into grid data) is mapped to the latent spacetime, and data in which the predicted data is mapped (projected) to the latent spacetime through the above-mentioned mapping to the latent spacetime. The latent spacetime mapping unit 40 can use the observed data transformed into grid data to map the predicted data to the latent spacetime. Therefore, "data in which the predicted data is mapped to the latent spacetime" can also include data obtained by mapping the observed data to the latent spacetime. On the other hand, when mapping the observed data, the latent spacetime mapping unit 40 maps the observed data to the latent spacetime alone.

[0048] By using the observation data transformed into the grid data to map the prediction data, data assimilation of the observation data and the prediction data is performed. That is, when the observation data transformed into the grid data and the prediction data are mapped to the potential space-time, they can be mixed (fused). In other words, in the potential space-time, the observation data transformed into the grid data is incorporated into the prediction data. More detailed functions of the potential space-time mapping unit 40 will be described later.

[0049] The nonlinear transformation unit 50 performs nonlinear transformation in the latent spacetime (step S50). Specifically, the nonlinear transformation unit 50 performs nonlinear transformation on the observed data and predicted data that have been mapped in the latent spacetime. More specifically, the nonlinear transformation unit 50 may also fuse the observed data and predicted data mapped to the latent spacetime to obtain data in the latent spacetime (latent spacetime data; fused data). Therefore, data assimilation of the observed data and predicted data is performed by the nonlinear transformation unit 50 (processing of S50). In addition, the nonlinear transformation unit 50 may also repeatedly perform nonlinear transformation to make the numerical distribution of the latent spacetime data mapped to the latent spacetime discontinuous. In addition, the nonlinear transformation unit 50 may also make the numerical distribution of the latent spacetime data mapped to the latent spacetime complex or simple through nonlinear transformation. In this case, the nonlinear transformation unit 50 may perform nonlinear transformation to appropriately perform super-resolution. It should be noted that the nonlinear transformation unit 50 may not change the number of elements when changing the numerical distribution. It should be noted that super-resolution can be performed by increasing the number of elements. More detailed functions of the nonlinear transformation unit 50 will be described later.

[0050] The high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data (step S60). Specifically, the high-resolution analysis data acquisition unit 60 maps the observation data and prediction data that have been subjected to nonlinear transformation from the potential space-time to the second actual space-time. Thus, the high-resolution analysis data acquisition unit 60 acquires high-resolution analysis data. That is, the high-resolution analysis data is analysis data that has been subjected to super-resolution in the temporal direction and the spatial direction.

[0051] Among them, the second actual space-time is an actual space-time with more elements than the potential space-time and higher resolution than the first actual space-time. Therefore, it can be said that the second actual space-time is a high-resolution space (HR space). In addition, high-resolution analysis data (HR analysis data) is grid data in space and time. In addition, high-resolution analysis data is data with a higher resolution in space and time than prediction data. In addition, high-resolution analysis data can also be time series data in the past and future time (time series) of the moment of observation data. That is, when the input prediction data exists in the future relative to the moment of observation data (reference moment), the high-resolution analysis data is analysis data including the same future moment. That is to say, the prediction data can also be prediction data in the moment of observation data and the past earlier than the moment and the future later than the moment. In this case, the high-resolution analysis data acquisition unit 60 can obtain high-resolution analysis data in the past and future time including the moment of observation data. More detailed functions of the high-resolution analysis data acquisition unit 60 will be described later.

[0052] The low-resolution analysis data calculation unit 70 calculates the low-resolution analysis data (step S70). Specifically, the low-resolution analysis data calculation unit 70 uses the high-resolution analysis data to calculate the low-resolution analysis data. Among them, the low-resolution analysis data (LR analysis data) is analysis data with a lower resolution than the high-resolution analysis data in time and space. More specifically, the low-resolution analysis data calculation unit 70 calculates the low-resolution analysis data by performing arithmetic operations (mathematical methods) such as algebraic interpolation on the high-resolution analysis data. More detailed functions of the low-resolution analysis data calculation unit 70 will be described later.

[0053] The simulation unit 20 takes the low-resolution analysis data as input and performs the next timing simulation (S20). Figure 5 The processing flow shown.

[0054] As described above, the information processing device 10 involved in this embodiment is configured to obtain high-resolution analysis data by performing data assimilation and super-resolution on observation data in time and space and low-resolution prediction data in time and space. In addition, the information processing device 10 is configured to process the high-resolution analysis data in latent time and space when obtaining it. Therefore, it is possible to predict the environmental state with high accuracy and efficiency.

[0055] Furthermore, the information processing device 10 according to the present embodiment is configured to calculate low-resolution analysis data, thereby enabling low-resolution simulation to be continued, thereby enabling efficient simulation.

[0056] In addition, the nonlinear transformation unit 50 may also perform super-resolution in the time direction by deforming the data array of the data mapped to the latent space-time. In this case, the high-resolution analysis data acquisition unit 60 may perform super-resolution in the space direction independently at each moment in the time direction on the data that has been super-resolution in the time direction in the latent space-time, thereby obtaining high-resolution analysis data. This enables more efficient processing. The details will be described later.

[0057] In addition, the structural transformation unit 30, the latent spatiotemporal mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 can also be implemented using a learning model obtained by learning through a machine learning algorithm. In this case, the structural transformation unit 30, the latent spatiotemporal mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 can be implemented using a learning model obtained by supervised learning using data with a higher resolution than the predicted data in time and space as supervised data. Alternatively, the structural transformation unit 30, the latent spatiotemporal mapping unit 40, the nonlinear transformation unit 50, and the high-resolution analysis data acquisition unit 60 can also be implemented using a learning model obtained by unsupervised learning in order to reduce the loss function. Thus, high-resolution analysis data can be efficiently obtained by using observation data that is not grid data and low-resolution predicted data. The details will be described later.

[0058] (Implementation Method 1)

[0059] The following description will be made with reference to the accompanying drawings. For the purpose of clear description, the following description and drawings are appropriately omitted and simplified. In addition, in each of the drawings, the same reference numerals are used for the same elements, and repeated descriptions are omitted as needed.

[0060] <Information processing device>

[0061] Figure 6 1 is a schematic diagram of the structure of the information processing device 100 involved in Embodiment 1. The information processing device 100 is, for example, a computer. The information processing device 100 includes a learning processing unit 110, a simulation unit 120, an observation data acquisition unit 122, a prediction data acquisition unit 124, a structure transformation unit 130, a potential space-time mapping unit 140, a nonlinear transformation unit 150, a high-resolution analysis data acquisition unit 160, and a low-resolution analysis data calculation unit 170. These components can be implemented by the hardware configuration described later.

[0062] The learning processing unit 110 performs machine learning such as neural network on the structure transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160. As a result, the structure transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 can realize their respective functions as a learning completion model.

[0063] Among them, when the learning processing unit 110 performs machine learning, it is also possible to collectively learn the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 instead of learning them separately. Specifically, the learning processing unit 110 learns the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 in a coherent manner through an end-to-end deep learning method. That is, the learning processing unit 110 regards the functions of the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as layers of a neural network and performs machine learning. In other words, the learning processing unit 110 regards the structure transformation unit 130, the latent space-time mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 as a neural network and performs machine learning. It should be noted that the pre-learning may also be performed separately by the structure transformation unit 130 , the latent spatiotemporal mapping unit 140 , the nonlinear transformation unit 150 , and the high-resolution analysis data acquisition unit 160 .

[0064] The learning processing unit 110 can perform supervised learning or unsupervised learning. In the case of supervised learning, the learning processing unit 110 can also use high-precision and high-resolution time series meteorological data as supervised data for learning. The details of the processing of the learning processing unit 110 will be described later. It should be noted that the structure transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160 are not limited to being implemented as a learning completion model obtained by machine learning performed by the learning processing unit 110.

[0065] The simulation unit 120 corresponds to the above-mentioned simulation unit 20. The simulation unit 120 performs low-resolution simulation in the time direction and the space direction. The simulation unit 120 generates low-resolution prediction data (simulation data) using the input initial state.

[0066] Low-resolution prediction data (LR prediction analog data) is digital data defined at grid points in space and time. That is, the prediction data is grid data defined in space and time. In other words, the prediction data has a data array defined by a grid structure. That is, the prediction data as grid data in space and time has data arrays for grid points at predetermined intervals in the spatial direction and the temporal direction. Figure 2In the four-dimensional calculation grid G2 shown, the grid points are arranged in the time axis direction, and in the three-dimensional calculation grid G1 in the four-dimensional calculation grid G2, the grid points are arranged in the three spatial axis directions. The grid data of the prediction data has a value at each grid point. The intervals between the grid points can be equal intervals or non-equal intervals. It should be noted that since the simulation unit 120 performs a low-resolution simulation, the grid data of the prediction data obtained has an average longer interval in the time direction and an interval in the space direction than the high-resolution grid data.

[0067] For example, suppose the prediction data consists of a data array A in four-dimensional space-time hijk Definition. Subscript h corresponds to the time direction (t direction), subscript i corresponds to the X-axis direction of the three-dimensional space, subscript j corresponds to the Y-axis direction of the three-dimensional space, and subscript k corresponds to the Z-axis direction of the three-dimensional space. At this time, the data array A hijk The subscripts h, i, j, and k take integer values. A grid point of the four-dimensional computational grid G2 corresponds to one of the integer values ​​in the group (h, i, j, k). Thus, the value A at the grid point can be uniquely specified. hijk In this way, the prediction data is obtained by using a data array A with integer subscripts for the directions of each dimension in the four-dimensional space. hijk is defined.

[0068] Among them, in the data array A hijk In the data array A, the number of elements in the time direction is H. That is, the subscript h takes H values. Similarly, the number of elements in the X-axis direction is I, that is, the subscript i takes I values. In addition, the number of elements in the Y-axis direction is J, that is, the subscript j takes J values. In addition, the number of elements in the Z-axis direction is K, that is, the subscript k takes K values. In this case, the data array A hijk The number of elements of A is H*I*J*K. If H=I=J=K=10, then the data array A hijk The number of elements is 10 4 .

[0069] The prediction data includes data of past time in the time series. In addition, the prediction data may also include data of future time in the time series. In addition, as described above, the prediction data is data of the time including the time (reference time) of the observation data described later.

[0070] The prediction data includes data representing physical variables (physical quantities) required to predict the state of the atmosphere and other environments based on physical equations. In the case of weather prediction simulation, the physical equations include the Navier-Stokes equations or thermodynamic equations of fluids, etc. In addition, physical quantities are, for example, air velocity, pressure, temperature, water vapor mixing ratio, cloud particle number density, etc. The four-dimensional data array can be set for each of these physical quantities. The data array can also be said to represent the "field" of each physical quantity in physics.

[0071] In addition, it should be noted that in the present embodiment, the prediction data (LR simulation result) is obtained through a single scenario. That is, the prediction data is a unique (single) simulation result obtained when simulating (physical simulation) from a unique initial state. In contrast, in the data assimilation involved in non-patent document 1, various similar situations are simulated in order to perform set calculations. That is, in non-patent document 1, prediction simulation is performed based on multiple scenarios.

[0072] The prediction data acquisition unit 124 corresponds to the above-mentioned prediction data acquisition unit 24. The prediction data acquisition unit 124 acquires the above-mentioned prediction data (low-resolution physical simulation prediction result) from the simulation unit 120. The prediction data is represented by the following equation (1).

[0073] [Calculation formula 1]

[0074]

[0075] Among them, x t L It is a vector field composed of all physical quantities of the predicted data. That is, x is a vector field representing a group (vector) of values ​​at each grid point in the three-dimensional space at a certain time t. And, formula (1) represents the set of vector fields of all physical quantities from time t=0 to time t=n. In other words, formula (1) represents the entire data array (numerical array) of the four-dimensional space-time related to the predicted data. In addition, the superscript L indicates low resolution in the spatial direction. In addition, t represents the timestamp of the predicted data. t represents the timestamp of the interval with a long time interval. That is, t represents low resolution in the time direction (i.e., long time interval). That is, formula (1) represents data with a small number of grids (small number of elements; low resolution) in four-dimensional space-time.

[0076] The observation data acquisition unit 122 corresponds to the observation data acquisition unit 22. The observation data acquisition unit 122, like the observation data acquisition unit 22, acquires observation data from sensors and cameras, drones, smartphones, etc. arranged in the environment. The observation data is represented by the following equation (2).

[0077] [Calculation formula 2]

[0078] {o τ}

[0079] …(2)

[0080] Formula (2) represents a set of digital data consisting of numerical values ​​(observation values) o representing a certain state at time τ. Here, τ represents the timestamp of the observation data. The time interval of τ may not be equal. In addition, o may represent a physical quantity or not. That is, o may also represent the numerical value of a non-physical quantity.

[0081] It should be noted that the observation data may also be grid data. Alternatively, the observation data does not need to be grid data. That is, the observation data may also be unstructured unstructured data (non-grid data). In the observation data as unstructured data, the observation values ​​correspond to time information and space information, but the respective intervals of the time and space are irregular. That is, the observation data as unstructured data does not have a spatial grid structure. Therefore, the observation data may also represent observation values ​​in random time and space.

[0082] In addition, the observation data may include various complex and different data. The observation data may be image data, sound data, point data, or log data. In addition, the observation data may also be a set of values ​​of physical quantities representing the state of the atmosphere, or may be digital data that can infer their physical quantities. For example, the observation data may also represent the observation values ​​of the automatic meteorological data acquisition system (AMeDAS). In this case, the observation values ​​may represent the temperature, humidity, wind speed, etc. at various locations. In addition, the observation data may also represent the radiant brightness of an object (building, etc.). Thus, the temperature at the position near the object and the observed time can be inferred. In addition, the observation data may also represent the acceleration log of an aircraft floating in the air such as a drone. Thus, the wind speed or turbulent dissipation rate at the position of the aircraft and the observed time can be inferred. In addition, the observation data may also represent an image of the sky taken. Thus, the cloud cover or precipitation at the location and time of the shooting can be inferred. In addition, the observation data may also represent the sales of frozen desserts (ice cream, sherbet, popsicles, shaved ice, etc.) at a certain location (convenience store, etc.). Thus, the local temperature of the area can be inferred. That is, it can be inferred that the higher the sales of frozen desserts, the higher the temperature in the area.

[0083] Furthermore, when a simulation is performed with sufficiently high accuracy, it can be said that the simulation result well represents the actual state. Therefore, the simulation result with sufficiently high accuracy can also be regarded as an observation result that well represents the actual state, and can be regarded as observation data. It should be noted that, depending on the setting, for example, as shown in micro-meteorological simulation, a "close to reality experiment" can be performed in which the shape of the building and the heat discharged from the building are also incorporated. This kind of simulation can be regarded as an observation result that well represents the actual state, that is, observation data, because the error is small enough.

[0084] In addition, the time and space resolution for observing the observation data can be high resolution or low resolution. In the case where the observation data represents satellite data or radar data, they are obtained by line (one-dimensional) observation or surface (two-dimensional) observation. In addition, in the case where the observation data is data obtained by Doppler radar (Light Detection And Ranging, LiDAR), this is obtained by three-dimensional observation. In these cases, it is possible to determine whether the resolution of the observation data is low resolution or high resolution based on the spatial resolution (and the time interval of observation).

[0085] In addition, in the case where the observation data corresponds to AMeDAS data, this is obtained through point (0-dimensional) observation. Even in this case, the resolution can be defined as follows. That is, even if it is a point observation, there is a representative scale that indicates how large a scale the observation value represents. In the case of observation data representing a large scale, it can be regarded as low-resolution observation data. In the case of observation data representing a small scale, it can be regarded as high-resolution observation data. For example, in a conventional meteorological model, observations represent observation values ​​with a coarse spatial resolution of 1km to 10km horizontally. Therefore, in the observation, strict conditions such as not being affected by artificial heat removal are set, such as places where shelters are not nearby, on grass, and where direct sunlight is not available. This type of observation data can be said to be low-resolution observation data. On the other hand, in a micrometeorological model, observation values ​​with fine spatial resolutions such as 1m to 5m horizontally that are affected by artificial heat removal can be observed. In a micrometeorological model, the impact of such observation data on the atmosphere can be taken into account and simulation can be performed. This type of observation data can be said to be high-resolution observation data.

[0086] It should be noted that the high-resolution observation data and the low-resolution prediction data are very different in resolution in the spatial direction and resolution in the temporal direction. Therefore, when performing data assimilation, it is necessary to match the low-resolution prediction data and average the high-resolution observation data in the spatiotemporal direction. Therefore, it should be noted that it is usually difficult to directly assimilate the high-resolution observation data with the low-resolution prediction data. In contrast, in this embodiment, data assimilation can be performed on any observation data and low-resolution prediction data.

[0087] The structure transformation unit 130 corresponds to the above-mentioned structure transformation unit 30. The structure transformation unit 130 has a function as a structurer. The structure transformation unit 130 can be implemented by an existing structurer. The structure transformation unit 130 transforms the observation data, which is unstructured data (non-grid data), into observation data with a structure of grid data. That is, the structure transformation unit 130 performs grid data on the observation data. The structure transformation unit 130 transforms the observation data into data on each grid point of the grid data in high-resolution space-time. In other words, the structure transformation unit 130 transforms the observation value into a physical quantity defined on the grid. It should be noted that the grid data obtained by transforming the observation data may also correspond to grid data having a larger number of elements than the grid data of the prediction data (i.e., high resolution).

[0088] If the function representing the structurer implemented by the structure conversion unit 130 is set to s(), the function of the structure conversion unit 130 is expressed by the following formula (3). The left side of formula (3) corresponds to the output data of the structure conversion unit 130. It should be noted that the observation data input to the structure conversion unit 130 (structurer) can be grid data. In addition, the observation data input to the structurer can represent physical quantities or numerical values ​​other than physical quantities.

[0089] [Calculation formula 3]

[0090]

[0091] Among them, T H It is a vector field composed of all the observation values ​​of the observation data. That is, o is a vector field representing a group (vector) of values ​​at each grid point in the three-dimensional space at a certain time T. In addition, the left side of formula (3) represents the set of vector fields of all observation values ​​(physical quantities, etc.) from time T=0 to time T=N. In other words, the left side of formula (3) represents the entire data array (numerical array) of the four-dimensional space-time corresponding to the observation data (physical quantities, etc.). In addition, the superscript H indicates high resolution in the spatial direction. In addition, T represents the timestamp of the observation data. T represents the timestamp of the interval with a short time interval. That is, T represents high resolution in the time direction (i.e., short time interval). In other words, the left side of formula (3) represents data with a large number of grids (large number of elements; high resolution) in the four-dimensional space-time.

[0092] The structurer of the structure transformation unit 130 can be implemented using a learning model obtained by learning through machine learning such as neural networks. In this case, the structure transformation unit 130 can be implemented by a linear projection operator, a fully coupled layer, or a graph convolutional network (the same is true for the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 described later). In this case, the structure transformation unit 130 is learned by the learning processing unit 110. For example, the structurer can also be implemented by the technology shown in non-patent document 2. As described above, the structure transformation unit 130, the latent spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 are learned in a coherent manner through an end-to-end deep learning method. In addition, in order to perform the super-resolution data assimilation involved in this embodiment, the structure transformation unit 130 (structurer) learns to transform the observed data into appropriate grid data.

[0093] In addition, the structurer may also be implemented without using a learning model obtained by machine learning. In this case, the structure transformation unit 130 may read the time and position information of the observation data, and project the observation data to the grid point closest to the read time and position information among the grid points arranged at a predetermined interval in space and time. In addition, in this case, the structure transformation unit 130 may also substitute missing values ​​for the grid points to which the observation data is not projected.

[0094] It should be noted that even when the observation data input to the structure conversion unit 130 is data representing non-physical quantities, the structure conversion unit 130 can also output grid data representing numerical values ​​such as physical quantities corresponding to the observation data. For example, when the observation data input to the structure conversion unit 130 is data representing the sales of frozen desserts at various locations at various times, the structure conversion unit 130 can output grid data representing the temperature at various times at various locations. In addition, the structure conversion unit 130 can also input various multiple observation data. In this case, the structure conversion unit 130 can output grid data representing numerical values ​​such as physical quantities corresponding to multiple observation data. For example, when the observation data input to the structure conversion unit 130 is the acceleration of the drone and the sales of frozen desserts at various locations at various times, the structure conversion unit 130 can output grid data representing the wind speed and temperature at various times at various locations. In addition, the structure conversion unit 130 is not limited to outputting grid data corresponding to the input observation data and representing physical quantities that humans can understand. The structure conversion unit 130 can also output grid data corresponding to numerical arrays that humans cannot understand (that is, only neural networks can understand). In other words, the structure transformation unit 130 outputs grid data (numerical array) representing numerical values ​​defined at grid points arranged at predetermined intervals in space and time. In addition, when the observed data is image data (camera data), the structure transformation unit 130 (or other components) may also perform processing using object recognition or pixel segmentation as preprocessing. Moreover, in order to perform this preprocessing, a neural network that has been pre-learned may also be used. Furthermore, the structure transformation unit 130 may also perform the above-mentioned structural transformation processing on the data that has been subjected to such preprocessing.

[0095] As described above, the structure transformation unit 130 is non-grid data or grid data, and takes observation data representing the observed value of physical quantity or non-physical quantity as input. In addition, the structure transformation unit 130 outputs grid data of a four-dimensional space-time data array (numerical array) related to a physical quantity (temperature or wind speed, etc.) representing an environmental state (e.g., an atmospheric state). According to this configuration, the structure transformation unit 130 can perform nonlinear transformation on the observation data of non-physical quantity and non-grid data that cannot be processed by the existing data assimilation method, and transform it into observation data of a numerical array that can perform data assimilation.

[0096] The latent space-time mapping unit 140 corresponds to the latent space-time mapping unit 40 described above. The latent space-time mapping unit 140 has a function as an encoder. The latent space-time mapping unit 140 can be implemented by an existing encoder. The latent space-time mapping unit 140 maps the observation data and prediction data in the first actual space-time to the latent space-time. The latent space-time mapping unit 140 transforms the observation data and prediction data structured in the four-dimensional space-time into data of the latent space-time at each moment. That is, in the latent space-time mapping unit 140, the output at each moment is calculated independently for the input at each moment.

[0097] If the function representing the encoder implemented by the latent spatiotemporal mapping unit 140 is set to e x (), e o (), the function of the latent spatiotemporal mapping unit 140 is expressed by the following equations (4) and (5). x () is a function used to map the predicted data x to the potential space-time. o () is a function used to map the observed data o to the latent space-time. Thus, at each time t, the observed data o and the predicted data x are mapped to the latent space-time respectively.

[0098] [Calculation formula 4]

[0099]

[0100] [Calculation formula 5]

[0101]

[0102] Thus, the latent space-time mapping unit 140 obtains the data of the latent space-time represented by the following equation (6): Equation (6) represents the set of data of the latent space-time at each time from time t=0 to t=n.

[0103] [Calculation formula 6]

[0104]

[0105] Wherein, formula (4) indicates: using the observation data o transformed into grid data t , the predicted data x t L The latent space-time mapping unit 140 may also perform pre-processing and processing on the observation data. For example, as pre-processing, the latent space-time mapping unit 140 may also convert the input observation data o into a latent space-time mapping data. T H The observed data o is transformed into a structure that matches the grid structure in space and time of the predicted data t . ot is the observation data o transformed into grid data T The latent spatiotemporal mapping unit 140 may also obtain the result by making the transformed o t The data reflected in the low-resolution prediction data is assimilated to obtain the mapping data p corresponding to the prediction data t .

[0106] In addition, formula (5) represents the observation data o which is transformed into grid data. t Mapped to potential space-time, the mapping data q is obtained t . Mapping data q t The latent spacetime mapping unit 40 maps the observation data to the latent spacetime.

[0107] Here, it can be seen from the subscript t in equations (4) to (6) that in the processing of the potential space-time mapping unit 140, the resolution in the time direction (time step t) matches the resolution of the predicted data inside the potential space-time mapping unit 140. On the other hand, the resolution in the spatial direction is smaller than the resolution of the predicted data. That is, as described above, the number of elements in the numerical array of the mapping data in the potential space-time is less than the number of elements in the numerical array of the predicted data. Therefore, the potential space-time mapping unit 140 reduces the number of elements in the numerical arrays of the observed data and the predicted data (dimensionality compression in the phase space).

[0108] In addition, equations (4) to (6) represent the fusion of observed data and predicted data in latent space-time. That is, equation (6) represents the data (fused data) after the observed data and predicted data are fused in latent space-time. Therefore, it can be said that the observed data and predicted data are assimilated by the latent space-time mapping unit 140.

[0109] The latent space-time mapping unit 140 (encoder) can also be implemented using a learning model obtained through machine learning such as neural networks. For example, the latent space-time mapping unit 140 can use convolution and pooling to perform nonlinear transformations while reducing the number of elements. In addition, the neural network can also be a neural network that reflects physical symmetry. In order to reflect spatial translation symmetry, a convolutional neural network can be used. In addition, in order to reflect spatial rotational symmetry, a group / convolutional neural network can be used. In addition, in order to reflect relabelling symmetry, a vision transformer or a graph convolutional neural network can be used. As a result, unlike existing data assimilation methods, a transformation that takes physical symmetry into account can be performed. This is also the same in the decoder of the high-resolution analysis data acquisition unit 160 described later.

[0110] As described above, the latent space-time mapping unit 140 takes as input a group of data arrays in four-dimensional space-time, namely, low-resolution prediction data and observation data transformed into grid data. In addition, the latent space-time mapping unit 140 outputs mapping data on the latent space-time with a small number of elements. Through this configuration, the latent space-time mapping unit 140 can improve the processing efficiency in the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160. That is, since the number of elements in the numerical array is reduced, the amount of data to be processed is reduced. Therefore, the processing amount of computing resources is suppressed.

[0111] The nonlinear transformation unit 150 corresponds to the above-mentioned nonlinear transformation unit 50. The nonlinear transformation unit 150 has a function as a time series transformer. The nonlinear transformation unit 150 can also be implemented by an existing time series transformer. The nonlinear transformation unit 150 performs nonlinear transformation on the observed data and predicted data that have been mapped in the latent space-time. More specifically, the nonlinear transformation unit 150 can also perform nonlinear transformation on the time series to generate data that fuses the observed data and predicted data mapped to the latent space-time. Moreover, the nonlinear transformation unit 150 involved in Implementation 1 can also perform super-resolution on the fused data in the time direction. Among them, as described above, in the latent space-time, the number of elements is reduced compared to the actual space-time. Therefore, since the amount of data of the processing object is suppressed, super-resolution can be efficiently performed in the time direction.

[0112] If the function representing the time series converter implemented by the nonlinear conversion unit 150 is set to F(), the function of the nonlinear conversion unit 150 is expressed by the following equation (7). The left side of equation (7) corresponds to the output data of the nonlinear conversion unit 150. As mentioned above, the subscript T indicates high resolution in the time direction. In addition, r TIt represents the data (fused data) after the observed data and predicted data mapped to the potential space-time at time T are fused through data assimilation. In addition, the left side of formula (7) represents the set of fused data from time T = 0 to time T = N. In other words, the left side of formula (7) represents the data array (numerical array) of the four-dimensional space-time corresponding to the fused data.

[0113] [Calculation formula 7]

[0114]

[0115] As described above, formula (7) represents the fusion of observed data and predicted data in latent space-time. That is, the left side of formula (7) represents the data (fused data) after the observed data and predicted data are fused in latent space-time. Therefore, it can be said that the observed data and predicted data are assimilated by the nonlinear transformation unit 150.

[0116] The time series transformer of the nonlinear transformer 150 can be implemented using a learning model obtained by machine learning such as a neural network. In this case, the nonlinear transformer 150 can be implemented by a neural network using an attention mechanism. In addition, the time series transformer of the nonlinear transformer 150 can also use a technique called a transformer described in non-patent document 3 to perform nonlinear transformation.

[0117] In addition, the nonlinear transformation unit 150 may also use information in the spatiotemporal direction to implicitly calculate the prediction error. That is, the nonlinear transformation unit 150 may also calculate the error inside the neural network based on the temporal change of the spatial pattern. In other words, the nonlinear transformation unit 150 may also match the spatiotemporal pattern, determine the size of the error relative to the correct data, and thereby calculate the prediction error. Moreover, the nonlinear transformation unit 150 may also be implemented by a neural network that uses the prediction error as a weight to assimilate the predicted data with the observed data. Thus, the data assimilation of the predicted data with the observed data can be performed by efficient calculations that do not require sets.

[0118] In addition, the time series transformer of the nonlinear transformation unit 150 may also repeatedly perform nonlinear transformations such as linear transformations (e.g., affine transformations) and linear rectifier functions (Rectified Linear Unit, ReLU). In the process of performing this processing, the nonlinear transformation unit 150 may also use the technology of non-patent document 4, for example, to deform the data array in a manner that refines the time step. The nonlinear transformation unit 150 may also divide the multiple elements in the potential space-time into two parts: elements in the time direction and elements in the space direction, and increase the number of elements in the time direction. For example, the number of elements in the time direction is set to H', the number of elements in the X-axis direction is set to I', the number of elements in the Y-axis direction is set to J', and the number of elements in the Z-axis direction is set to K'. In this case, the nonlinear transformation unit 150 deforms the four-dimensional array H'×I'×J'×K' into a two-dimensional array H'×M. It should be noted that M is the number of elements in the space direction, M=I'×J'×K'. In addition, the nonlinear transformation unit 150 may also deform the array into an array of 2H'×M / 2. Thus, the number of elements in the time direction is doubled, and the time step is 1 / 2. Therefore, super-resolution is performed in the time direction for the data mapped to the potential space-time. In this way, the nonlinear transformation unit 150 can perform super-resolution in the time direction by deforming the data array of the data mapped to the potential space-time.

[0119] As described above, the nonlinear transformation unit 150 takes as input a group of numerical arrays (data arrays) corresponding to the observed data and predicted data in the latent spacetime. Moreover, the nonlinear transformation unit 150 can also output time series data on the latent spacetime that has been super-resolutioned in the time direction. The time series data on the latent spacetime that has been super-resolutioned in the time direction has a higher resolution in the time direction than the resolution in the time direction of the predicted data. Thus, the nonlinear transformation unit 150 performs super-resolution in the time direction on the latent spacetime with a small number of elements (i.e., low-dimensional in the phase space), thereby reducing the amount of calculation, thereby reducing the processing cost. Therefore, the calculation efficiency can be improved.

[0120] In addition, super-resolution in the time direction is performed by the time series converter of the nonlinear transformation unit 150. As described above, since the number of elements of the numerical array in the potential space-time is small, the required storage amount can be saved, so that a long time series can be processed. As a result, super-resolution in the time direction can be efficiently performed. Moreover, by the processing of the nonlinear transformation unit 150, super-resolution in the time direction can be performed, and data assimilation of observation data and prediction data can be performed at the same time. Therefore, data assimilation and super-resolution in the time direction can be efficiently performed.

[0121] The high-resolution analysis data acquisition unit 160 corresponds to the above-mentioned high-resolution analysis data acquisition unit 60. The high-resolution analysis data acquisition unit 160 has a function as a decoder. The high-resolution analysis data acquisition unit 160 can be implemented by an existing decoder. The high-resolution analysis data acquisition unit 160 maps the observation data and the prediction data that have been subjected to nonlinear transformation from the latent space-time to the second actual space-time. The high-resolution analysis data acquisition unit 160 maps the data obtained by the processing of the nonlinear transformation unit 150 from the latent space-time to the second space-time at each moment. Thus, the high-resolution analysis data acquisition unit 160 performs super-resolution on the data obtained by the processing of the nonlinear transformation unit 150 in the spatial direction. Thus, the high-resolution analysis data acquisition unit 160 transforms the data in the latent space-time into high-resolution analysis data at each moment. That is, in the high-resolution analysis data acquisition unit 160, the output at each moment is independently calculated for the input at each moment.

[0122] In this way, the high-resolution analysis data acquisition unit 160 obtains high-resolution analysis data Da1. As described above, the high-resolution analysis data Da1 is data having a higher resolution in time and space than the prediction data. In addition, the high-resolution analysis data Da1 may also be analysis data in the past and future time (time series) including the time of the observation data according to the time range of the input prediction data. Therefore, the high-resolution analysis data Da1 may be analysis data that has been extrapolated (time extrapolation) in the time direction relative to the time (reference time) of the observation data.

[0123] If the function representing the decoder implemented by the high-resolution analysis data obtaining unit 160 is d(), the function of the high-resolution analysis data obtaining unit 160 is expressed by the following equation (8). d() is a function for converting the fused data r related to the time T into T A function for mapping from the latent space-time to the second actual space-time. Thus, at each time T, the observed data and the predicted data are mapped to the second actual space-time with high resolution through the fused data after data assimilation fusion. Thus, the fused data that has been super-resolved in the time direction by the nonlinear transformation unit 150 is super-resolved in the spatial direction.

[0124] [Calculation formula 8]

[0125]

[0126] Thus, the high-resolution analysis data acquisition unit 160 obtains the data of the second real time space represented by the following formula (9). Formula (9) represents the set of high-resolution analysis data y at each time from time T = 0 to T = N. Formula (9) corresponds to the output data of the high-resolution analysis data acquisition unit 160, namely, high-resolution analysis data Da1. The high-resolution analysis data represented by formula (9) is time series data on a high-resolution grid. The high-resolution analysis data Da1 may include past, present, and future data.

[0127] [Calculation formula 9]

[0128]

[0129] Among them, y T H It is a vector field composed of all physical quantities of high-resolution analysis data. That is, y is a vector field representing a group (vector) of values ​​at each grid point in the three-dimensional space at a certain time T. And, formula (9) represents the set of vector fields of all physical quantities from time T = 0 to time T = N. In other words, formula (9) represents the entire data array (numerical array) of the four-dimensional space-time related to the high-resolution analysis data. In addition, the superscript H indicates high resolution in the spatial direction. In addition, T represents the timestamp of the high-resolution analysis data. T represents the timestamp of the interval with a short time interval. That is, T represents high resolution in the time direction (i.e., short time interval). That is, formula (9) represents data with a large number of grids (large number of elements; high resolution) in the four-dimensional space-time.

[0130] The decoder of the high-resolution analysis data acquisition unit 160 can also be implemented using a learning model obtained by machine learning such as a neural network. For example, the neural network related to the high-resolution analysis data acquisition unit 160 can also be a neural network that reflects physical symmetry. Thus, unlike the existing data assimilation method, a transformation that takes physical symmetry into account can be performed. In addition, the high-resolution analysis data acquisition unit 160 can also generate high-resolution analysis data in which the time (reference time) of the observed data is extrapolated in the time direction (time extrapolation) through the neural network. That is, the neural network of the high-resolution analysis data acquisition unit 160 can use time series data with higher resolution and higher accuracy than the predicted data in time and space as supervisory data, and learn to output high-resolution analysis data that has been time extrapolated relative to the reference time.

[0131] In addition, the decoder of the high-resolution analysis data acquisition unit 160 may also repeatedly perform linear transformation and nonlinear transformation (such as ReLU) to increase the resolution in the spatial direction. In the process of performing this processing, the high-resolution analysis data acquisition unit 160 may also use, for example, the Pixel Shuffle technology shown in non-patent document 5 to deform the data array (numerical array) to increase the resolution in the spatial direction. At this time, for example, the fusion data is set to be a numerical array with the number of elements in the time direction set to n and the number of elements in the spatial direction set to m. In this case, the high-resolution analysis data acquisition unit 160 can deform the n×m array into an n / 2×2m array. As a result, the number of elements in the spatial direction is doubled (m→2m), and thus the resolution in the spatial direction is doubled.

[0132] As described above, the high-resolution analysis data acquisition unit 160 inputs the time series data on the potential spacetime corresponding to the fused data, that is, the numerical array (data array). In addition, the high-resolution analysis data acquisition unit 160 outputs the time series data on the high-resolution grid in the second real spacetime with a large number of elements, that is, the four-dimensional numerical array. Through this configuration, the high-resolution analysis data acquisition unit 160 can efficiently perform super-resolution in the spatial direction in the real spacetime on the fused data that has been data assimilated and super-resolved in the time direction.

[0133] That is, the high-resolution analysis data acquisition unit 160 independently processes the numerical array (fused data) in the potential space-time at each moment. Therefore, in the processing of the high-resolution analysis data acquisition unit 160, super-resolution in the spatial direction can be performed without referring to the information in the time direction. Therefore, in the processing of the high-resolution analysis data acquisition unit 160, since the required computing resources (storage amount, etc.) are saved, super-resolution in three-dimensional space can be efficiently performed. Therefore, high-resolution analysis data that has been super-resolutioned in the time direction and the space direction can be efficiently obtained. In addition, when the prediction data is prediction data in the time including the observation data and the past and future time earlier than the time, the high-resolution analysis data acquisition unit 160 can obtain high-resolution analysis data in the past and future time including the observation data. Thus, while providing high-resolution analysis data, future prediction information that has been extrapolated in time can also be provided. Therefore, a service with higher added value can be provided.

[0134] In addition, by obtaining high-resolution analysis data that has been super-resolution in the time direction and the space direction, as shown below, it is possible to supplement the low-resolution prediction simulation. That is, in practical applications, the resolution of prediction simulations such as weather and sea state forecasts is often insufficient in the time direction and the space direction. For example, in aquaculture, if the water temperature rises, the activity of the fish will increase, so if the fish are fed too much fish food, it is possible that the fish will die due to lack of oxygen. Therefore, in order to adjust the feeding amount, aquaculture farmers will want to know the timing of water temperature rise. However, in the case of current ocean forecasts, the resolution in the plane direction is about 10km, which is a low resolution. In the prediction simulation with this degree of resolution, it may not be possible to predict the water temperature near the fish cage of the aquaculture farmer with high accuracy. In order to predict the water temperature of the fish cage of each aquaculture farmer with high accuracy, a fine resolution of at least 1km is required, and if possible, a fine resolution of about 100m (high resolution) is required. In contrast, in the present embodiment, since high-resolution analysis data with super-resolution in the temporal and spatial directions can be obtained, it is possible to provide information with high accuracy to practitioners who want to know pin point weather and sea state forecasts.

[0135] The low-resolution analysis data calculation unit 170 corresponds to the above-mentioned low-resolution analysis data calculation unit 70. The low-resolution analysis data calculation unit 170 uses the high-resolution analysis data Da1 to calculate the low-resolution analysis data Da2. It should be noted that the resolution of the low-resolution analysis data Da2 can also correspond to the resolution of the prediction data. The low-resolution analysis data Da2 is grid data of a data array having a lower resolution than the high-resolution analysis data Da1. In addition, the low-resolution analysis data Da2 can also be snapshot data at a certain moment.

[0136] Specifically, the low-resolution analysis data calculation unit 170 calculates the low-resolution analysis data Da2 by using a predetermined function f(y) to input the above-mentioned high-resolution analysis data y into the function f. Function f, for example, can be a function representing algebraic interpolation. Function f can be a function that performs algebraic interpolation operations such as linear interpolation. In addition, function f can also be a function defined by linear interpolation, bicubic interpolation or Lanczos method. For example, the low-resolution analysis data calculation unit 170 can calculate the low-resolution analysis data Da2 from the high-resolution analysis data Da1 by resizing (reducing the resolution) the high-resolution analysis data. Specifically, the low-resolution analysis data calculation unit 170 locally interpolates the values ​​corresponding to each grid point of the grid data using a polynomial, thereby expanding or reducing the grid data. Thus, the low-resolution analysis data Da2 is calculated.

[0137] As described above, the low-resolution analysis data Da2 is input to the simulation unit 120 and used to perform the next timing prediction simulation. In addition, the low-resolution analysis data Da2 can also be used to numerically solve the physical equation to obtain future predictions. Thus, the time evolution of the state of the prediction simulation can be performed. In this case, the low-resolution analysis data Da2 can show the state at the current moment.

[0138] As described above, the information processing device 100 involved in Embodiment 1 is configured to perform data assimilation and super-resolution in a latent spacetime in which the number of elements is reduced. Thus, it is possible to perform processing by reducing the number of elements to be processed in the calculation, that is, the amount of data, so that data assimilation and super-resolution can be performed efficiently. Therefore, observation data representing non-physical quantities and observation data that are irregular in the time / space direction can be efficiently assimilated relative to the predicted data. In addition, as described above, in the learning processing unit 110, a large amount of computing resources are required when the structure transformation unit 130, the latent spacetime mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160 are learned in a coherent manner by an end-to-end learning method. Therefore, as described above, by performing processing in the latent spacetime to reduce the amount of data of the processing object, such learning can be performed efficiently.

[0139] In addition, the information processing device 100 involved in the first embodiment is configured so that the nonlinear transformation unit 150 performs super-resolution in the time direction, and the high-resolution analysis data acquisition unit 160 performs super-resolution in the space direction. Thus, super-resolution in the time direction and the space direction can be efficiently performed. That is, in order to perform super-resolution in both the time direction and the space direction at the same time, a large amount of storage and calculation time are required. In particular, in the learning phase, in order to make the error back propagate, it is necessary to store the calculation graph and the gradient value. Therefore, a huge amount of storage and calculation time are required. In contrast, in the information processing device 100 involved in the first embodiment, in the nonlinear transformation unit 150, the dimension in the time direction is efficiently referred to, so that super-resolution is performed in the time direction on the latent space-time with a small number of elements. On the other hand, in the high-resolution analysis data acquisition unit 160, since snapshots at each moment are processed, there is no need to consider the dimension in the time direction. In addition, since snapshots at each moment are processed, these snapshots can be processed simultaneously. Therefore, the necessary calculation resources can be suppressed.

[0140] <Comparison with Comparative Example>

[0141] Next, the technology of Comparative Example 1 according to Non-Patent Document 1 and that of Embodiment 1 are compared.

[0142] As described above, the technique of Non-Patent Document 1 simply combines the super-resolution method and the data assimilation method. That is, in Non-Patent Document 1, super-resolution and data assimilation are performed independently.

[0143] Figure 7 It is a diagram for explaining the technique according to the comparative example. Figure 7 The technology involved in non-patent document 1 is described as a comparative example. In the comparative example, as shown by the white dots, physical simulation is performed on various conditions through set calculation. And, as shown by the black dots, it is assumed that there is observation data at time t1. At this time, as shown by the white triangle icon, super-resolution is performed using the physical simulation result at the time t1 to perform high-resolution prediction. This super-resolution is performed independently at the time t1 for the set calculation result, that is, each condition of each condition. Moreover, at the time t1, data assimilation is performed on the high-resolution set prediction result and the observation data, and as the final output, high-resolution prediction data obtained by assimilating the observation data is obtained, as shown by the black triangle icon. In this way, in the comparative example, set calculation is utilized. In addition, in the comparative example, super-resolution at a certain moment is performed independently from data assimilation at the same moment. And, data assimilation is performed in a high-resolution space.

[0144] Figure 8 This is a diagram for explaining super-resolution and data assimilation involved in Implementation Example 1. In Implementation Example 1, a single scene is used. That is, physical simulation is performed from a unique initial state, and a unique simulation result in time series is obtained as shown by the white dots. Figure 8 In the example above, the physical simulation results (prediction data) corresponding to the time series data t1, t2, t3, and t4 are obtained. In addition, super-resolution is not performed at a certain time, but the observation data represented by the black dots and the prediction data as the time series data are used as input, and super-resolution and data assimilation are performed simultaneously in the potential space-time with the number of elements reduced. As a result, high-resolution data as time series data is obtained, as shown in the black triangle icon. In addition, Figure 8 As shown, in the high-resolution data, the resolution in the time direction is higher than that in the physical simulation. Thus, in the first embodiment, unlike the comparative example, no ensemble calculation is performed in the simulation. In addition, in the first embodiment, unlike the comparative example, super-resolution and data assimilation are performed simultaneously in the latent space-time. In addition, in the first embodiment, unlike the comparative example, super-resolution and data assimilation are performed using time series information.

[0145] Fig. 9 This is a diagram comparing the experimental results according to Embodiment 1 and the experimental results according to the comparative example. Fig. 9is a schematic diagram of the time series of the mean absolute error MAE (Mean Absolute Error) of the vorticity field ω. Curve A represents the case where neither data assimilation nor super-resolution is performed. Curve B represents the case of the comparative example. Curve C represents the case of implementation example 1. Fig. 9 As shown, the error is the smallest in the case of Embodiment 1. Therefore, the technique according to Embodiment 1 can realize high-precision prediction.

[0146] In addition, in the experimental results of the comparative example, the calculation time for each experiment was 320 seconds. In contrast, in the experimental results of the first embodiment, the calculation time for each experiment was 61 seconds. Thus, compared with the comparative example, the technology of the first embodiment can significantly shorten the calculation time.

[0147] <Learning Method>

[0148] Next, the learning method of the learning processing unit 110 of the information processing device 100 involved in Embodiment 1 is described. As described above, the learning processing unit 110 learns the structure transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160 in a coherent manner through the end-to-end learning method. That is, the learning processing unit 110 can also learn the structure transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160 as a neural network, in which each unit is a layer. In other words, the learning processing unit 110 takes the observation data and the prediction data as input, and collectively learns the structure transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160 to output appropriate high-resolution analysis data.

[0149] First, the first learning method is explained. The first learning method is supervised learning. The supervised data (correct answer data) is, for example, high-precision and high-resolution data. In the case of a system for weather forecasting, the supervised data is, for example, high-precision and high-resolution weather data. That is, the supervised data is time series data of physical variables of the atmosphere with high precision and high resolution. The supervised data is, for example, time series data (four-dimensional numerical arrays) such as velocity fields, temperature fields, and density fields. In addition, ultra-high-resolution micro-meteorological simulation results can also be used as supervised data. The learning processing unit 110 takes the above-mentioned prediction data and observation data as input, and uses the above-mentioned supervised data to update the parameters (weights, etc.) of the neural network in the gradient direction of the error between the supervised data and the final output (high-resolution analysis data) through the error back propagation method. By repeating this process, the learning processing unit 110 learns the neural network that constitutes the structure transformation unit 130, the potential spatiotemporal mapping unit 140, the nonlinear transformation unit 150, and the high-resolution analysis data acquisition unit 160.

[0150] Next, the second learning method is described. The second learning method is unsupervised learning. In the above-mentioned supervised learning, as supervisory data, for example, time series data of physical variables of the atmosphere with high precision and high resolution is required. However, it may be difficult to obtain such data. In contrast, in unsupervised learning, such supervisory data is not required. Unsupervised learning can be performed, for example, by the variational Bayes method. In addition, unsupervised learning can also be performed by adversarial learning. Below, a learning method based on the variational Bayes method is described.

[0151] The variational Bayesian method is an approximation method that uses a simple probability distribution q to approximate the true probability distribution p. Then, the parameters of q are estimated by minimizing the KL divergence. The variational Bayesian method sets the real physical variables of the environmental state such as the atmosphere as hidden states, and based on the hidden states, gives a probability model of the input observation value or low-resolution prediction value.

[0152] As one of the implementation examples of the variational Bayes method, a lower bound of the log-likelihood ln(p(o|x)) is introduced and the lower bound is maximized. The log-likelihood ln(p(o|x)) can be transformed using the Jensen inequality as shown in the following equation (10). Wherein o corresponds to the observed data and x corresponds to the low-resolution predicted data. In the transformation process of equation (10), a latent variable y is introduced. The latent variable corresponds to the high-resolution analysis data Da1 represented by equation (9). As a result, it is possible to derive a loss function that allows the latent variable y to be inferred from the observed data o and the low-resolution prediction x.

[0153] [Calculation formula 10]

[0154]

[0155]

[0156] In formula (10), the observation value o corresponds to low-resolution or high-resolution observation data. o corresponds to the above formula (2) (or the left side of formula (3)). In addition, the latent variable y corresponds to the high-resolution analysis data Da1. It should be noted that the observation value o can also be high-precision prediction data. The variable x corresponds to the input low-resolution prediction data (formula (1)). In addition, the probability model is composed of a recognition model and a generation model. In addition, the recognition model (the part that calculates the latent variables) corresponds to the structural transformation unit 130, the potential space-time mapping unit 140, the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160 (encoder, decoder model) in the information processing device 100. The generation model will be described later.

[0157] The fourth equation in (10) (the rightmost; the final form of the deformation) corresponds to the lower bound of the log-likelihood, namely the variational lower bound (VLB). The first term of the variational lower bound, E q [p(o|x,y)] represents the reconstruction error, which corresponds to the log-likelihood of the observed data o. The second term of the variational lower bound, KL(q(y|x,o)|p(y|x)), represents the KL divergence. The KL divergence is an indicator corresponding to the distance between the distribution q and the distribution p.

[0158] The learning processing unit 110 performs learning by maximizing (increasing) the variational lower bound represented by the fourth equation of equation (10) and updating the model parameters (parameters of the neural network) by error back propagation and gradient descent. The variational lower bound corresponds to the training error. Maximizing the variational lower bound corresponds to minimizing the loss function in machine learning. In other words, increasing the variational lower bound corresponds to reducing the loss function in machine learning. At this time, learning is performed in a way that the reconstruction error and the KL divergence are balanced. Balancing the reconstruction error and the KL divergence corresponds to fusing the observed data with the predicted data in data assimilation.

[0159] It should be noted that in the maximization of the variational lower bound, a new aspect of the generalization of the minimum variance estimation has been discovered. That is, by averaging the errors of the observed value o and the predicted data x as weights (performing a weighted average), the high-resolution analysis data y can be estimated. In other words, since the observed value is usually more accurate than the predicted data, the analysis data can be estimated by making the weight of the observed value larger than the weight of the predicted data and averaging them.

[0160] Fig.101 is a diagram for explaining a method of learning the components involved in Embodiment 1 according to the variational Bayesian method. When learning is performed based on the variational Bayesian method, the learning processing unit 110 includes a sampler 112 and an observation data generating unit 114. The sampler 112 and the observation data generating unit 114 can be regarded as a generation model for generating observation values ​​(pseudo observation data Da3). The reconstruction error E can be calculated by using this generation model. q [p(o|x, y)] Specifically, p(o|x, y) is calculated by the observation data generation unit 114. P(o|x, y) represents the distribution of pseudo observation data o in a state where the prediction data x and the high-resolution analysis data y are determined.

[0161] On the other hand, as described above, the structural transformation unit 130, the potential spatiotemporal mapping unit 140, the nonlinear transformation unit 150 and the high-resolution analysis data acquisition unit 160 can be regarded as a recognition model. Through the recognition model, the KL divergence can be calculated. Specifically, q(y|x,o) in the KL divergence is calculated by the recognition model. q(y|x,o) represents the distribution of the high-resolution analysis data y when the predicted data x and the observed data o are taken as input. It should be noted that p(y|x) is a prior distribution, which is obtained by appropriate assumptions or by pre-learning.

[0162] The sampler 112 samples the probability distribution corresponding to the high-resolution analysis data generated by learning. Thus, the sampler 112 performs a substitution from the neural network to the probability distribution (probability model). That is, the neural network usually performs deterministic output, so it is difficult to perform random output such as probability distribution. Therefore, the sampler 112 combines the random numbers sampled from the Gaussian distribution with the output from the neural network (high-resolution analysis data) to simulate the probability distribution (re-parametrization trick). Thus, the error of the high-resolution analysis data can be inferred. Specifically, as shown in the following formula (11), the probability distribution is simulated. It should be noted that μ and σ are deterministic variables given by the neural network, and ε is a random number sampled from the Gaussian distribution.

[0163] [Calculation formula 11]

[0164] μ+σ×∈

[0165] …(11)

[0166] Specifically, the sampler 112 receives high-resolution analysis data Da1 (corresponding to y in formula (11)) from the high-resolution analysis data acquisition unit 160. The sampler 112 samples random numbers from a Gaussian distribution and adds noise to the high-resolution analysis data. Thus, since randomness is added to the high-resolution analysis data provided by the neural network, data that can be regarded as values ​​sampled from a probability distribution can be obtained. Therefore, the high-resolution analysis data can be expressed as a probability distribution. That is, the sampler 112 obtains sampling data when the high-resolution analysis data is expressed as a probability distribution. And the sampler 112 outputs the sampling data of the high-resolution analysis data to the observation data generation unit 114. It should be noted that in the case of expressing a more complex probability distribution, that is, in the case of generating noise from a distribution more complex than a Gaussian distribution, a mixed distribution or normalizing flows can also be used. In this case, a complex distribution can be obtained by applying a nonlinear transformation to a random variable that follows a simple probability distribution such as a Gaussian distribution.

[0167] The observation data generation unit 114 uses the sampling data of the high-resolution analysis data generated by the sampler 112 to generate pseudo observation data Da3. That is, the observation data generation unit 114 transforms the sampling data of the high-resolution analysis data into pseudo observation data Da3. The observation data generation unit 114 can generate pseudo observation data Da3 without defects in the spatiotemporal direction. The observation data generation unit 114 can also be implemented using a neural network learned in advance through machine learning. The observation data generation unit 114 can also be implemented using a neural network that reflects physical symmetry. By generating pseudo observation data Da3, unsupervised learning can be achieved.

[0168] Specifically, the observation data generating unit 114 may also perform processing opposite to that performed by the above-mentioned structurer (structure transformation unit 130) to generate pseudo observation data Da3. That is, the observation data generating unit 114 may also generate pseudo observation data Da3 using substantially the same technology as the structurer. More specifically, the observation data generating unit 114 extracts data of grid points at arbitrary times and positions from the sampling data of high-resolution analysis data as grid data, and repeats linear transformation and nonlinear transformation on the data. Thus, the observation data generating unit 114 obtains pseudo observation data Da3 in a form substantially the same as that of the observation data o obtained by the observation data obtaining unit 122. Therefore, the pseudo observation data Da3 may also be non-grid data. In addition, the pseudo observation data Da3 may also represent the numerical value of a non-physical quantity.

[0169] As described above, the learning processing unit 110 to which the variational Bayesian method is applied generates pseudo observation data Da3 from the high-resolution analysis data Da1. In the variational lower bound represented by the fourth equation of equation (10), the inferred high-resolution analysis data y is a hidden state. It should be noted that in the framework of the variational Bayesian method, the hidden state is not the final output. In the variational Bayesian method, in the learning stage, the pseudo observation data Da3 is the final output. In addition, since the high-resolution analysis data y is in a hidden state, there is no need to prepare the correct solution data corresponding to the high-resolution analysis data y in the learning stage. Therefore, there is no need to prepare high-precision and high-resolution weather data required for supervised learning.

[0170] (Hardware Configuration Example)

[0171] An example of configuring the hardware resources of the apparatus and system involved in each of the above-mentioned embodiments using one computing device (information processing device, computer) is described. However, the apparatus (information processing device) involved in each embodiment can also be implemented physically or functionally using at least two computing devices. In addition, the apparatus involved in each embodiment can be implemented as a dedicated device or by a general-purpose information processing device.

[0172] Fig.11 1000 is a block diagram schematically showing an example of a hardware configuration of a computing device that can realize the device and system involved in each embodiment. The computing device 1000 has a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF 1007 (IF: Interface). Therefore, it can be said that the device involved in each embodiment has a CPU 1001, a volatile storage device 1002, a disk 1003, a non-volatile recording medium 1004, and a communication IF 1007. The computing device 1000 can be connected to an input device 1005 and an output device 1006. The computing device 1000 can also have an input device 1005 and an output device 1006. In addition, the computing device 1000 can send and receive information with other computing devices and communication devices through the communication IF 1007.

[0173] The non-volatile recording medium 1004 is computer-readable, such as a compact disc or a digital versatile disc. In addition, the non-volatile recording medium 1004 may also be a universal serial bus (USB) memory, a solid state drive (Solid State Drive), etc. The non-volatile recording medium 1004 will also maintain the relevant programs and support portability without providing power. It should be noted that the non-volatile recording medium 1004 is not limited to the above-mentioned media. In addition, instead of the non-volatile recording medium 1004, the relevant programs may also be provided through the communication IF 1007 and the communication network.

[0174] The volatile storage device 1002 can be read by a computer and can temporarily store data. The volatile storage device 1002 is a memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0175] That is, when executing, the CPU 1001 copies the software program (computer program, hereinafter referred to as "program") stored in the disk 1003 to the volatile storage device 1002, and performs arithmetic processing. The CPU 1001 reads the data required to execute the program from the volatile storage device 1002. When display is required, the CPU 1001 displays the output result on the output device 1006. When the program is input from the outside, the CPU 1001 obtains the program from the input device 1005. The CPU 1001 interprets and executes the above-mentioned Figure 4 , Figure 6 , Fig.10 The CPU 1001 executes the processing described in the above embodiments. Figure 4 , Figure 6 , Fig.10 The functions of the components shown can be realized by the CPU 1001 executing a program stored in the magnetic disk 1003 or the volatile storage device 1002 .

[0176] That is, it can be understood that each embodiment can also be implemented by the above-mentioned program. In addition, it can be understood that each embodiment described above can also be implemented by a computer-readable non-volatile recording medium having the above-mentioned program recorded thereon.

[0177] (Variation Example)

[0178] It should be noted that the present invention is not limited to the above-mentioned embodiments, and various modifications can be made without departing from the scope of the present invention. For example, in the above-mentioned flowchart, the order of each process (step) can be appropriately changed. In addition, one or more of the multiple processes (steps) can also be omitted. For example, in Figure 5 In the flowchart of , the process of S22 may be performed before the process of S20. In addition, the process of S24 may be performed before the process of S22. In addition, the process of S70 may be omitted.

[0179] In addition, in the above embodiment, the case of performing weather forecasting is described, but the present embodiment is not limited to the case of performing weather forecasting. The present embodiment can be applied to any forecast simulation using grid data. For example, the present embodiment can also be applied to ocean forecasting. In addition, the present embodiment can also be applied to cosmic physics simulation.

[0180] In addition, in this embodiment, the dimension of "spacetime" is not limited to four dimensions consisting of three-dimensional space and one-dimensional time. The dimension of "spacetime" can be three dimensions consisting of two-dimensional space and one-dimensional time. Alternatively, the dimension of "spacetime" can also be ten dimensions, etc., which are larger than four dimensions.

[0181] In the above examples, the program includes an instruction group (or software code) for causing the computer to perform one or more functions described in the embodiment when it is read into the computer. The program can be stored in a non-temporary computer-readable medium or a physical recording medium. As an example and not limitation, a computer-readable medium or a physical storage medium includes a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory, a solid-state drive (Solid-State Drive, SSD) or other memory technology, a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a digital versatile disc (Digital Versatile Disc, DVD), a Blu-ray (Blu-ray (registered trademark)) disk or other optical disk storage, a magnetic cassette, a magnetic tape, a magnetic disk storage or other magnetic storage device. The program can be sent on a temporary computer-readable medium or a communication medium. As an example and not limitation, a temporary computer-readable medium or a communication medium includes electrical, optical, acoustic or other forms of propagation signals.

[0182] This application claims the priority based on Japanese patent application No. 2022-155991 filed on September 29, 2022, and incorporates all the contents disclosed therein into this application.

[0183] Description of Reference Numerals

[0184] 10 Information processing device

[0185] 20 Simulation Department

[0186] 22 Observation Data Acquisition Department

[0187] 24 Forecast data acquisition department

[0188] 30 Structural transformation department

[0189] 40 Latent Space-Time Mapping

[0190] 50 Nonlinear transformation unit

[0191] 60 High-resolution analysis data acquisition unit

[0192] 70 Low resolution analysis data calculation unit

[0193] 100 Information processing device

[0194] 110 Learning Processing Department

[0195] 112 Sampler

[0196] 114 Observation data generation unit

[0197] 120 Simulation Department

[0198] 122 Observation Data Acquisition Department

[0199] 124 Prediction Data Acquisition Department

[0200] 130 Structural Transformation Department

[0201] 140 Latent Space-Time Mapping

[0202] 150 Nonlinear transformation unit

[0203] 160 High-resolution analysis data acquisition unit

[0204] 170 Low-resolution analysis data calculation unit

Claims

1. An information processing device, wherein: The information processing device comprises: a structure conversion unit that converts the structure of observation data obtained as data on the observed spatiotemporal state into observation data having a structure of grid data, wherein the grid data represents numerical values ​​defined at grid points arranged at predetermined intervals in the spatiotemporal state; a potential space-time mapping unit that maps the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time from a first actual space-time to a potential space-time having a smaller number of elements than the first actual space-time, the prediction data being the grid data on the space-time obtained by simulation; a nonlinear transformation unit for performing nonlinear transformation on the mapped observation data and prediction data in the latent space-time; as well as a high-resolution analysis data acquisition unit, which maps the observed data and the predicted data, which have been subjected to the nonlinear transformation, from the potential space-time to a second actual space-time with a larger number of elements than the potential space-time and a higher resolution than the first actual space-time, thereby obtaining high-resolution analysis data having a higher resolution in space-time than the predicted data, wherein the high-resolution analysis data is grid data in space-time, The data assimilation of the observed data and the predicted data is performed by the latent spatiotemporal mapping unit and the nonlinear transformation unit.

2. The information processing device according to claim 1, wherein: The information processing device further comprises: A low-resolution analysis data calculation unit calculates low-resolution analysis data having a lower resolution than the high-resolution analysis data in terms of time and space, using the high-resolution analysis data.

3. The information processing device according to claim 1, wherein: The nonlinear transformation unit performs super-resolution in the time direction by deforming the data array of the data mapped to the latent space-time.

4. The information processing device according to claim 3, wherein: The high-resolution analysis data obtaining unit obtains the high-resolution analysis data by independently performing super-resolution in the spatial direction at each time in the temporal direction on the data on which super-resolution has been performed in the temporal direction in the latent space-time.

5. The information processing device according to claim 1, wherein: The prediction data is prediction data at a time including the time of the observation data and a time in the past earlier than the time and a time in the future later than the time. The high-resolution analysis data acquisition unit acquires high-resolution analysis data at a time in the past and future including a time of the observation data.

6. The information processing device according to claim 1, wherein: The structure transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are implemented using a learning model obtained by learning through a machine learning algorithm.

7. The information processing device according to claim 6, wherein: The structural transformation unit, the latent spatiotemporal mapping unit, the nonlinear transformation unit and the high-resolution analysis data acquisition unit are implemented using a learning completion model obtained by performing supervised learning using data having a higher spatiotemporal resolution than the prediction data as supervised data.

8. The information processing device according to claim 6, wherein: The structure transformation unit, the latent space-time mapping unit, the nonlinear transformation unit, and the high-resolution analysis data acquisition unit are implemented using a learning model obtained by learning through unsupervised learning in order to reduce a loss function.

9. An information processing method, wherein: transforming the structure of observation data obtained as data obtained by observing the state in space and time into observation data with a structure of grid data, wherein the grid data represents numerical values ​​defined at grid points arranged at predetermined intervals in space and time; Mapping the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time from a first actual space-time to a potential space-time having a smaller number of elements than the first actual space-time, wherein the prediction data is the grid data on the space-time obtained by simulation; In the latent space-time, nonlinearly transform the mapped observation data and the predicted data; as well as The observed data and the predicted data to which the nonlinear transformation has been applied are mapped from the potential space-time to a second actual space-time with a larger number of elements than the potential space-time and a higher resolution than the first actual space-time, thereby obtaining high-resolution analysis data having a higher resolution in space-time than the predicted data, wherein the high-resolution analysis data is grid data in space-time, The data assimilation of the observed data and the predicted data is performed by mapping from the first actual space to the latent space-time and performing nonlinear transformation on the observed data and the predicted data in the latent space-time.

10. A program, wherein: The program causes the computer to execute the following processing: transforming the structure of observation data obtained as data obtained by observing the state in space and time into observation data with a structure of grid data, wherein the grid data represents numerical values ​​defined at grid points arranged at predetermined intervals in space and time; Mapping the observation data transformed into the grid data and the prediction data at a time at least including the observation data and a past time earlier than the time from a first actual space-time to a potential space-time having a smaller number of elements than the first actual space-time, wherein the prediction data is the grid data on the space-time obtained by simulation; In the latent space-time, nonlinearly transform the mapped observation data and the predicted data; as well as The observed data and the predicted data to which the nonlinear transformation has been applied are mapped from the potential space-time to a second actual space-time with a larger number of elements than the potential space-time and a higher resolution than the first actual space-time, thereby obtaining high-resolution analysis data having a higher resolution in space-time than the predicted data, wherein the high-resolution analysis data is grid data in space-time, The data assimilation of the observed data and the predicted data is performed by mapping the first actual space to the latent space-time and performing nonlinear transformation on the observed data and the predicted data in the latent space-time.

Images