Method for generating Riemannian manifold data set based on geodesic interpolation technology

Abstract

The invention discloses a method for generating a Riemannian manifold data set based on a geodesic interpolation technology, and the method comprises the steps: S1, determining a task manifold, and initializing a neural network; s2, establishing a geodesic interpolation model based on a task manifold, and obtaining an interpolation point set; s3, based on the interpolation point set, calculating and optimizing two neural networks used for approaching a tangent vector field on the task manifold to obtain an optimal neural velocity field network and an optimal neural score field network; s4, establishing a data generation framework on a manifold according to the optimal neural velocity field network and the optimal neural fractional field network; and S5, generating a new priori initial value sample, and using the new priori sample as an initial value to input a data generation framework on the task manifold, simulating unknown distribution data to be generated, and obtaining sufficient new data points as a Riemannian manifold dataset. And a way is provided for further unifying the flow model and the diffusion model on the manifold in the future.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and in particular to a method for generating a Riemannian manifold data set based on geodesic interpolation technology. Background Art

[0002] Today's world is highly data-driven, with massive amounts of data being generated, processed, transformed, and applied every second. In many fields of science and engineering, it is often necessary to study data with a certain intrinsic geometric structure (for example, geographical events on the earth's surface, such as volcanic eruptions and forest fires, which grow on spherical manifolds). The living space of this type of data can be modeled using Riemann manifolds, so that the corresponding theoretical tools can be used to study manifold data in a targeted manner, which is very helpful for solving such problems.

[0003] The research field of generative models in machine learning aims to generate new samples of unknown distribution data sets through high-precision simulation algorithms, and has a very broad application space. However, when dealing with data generation problems on Riemannian manifolds, traditional machine learning generative algorithms (such as the diffusion generative model of Euclidean space) often find it difficult to capture the inherent geometric structure of the data and are not suitable for direct application to such problems; previously existing generative algorithms specially designed for Riemannian manifolds (such as the RSGM algorithm) often find it difficult to balance training speed and generation effect, and there is still room for further improvement in practical applications. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for generating a Riemannian manifold data set based on geodesic interpolation technology, which solves the problem that traditional machine learning models based on Euclidean space data sets often face the problems of unstable training and meaningless generation when processing manifold data sets due to insufficient consideration of the constraints of the geometric structure.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a method for generating a Riemann manifold data set based on geodesic interpolation technology, comprising:

[0006] S1. Determine the task manifold and initialize the neural network;

[0007] S2. Establish a geodesic interpolation model based on the task manifold and obtain a set of interpolation points;

[0008] S3. Based on the interpolation point set, two neural networks for approximating the tangent vector field on the task manifold are calculated and optimized to obtain the optimal neural velocity field network and the optimal neural score field network;

[0009] S4, establishing a data generation framework on the manifold based on the optimal neural velocity field network and the optimal neural score field network;

[0010] S5. Generate new prior initial value samples, and use the new prior samples as the initial value input into the data generation framework on the task manifold, simulate the unknown distribution data to be generated, and obtain sufficient new data points as the Riemann manifold data set.

[0011] Further: S1 includes:

[0012] S11. Confirm the task manifold and obtain the Riemann manifold M;

[0013] S12. Obtain prior data to obtain a prior data set {x0} containing N sample data points i ;

[0014] Where i represents the i-th sample, 1≤i≤N;

[0015] S13, obtain the target data, and obtain the target data set {x1} containing N sample data points i ;

[0016] S14, obtain the time intermediate value point, and obtain a data set {t} containing N points i ;

[0017] S15. Based on the prior data set {x0} i , target dataset {x1} i And the dataset {t} i , determine the input and output shapes of the neural network, and initialize the parameters to obtain randomly initialized fully connected neural networks vN(x, t) and sN(x, t) with spatiotemporal information as input.

[0018] Further: S2 includes:

[0019] S21. Establish a geodesic interpolation model I(t,x0,x1) based on the task manifold, which is expressed as:

[0020]

[0021] S22, couple the interpolation tuples to obtain a triple set {x0, x1, t};

[0022] S23. Calculate the geodesic interpolation point x t =I(t), and obtain the interpolation point set {x t ,t}.

[0023] Further: S3 includes:

[0024] S31, randomly select the interpolation point set {x t,t} are input into the fully connected neural network vN(x,t), the neural velocity field is calculated, and the output V(x,t) of the fully connected neural network vN(x,t) is obtained;

[0025] S32. According to the output V(x, t) of the fully connected neural network vN(x, t), the neural velocity field loss function Loss_V is calculated, and its expression is:

[0026] Loss_V=E[0.5(V(x,t)) 2 -partial_t(I(t))·(V(x,t)]

[0027] Among them, E(.) represents mathematical expectation, partial_t(.) represents time derivative;

[0028] S33. According to the value of the neural velocity field loss function Loss_V, the Adam algorithm is used to optimize the parameters of the fully connected neural network vN(x, t);

[0029] S34, repeatedly executing S31-S33 until the preset number of iterations is reached to obtain the optimal neural velocity field network vN*;

[0030] S35, randomly select the interpolation point set {x t ,t} are input into the fully connected neural network vN(x,t), the neural score field is calculated, and the output S(x,t) of the fully connected neural network sN(x,t) is obtained;

[0031] S36. According to the output S(x, t) of the fully connected neural network sN(x, t), the neural fractional field loss function Loss_S is calculated, and its expression is:

[0032] Loss_S=E[0.5(S(x,t)) 2 +div(V(x,t)]

[0033] Where div(.) represents the divergence operator on the Riemann manifold M;

[0034] S37. According to the value of the neural fractional field loss function Loss_S, the Adam algorithm is used to optimize the parameters of the fully connected neural network sN(x, t);

[0035] S38. Repeat S35-S37 until the preset number of iterations is reached to obtain the optimal neural fractional field network sN*.

[0036] Further: S4 includes:

[0037] S41, determining a generation strategy to obtain a univariate function e(t) that changes with time t;

[0038] S42. Calculate the neural space-time tangent vector field V on the Riemann manifold M based on the optimal neural velocity field network vN* and the optimal neural score field network sN*. t , whose expression is:

[0039] V t =vN*+e·sN*

[0040] Among them, e is a function that changes continuously and smoothly with time t;

[0041] S43, according to the univariate function e(t) and the neural space-time tangent vector field V on the Riemann manifold M t , establish a data generation framework on the manifold.

[0042] Further: In S43, the expression of the data generation framework on the manifold is:

[0043]

[0044] where dBt represents the standard Brownian motion on the manifold.

[0045] Further: S5 includes:

[0046] S51, generate new priori initial value samples, and obtain the priori data set {x0} i The data point x'0 of the distribution;

[0047] S52, starting from the data point x'0, the data generation framework on the numerical simulation manifold is obtained, and the data point x at time t' is obtained. t ';

[0048] S53, step from the last moment t' to t'=1, and obtain the target data set {x1} i The data point x1' of the distribution;

[0049] S54. Repeat S51-S53 until sufficient data points x1' are obtained as the Riemann manifold data set.

[0050] The beneficial effects of the present invention are:

[0051] 1. The present invention can achieve accurate modeling and rapid generation of high-quality manifold data, and the generation performance on the global geographic data set is optimal in the sense of likelihood. At the same time, in terms of theory, it provides a way to further unify the flow model and diffusion model on the manifold in the future;

[0052] 2. The present invention can theoretically prove that the algorithm can generate unbiased sampling of the target distribution in a limited time, and the computing power required in the training stage and the generation stage is relatively small. Taking the commonly used global geographic data set as an example, the training process of the present invention on the geographic data set can converge after 50,000 iterations, and the training time can be controlled to about 1 hour. Effective cost and energy cost control can be achieved when deployed on a large scale;

[0053] 3. The model does not depend on the prior distribution of the problem and can generate data from any distribution as a starting point, which ensures the flexibility and versatility of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Flowchart of the method for generating a Riemannian manifold dataset based on geodesic interpolation technology. DETAILED DESCRIPTION

[0055] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0056] like Figure 1 As shown, in one embodiment of the present invention, a method for generating a Riemannian manifold data set based on a geodesic interpolation technique is provided, which is applied to geography, and includes:

[0057] S1. Determine the task manifold and initialize the neural network;

[0058] S2. Establish a geodesic interpolation model based on the task manifold and obtain a set of interpolation points;

[0059] S3. Based on the interpolation point set, two neural networks for approximating the tangent vector field on the task manifold are calculated and optimized to obtain the optimal neural velocity field network and the optimal neural score field network;

[0060] S4, establishing a data generation framework on the manifold based on the optimal neural velocity field network and the optimal neural score field network;

[0061] S5. Generate new prior initial value samples, and use the new prior samples as the initial value input into the data generation framework on the task manifold, simulate the unknown distribution data to be generated, and obtain sufficient new data points as the Riemann manifold data set.

[0062] The N-1-dimensional unit sphere in N-dimensional space is one of the most typical Riemann manifolds. Naturally, we can model the earth we live on as a three-dimensional space R 3The two-dimensional unit sphere S in 2 , and thus any spatial point on the surface can be assigned a unique spherical coordinate.

[0063] For some geographical events, including but not limited to volcanic eruptions, forest fires, floods, and earthquakes, we can regard the location where they occur as a point on the sphere S 2 The random variable with values ​​on satisfies a certain unknown probability distribution, so we can use the geodesic interpolation model to model this unknown distribution and extract new samples from it, that is, to predict the locations where similar geographical events may occur, so as to guide us to prevent natural disasters more specifically.

[0064] Specifically, S1 includes:

[0065] S11. Confirm the task manifold and obtain the Riemann manifold M;

[0066] S12. Obtain prior data to obtain a prior data set {x0} containing N sample data points i ;

[0067] Where i represents the i-th sample, 1≤i≤N;

[0068] S13, obtain the target data, and obtain the target data set {x1} containing N sample data points i ;

[0069] S14, obtain the time intermediate value point, and obtain a data set {t} containing N points i ;

[0070] S15. Based on the prior data set {x0} i , target dataset {x1} i And the dataset {t} i , determine the input and output shapes of the neural network, and initialize the parameters to obtain randomly initialized fully connected neural networks vN(x, t) and sN(x, t) with spatiotemporal information as input.

[0071] In this embodiment, the Riemann manifold M is a three-dimensional space R 3 The unit sphere S in 2 , the prior data set {x0} i Obey S 2 uniform distribution on the target dataset {x1} i Obey S 2 von-Mises-Fisher distribution on the dataset {t} i It obeys a uniform distribution on [0,1]; the dimension of the neural network input is 4, including a spatial dimension of 3 and a temporal dimension of 1, and the neural network output is a spatial dimension of 3.

[0072] The data generation problem on the Riemann manifold M can be summarized as generating a new data point x1 for the target data set S, where all data points in S are M-valued random variables that obey an unknown probability density function p1 on M, so that x1 is also an M-valued random variable that obeys the probability density function p1.

[0073] To do this, use an M-valued random process x t Establish a mathematical model to describe the transformation process of data, random process x t At t = 0, the random variable x0 obeys the prior data distribution p0, and at t = 1, the random variable x1 to be calculated obeys the (usually unknown) data distribution p1. In the process of t continuously changing from 0 to 1, x t The position on the manifold M also changes continuously and smoothly.

[0074] At any time t, x t Randomly arrive at different positions on M, and have a random instantaneous speed v t , the distribution law of this randomness can be expressed by the probability distribution density function p on the manifold t Description, and p t As time t changes, the transmission equation on M can be used to calculate partial_t(p t )+div(v t *p t )=0, for this transport equation, by t The linear transformation V t =v t +e·s t calculate;

[0075] Among them, e is an artificially determined function that changes continuously and smoothly with time t, s t is the score function at time t; V t is the neural spacetime tangent vector field on the Riemannian manifold M;

[0076] We can also transform the transmission equation on M into partial_t(p t )+div(v t *p t )=0 is transformed into a Fokker-Planck equation: partial_t(p t )+div(V t *p t )=eD(p t );

[0077] Among them, D(p t ) represents the function p tCalculate the value of its Laplace-Beltrami operator after mapping, and then this Fokker-Planck equation corresponds to an M-valued stochastic differential equation dBt represents the standard Brownian motion on the manifold. For each time t, the solution function p of the Fokker-Planck equation is t is the stochastic differential equation solution process X t The probability density function of .

[0078] Specifically, S2 includes:

[0079] S21. Establish a geodesic interpolation model I(t,x0,x1) based on the task manifold, which is expressed as:

[0080]

[0081] S22, couple the interpolation tuples to obtain a triple set {x0, x1, t};

[0082] S23. Calculate the geodesic interpolation point x t =I(t), and obtain the interpolation point set {x t ,t}.

[0083] Specifically, S3 includes:

[0084] S31, randomly select the interpolation point set {x t ,t} are input into the fully connected neural network vN(x,t), the neural velocity field is calculated, and the output V(x,t) of the fully connected neural network vN(x,t) is obtained;

[0085] S32. According to the output V(x, t) of the fully connected neural network vN(x, t), the neural velocity field loss function Loss_V is calculated, and its expression is:

[0086] Loss_V=E[0.5(V(x,t)) 2 -partial_t(I(t))·(V(x,t)]

[0087] Among them, E(.) represents mathematical expectation, partial_t(.) represents time derivative;

[0088] S33. According to the value of the neural velocity field loss function Loss_V, the Adam algorithm is used to optimize the parameters of the fully connected neural network vN(x, t);

[0089] S34, repeatedly executing S31-S33 until the preset number of iterations is reached to obtain the optimal neural velocity field network vN*;

[0090] S35, randomly select the interpolation point set {x t,t} are input into the fully connected neural network vN(x,t), the neural score field is calculated, and the output S(x,t) of the fully connected neural network sN(x,t) is obtained;

[0091] S36. According to the output S(x, t) of the fully connected neural network sN(x, t), the neural fractional field loss function Loss_S is calculated, and its expression is:

[0092] Loss_S=E[0.5(S(x,t)) 2 +div(V(x,t)]

[0093] Where div(.) represents the divergence operator on the Riemann manifold M;

[0094] S37. According to the value of the neural fractional field loss function Loss_S, the Adam algorithm is used to optimize the parameters of the fully connected neural network sN(x, t);

[0095] S38. Repeat S35-S37 until the preset number of iterations is reached to obtain the optimal neural fractional field network sN*.

[0096] Specifically, S4 includes:

[0097] S41, determining a generation strategy to obtain a univariate function e(t) that changes with time t;

[0098] S42. Calculate the neural space-time tangent vector field V on the Riemann manifold M based on the optimal neural velocity field network vN* and the optimal neural score field network sN*. t , whose expression is:

[0099] V t =vN*+e·sN*

[0100] Among them, e is a function that changes continuously and smoothly with time t and is determined artificially;

[0101] S43, according to the univariate function e(t) and the neural space-time tangent vector field V on the Riemann manifold M t , establish a data generation framework on the manifold, and its expression is:

[0102]

[0103] Specifically, S5 includes:

[0104] S51, generate new priori initial value samples, and obtain the priori data set {x0} i The data point x'0 of the distribution;

[0105] S52, starting from the data point x'0, the data generation framework on the numerical simulation manifold is obtained, and the data point x at time t' is obtained.t ';

[0106] S53, step from the last moment t' to t'=1, and obtain the target data set {x1} i The data point x1' of the distribution;

[0107] S54. Repeat S51-S53 until sufficient data points x1' are obtained as the Riemann manifold data set.

[0108] In one embodiment of the present application, a simulation experiment was conducted. In the simulation experiment, global earthquake data and global volcanic eruption data collected by the National Oceanic and Atmospheric Administration (NOAA), global flood data collected by the University of Colorado, and global fire data collected by the National Aeronautics and Space Administration (NASA) were used. Based on these four types of geographic data sets, corresponding geodesic interpolation models were trained respectively, and then the trained models were used to generate new data points that conform to the distribution of the training data:

[0109] In the simulation experiment, we use negative log-likelihood (NLL) as an indicator to characterize the quality of model generation. Likelihood is a statistic that measures the probability of a random sample. The larger the likelihood value at point x calculated based on the probability density function p(x, t), the more the random sample conforms to the given probability density function p(x, t). From the perspective of calculation accuracy, the negative logarithm of the obtained likelihood value is calculated as the final metric. From the basic properties of the logarithmic function, it can be seen that a lower negative log-likelihood value represents better data generation quality. Experiments show that compared with similar models, our model has significant improvements in both training speed and generation quality. A comparison of the geodesic interpolation model proposed in this application and the diffusion model on manifold (Riemannian Score-based Generative Modelling, RSGM) is shown in Table 1:

[0110] Table 1 Comparison of negative log-likelihood index between geodesic interpolation model and diffusion model

[0111] NLL volcano earthquake flood Fire RSGM Model -4.92 -0.19 0.45 -1.33 Geodesic interpolation model -12.37 -0.70 0.17 -1.37

[0112] It should be emphasized that NLL decreases as the degree of model optimization increases. To achieve the NLL value shown in the table, the RSGM model requires 600,000 training iterations, while our geodesic interpolation model only requires 60,000 training iterations, achieving a speedup effect of more than 10 times, thereby saving a lot of computing resources.

[0113] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a Riemannian manifold dataset based on geodesic interpolation technology, applied to geography, characterized in that: include: S1. Determine the task manifold and initialize the neural network; S2. Establish a geodesic interpolation model based on the task manifold and obtain a set of interpolation points; S3. Based on the interpolation point set, two neural networks for approximating the tangent vector field on the task manifold are calculated and optimized to obtain the optimal neural velocity field network and the optimal neural score field network; S4, establishing a data generation framework on the manifold based on the optimal neural velocity field network and the optimal neural score field network; S5. Generate new prior initial value samples, and use the new prior samples as the initial value input into the data generation framework on the task manifold, simulate the unknown distribution data to be generated, and obtain sufficient new data points as the Riemann manifold data set.

2. The method for generating a Riemannian manifold data set based on geodesic interpolation technology according to claim 1, characterized in that: S1 includes: S11. Confirm the task manifold and obtain the Riemann manifold M; S12. Obtain prior data to obtain a prior data set {x0} containing N sample data points i ; Where i represents the i-th sample, 1≤i≤N; S13, obtain the target data, and obtain the target data set {x1} containing N sample data points i ; S14, obtain the time intermediate value point, and obtain a data set {t} containing N points i ; S15. Based on the prior data set {x0} i , target dataset {x1} i And the dataset {t} i , determine the input and output shapes of the neural network, and initialize the parameters to obtain randomly initialized fully connected neural networks vN(x, t) and sN(x, t) with spatiotemporal information as input.

3. The method for generating a Riemannian manifold data set based on geodesic interpolation technology according to claim 2, characterized in that S2 include: S21. Establish a geodesic interpolation model I(t,x0,x1) based on the task manifold, which is expressed as: S22, couple the interpolation tuples to obtain the triple set {x0,x1,t} i ; S23. Calculate the geodesic interpolation point x t =I(t), and obtain the interpolation point set {x t ,t} i .

4. The method for generating a Riemannian manifold data set based on geodesic interpolation technology according to claim 3, characterized in that S3 include: S31, randomly select the interpolation point set {x t ,t} are input into the fully connected neural network vN(x,t), the neural velocity field is calculated, and the output V(x,t) of the fully connected neural network vN(x,t) is obtained; S32. According to the output V(x, t) of the fully connected neural network vN(x, t), the neural velocity field loss function Loss_V is calculated, and its expression is: Loss_V=E[0.5(V(x,t)) 2 -partial_t(I(t))·(V(x,t)] Among them, E(.) represents mathematical expectation, partial_t(.) represents time derivative; S33. According to the value of the neural velocity field loss function Loss_V, the Adam algorithm is used to optimize the parameters of the fully connected neural network vN(x, t); S34, repeatedly executing S31-S33 until the preset number of iterations is reached to obtain the optimal neural velocity field network vN*; S35, randomly select the interpolation point set {x t ,t} are input into the fully connected neural network vN(x,t), the neural score field is calculated, and the output S(x,t) of the fully connected neural network sN(x,t) is obtained; S36. According to the output S(x, t) of the fully connected neural network sN(x, t), the neural fractional field loss function Loss_S is calculated, and its expression is: Loss_S=E[0.5(S(x,t)) 2 +div(V(x,t)] Where div(.) represents the divergence operator on the Riemann manifold M; S37. According to the value of the neural fractional field loss function Loss_S, the Adam algorithm is used to optimize the parameters of the fully connected neural network sN(x, t); S38. Repeat S35-S37 until the preset number of iterations is reached to obtain the optimal neural fractional field network sN*.

5. The method for generating a Riemannian manifold data set based on geodesic interpolation technology according to claim 4, characterized in that S4 include: S41, determining a generation strategy to obtain a univariate function e(t) that changes with time t; S42. Calculate the neural space-time tangent vector field V on the Riemann manifold M based on the optimal neural velocity field network vN* and the optimal neural score field network sN*. t , whose expression is: V t =vN*+e·sN* Among them, e is a function that changes continuously and smoothly with time t; S43, according to the univariate function e(t) and the neural space-time tangent vector field V on the Riemann manifold M t , establish a data generation framework on the manifold.

6. The method for generating a Riemannian manifold data set based on geodesic interpolation technology according to claim 5, characterized in that: In S43, the expression of the data generation framework on the manifold is: where dBt represents the standard Brownian motion on the manifold.

7. The method for generating a Riemannian manifold data set based on geodesic interpolation technology according to claim 1, characterized in that S5 include: S51, generate new priori initial value samples, and obtain the priori data set {x0} i The data point x'0 of the distribution; S52, starting from the data point x'0, the data generation framework on the numerical simulation manifold is obtained, and the data point x at time t' is obtained. t '; S53, step from the last moment t' to t'=1, and obtain the target data set {x1} i The data point x1' of the distribution; S54. Repeat S51-S53 until sufficient data points x1' are obtained as the Riemann manifold data set.

Images