A training method and device of a manifold geodesic calculation model, equipment and medium

Abstract

The application provides a manifold geodesic calculation model training method and device, equipment and medium, relates to the model training technical field, through the first stage training, the weight of each weight value of the mean function of the variational autoencoder to be trained is obtained, and through the second stage training, the weight of each weight value of the difference function of the variational autoencoder trained through the first stage is obtained, and after determining the weight of each weight value of the mean function and the weight of each weight value of the variance function, the parameter value of the randomization manifold space is learned, and data is generated through the parameter value of the randomization manifold space. The variance function after training can accurately reflect the original data sample space relationship, and new data can be effectively generated based on the potential variability of the original data.

Description

Technical Field

[0001] This application relates to the field of model training technology, and in particular to a training method, apparatus, equipment and medium for a manifold geodesic calculation model. Background Technology

[0002] The VAE (Variational Auto-Encoder) model generates a network that constructs random variables and a randomized space. If the reconstructed original data sample contains a manifold structure, the generated sample in the space is a special manifold with random information, which is called a randomized manifold.

[0003] A randomized manifold will have different randomization components, i.e., uncertainties, at different locations in its space. If the known sample size of a certain region in the original data space is large, the random component in the corresponding generated region will be weak; conversely, if the known sample size of a certain region in the original data space is small, the generated data will be highly random, i.e., noisy. From the VAE encoder network output formula f(z) = μ(z) + σ(z)·ε, it can be seen that the VAE encoder network outputs a mean function μ(z) and a variance function σ(z), defining the probability distribution of each sample in the latent space. The amount of noise introduced by the ε term depends on the result of the variance function.

[0004] Current variance functions cannot accurately reflect the spatial relationships of the original input data samples, and therefore cannot effectively generate new data based on the potential variability of the original input data. Summary of the Invention

[0005] In view of this, embodiments of this application provide a training method, apparatus, device, and medium for a manifold geodesic calculation model, in order to overcome the above problems or at least partially solve the above problems.

[0006] The first aspect of this application provides a method, apparatus, device, and medium for training a manifold geodesic calculation model, the method comprising:

[0007] Based on the variance function and the mean function, a variational autoencoder to be trained is constructed;

[0008] With the output value of the variance function set to a constant, the variational autoencoder to be trained is trained in the first stage. At the end of the first stage of training, the weights of the mean function are obtained.

[0009] While keeping the weights of the mean function unchanged, the variational autoencoder trained in the first stage is trained in the second stage. At the end of the second stage of training, the weights of the variance function are obtained.

[0010] While keeping the weights of the obtained mean function and variance function unchanged, the parameter values ​​of the randomized manifold space are learned, and the parameter values ​​of the randomized manifold space are used for data generation.

[0011] Optionally, the step of learning the parameter values ​​of the randomized manifold space while keeping the weights of the obtained mean function and variance function unchanged includes:

[0012] While keeping the weights of the mean function and the variance function unchanged, the parameter set of the cubic spline curve is updated using the parameter set optimization formula.

[0013] In the process of updating the parameter set of the cubic spline curve, a target parameter set is determined. The target parameter set is the parameter set that enables the cubic spline curve to reach the optimal state in the latent variable space.

[0014] Based on the target parameter set, obtain the parameter set expression for the target cubic spline curve;

[0015] The method further includes:

[0016] After the second phase of training is completed, the reconstructed network is obtained based on the weights of the mean function and the weights of the variance function.

[0017] Based on the reconstructed network, the target cubic spline curve is mapped into the generation space so that the distance between two given points is connected by the shortest distance through the target cubic spline curve in the generation space, and the relationship between the two given points is represented to complete the reconstruction and generation of the data.

[0018] Optionally, the parameter set optimization formula is:

[0019]

[0020] The method further includes: [the following steps are taken] for S * The derivative is calculated, and the stochastic gradient descent method is used to calculate the update amount of the parameter set, thus obtaining the target parameter set.

[0021] Optionally, the step of performing a second stage of training on the variational autoencoder trained in the first stage while keeping the weights of the mean function unchanged, and obtaining the weights of the variance function at the end of the second stage of training, includes:

[0022] Establish a weighted Gaussian mixture model;

[0023] The variance function is characterized based on the weighted Gaussian mixture model.

[0024] The weighted Gaussian mixture model is as follows: K represents the number of Gaussian components in the weighted Gaussian mixture model, and c i Represents the sum of the mean and sum of the i-th components. i Let ω represent the covariance of the i-th component. i W represents the mixing coefficient. g ∈R D Let z represent the weight vector, and z represent the latent variable.

[0025] Optionally, the method further includes:

[0026] The weight vector W for training the weighted Gaussian mixture model g ;

[0027] In the weight vector W g After training is completed, based on the weight vector W obtained after training... g Then update the weights of the variance function.

[0028] Optionally, the method further includes:

[0029] The covariance in the weighted Gaussian mixture model is set to a diagonal matrix.

[0030] via log(σ) 2 (z) is used to obtain the output value of the variance function.

[0031] Optionally, the step of performing a first-stage training on the variational autoencoder to be trained while setting the output value of the variance function to a constant, and obtaining the weights of the mean function at the end of the first-stage training, includes:

[0032] Set the output value of the variance function to a constant;

[0033] Obtain the output value of the mean function, and based on the output value of the mean function, determine that all weights of the mean function have been updated.

[0034] A second aspect of this application provides a training apparatus for a manifold geodesic calculation model, the apparatus comprising:

[0035] The building module is used to construct the variational autoencoder to be trained based on the variance function and the mean function;

[0036] The first-stage training module is used to perform the first-stage training on the variational autoencoder to be trained with the output value of the variance function set to a constant, and to obtain the weights of the mean function at the end of the first-stage training.

[0037] The second-stage training module is used to perform a second-stage training on the variational autoencoder that has been trained in the first stage while keeping the weights of the mean function unchanged. At the end of the second-stage training, the weights of the variance function are obtained.

[0038] The learning module is used to learn the parameter values ​​of the randomized manifold space while keeping the weights of the obtained mean function and variance function unchanged. The parameter values ​​of the randomized manifold space are used for data generation.

[0039] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the training method for the manifold geodesic calculation model as described in the first aspect.

[0040] A fourth aspect of this application provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the training method for the manifold geodesic calculation model as described in the first aspect.

[0041] The beneficial effects of this application are:

[0042] This application provides a training method for a manifold geodesic calculation model. The method includes: constructing a variational autoencoder to be trained based on a variance function and a mean function; performing a first-stage training on the variational autoencoder to be trained while setting the output value of the variance function to a constant, and obtaining the weights of the mean function at the end of the first-stage training; performing a second-stage training on the variational autoencoder after the first-stage training while keeping the weights of the mean function unchanged, and obtaining the weights of the variance function at the end of the second-stage training; and learning the parameter values ​​of a randomized manifold space while keeping the obtained weights of the mean function and the variance function unchanged, wherein the parameter values ​​of the randomized manifold space are used for data generation. The first training stage obtains the weights of the mean function of the variational autoencoder to be trained. The second training stage obtains the weights of the difference function of the variational autoencoder trained in the first stage. After determining the weights of the mean and variance functions, the parameters of the randomized manifold space are learned, and data is generated using these parameters. This ensures that the trained variance function accurately reflects the spatial relationships of the original data samples and effectively generates new data based on the potential variability of the original data. Attached Figure Description

[0043] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0044] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This application provides a method for training a manifold geodesic calculation model according to one embodiment.

[0046] Figure 2 This is a schematic diagram of the training stages and structure of a training method for a manifold geodesic calculation model provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of a stochastic optimization algorithm for training the parameter set S of a cubic spline curve C in a training method for a manifold geodesic calculation model provided in an embodiment of this application.

[0048] Figure 4 This application provides a training device for a manifold geodesic calculation model. Detailed Implementation

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Instead of compressing each sample to a single point, the encoder network of a Variational Autoencoder (VAE) maps each sample to a standard normal distribution. This means the encoder network outputs a mean vector and a covariance matrix, defining the probability distribution of each sample in the latent space. By mapping samples to probability distributions, VAEs are able to capture the latent variability of the data. This means VAEs can generate more diverse and realistic data than traditional autoencoders. The latent space of a VAE is designed to be a continuous distribution. This means we can sample and interpolate the latent representation to generate new, realistic data points. This makes VAEs effective generative models because they can generate new data points similar to the training data.

[0052] Suppose we have a vector of latent variables z in a high-dimensional space, which can be easily sampled according to some probability density function P(z) defined in the high-dimensional space. Given a function f(z; θ), parameterized by a vector θ in some space Θ, where f: Z × Θ → X. f is deterministic; therefore, if the latent variable z is random and θ is fixed, then f(z; θ) is a random variable in the space χ. Maximizing the probability of each X in the training set throughout the generation process is: P(X) = ∫P(X|z; θ)P(z)dz

[0053] In VAEs, the output distribution is typically chosen to be a Gaussian distribution, i.e., P(X|z;θ)=N(X|f(z;θ),σ 2 *I), that is, its mean is f(z; θ), and its covariance is equal to the identity matrix I multiplied by a scalar σ.

[0054] Therefore, a new function is needed that takes the value of X as input and provides the distribution of z values ​​that may produce X. The aim is to make the space of latent variable z values ​​generated by Q smaller than the space generated under the prior P(z).

[0055] First, define the Kullback-Leibler divergence (KL divergence) between P(z|X) and Q(z): for any Q: KL(Q(z)|P(z|X))=E z～Q [logQ(z)-logP(z|X)].

[0056] Furthermore, applying Bayes' theorem to P(z|X), we obtain: KL(Q(z)|P(z|X))=E z～Q [logQ(z)-logP(X|z)-logP(z)]+logP(X).

[0057] Here, logP(X) does not depend on the latent variable z and is outside the expectation. Reversing both sides and rearranging them, we get:

[0058] logP(X)-KL(Q(z)|P(z|X))=E z～Q [logP(X|z)]-D[Q(z)||P(z)]

[0059] At this point, X is fixed, and Q can be any distribution. Since we are interested in inferring P(X), it makes sense to construct a Q that depends on X in order to minimize KL(Q(z)|P(z|X)).

[0060] logP(X)-KL(Q(z|X)|P(z|X))=E z～Q [logP(X|z)]-KL(Q(z|X)|P(z))

[0061] This equation is the core of the variational autoencoder. The left side has the quantity we want to maximize: logP(X), and the right side can be optimized by stochastic gradient descent to make the correct choice of Q.

[0062] Usually chosen The KL divergence between two multivariate Gaussian distributions can be calculated using a closed-form method:

[0063]

[0064] In our example, it is simplified to

[0065]

[0066] The first term on the right side of the formula, E z～Q [logP(X|z)], as the standard for stochastic gradient descent, takes a sample z and considers z as E. z～Q The approximate value of [logP(X|z)].

[0067] The complete equation we want to optimize is: E X～D [logP(X)-KL(Q(z|X)|P(z|X))]= E X～D [E z～Q [logP(X|z)]-KL(Q(z|X)|P(z))]

[0068] We can sample individual X values ​​and individual z values ​​from the distribution Q(z|X) and compute the gradient:

[0069] logP(X|z)-KL(Q(z|X)|P(z))

[0070] Since latent variables need to follow a certain distribution, if this distribution is generated directly, the computation graphs of the encoder and decoder will be broken, making it impossible for the backpropagation method to optimize the encoder and decoder.

[0071] To address this issue, a reparameterization technique was employed. In the actual model, the encoder does not directly generate latent variables, but instead outputs parameters that the latent variables follow, and then uses these parameters to generate latent variables that follow a certain distribution.

[0072] Given the mean μ(X) and covariance Σ(X) of Q(z|X), a linear transformation z = μ(X) + Σ is performed by sampling ε ~ N(0,I). 1 / 2 We use (X)*ε to sample from N(μ(X),Σ(X)). The actual gradient equation is:

[0073] E X～D [E ε～N(0,I) [logP(X|z=μ(X)+Σ 1 / 2 [(X)*ε)]-KL(Q(z|X)|P(z))]

[0074] The two networks exhibit identical feedforward behavior, but backpropagation can only be applied to… Figure 2 The network.

[0075] During testing, when new samples need to be generated, simply input the values ​​of z ~ N(0, I) into the decoder. In other words, the "encoder," including multiplication and addition operations that would alter the z distribution, has been removed.

[0076] A randomized manifold will have different randomization components, or uncertainties, at different locations in its space. Intuitively, if the known sample size of a certain region in the original data space is large, the random component in the corresponding generated region will be weak; conversely, if the known sample size of a certain region in the original data space is small, the randomness of the generated data will be strong, i.e., high noise. From f(z)=μ(z)+σ(z)·ε, we know that the amount of noise term ε introduced depends on the variance function σ. 2 The result of (·); therefore, how to establish a network structure that can accurately reflect the spatial relationships of the original data samples to characterize the variance function σ 2 (·) is crucial.

[0077] Therefore, we expect to achieve the following objective: variance function σ 2 (·) In regions with high data density in the latent variable space, its value is small, and vice versa. In related technologies, there is a method using RBF networks to calculate the reciprocal of the variance function to achieve the desired effect. Through our simulation experiments and analysis of the data, we found that RBF networks have certain limitations, and the initial center point has a significant impact.

[0078] Based on the above-mentioned technical problems, this application proposes a training method, apparatus, equipment, and medium for a manifold geodesic calculation model. Figure 1This application provides a method for training a manifold geodesic calculation model according to one embodiment. Figure 2 This is a schematic diagram of the training stages and structure of a training method for a manifold geodesic calculation model provided in an embodiment of this application, as shown below. Figure 1 and Figure 2 As shown in the embodiment of this application, a training method for a manifold geodesic calculation model is provided, the method comprising:

[0079] In step S101, a variational autoencoder to be trained is constructed based on the variance function and the mean function;

[0080] In step S102, with the output value of the variance function set to a constant, the variational autoencoder to be trained is trained in the first stage. At the end of the first stage of training, the weights of the mean function are obtained.

[0081] In step S103, while keeping the weights of the mean function unchanged, the variational autoencoder trained in the first stage is trained in the second stage. At the end of the second stage of training, the weights of the variance function are obtained.

[0082] In step S104, while keeping the weights of the obtained mean function and variance function unchanged, the parameter values ​​of the randomized manifold space are learned, and the parameter values ​​of the randomized manifold space are used for data generation.

[0083] First, in step S101, a variational autoencoder to be trained is constructed based on the variance function and the mean function.

[0084] In this embodiment, to enable the data generation model to learn the latent representation of the data and generate new samples with a similar distribution, a variational autoencoder (VAE) utilizes the concepts of encoding and decoding. These are used to map input data to the latent space and latent variables to the visible space, respectively. The model is trained by maximizing the marginal probability of the observed data. The variance function and mean function define the loss function of the VAE, with the variance and mean functions playing crucial roles. The variance function measures the dispersion of the latent variables, while the mean function measures their mean. By optimizing the variance and mean functions, the VAE can learn a representation with a suitable distribution in the latent space. Therefore, before training the VAE, the variance and mean functions need to be constructed, i.e., initialized. Constructing the VAE using the variance and mean functions enables the model to learn the latent representation of the input data, facilitating subsequent optimization of the variance and mean functions to achieve reasonable modeling of the latent space.

[0085] Furthermore, in step S102, with the output value of the variance function set to a constant, the variational autoencoder to be trained is subjected to a first stage of training, and at the end of the first stage of training, the weights of the mean function are obtained.

[0086] In this embodiment of the application, the variational autoencoder to be trained is trained in a multi-stage training manner, such as... Figure 2 As shown, the encoding network consists of three parts: H-enc represents the number of hidden layer neural network structures through which the input data passes; M-enc and S-enc represent the mean function network and variance function network that constitute the latent variable space during the VAE model encoding process, respectively. The generator network also consists of three parts: H-dec is the number of hidden layer neural network structures through which the latent variable data, as input, passes; M-dec and S-dec represent the mean function network and variance function network when generating the randomization space, respectively.

[0087] The output value of the variance function is calculated based on the actual distribution of the input data and is used to measure the degree of dispersion or variation of the data. In the first stage of training, to disregard changes in the input data, the output value of the variance function is set to a constant value less than a preset threshold, making the variance function insensitive to changes in the input data. That is, regardless of the actual distribution of the input data, the output value of the variance function remains constant. With the output value of the variance function remaining constant, the mean function of the variational autoencoder to be trained is trained in the first stage. Thus, at the end of the first stage of training, the weights of the mean function of the variational autoencoder to be trained are obtained. The mean function is part of the encoder network of the variational autoencoder. The mean function can map the input data to the mean vector in the latent space. By training the weights of the mean function, the generated data can better approximate the distribution of the input data.

[0088] Furthermore, in step S103, while keeping the weights of the mean function unchanged, the variational autoencoder trained in the first stage is trained in the second stage, and at the end of the second stage of training, the weights of the variance function are obtained.

[0089] In this embodiment, after the training of each weight of the mean function in the first stage of training is completed, the variational autoencoder trained in the first stage needs to be further trained in the second stage. Before the second stage of training, the weights of the mean function that have been trained in the first stage need to be kept unchanged, and the variance function of the variational autoencoder trained in the first stage needs to be trained in the second stage. Training the variance function means optimizing the variance function to minimize the difference between the generated data and the original input data. At the end of the second stage of training, the weights of the variance function are obtained.

[0090] Finally, in step S104, while keeping the weights of the obtained mean function and variance function unchanged, the parameter values ​​of the randomized manifold space are learned, and the parameter values ​​of the randomized manifold space are used for data generation.

[0091] In this embodiment, after training the weights of the mean function of the variational autoencoder to be trained in the first stage and the weights of the variance function of the variational autoencoder to be trained in the second stage, while keeping the weights of the mean function and variance function unchanged, the parameter values ​​of the randomized manifold space are learned. Learning the parameter values ​​of the randomized manifold space refers to determining the parameter values ​​in the manifold space through model training. In machine learning, the manifold space refers to the nonlinear structure of data, containing complex relationships and patterns. Learning the parameter values ​​of the manifold space means adjusting the model's parameters to better fit, describe, or capture this nonlinear structure in the data. By learning the parameter values ​​of the manifold space, the model can better model and predict data, thereby improving its performance and generalization ability. This process typically involves using optimization algorithms, such as gradient descent, to minimize the model's loss function and update the parameter values ​​to better adapt to the manifold structure of the data. The learned parameter values ​​of the randomized manifold space are used for data generation. Finally, the trained variational autoencoder can be used to complete various downstream tasks, including data generation.

[0092] Generating new samples: By learning the latent representation of data, new samples can be generated that are similar to, but not exactly the same as, the training data. This has great potential for generating realistic images, text, music, etc.

[0093] Data reconstruction and denoising: Used to reconstruct original input data, including images, text, etc. It can recover original data from damaged or noisy samples, aiding in data recovery and denoising tasks.

[0094] Feature learning and dimensionality reduction: Learning meaningful low-dimensional representations of data to achieve dimensionality reduction and feature learning. This representation can then be used for subsequent tasks such as classification and clustering.

[0095] Anomaly detection: Learning the distribution of normal data to detect anomalous data. By comparing the reconstruction error or the distance to the latent space, anomalous samples that differ from the training data can be identified.

[0096] Semi-supervised learning and transfer learning: Semi-supervised learning is achieved by introducing label information into the encoder. It can also perform transfer learning between data from different domains, applying the learned latent representations to new tasks.

[0097] Through the above embodiments, the weights of the mean function of the variational autoencoder to be trained can be obtained through the first stage of training, and the weights of the difference function of the variational autoencoder trained in the first stage can be obtained through the second stage of training. After determining the weights of the mean function and the variance function, the parameter values ​​of the randomized manifold space are learned, and data is generated through the parameter values ​​of the randomized manifold space. This enables the trained variance function to accurately reflect the sample space relationship of the original input data and effectively generate new data based on the potential variability of the original input data.

[0098] Figure 3 This is a schematic diagram of a stochastic optimization algorithm for training the parameter set S of a cubic spline curve C in a training method for a manifold geodesic calculation model provided in an embodiment of this application. Figure 3 As shown.

[0099] Optionally, step S104 includes:

[0100] While keeping the weights of the mean function and the variance function unchanged, the parameter set of the cubic spline curve is updated using the parameter set optimization formula.

[0101] In the process of updating the parameter set of the cubic spline curve, a target parameter set is determined. The target parameter set is the parameter set that enables the cubic spline curve to reach the optimal state in the latent variable space.

[0102] Based on the target parameter set, obtain the parameter set expression for the target cubic spline curve;

[0103] The method further includes:

[0104] After the second phase of training is completed, the reconstructed network is obtained based on the weights of the mean function and the weights of the variance function.

[0105] Based on the reconstructed network, the target cubic spline curve is mapped into the generation space so that the distance between two given points is connected by the shortest distance through the target cubic spline curve in the generation space, and the relationship between the two given points is represented to complete the reconstruction and generation of the data.

[0106] Specifically, in this embodiment, while keeping the weights of the obtained mean function and variance function unchanged, learning the parameter values ​​of the randomized manifold space includes the following steps:

[0107] Step 1: Keep the weights of the neural network representing the mean and variance functions unchanged.

[0108] Step 2: Train and update the parameter set of the cubic spline curve, and obtain the optimal parameter values ​​through an optimization algorithm.

[0109] Step 3: In the latent variable space, the learned curve is represented by the optimal curve parameter expression.

[0110] Step 4: Map the learned curves into the generative space through the reconstruction network.

[0111] Step 5: In the generated space, use the most probable way to connect the distances between two points so that the generated curve matches the original data as closely as possible.

[0112] The following is a detailed explanation of steps 1-5:

[0113] First, according to step 1, keep the weights of the mean function μ and the variance function σ. 2 With all weights unchanged, the parameter set S of the cubic spline curve C is updated using the parameter set optimization formula to obtain its optimal learning result. According to step 2, in this training and update process, the parameter set S of the cubic spline curve C is updated using training data so that it can better fit the training data and obtain a more accurate curve representation. Further, according to step 3, in the process of updating the parameter set of the cubic spline curve, a target parameter set is determined. The target parameter set is the parameter set that enables the cubic spline curve to reach the optimal state in the latent variable space. By iteratively updating the parameter set of the spline curve, the goal is to find the parameter set that can best fit the training data as the target parameter set. The target parameter set enables the cubic spline curve to accurately represent the features and patterns of the training data. Further, the parameter set expression of the target cubic spline curve is determined by obtaining the target parameter set.

[0114] Furthermore, according to step 4, after the training in the second stage is completed, the reconstruction network f is obtained through the weights of the mean function and the weights of the variance function, and the target cubic spline curve is mapped to the generation space through the reconstruction network f; according to step 5, the distance between two given points is connected by the shortest distance through the target cubic spline curve in the generation space, and the relationship between the two given points is represented to complete the reconstruction and generation of the data.

[0115] Through the above embodiments, the optimal cubic spline curve parameter expression can be learned through training and optimization, and the learned curve can be mapped into the generative space using a generative model to obtain a curve that is most similar to the original data.

[0116] Optionally, the parameter set is differentiated, and the stochastic gradient descent method is used to calculate the update amount of the parameter set to obtain the target parameter set.

[0117] Specifically, in this embodiment, the parameter values ​​in the cubic spline curve are updated by learning the weights of a neural network to minimize the distance. Based on the distance of a randomized manifold, the optimization formula for this parameter set can be expressed as:

[0118]

[0119] Among them, S * This represents a parameter set, which contains parameter a. * ,b * ,c * , σ 2 Let μ denote the variance function, and μ denote the mean function.

[0120] Through the above embodiments, the generated cubic spline curves can be better adapted to the required distance conditions by training and updating the weights of the neural network, making them suitable for various application fields such as image processing and robot path planning, so as to achieve precise control and optimization of the curve shape.

[0121] Optionally, step S103 includes:

[0122] Establish a weighted Gaussian mixture model;

[0123] The variance function is characterized based on the weighted Gaussian mixture model.

[0124] The weighted Gaussian mixture model is as follows: K represents the number of Gaussian components in the weighted Gaussian mixture model, and c i Represents the sum of the mean and sum of the i-th components. i Let ω represent the covariance of the i-th component. i W represents the mixing coefficient. g∈R D Let z represent the weight vector, and z represent the latent variable.

[0125] Specifically, in this embodiment, the second stage of training is actually training the parameters of the established weighted Gaussian mixture model. Therefore, in this embodiment, it is first necessary to establish a weighted Gaussian mixture model, which represents the variance function. The weighted Gaussian mixture model is as follows: K represents the number of Gaussian components in the weighted Gaussian mixture model, and c i Represents the sum of the mean and sum of the i-th components. i Let ω represent the covariance of the i-th component. i W represents the mixing coefficient. g ∈R D Let z represent the weight vector, and z represent the latent variable.

[0126] Through the above embodiments, the variance function can be characterized by a rescaled Gaussian mixture model. A larger variance function will result in a smaller weight for the corresponding Gaussian component, thus reducing its impact on the entire model. Conversely, a smaller variance function will result in a larger weight for the corresponding Gaussian component, increasing its impact on the entire model. By adjusting the contribution of each Gaussian component to the entire model, flexible modeling of the data distribution can be achieved.

[0127] Optionally, the method further includes:

[0128] The weight vector W for training the weighted Gaussian mixture model g ;

[0129] In the weight vector W g After training is completed, based on the weight vector W obtained after training... g Then update the weights of the variance function.

[0130] Specifically, in this embodiment, the weight vector determines the proportion of each Gaussian distribution in the weighted Gaussian mixture model. A larger weight means that the corresponding Gaussian distribution has a higher probability when generating data, indicating that the generated samples are more important. Conversely, a smaller weight means that the corresponding Gaussian distribution contributes less to the generated data.

[0131] The weight vector W of the established weighted Gaussian mixture model g Training allows control over the relative importance of each Gaussian distribution in the weighted Gaussian mixture model. The weight vector defines the weight of each Gaussian distribution, representing the degree to which each distribution contributes to the final result when generating the data. Larger weights can be used to capture the main patterns and structure in the data, while smaller weights can be used to handle noise or minor components in the data.

[0132] And in the weight vector W g After training, the weight vector W is used. g Then update the weights of the variance function to complete the second stage of training.

[0133] Through the above embodiments, it is possible to adjust the weight vector W g It can adjust the complexity and flexibility of the weighted Gaussian mixture model to adapt to the modeling needs of different data distributions.

[0134] Optionally, the method further includes:

[0135] The covariance in the weighted Gaussian mixture model is set to a diagonal matrix.

[0136] via log(σ) 2 (z) is used to obtain the output value of the variance function.

[0137] Specifically, in this embodiment, considering computational efficiency, the covariance of each Gaussian component in the weighted Gaussian mixture model can be set to a diagonal matrix. The latent variables are used as the object, and their weighted Gaussian mixture models and corresponding weights are established and calculated. To avoid inconvenience in numerical expression due to excessively small values, logarithmic transformation is often used in practice, i.e., by using log(σ... 2 (z) can be used to observe the output value of the variance function.

[0138] Through the above embodiments, by setting the covariance in the weighted Gaussian mixture model as a diagonal matrix, the computational complexity is reduced and computational efficiency is improved due to the relatively simple calculation of the diagonal covariance matrix. The diagonal covariance matrix only needs to store the variance of each feature, without needing to store the covariance between features. This reduces the number of model parameters that need to be estimated, especially in the case of high-dimensional data, thereby reducing model complexity and storage requirements. Furthermore, the diagonal covariance matrix makes the variance and importance of each feature more explicit. By observing the variance of each feature, we can understand the range of variation of that feature in the data and its contribution to the model.

[0139] Optionally, step S102 includes:

[0140] Set the output value of the variance function to a constant;

[0141] Obtain the output value of the mean function, and based on the output value of the mean function, determine that all weights of the mean function have been updated.

[0142] Specifically, in this embodiment, the output value of the variance function is set to a constant to prevent the output value of the variance function from changing with the training of the mean function. The mean function is trained when the variance function is set to a constant. The output value of the mean function can be used to determine that the weights of the mean function have been updated.

[0143] Through the above embodiments, a stable generation space can be obtained while keeping the constraint noise relatively low.

[0144] Figure 4 This application provides a training device for a manifold geodesic calculation model, as shown in one embodiment. Figure 4 As shown in the figure, this application provides a training device for a manifold geodesic calculation model, the device comprising:

[0145] Module 11 is used to construct the variational autoencoder to be trained based on the variance function and the mean function;

[0146] The first-stage training module 12 is used to perform the first-stage training on the variational autoencoder to be trained with the output value of the variance function set to a constant, and to obtain the weights of the mean function at the end of the first-stage training.

[0147] The second-stage training module 13 is used to perform a second-stage training on the variational autoencoder that has been trained in the first stage while keeping the weights of the mean function unchanged. At the end of the second-stage training, the weights of the variance function are obtained.

[0148] Learning module 14 is used to learn the parameter values ​​of the randomized manifold space while keeping the weights of the obtained mean function and variance function unchanged. The parameter values ​​of the randomized manifold space are used for data generation.

[0149] Optionally, the learning module 14 includes:

[0150] The update module is used to update the parameter set of the cubic spline curve using a parameter set optimization formula while keeping the weights of the mean function and the variance function unchanged.

[0151] The determination module is used to determine a target parameter set during the process of updating the parameter set of the cubic spline curve. The target parameter set is a parameter set that enables the cubic spline curve to reach the optimal state in the latent variable space.

[0152] The parameter set expression acquisition module is used to acquire the parameter set expression of the target cubic spline curve based on the target parameter set.

[0153] The device further includes:

[0154] The reconstructed network acquisition module is used to acquire the reconstructed network based on the weights of the mean function and the weights of the variance function after the training in the second stage is completed.

[0155] The mapping module is used to map the target cubic spline curve into the generation space based on the reconstruction network, so as to connect the distance between two given points in the generation space with the shortest distance through the target cubic spline curve, and to represent the relationship between the two given points, so as to complete the reconstruction and generation of data.

[0156] Optionally, the parameter set optimization formula is:

[0157]

[0158] The device further includes: a computing module for processing S * The derivative is calculated, and the stochastic gradient descent method is used to calculate the update amount of the parameter set, thus obtaining the target parameter set.

[0159] Optionally, the second-stage training module 13 includes:

[0160] Establish a module for building a weighted Gaussian mixture model;

[0161] The characterization module is used to characterize the variance function based on the weighted Gaussian mixture model;

[0162] The weighted Gaussian mixture model is as follows: K represents the number of Gaussian components in the weighted Gaussian mixture model, and c i Represents the sum of the mean and sum of the i-th components. i Let ω represent the covariance of the i-th component. i W represents the mixing coefficient. g ∈R D Let z represent the weight vector, and z represent the latent variable.

[0163] Optionally, the device further includes:

[0164] Training unit, used to train the weight vector W of the weighted Gaussian mixture model. g ;

[0165] Update unit, used in the weight vector W g After training is completed, based on the weight vector W obtained after training... g Then update the weights of the variance function.

[0166] Optionally, the device further includes:

[0167] The covariance setting module is used to set the covariance in the weighted Gaussian mixture model to a diagonal matrix.

[0168] The output module is used to output log(σ) 2 (z) is used to obtain the output value of the variance function.

[0169] Optionally, the first-stage training module 12 includes:

[0170] The variance function setting module is used to set the output value of the variance function to a constant;

[0171] The mean function weight determination module is used to obtain the output value of the mean function and, based on the output value of the mean function, determine that each weight of the mean function has been updated.

[0172] Based on the same inventive concept, another embodiment of this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the training method of the manifold geodesic calculation model as described in any of the above embodiments.

[0173] Based on the same inventive concept, another embodiment of this application also provides a computer program product, including a computer program that is executed by a processor using the training method for the manifold geodesic calculation model as described in any of the above embodiments.

[0174] Based on the same inventive concept, another embodiment of this application provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, it implements the training method for the manifold geodesic calculation model as described in any of the above embodiments.

[0175] As the apparatus is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment.

[0176] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0179] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0182] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0183] The training method, apparatus, equipment, and medium for a manifold geodesic calculation model provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for a manifold geodesic calculation model, characterized in that, The method includes: Based on the variance function and the mean function, a variational autoencoder to be trained is constructed; With the output value of the variance function set to a constant, the variational autoencoder to be trained is trained in the first stage. At the end of the first stage of training, the weights of the mean function are obtained. While keeping the weights of the mean function unchanged, the variational autoencoder trained in the first stage is trained in the second stage. At the end of the second stage of training, the weights of the variance function are obtained. While keeping the weights of the obtained mean function and variance function unchanged, the parameter values ​​of the randomized manifold space are learned, and the parameter values ​​of the randomized manifold space are used for data generation. The step of learning the parameter values ​​of the randomized manifold space while keeping the weights of the obtained mean function and variance function unchanged includes: While keeping the weights of the mean function and the variance function unchanged, the parameter set of the cubic spline curve is updated using the parameter set optimization formula. In the process of updating the parameter set of the cubic spline curve, a target parameter set is determined. The target parameter set is the parameter set that enables the cubic spline curve to reach the optimal state in the latent variable space. Based on the target parameter set, obtain the parameter set expression of the cubic spline curve; The method further includes: After the second phase of training is completed, the reconstructed network is obtained based on the weights of the mean function and the weights of the variance function. Based on the reconstructed network, the cubic spline curve is mapped into the generation space so that the distance between two given points is connected by the cubic spline curve in the generation space with the shortest distance, and the relationship between the two given points is represented to complete the reconstruction and generation of the data. The optimization formula for the parameter set is: ； The method further includes: [the following] The derivative is calculated, and the stochastic gradient descent method is used to calculate the update amount of the parameter set, thus obtaining the target parameter set; The variational autoencoder is used to generate new samples from the original data, reconstruct the original data, and perform dimensionality reduction and feature learning on the original data. The output features can be used for classification tasks and clustering tasks. The types of new samples generated include at least: images, text, and music.

2. The training method for the manifold geodesic calculation model according to claim 1, characterized in that, While keeping the weights of the mean function unchanged, the variational autoencoder trained in the first stage is trained in the second stage. At the end of the second stage of training, the weights of the variance function are obtained, including: Establish a weighted Gaussian mixture model; The variance function is characterized based on the weighted Gaussian mixture model. The weighted Gaussian mixture model is as follows: , This indicates the number of Gaussian components in the weighted Gaussian mixture model. Indicates the first The mean of each component and Indicates the first The covariance of each component, Represents the mixing coefficient. Represents a weight vector. This represents a latent variable.

3. The training method for the manifold geodesic calculation model according to claim 2, characterized in that, The method further includes: The weight vectors of the weighted Gaussian mixture model are trained. ; In the weight vector After training is completed, based on the weight vector obtained after training... Then update the weights of the variance function.

4. The training method for the manifold geodesic calculation model according to claim 2, characterized in that, The method further includes: The covariance in the weighted Gaussian mixture model is set to a diagonal matrix. pass To obtain the output value of the variance function.

5. The training method for the manifold geodesic calculation model according to claim 1, characterized in that, The variational autoencoder to be trained is trained in the first stage with the output value of the variance function set to a constant. At the end of the first stage of training, the weights of the mean function are obtained, including: Set the output value of the variance function to a constant; Obtain the output value of the mean function, and based on the output value of the mean function, determine that all weights of the mean function have been updated.

6. A training device for a manifold geodesic calculation model, characterized in that, The device includes: The building module is used to construct the variational autoencoder to be trained based on the variance function and the mean function; The first-stage training module is used to perform the first-stage training on the variational autoencoder to be trained with the output value of the variance function set to a constant, and to obtain the weights of the mean function at the end of the first-stage training. The second-stage training module is used to perform a second-stage training on the variational autoencoder that has been trained in the first stage while keeping the weights of the mean function unchanged. At the end of the second-stage training, the weights of the variance function are obtained. The learning module is used to learn the parameter values ​​of the randomized manifold space while keeping the weights of the obtained mean function and variance function unchanged. The parameter values ​​of the randomized manifold space are used for data generation. The learning module includes: The update module is used to update the parameter set of the cubic spline curve using a parameter set optimization formula while keeping the weights of the mean function and the variance function unchanged. The determination module is used to determine a target parameter set during the process of updating the parameter set of the cubic spline curve. The target parameter set is a parameter set that enables the cubic spline curve to reach the optimal state in the latent variable space. The parameter set expression acquisition module is used to acquire the parameter set expression of the cubic spline curve based on the target parameter set. The device further includes: The reconstructed network acquisition module is used to acquire the reconstructed network based on the weights of the mean function and the weights of the variance function after the training in the second stage is completed. The mapping module is used to map the cubic spline curves into the generation space based on the reconstruction network, so as to connect the distances between two given points in the generation space with the shortest distance through the cubic spline curves, and to represent the relationship between the two given points, so as to complete the reconstruction and generation of data; The optimization formula for the parameter set is: ； The device further includes: a computing module, used for... The derivative is calculated, and the stochastic gradient descent method is used to calculate the update amount of the parameter set, thus obtaining the target parameter set; The variational autoencoder is used to generate new samples from the original data, reconstruct the original data, and perform dimensionality reduction and feature learning on the original data. The output features can be used for classification tasks and clustering tasks. The types of new samples generated include at least: images, text, and music.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement a training method for a manifold geodesic calculation model as described in any one of claims 1-5.

8. A computer-readable medium, characterized in that, It stores a computer program, wherein when the computer program is executed by a processor, it implements the training method for the manifold geodesic calculation model as described in any one of claims 1-5.

Images