Learning data density functions

Abstract

A data processing system for forming a mapping from an input data domain to a compressed representation of the input domain, the system receiving a set of input data representing the input domain and candidate mappings, and iteratively performing the following steps: forming a set of vectors representing the input data set mapped by the candidate mappings; comparing the density distribution of the vector set with a predetermined density distribution; and adjusting the candidate mappings based on the comparison. Thus, underlying density data can be learned, for example, to generate samples and compress data. This technique can be applied to tasks such as image generation, improving image quality compared to existing autoencoders.

Description

Technical Field

[0001] This invention relates to learning data density functions, for example, for sample generation and data compression. Background Technology

[0002] Deep generative autoencoders have shown promising results in many applications involving unsupervised density estimation, such as image generation, processing and editing, image style transfer, learning world models in model-based reinforcement learning, and other applications involving data compression and generation.

[0003] These models include encoder-decoder architectures, where the encoder aims to learn a compressed representation of the data, while the decoder ensures that the data can be reconstructed from this representation with high fidelity. Deep generative autoencoders have two main functions: sample generation and data compression. The former can be used in many applications of computer vision (e.g., for realistic generation of natural images) and machine learning (e.g., for training reinforcement learning agents in simulated environments), while the latter can be used for data transmission and storage.

[0004] While these models have achieved success in many applications, their lack of theoretical guarantees and understanding directly impacts their practical performance. Current techniques also fail to address the problem of local minima, thus resulting in suboptimal performance. For example, in image generation tasks, autoencoder models often produce blurred images.

[0005] I. Tolstikhin, O. Bousquet, S. Gelly, and B. Schoelkopf, “Wasserstein Auto-Encoders,” International Conference on Learning Representations (ICLR), 2018, describe an autoencoder with an MMD regularizer. Improvements over traditional VAEs are observed in FID and blurring of the resulting images.

[0006] B. Dai and D. Wipf's "Diagnosing and Enhancing VAE Models," ICLR, 2019, describes a two-stage algorithm that uses two cascaded VAEs to enhance the performance of variational autoencoders. The first stage involves training the VAEs in a conventional manner. The second stage involves training a second VAE on the learned latent representations to further reduce the distance between the latent density and the previous density. The second VAE is used to overcome the training limitations of the first stage, thus introducing more complexity into the model in terms of capability / training and inference time. Therefore, the problem of local minima is mitigated by acting on the structure of the neural network.

[0007] US 2018 / 0314917 A1 describes a data augmentation technique for improving the generalization performance of an autoencoder by applying predefined transformations to augment the data (e.g., rotation). The objective is defined as being robust to such transformations (e.g., recovering a true image). The improvement is achieved by (i) augmenting the dataset with a predefined set of transformations and (ii) including further reconstruction terms in the objective function.

[0008] US 2019 / 0026631A1 describes imposing constraints (i.e., matrix factorization) on the latent representation with the aim of ensuring that the learned representation is low-dimensional and more interpretable. Improvements are achieved by introducing new terms into the objective and by increasing the model capacity.

[0009] For sample generation and data compression, a method is needed to accurately learn the density of underlying data. This method can be applied to image generation tasks and improves image quality compared to existing autoencoder models. Summary of the Invention

[0010] According to a first aspect, a data processing system is provided for forming a mapping from an input data domain to a compressed representation of the input domain, the system being configured to receive an input data set representing the input domain and a candidate mapping, and iteratively performing the following steps: forming a set of vectors representing the input data set mapped by the candidate mapping; comparing a density distribution of the vector set with a predetermined density distribution; and adjusting the candidate mapping based on the comparison.

[0011] The candidate mapping can be an artificial intelligence model, and the step of adjusting the candidate mapping includes adjusting the parameters of the model. This allows the system to learn the optimal density distribution. By iteratively updating the parameters of the model, the quality of the learned density distribution can be further improved.

[0012] The step of adjusting the candidate mapping can be performed based on the gradient of the reconstruction error of the input data set and the distance between the density distribution of the vector set and the predetermined density distribution in the objective function of the model. The step of adjusting the candidate mapping can also be performed based on the dot product of the gradients of the reconstruction error of the input data set and the distance between the density distribution of the vector set and the predetermined density distribution in the objective function of the model. This avoids local minima in the function space.

[0013] The system can be used to determine the difference between the density distribution of the vector set and a predetermined density distribution based on a distance metric. The distance metric can be a Coulomb kernel. This can reduce the influence of local minima in the function space. This can have a direct impact on practical performance. For example, it may improve image quality in image generation tasks.

[0014] The artificial intelligence model could be a neural network. This could be a convenient implementation.

[0015] The input dataset may include images. This allows the system to be used in image generation tasks.

[0016] The system can also be used to reconstruct a set of vectors representing an input data set into an output data set.

[0017] The system can also be used to select a vector based on the predetermined density distribution and apply the inverse mapping of the mapping to the vector to generate data in the input data domain. This allows the system to be used in sample generation tasks.

[0018] According to a second aspect, a method is provided for forming a compressed representation of an input data domain to a input domain based on an input data set representing an input domain and a candidate mapping, characterized in that the method includes iteratively performing the following steps: forming a set of vectors representing the input data set mapped by the candidate mapping; comparing the density distribution of the vector set with a predetermined density distribution; and adjusting the candidate mapping based on the comparison.

[0019] The candidate mapping can be an artificial intelligence model, and the step of adjusting the candidate mapping includes adjusting the parameters of the model. Therefore, the method can learn an optimal density distribution. By iteratively updating the parameters of the model, the quality of the learned density distribution can be further improved.

[0020] The step of adjusting the candidate mapping can be performed based on the gradient of the reconstruction error of the input data set and the distance between the density distribution of the vector set and the predetermined density distribution in the objective function of the model. The step of adjusting the candidate mapping can also be performed based on the dot product of the gradients of the reconstruction error of the input data set and the distance between the density distribution of the vector set and the predetermined density distribution in the objective function of the model. This avoids local minima in the function space.

[0021] The step of comparing the density distribution of the vector set with a predetermined density distribution may include determining the difference between the density distribution of the vector set and the predetermined density distribution based on a distance metric. The distance metric may be a Coulomb kernel. This can reduce the influence of local minima in the function space. This can have a direct impact on practical performance. For example, it may improve image quality in image generation tasks.

[0022] The artificial intelligence model could be a neural network. This could be a convenient implementation.

[0023] According to a third aspect, a computer program is provided that, when executed by a computer, causes the computer to perform the methods described above. The computer program may be provided on a non-transitory computer-readable storage medium. Attached Figure Description

[0024] The invention will now be described by way of example with reference to the accompanying drawings. In the drawings:

[0025] Figure 1 An overview diagram of the method for updating the encoder and decoder neural networks in this invention is provided.

[0026] Figure 2 An example of a one-dimensional case with a single ((a)-(c)) and a pair of negatively charged particles ((d)-(f)) is shown. (ab) and (de) are plots of the regularizer in equation (1) at different positions of the negative particles with Gaussian and Coulomb kernel functions, respectively. (c) and (f) show the possible minimum values.

[0027] Figure 3 An example of a local minimum problem in the parameter space of a neural network is shown.

[0028] Figure 4 An algorithm outlining the complete process for training an autoencoder is shown.

[0029] Figure 5 The elements used in the objective function and optimization strategy described in this paper are highlighted.

[0030] Figure 6An example of a method is shown that forms a compressed representation of an input data domain to a set of input data representing an input domain and candidate mappings.

[0031] Figure 7 (a)-(c) show Figure 7 (c) uses the Coulomb kernel function and Figure 7 The true value in (a) and Figure 7 (b) The results obtained using Gaussian kernels improved the recovery of the true density compared to the results obtained using Gaussian kernels.

[0032] Figure 8 The results of synthetic experiments using different models on a grid dataset are shown.

[0033] Figure 9 The test log-likelihood on the grid and the low-dimensionality of the embedded dataset are shown (higher scores indicate better performance).

[0034] Figure 10 (a)-(d) show the results of real-world experiments conducted using the method described herein, compared to other known methods.

[0035] Figure 11 The FID scores of different models on the CelebA 64×64 dataset are shown (the lower the score, the better the performance).

[0036] Figure 12 An example of a data processing system is shown. Detailed Implementation

[0037] The present invention relates to a training strategy for learning density-based underlying data in an autoencoder, for example for sample generation and data compression.

[0038] Models used for this purpose include encoder-decoder architectures, where the encoder learns a compressed representation of the data, while the decoder ensures that the data can be reconstructed from such representation with high fidelity.

[0039] In an encoder neural network, the model takes a dataset representing the input domain as input. For example, this input dataset could be a set of images. The input dataset is compressed to form a set of vectors with a density distribution. To provide a reliable output when the decoder reconstructs the data, the model is trained to learn the optimal density distribution for the application.

[0040] Therefore, during training, the parameters of the model used to transform the input data into a compressed representation are iteratively updated to give the optimal density distribution.

[0041] Learning the unknown density distribution in an autoencoder requires minimizing two terms: the error between the input data and its corresponding reconstructed version, and the distance between the density functions resulting from the previous density function and the encoder function.

[0042] In this invention, the optimization strategy minimizes the aforementioned objective (in the conditional update of the encoder, based on the error gradient between the input data and its corresponding reconstructed version, and the distance between the density functions resulting from the previous density function and the encoder function) through encoder conditional updates. Figure 1 (As shown in 101 and 102).

[0043] Then, as shown in 104, the decoder is updated, and this process is performed iteratively to update the model's parameters until it converges to a predetermined threshold. The update process ends in 105.

[0044] The objective function used to train the autoencoder and the optimization strategy used to update the encoder network will now be described in more detail.

[0045] The objective function of an autoencoder is typically non-convex. Therefore, training using local search methods may get stuck in local minima. This problem can occur at three different levels: at the objective function level (i), at the optimizer level (ii), and at the architecture design level (iii).

[0046] The goal is to estimate the unknown density function p. X (x), its support is limited.

[0047] Consider two continuous functions f∶Ω x →Ω z and g∶Ω z →Ω x ,in And h equals Ω x The eigendimensionality of Ω. Furthermore, it is assumed that for every x∈Ω x , g(f(x)) = x. That is, g is the domain Ω of f. x The left-hand reverse.

[0048] f and g are neural networks parameterized by vectors θ and γ, respectively. f is called the encoding function, and it uses a density of p. X The generation density of the random input x of (x) is q Z (z) is a random vector z, and g is a decoding function that takes z as input and generates a function based on q. Y A random vector y with a distribution of (y). Note: p X (x)=q Y (y), because for every x∈Ω xy = g(z) = g(f(x)) = x. This is already a density estimator, but its drawback lies in q Z (z) cannot usually be written in a closed form. Now, p Z (z) is defined as having a support Ω z The support has an arbitrary density and a closed form (p) Z (z) is considered to be the standard multivariable Gaussian density.

[0049] The goal is to ensure q across the entire support. Z (z)=p Z (z), and for every x∈Ω x Keep g(f(x)) = x. This allows the decoding function to be used as a generator and to produce results based on p. X Samples with a (x) distribution.

[0050] Therefore, the density estimation problem in a high-dimensional feature space is transformed into an estimation problem in a low-dimensional vector space, thus overcoming the dimensionality problem.

[0051] The objective of the minimization problem is defined as:

[0052]

[0053] Where φ(z)=p Z (z)-q z (z), k(.,.′) are kernel functions, and λ is a positive scalar hyperparameter weighting the two addends. Note that when the encoding and decoding functions are supported by Ω... x When q is invertible, the first term in equation (1) reaches its global minimum, while when q Z (z) equals p Z When (z), the second term in equation (1) is globally optimal. Therefore, the global minimum of equation (1) satisfies the initial requirement, and the optimal solution corresponds to q. Y (y)=p X The case of (x).

[0054] Throughout the analysis described herein, it is assumed that the encoder and decoder networks have sufficient capacity to achieve the global minimum of the objective in equation (1).

[0055] To improve the global convergence of training to the global minimum of the second addend in equation (1), it is advantageous that the kernel function satisfies the Poisson equation (see Theorem 2 in S. Hochreiter and K. Obermayer, “Optimal Kernels for Unsupervised Learning”, IEEE International Conference on Neural Networks (IJCNN2005), pp. 1895-1899, 2005).

[0056] Assume the kernel function satisfies the Poisson equation, i.e. Where δ(.) is the delta function and but:

[0057]

[0058] Where, φ t (.) represents φ(.) for iteration t, while z max =argmax z ||φ t (z)‖. Therefore, gradient descent-based training converges to the global minimum of the second addend in equation (1) and for all z∈Ω. z The global minimum value φ(z) = 0.

[0059] Note that the previous results are valid for gradient descent optimization performed in the encoder's function space and are also independent of the initialization of f. This means that the regularizer in equation (1) has only a global minimum.

[0060] The solution to the Poisson equation can be written in closed-form, i.e.:

[0061]

[0062] Among them, S h It is the surface area of ​​an h-dimensional unit sphere, β = h⁻². These functions are called Coulomb kernels. In this embodiment, ‖zz′‖ is replaced with And ∈-1e-3 to avoid the singularity of z=z′.

[0063] The first important property of equation (3) is that the Coulomb kernel represents the generalization of Coulomb's law to any h-dimensional Euclidean space. To illustrate this, consider that for h = 3, the kernel function in equation (3) exactly conforms to Coulomb's law. In fact, the regularizer in equation (1) represents... The energy function of an electrostatic system in p. Z (z) samples and from q ZThe samples of (z) can be interpreted as positively charged particles and negatively charged particles, respectively, with the Coulomb nucleus causing some global attraction and repulsion between them. Therefore, with regard to the position of the negatively charged particles, minimizing the regularizer in equation (1) allows for low-energy configurations where the effect of negative particles balancing positive particles can be found.

[0064] The second important property is that, unlike the kernel function in equation (3), this kernel function is not a solution to the Poisson equation and may therefore introduce other local optima. This includes the kernel functions used in the work of I. Tolstikhin, O. Bousquet, S. Gelly and B. Schoelkopf, “Wasserstein Autoencoder”, ICLR, 2018, namely the Gaussian and inverse quadratic kernels used in the experiments.

[0065] To gain an intuitive understanding of these two properties, we will now analyze the effects of using Gaussian and Coulomb kernels on two simple one-dimensional cases (h=1).

[0066] like Figure 2 As shown in (c), the first example includes three positive particles located at -4, 0, and 4, and a single negative particle allowed to move freely. In this case, p Z (z)=δ(z+4)+δ(z)+δ(z-4) and q Z (z) = δ(z-z1), where z1 represents the variable position of the negative particle.

[0067] Figure 2 (a) and Figure 2 (b) shows the curves of the regularizer in Equation (1) evaluated under different z1 conditions for the Gaussian and Coulomb nuclei, respectively. The Gaussian nucleus introduces a new local optimum, where negative particles are locally attracted to one of the positive charges, unaffected by the others. In contrast, the Coulomb nucleus has only a single minimum. This minimum configuration is the best configuration if all positive particles exert an attractive force on the negative particles is considered. As a result, the Coulomb nucleus induces a global attractive force.

[0068] like Figure 2 As shown in (f), the second example includes the same three positive particles and a pair of free negative charges. In this case, q Z (z) = δ(z-z1) + δ(z-z2), where z1 and z2 are the positions of the two negative particles.

[0069] Figure 2 (d) and Figure 2(e) represents the graphs of the regularizer in equation (1) evaluated under different z1 and z2 conditions for the Gaussian and Coulomb kernels, respectively. Following the same reasoning as in the previous example, the Coulomb kernel induces a global repulsive force. In this case, there exist minimum pairs corresponding to permutations of a single configuration.

[0070] The solution to the Poisson equation guarantees convergence to the global minimum of the regularizer in equation (1), while other alternatives such as the Gaussian kernel do not have this property because there are new local optima that make the optimization strongly correlated with the initial conditions.

[0071] The objective in equation (1) is to minimize the two addends, i.e., the reconstruction error. Distance between and potential density Note that the combination of these two terms can introduce a local minimum in the optimization. This can be seen from a simple counterexample with a single training sample, i.e., x = z = 1, and two simple linear networks, i.e., f(x) = θx and g(z) = γz, where the objective in equation (1) is rewritten as (In this case, λ is assumed to be 1).

[0072] Figure 3 This illustrates the local minima problem in the parameter space of a neural network and demonstrates how to solve it using a traditional global objective method. The quality of the solution minimized strongly depends on parameter initialization. In fact, when both network parameters are negatively initialized, training converges to a suboptimal local minimum. This failure is due to the fact that during convergence... However, these two gradient terms can be non-zero. This problem can be solved by checking... and To avoid this, we need to determine if their correlation is positive, i.e., whether their dot product is positive. If the dot product is positive in all training iterations, we can guarantee that the training converges to the desired solution.

[0073] This observation allows for the formulation of a simple optimization strategy, where, at each iteration, the dot product between the reconstruction and regularizer terms is computed, and if the dot product is regularized according to... Update the encoder; otherwise (i.e., if the dot product is zero or negative), then... (Update encoder). Only the reconstruction term depends on γ. Therefore, the decoder update remains unchanged compared to the update rule used in joint optimization.

[0074] Figure 3 The global minimum of the reconstruction term in (a) lies on two discontinuous communicative shapes parameterized by the curve θ = 1 / γ. Figure 3 (b) The global minimum of the regularizer term lies on the manifold θ = 1. Figure 3The combination of the two terms in (c) produces two distinct local minima, only one of which is global. Based on parameter initialization, joint optimization (shown at 301) Figure 3 The white trajectory in (d) converges to a suboptimal solution. The conditional update (the trajectory shown at 302) is similar to the global minimum.

[0075] Therefore, the proposed training strategy addresses the local minima problem introduced by the combination of reconstruction and MMD terms. This problem still exists at the level of each individual objective term, as it is a common issue in deep learning.

[0076] In Algorithm 1, as follows Figure 4 As shown, the complete process for training an autoencoder is outlined.

[0077] Figure 5 This paper emphasizes the elements used in the objective function and optimization strategy described herein, namely, the use of the Coulomb kernel in the objective function and the conditional update of the encoder as part of the optimization strategy. The conditional update can be applied to any generative autoencoder model, such as a variational autoencoder. The use of the Coulomb kernel in the regularizer term of the objective function is found to be particularly advantageous for autoencoders using the MMD distance as a regularizer. Other distance functions may also be used.

[0078] Figure 6 An example of a method for forming a compressed representation of an input data domain to a candidate mapping based on an input data set representing an input domain and a candidate mapping is shown. Step 601, the method includes iteratively performing the following steps: forming a set of vectors representing the input data set mapped by the candidate mapping. Step 602, the method further includes comparing the density distribution of the vector set with a predetermined density distribution. Step 603, the method includes adjusting the candidate mapping based on the comparison.

[0079] The described framework for deep generative autoencoders alleviates the problem of local minima at the objective function and optimizer levels while maintaining the same model complexity. The framework includes the underlying principles of the objective function and the training encoder / decoder architecture. The advantageous use of a Coulomb kernel in the objective function ensures that no local minima exist in the function space except for the global minimum.

[0080] Figure 7 (a)-(c) show Figure 7 (c) uses the Coulomb kernel function and Figure 7 The true value in (a) and Figure 7 (b) The results obtained using Gaussian kernels improved the recovery of the true density compared to the results obtained using Gaussian kernels.

[0081] Figure 8The results of the synthetic experiments obtained using the above method are shown, comparing them with known methods. The experiments used a two-dimensional dataset consisting of 25 isotropic Gaussians placed according to a grid, such as... Figure 8 As shown in (a). This dataset is hereby referred to as the grid dataset. The training dataset consists of 500 samples generated from the true density.

[0082] exist Figure 8 (d) shows the performance of the model trained with Joint Optimization (CouAE). Figure 8 (e) shows the performance of the model trained using the conditional updates from Algorithm 1 (CouAE cond.). Figure 8 (b) shows a comparison with the baseline of Variational Autoencoders (VAEs), such as D.P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” ICLR, 2014; DJ. Rezende, S. Mohamed, and D. Wierstra, “Stochastic Backpropagation and Approximate Inference in Deep Generative Models,” International Conference on Machine Learning (ICML), pp. 1278-1286, 2014; Wasserstein Autoencoders (WAEs), such as… Figure 8 (c) As shown in I. Tolstikhin, O. Bousquet, S. Gelly and B. Schoelkopf., “Wasserstein Autoencoder”, ICLR, 2018.

[0083] Following the methodology of other works (see, for example, JHLim and JCYe, “Geometric Gan”, arXiv preprint arXiv:1705.02894, 2017; T. Unterthiner, B. Nessler, G. Klambauer, M. Heusel, H. Ramsauer and S. Hochreiter, “Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields”, ICLR, 2018), fully connected multilayer perceptrons with two hidden layers (each with 128 neurons) are chosen at both the encoder and decoder, and h is set to h=2. All models are run at 3×10⁻⁶. 6 This is the second iteration of training, using a learning rate of 10. -3 The Adam optimizer. The model is quantitatively evaluated by visually inspecting the generated samples and calculating the log-likelihood of the test data. To calculate the log-likelihood, first, in 10... 4 Apply kernel density estimation using a Gaussian kernel to the generated samples, and then apply it to 10 samples. 4 The log-likelihood was evaluated on 10 test samples from the true distribution. The results were averaged after 10 trials.

[0084] Figure 9 The first row of the table provides quantitative results for the grid dataset by testing log-likelihood. It can be seen that WAE performs significantly worse than the proposed solution and VAE.

[0085] As the feature dimension increases, conditional updates (CouAE cond.) have a further positive effect, as shown in the second synthetic case. The second dataset consists of 10-dimensional isotropic Gaussians embedded in a 1000-dimensional vector space (hereinafter referred to as the low-dimensional embedding dataset). 500 samples are generated from the true density, and all models are subjected to 2×10⁻⁶ iterations. 6 This is the second iteration of training. The method is similar to that used in the previous case. The results for this dataset are shown... Figure 9 In the second row of the table.

[0086] exist Figure 9 As can be seen from the results, the two proposed solutions (CouAE and CouAE cond.) achieve the best performance in testing log-likelihood. In particular, the Coulomb kernel significantly improves WAE, and the conditional update further enhances performance.

[0087] Figure 10(a)-(d) show the results of real-world experiments using the method described in this paper compared to other known methods. For this set of experiments, the Celeb 64×64 dataset was used, and the setup used was the same as that used by Tolstikhin et al. 2018. The DCGAN architecture, as described in A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks”, arXiv preprint arXiv:1511.06434, 2015, was used, and all models were trained for 10⁵ iterations with a learning rate of 0.0005. For VAE and WAE, the implementation from A. Radford et al. 2015 was used to run the simulations.

[0088] Figure 11 Quantitative results regarding the testing of FID are provided, as in M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” Advances in Neural Information Processing Systems (NIPS), pp. 6629-6640, 2017. Lower results indicate better performance. These results demonstrate that the method of this invention outperforms known techniques such as VAE and WAE.

[0089] These experiments demonstrate that, in some implementations, conditional updates improve performance compared to known methods. Therefore, the encoder's gradient conditions and the encoder's initial update can improve the performance of the autoencoder. The use of the Coulomb kernel distance function allows for further improvements.

[0090] The systems and methods described above can be used in applications such as sample generation to select a vector based on a predetermined density distribution and apply the inverse mapping of that vector to generate other data in the input data domain (e.g., generating an image).

[0091] This invention reduces the impact of local minima, directly affecting actual performance, such as improving image quality in image generation tasks. No modifications to the network architecture are required. In other words, no increase in model capacity is needed. Therefore, this invention is more efficient than existing methods in terms of both storage and execution time.

[0092] Figure 12 A schematic diagram of a data processing system 1200 for implementing the methods described above and their related components is shown. The system may include a processor 1201 and a non-volatile memory 1202. The system may include more than one processor and more than one memory. The memory may store data executable by the processor. The processor may be used to run according to a computer program stored in a non-transitory form on a machine-readable storage medium. The computer program may store instructions for causing the processor to perform its methods in the manner described herein.

[0093] The applicant hereby discloses individually each individual feature described herein, as well as any combination of two or more such features. With ordinary knowledge of those skilled in the art, such features or combinations can be implemented as a whole based on this specification, regardless of whether such features or combinations of features solve any problem disclosed herein, and without affecting the scope of the claims. This application demonstrates that aspects of the invention can be constituted by any such individual features or combinations of features. Various modifications that can be made within the scope of the invention, given the foregoing description, will be apparent to those skilled in the art.

Claims

1. A data processing system for forming a compressed representation of an input data domain to a candidate mapping based on an input data set representing an input domain and a candidate mapping, characterized in that, The system is configured to receive the set of input data representing the input domain and the candidate mapping, the set of input data including images, the candidate mapping being an artificial intelligence model, and iteratively execute the following steps: Form a set of vectors representing the input data set mapped by the candidate mapping; The density distribution of the vector set is compared with the predetermined density distribution in the objective function of the model; and The candidate mappings are adjusted based on the comparison; When the dot product result is positive, the encoder of the model is adjusted according to the gradient of the reconstruction error of the input data set, wherein the dot product result is the dot product of the gradient of the reconstruction error of the input data set and the gradient of the target distance, and the target distance is the distance between the density distribution of the vector set and the predetermined density distribution in the objective function of the model. If the dot product result is negative or zero, the encoder of the model is adjusted according to the gradient of the target distance.

2. The system as described in claim 1, characterized in that, The system is used to determine the difference between the density distribution of the vector set and the predetermined density distribution based on a distance metric.

3. The system as described in claim 2, characterized in that, The distance metric is the Coulomb kernel.

4. The system as described in claim 1, characterized in that, The artificial intelligence model is a neural network.

5. The system as described in claim 1, characterized in that, The system is also used to reconstruct a set of vectors representing an input data set into an output data set.

6. The system as described in claim 1, characterized in that, The system is also used to select a vector according to the predetermined density distribution and apply the inverse mapping of the mapping to the vector to generate data in the input data domain.

7. A method for forming a compressed representation of an input data domain to a candidate mapping based on an input data set representing an input domain and a candidate mapping, wherein the input data set includes images and the candidate mapping is an artificial intelligence model, characterized in that, The method includes iteratively performing the following steps: Form a set of vectors representing the input data set mapped by the candidate mapping; The density distribution of the vector set is compared with a predetermined density distribution; and The candidate mappings are adjusted based on the comparison; When the dot product result is positive, the encoder of the model is adjusted according to the gradient of the reconstruction error of the input data set, wherein the dot product result is the dot product of the gradient of the reconstruction error of the input data set and the gradient of the target distance, and the target distance is the distance between the density distribution of the vector set and the predetermined density distribution in the objective function of the model. If the dot product result is negative or zero, the encoder of the model is adjusted according to the gradient of the target distance.

8. The method as described in claim 7, characterized in that, The step of comparing the density distribution of the vector set with a predetermined density distribution includes: determining the difference between the density distribution of the vector set and the predetermined density distribution based on a distance metric.

9. The method as described in claim 8, characterized in that, The distance metric is the Coulomb kernel.

10. The method as described in claim 7, characterized in that, The artificial intelligence model is a neural network.

11. A computer program product, characterized in that, When the computer program product is executed by a computer, the computer program causes the computer to perform the method of any one of claims 7 to 10.

Images