Image data augmentation method and system based on optimal transport theory

Abstract

The application provides an image data expansion method and system based on optimal transmission theory, belongs to the technical field of image processing, combines an auto-encoder with optimal transmission theory, replaces a second task of image generation, i.e. probability distribution conversion, with optimal transmission theory. Manifold learning is completed by the auto-encoder, and probability distribution conversion is completed by optimal transmission mapping. A semi-discrete optimal transmission mapping network is established by using a Monte Carlo method, the network maps continuous white noise distribution into discrete distribution of data in a hidden space, and solves the problem of using continuous mapping of a neural network to approximate a discontinuous distribution. Finally, the image data generated by the image data expansion method based on optimal transmission theory is tested and analyzed, the application solves the problems of mode collapse and mode mixing in image generation, and the effectiveness of the application is verified through the test result analysis.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an image data augmentation method and system based on optimal transmission theory. Background Technology

[0002] In recent years, deep learning has developed rapidly, and its applications in computer vision have matured. With the help of deep learning, such as convolutional neural networks, the accuracy and precision of image processing have improved. This includes image classification, feature extraction, distinguishing between real and fake samples, and generating images that closely resemble real training samples. However, despite numerous analyses of the interpretability of neural networks by scientists both domestically and internationally, their application in technological fields with high interpretability requirements, such as military or medical fields, remains limited. Generative Adversarial Networks (GANs), the most widely used network in image generation, suffers from an adversarial relationship between the generator responsible for image generation and the discriminator that distinguishes between real and virtual samples, which is detrimental to image generation and leads to computational redundancy. Existing image generation models suffer from problems such as mode collapse. The generated images fail to reflect the complexity and diversity of the real world or are of low quality, making them unsuitable for practical applications. Optimal transport theory reveals that the main reason for model collapse is the contradiction between discontinuous optimal transport mappings and the fact that convolutional neural networks can only express continuous mappings. From a geometric perspective, the image generation model completes two parts: manifold learning and probability distribution transformation. Since generative models are built based on neural networks, their interpretability is relatively poor. Summary of the Invention

[0003] The purpose of this invention is to provide an image data augmentation method and system based on optimal transmission theory, which applies optimal transmission theory to an image generation model and uses the Monte Carlo method to solve a semi-continuous optimal transmission algorithm, thereby solving at least one of the technical problems existing in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] On one hand, the present invention provides an image data augmentation method based on optimal transmission theory, comprising:

[0006] The original image data is dimensionality reduced by using the encoder in the pre-built autoencoder;

[0007] Using a pre-constructed optimal transport mapping network, the data distribution of the dimensionality-reduced data is transformed from the sampling space to the latent space of the target data, and the transformed data distribution result is output.

[0008] The output of the optimal transport mapping network is processed and decoded using the decoder in the pre-built autoencoder to output the generated image.

[0009] Furthermore, the encoder contains one input layer, four regular convolutional layers, one dilated convolutional layer, and one fully connected layer; the decoder contains one input layer, one upsampling layer, and seven deconvolutional layers.

[0010] Furthermore, the data distribution transformation from the sampling space to the target data latent space is achieved, including: using the [0,1] subspace of the N-dimensional space as the sampling space Ω, randomly sampling n points in the sampling space and assigning a weight of 1 / n; taking the manifold distribution of the data in the latent space as the target space Y, assuming there are m target points, assigning a weight of 1 / m to each target point; using the Monte Carlo method to obtain m cavity decompositions of the sampling space Ω, while ensuring that the m cavities have the same area, which is 1 / n; selecting the centroid of the decomposed cavities, summing the sampling points distributed in each cavity and taking their arithmetic mean to obtain a uniform distribution of the N-dimensional subspace Ω.

[0011] Furthermore, the Monte Carlo method is used to obtain the m-cavity decomposition of the sampling space Ω, including: initializing the height between the hyperplane and the sampling plane in each upper envelope according to the number of target discrete points; sampling an arbitrary number of points in the sampling space to fit the total measure of the sampling space; and through... Calculate the location of the transmitted sampling points to give the label of each sampling point; update the height h corresponding to the upper envelope hyperplane using gradient descent until convergence.

[0012] Furthermore, sampling is performed in the latent space of the data. The area of ​​the sampling space is fitted by calculating the number of discrete points within the cavity. The Brenier potential function is adjusted so that the area of ​​each cavity is equal to the target measure.

[0013] Assume that X and Y are both d-dimensional Euclidean spaces R. d If a subset of X is given and the transmission cost is Euclidean distance, then there exists a convex function u:X→R called the Brenier potential. The gradient of this function gives the solution to the Monge problem: That is, the solution to the Monge-Ampère equation is the gradient of a continuous function.

[0014] Furthermore, under discrete measure, sampling within the n-dimensional space [0,1] and assigning a weight to each sampling point yields the probability measure μ over Ω; treating the latent space of the image as the target space, with each target point having a weight, we obtain the target space and the target measure υ; ensuring that the sum of the probability measures in the target space and the source space is the same, i.e. Suppose Y = {y1, y2, ..., y} n If} is the training data, then for each training sample y i Each corresponds to a supporting plane of a Brenier potential: π h,i (x):=<x,yi >+h i ;

[0015] h is calculated iteratively for each sample point. i Given the top-level envelope, the Brenier potential u h :Ω→R is a continuous convex function, and the height of each hyperplane is determined by h:

[0016]

[0017] Furthermore, the Brenier potential is an upper envelope composed of multiple convex polygons, each supporting a plane π. h,i Each of these is a convex plane, and each convex plane in the upper envelope induces a cavity decomposition of the sampling space Ω. Each supporting plane π is defined as... i The projection is W i (h), the hyperplane clusters constitute the upper envelope, and the cavities induced by each upper envelope constitute the sampling space:

[0018]

[0019] Furthermore, the height between the hyperplane and the sampling plane in each upper envelope is initialized according to the number of target discrete points. Calculate the location of the transmitted sampling point, and use this to assign a label to each sampling point. Divide the sampling space into several cavities, and update the height h corresponding to the upper envelope hyperplane using the gradient descent method until convergence.

[0020] Secondly, the present invention provides an image data augmentation system based on optimal transmission theory, comprising:

[0021] The dimensionality reduction module is used to reduce the dimensionality of the original image data using the encoder in the pre-built autoencoder;

[0022] The transformation module is used to transform the data distribution from the sampling space to the latent space of the target data using a pre-selected optimal transport mapping network, and outputs the transformed data distribution result.

[0023] The decoding module is used to process and decode the output of the optimal transport mapping network using the decoder in the pre-built autoencoder, and output the generated image.

[0024] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the image data augmentation method based on optimal transmission theory as described above.

[0025] Fourthly, the present invention provides a computer device including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the image data augmentation method based on optimal transmission theory as described above.

[0026] Fifthly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the image data augmentation method based on the optimal transmission theory as described above.

[0027] The beneficial effects of this invention are as follows: It combines the optimal transmission algorithm of arbitrary dimensions with the current neural network to propose a semi-transparent image generation model; it achieves the generation of high-quality image data; it effectively solves the mode collapse and mode mixing phenomena that occur in the current image generation model; and it realizes the optimal transmission algorithm in arbitrary dimensions, breaking the bottleneck of the previous geometric optimal transmission algorithm to high dimensions.

[0028] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a diagram of the image generation model based on optimal transmission as described in an embodiment of the present invention.

[0031] Figure 2 This is the Brenier potential diagram described in an embodiment of the present invention.

[0032] Figure 3 This is a diagram of the semi-discrete optimal transmission network algorithm described in an embodiment of the present invention.

[0033] Figure 4 This is a diagram of the encoder network architecture described in an embodiment of the present invention.

[0034] Figure 5 This is a diagram of the decoder network architecture described in an embodiment of the present invention.

[0035] Figure 6This is a diagram showing the MNIST image and the generated MNIST data results as described in an embodiment of the present invention.

[0036] Figure 7 This is a flowchart of the image data augmentation method based on optimal transmission theory according to an embodiment of the present invention. Detailed Implementation

[0037] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0038] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.

[0040] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0041] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0042] To facilitate understanding of the present invention, the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0043] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.

[0044] Example 1

[0045] In this embodiment 1, an image data augmentation system based on optimal transport theory is first provided, including: a dimensionality reduction module, used to reduce the dimensionality of the original image data using the encoder in a pre-built autoencoder; a transformation module, used to transform the data distribution from the sampling space to the latent space of the target data using a pre-selected optimal transport mapping network, and output the transformed data distribution result; and a decoding module, used to process and decode the output result of the optimal transport mapping network using the decoder in a pre-built autoencoder, and output the generated image.

[0046] In this embodiment, the above-described system is used to implement an image data augmentation method based on optimal transport theory. The method includes: using the encoder in a pre-constructed autoencoder to reduce the dimensionality of the original image data; using a pre-selected optimal transport mapping network to transform the data distribution from the sampling space to the latent space of the target data after dimensionality reduction, and outputting the transformed data distribution result; and using the decoder in the pre-constructed autoencoder to process and decode the output result of the optimal transport mapping network, and outputting the generated image. The autoencoder contains one input layer, four ordinary convolutional layers, one dilated convolutional layer, and one fully connected layer; the decoder contains one input layer, one upsampling layer, and seven deconvolutional layers.

[0047] The data distribution transformation from the sampling space to the target data latent space includes: using the [0,1] subspace of the N-dimensional space as the sampling space Ω, randomly sampling n points in the sampling space and assigning a weight of 1 / n; taking the manifold distribution of the data in the latent space as the target space Y, assuming there are m target points, and assigning a weight of 1 / m to each target point; using the Monte Carlo method to obtain m cavity decompositions of the sampling space Ω, while ensuring that the m cavities have the same area, which is 1 / n; selecting the centroid of the decomposed cavities, summing the sampling points distributed in each cavity and taking their arithmetic mean to obtain a uniform distribution of the N-dimensional subspace Ω.

[0048] The Monte Carlo method is used to obtain the m-cavity decomposition of the sampling space Ω, including: initializing the height between the hyperplane and the sampling plane in each upper envelope according to the number of target discrete points; sampling an arbitrary number of points in the sampling space to fit the total measure of the sampling space; and then... Calculate the location of the transmitted sampling points to give the label of each sampling point; update the height h corresponding to the upper envelope hyperplane using gradient descent until convergence.

[0049] Sampling is performed in the latent space of the data. The area of ​​the sampling space is fitted by calculating the number of discrete points within the cavity. The Brenier potential function is adjusted so that the area of ​​each cavity is equal to the target measure.

[0050] Assume that X and Y are both d-dimensional Euclidean spaces R. d If a subset of X is given and the transmission cost is Euclidean distance, then there exists a convex function u:X→R called the Brenier potential. The gradient of this function gives the solution to the Monge problem: That is, the solution to the Monge-Ampère equation is the gradient of a continuous function.

[0051] In discrete measure, sampling within the n-dimensional space [0,1] and assigning weights to each sampling point yields the probability measure μ over Ω. Treating the latent space of the image as the target space, with each target point having a weight, we obtain the target space and the target measure υ. It is ensured that the sum of the probability measures in the target space and the source space is the same, i.e. Suppose Y = {y1, y2, ..., y} n If} is the training data, then for each training sample y i Each corresponds to a supporting plane of a Brenier potential: π h,i (x):=<x,y i >+h i ;

[0052] h is calculated iteratively for each sample point. i Given the top-level envelope, the Brenier potential u h :Ω→R is a continuous convex function, and the height of each hyperplane is determined by h:

[0053]

[0054] The Brenier potential is an upper envelope composed of multiple convex polygons, each supporting a plane π. h,i Each of these is a convex plane, and each convex plane in the upper envelope induces a cavity decomposition of the sampling space Ω. Each supporting plane π is defined as... i The projection is W i (h), the hyperplane clusters constitute the upper envelope, and the cavities induced by each upper envelope constitute the sampling space:

[0055]

[0056] The height between the hyperplane and the sampling plane in each upper envelope is initialized based on the number of target discrete points. Calculate the location of the transmitted sampling point, and use this to assign a label to each sampling point. Divide the sampling space into several cavities, and update the height h corresponding to the upper envelope hyperplane using the gradient descent method until convergence.

[0057] Example 2

[0058] To address the limitations of neural network applications in fields with high interpretability requirements, this embodiment focuses on establishing interpretable neural networks using reliable mathematical theories. Based on data manifolds and optimal transport theory, it analyzes optimal transport networks using mathematical theory to establish new models. This not only solves the problem of model interpretability but also more effectively utilizes neural networks to complete image generation tasks, resolving the pattern collapse problem in generative adversarial networks and providing a theoretical guarantee for interpretability in deep learning networks. Optimal transport theory is based on transforming data distribution with minimal transport cost, combined with the idea of ​​reducing data dimensionality to the latent space to observe its distribution manifold in data manifolds.

[0059] like Figures 1 to 7 As shown, in this embodiment, a semi-discrete optimal transport network is established using the Monte Carlo method. This network is then combined with an autoencoder in a convolutional neural network, replacing the autoencoder's sampling method in the latent space with a more interpretable optimal transport mapping method to establish a new autoencoder-optimal transport model. The feature extraction from the autoencoder is then visualized to further improve the model's interpretability.

[0060] This embodiment 2 provides a novel image data augmentation method, including:

[0061] The original image data is input into the encoder of the constructed autoencoder to reduce the dimensionality of the data.

[0062] The dimensionality-reduced data is input into the constructed optimal transport mapping network to realize the data distribution transformation from the sampling space to the target data latent space, and the result of the data distribution transformation is output.

[0063] The output of the optimal transport mapping network is input into the decoder of the constructed autoencoder to output the generated image.

[0064] An autoencoder comprises: an encoder containing one input layer, four regular convolutional layers, one dilated convolutional layer, and one fully connected layer; and a decoder containing one input layer, one upsampling layer, and seven deconvolutional layers.

[0065] The optimal transport mapping network transforms the data distribution from the sampling space to the target data latent space. This includes: using a subspace of [0,1] in an N-dimensional space as the sampling space Ω, randomly sampling n points with a weight of 1 / n. The manifold distribution of data in the latent space is treated as the target space Y, assuming m target points, each with a weight of 1 / m. The Monte Carlo method is used to decompose the sampling space Ω into m cavities, ensuring that the m cavities have the same area, 1 / n. The centroids of the decomposed cavities are selected, and the sampling points distributed in each cavity are summed and their arithmetic mean is taken to obtain a uniform distribution in the N-dimensional subspace Ω. The centroids of the decomposed cavities correspond to the data points in the latent space, achieving the data distribution transformation. The Monte Carlo method for obtaining the m-cavity decomposition of the sampling space Ω includes: initializing the height between the hyperplane and the sampling plane in each upper envelope based on the number of target discrete points. Sampling an arbitrary number of points in the sampling space to fit the total measure of the sampling space. Calculate the location of the transmitted sampling points to assign a label to each sampling point. Update the height h corresponding to the upper envelope hyperplane using gradient descent until convergence.

[0066] In this embodiment, an arbitrary-dimensional optimal transmission algorithm is combined with a deep neural network to propose a semi-transparent image generation model. For example... Figure 1 As shown, the specific steps are as follows:

[0067] Step 1: Solve for the optimal transport map using the Monte Carlo method.

[0068] Sampling is performed in the latent space of the data. The area of ​​the sampling space is fitted by calculating the number of discrete points within each cavity. The Breniier potential function is adjusted so that the area of ​​each cavity equals the target measure. The mathematical calculation method is as follows:

[0069] Assume that X and Y are both d-dimensional Euclidean spaces R. d If a subset of X is given, and the transmission cost is always Euclidean distance, then there exists a convex function u:X→R called the Breniier potential. The gradient of this function gives the solution to the Monge problem. It can be expressed as:

[0070]

[0071] That is, the solution to the Monge-Ampère equation is the gradient of a continuous function (Brenier potential).

[0072] In discrete measure, sampling within the n-dimensional space [0,1] (denoted as Ω) and assigning a weight to each sampling point yields the probability measure μ over Ω. Treating the latent space of the image as the target space, each target point has a weight, resulting in the target space and target measure υ. It is ensured that the sum of the probability measures in the target space and the source space is the same, i.e. Suppose Y = {y1, y2, ..., y} n If} is the training data, then for each training sample y i Each corresponds to a supporting plane of the Brenier potential, which can be represented as:

[0073] π h,i (x):=<x,y i >+h i

[0074] h is calculated iteratively for each sample point. i Obtain the top-level envelope. For example... Figure 2 As shown, the Brenier potential u h :Ω→R is a continuous convex function, and the height of each hyperplane is determined by h.

[0075]

[0076] The Brenier potential is an upper envelope composed of multiple convex polygons, each supporting a plane π. h,i Each of these is a convex plane, and each convex plane in the upper envelope induces a cavity decomposition of the sampling space Ω. Each supporting plane π is defined as... i The projection is W i (h) represents the hyperplane clusters that form the upper envelope, and the cavities induced by each upper envelope constitute the sampling space. This can be represented as:

[0077]

[0078] The height between the hyperplane and the sampling plane in each upper envelope is initialized based on the number of target discrete points. The location of the transmitted sampling points is calculated, and a label is assigned to each sampling point accordingly. The sampling space is divided into several cavities. The flowchart of the cavity decomposition algorithm is as follows: Figure 3 As shown, the height h corresponding to the upper envelope hyperplane is updated iteratively using gradient descent until convergence.

[0079] Step 2: Use an autoencoder for manifold learning.

[0080] Construct an autoencoder, the structure of which is as follows: Figure 4 As shown, the encoder contains one input layer, four regular convolutional layers, one dilated convolutional layer, and one fully connected layer. The decoder structure is as follows. Figure 5 As shown, the decoder contains one input layer, one upsampling layer, and seven deconvolutional layers.

[0081] During training, the sigmoid function was used as the activation function, and the loss function was defined as the L2 norm (MSE) of the difference between the generated and real images. The Adam optimizer was used for optimization. When the latent space data dimension was set to 2D, the loss after 6000 reconstructed images was 0.033. When the latent space data was increased to 10D, the loss after 6000 reconstructed images on the training dataset was 0.010.

[0082] Step 3: Construct an image generation model

[0083] The probability distribution transformation of the latent space of image data is achieved by using the optimal transport mapping. The manifold data after dimensionality reduction of the MNIST dataset is taken as the target distribution. Given each target point with the same weight, the equal area cavity decomposition in the N-dimensional data latent space can be obtained. That is, the latent space distribution of image data (target space Y) is transformed into a uniform distribution on the N-dimensional latent space [0,1], thereby realizing the data distribution transformation.

[0084] The semi-continuous optimal transport mapping achieved through the Monte Carlo method transforms the data latent space distribution into a uniform distribution, and the generator is retrained to achieve the purpose of image generation.

[0085] In this embodiment, the experiments used the open-source datasets MNIST, CelebA-HQ, UAVid, and VisDrone2019 for validation. The image data augmentation method based on optimal transmission theory in this embodiment was tested and validated using the FID image evaluation criterion.

[0086] The calculation method for the FID evaluation criteria is as follows:

[0087] The Inception network is used to extract meaningful features from real and generated images.

[0088] Use a Gaussian distribution (normal distribution) to fit the feature distribution of the real data and the generated data.

[0089] The distance between two distributions and the Gaussian distribution is calculated using the following formula:

[0090]

[0091] The generated MNIST data results are as follows Figure 6 As shown in the figure. On the MNIST dataset, the decoder is retrained using the correspondence between the uniform centroid and the background spatial image data, and the reconstruction error dimension of the decoder is 0.0068. Then, uniform sampling is performed in the latent space and fed into the decoder to complete the image generation task. The FID distance between the generated image data and the original image data is calculated, and the result is 8.39.

[0092] The autoencoder proposed in this embodiment was used to train the autoencoder on the CelebA dataset. The latent space data dimension was set to 128 dimensions. The error between the reconstructed image and the real image after training the autoencoder was 0.0013, and the FID distance between the generated image and the real image was calculated to be 67.32.

[0093] Table 1 shows a comparison of the FID scores between the image data augmentation method based on optimal transport theory proposed in this embodiment and the output results of three generative models (WGAN, AE, and VAE) trained on four datasets: MNIST, UAvid, CelebA, and VisDrone2019. The simpler the images in the dataset, the lower the FID score, indicating a more realistic generation. The table shows that the FID score of the model proposed in this embodiment is the lowest in all four training sets, demonstrating that this model can generate more realistic images.

[0094] Table 1. FID scores of the four models on different datasets.

[0095]

[0096] The experimental results show that the image data augmentation method based on optimal transmission theory in this embodiment can effectively generate high-quality images and complete the data augmentation task, demonstrating significant practical application value.

[0097] Example 3

[0098] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When executed by a processor, the computer instructions implement the image data augmentation method based on optimal transmission theory as described above. The method includes:

[0099] The original image data is dimensionality reduced by using the encoder in the pre-built autoencoder;

[0100] Using a pre-constructed optimal transport mapping network, the data distribution of the dimensionality-reduced data is transformed from the sampling space to the latent space of the target data, and the transformed data distribution result is output.

[0101] The output of the optimal transport mapping network is processed and decoded using the decoder in the pre-built autoencoder to output the generated image.

[0102] Example 4

[0103] This embodiment 4 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, and the memory stores program instructions that can be executed by the processor. The processor calls the program instructions to execute an image data augmentation method based on optimal transmission theory, the method including:

[0104] The original image data is dimensionality reduced by using the encoder in the pre-built autoencoder;

[0105] Using a pre-constructed optimal transport mapping network, the data distribution of the dimensionality-reduced data is transformed from the sampling space to the latent space of the target data, and the transformed data distribution result is output.

[0106] The output of the optimal transport mapping network is processed and decoded using the decoder in the pre-built autoencoder to output the generated image.

[0107] Example 5

[0108] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the image data augmentation method based on optimal transmission theory as described above. The method includes:

[0109] The original image data is dimensionality reduced by using the encoder in the pre-built autoencoder;

[0110] Using a pre-constructed optimal transport mapping network, the data distribution of the dimensionality-reduced data is transformed from the sampling space to the latent space of the target data, and the transformed data distribution result is output.

[0111] The output of the optimal transport mapping network is processed and decoded using the decoder in the pre-built autoencoder to output the generated image.

[0112] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. An image data augmentation method based on optimal transmission theory, characterized in that, include: The original image data is dimensionality reduced by using the encoder in the pre-built autoencoder; Using a pre-constructed optimal transport mapping network, the data distribution transformation from the sampling space to the latent space of the target data is achieved on the dimensionality-reduced data, and the transformed data distribution result is output. The data distribution transformation from the sampling space to the latent space includes: using N-dimensional space... The subspace is taken as the sampling space Ω, and random sampling is performed in the sampling space. Given points and weights Treat the manifold distribution of data in the latent space as the target space. Assuming there is There are 1 target points, each with a weight. The sampling space Ω was obtained using the Monte Carlo method. Each cavity is decomposed, while ensuring Each cavity has the same area, which is Select the centroid of the decomposed cavity, sum the sampling points distributed in each cavity, and take their arithmetic mean to obtain... A uniform distribution in the dimensional subspace Ω; wherein, the sampling space Ω is obtained using the Monte Carlo method. Each cavity decomposition includes: initializing the height between the hyperplane and the sampling plane in each upper envelope based on the number of target discrete points; sampling an arbitrary number of points in the sampling space to fit the total measure of the sampling space; and through... Calculate the location of the transmitted sampling points to assign a label to each sampling point; update the height of the upper envelope hyperplane iteratively using gradient descent. Until convergence; The output of the optimal transport mapping network is processed and decoded using the decoder in the pre-built autoencoder to output the generated image.

2. The image data augmentation method based on optimal transmission theory according to claim 1, characterized in that, The encoder contains one input layer, four regular convolutional layers, one dilated convolutional layer, and one fully connected layer; the decoder contains one input layer, one upsampling layer, and seven deconvolutional layers.

3. The image data augmentation method based on optimal transmission theory according to claim 1, characterized in that, Sampling is performed in the latent space of the data. The area of ​​the sampling space is fitted by calculating the number of discrete points within the cavity. The Brenier potential function is adjusted so that the area of ​​each cavity is equal to the target measure. assumed and All Vieux Space If a subset of the set of objects has a transmission cost of Euclidean distance, then there exists a convex function called the Brenier potential. The gradient of this function gives the solution to the Monge problem: That is, the solution to the Monge-Ampère equation is the gradient of a continuous function.

4. The image data augmentation method based on optimal transmission theory according to claim 3, characterized in that, In discrete measure, 3D space Internal sampling, assigning weights to each sampling point, yields... probability measure on Treating the latent space of the image as the target space, with each target point having a weight, yields the target space and target measure. ; Ensure that the sum of the probability measures in the target space and the source space is the same, that is Assuming If it's training data, then each training sample... Each corresponds to a supporting plane of the Brenier potential: ; Each sample point is calculated iteratively. Given the top-level envelope, the Brenier potential It is a continuous convex function, and the height of each hyperplane is determined by... Decide: 。 5. The image data augmentation method based on optimal transmission theory according to claim 4, characterized in that, The Brenier potential is an upper envelope composed of multiple convex polygons, each supporting a plane. Each of these is a convex plane, and each convex plane in the upper envelope induces a sampling space. The cavity is decomposed, and each supporting plane is defined. The projection is The hyperplane clusters form the upper envelope, and the cavities induced by each upper envelope constitute the sampling space: 。 6. The image data augmentation method based on optimal transmission theory according to claim 5, characterized in that, The height between the hyperplane and the sampling plane in each upper envelope is initialized based on the number of target discrete points. The location of the transmitted sampling points is calculated, and the label of each sampling point is given accordingly. The sampling space is divided into several cavities, and the height corresponding to the upper envelope hyperplane is updated iteratively using the gradient descent method. Until convergence.

7. An image data augmentation system based on optimal transmission theory, characterized in that, include: The dimensionality reduction module is used to reduce the dimensionality of the original image data using the encoder in the pre-built autoencoder; The transformation module utilizes a pre-selected, optimal transport mapping network to transform the data distribution from the sampling space to the latent space of the target data after dimensionality reduction, and outputs the transformed data distribution result. The transformation from the sampling space to the latent space includes: using an N-dimensional space... The subspace is taken as the sampling space Ω, and random sampling is performed in the sampling space. Given points and weights Treat the manifold distribution of data in the latent space as the target space. Assuming there is There are 1 target points, each with a weight. The sampling space Ω was obtained using the Monte Carlo method. Each cavity is decomposed, while ensuring Each cavity has the same area, which is Select the centroid of the decomposed cavity, sum the sampling points distributed in each cavity, and take their arithmetic mean to obtain... A uniform distribution in the dimensional subspace Ω; wherein, the sampling space Ω is obtained using the Monte Carlo method. Each cavity decomposition includes: initializing the height between the hyperplane and the sampling plane in each upper envelope based on the number of target discrete points; sampling an arbitrary number of points in the sampling space to fit the total measure of the sampling space; and through... Calculate the location of the transmitted sampling points to assign a label to each sampling point; update the height of the upper envelope hyperplane iteratively using gradient descent. Until convergence; The decoding module is used to process and decode the output of the optimal transport mapping network using the decoder in the pre-built autoencoder, and output the generated image.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the image data augmentation method based on optimal transmission theory as described in any one of claims 1-6.

Images