An image generation method based on shape space theory

Abstract

This invention relates to an image generation method based on shape space theory, comprising the following steps: inputting an image dataset and random noise; constructing an image generation model and a pre-shape space; the image generation model includes a generator and a discriminator; randomly selecting a set of images from the image dataset, inputting them into the discriminator to obtain training features, projecting the training features to obtain data features, constructing a geodesic surface on the pre-shape space, and obtaining sample features based on randomly generated sampling weights; generating random noise through Gaussian distribution sampling, calculating anchor point features through the random noise and sample features, and constructing anchor point images and interpolated images, extracting anchor point features and interpolated image features; constructing a loss function using sample features, anchor point features, and interpolated image features, and updating the generator and discriminator; and generating an image by inputting random noise through the updated generator and discriminator. Compared with existing technologies, this invention does not require additional information for training and generates high-quality images.

Description

Technical Field

[0001] This invention relates to the field of machine learning, and in particular to an image generation method based on shape space theory. Background Technology

[0002] When using machine learning, especially deep learning, for data processing, a large number of training samples are required. However, due to factors such as equipment costs, labor costs, and time costs, the availability of labeled, high-quality training samples is often limited. The quantity and diversity of training samples can be increased by training an image generation model to generate a sufficient number of high-quality samples.

[0003] Current few-shot image generation models mainly fall into two categories: one is the few-shot adaptation method, which uses a pre-trained model that is semantically related to the training set for training. This type of method can achieve good image generation results with a very small number of training samples, but semantically related pre-trained models are not easy to obtain. The other type does not use any additional information and trains the model through data augmentation, changing the model structure, adding regularization terms, etc. However, this type of method does not achieve the expected generation results in extremely few-shot cases with fewer than 10 training samples.

[0004] To overcome the limitations of current few-sample image generation methods, it is necessary to propose an image generation method that can generate high-quality images even in extremely small-sample training scenarios without requiring any additional information for training. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies in image generation that fail to achieve the expected results, and to provide an image generation method based on shape space theory.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] An image generation method based on shape space theory includes the following steps:

[0008] S1: Input image dataset and random noise, construct image generation model and pre-shape space. Image generation model includes generator and discriminator;

[0009] S2: Randomly select a set of images from the image dataset, input them into the discriminator to obtain training features, project the training features to obtain data features, construct a geodesic surface in the pre-shape space based on the data features, and obtain sample features from the geodesic surface according to the randomly generated sampling weights.

[0010] S3: Generate random noise through Gaussian distribution sampling, calculate anchor point features through random noise and sample features, construct anchor point images and interpolated images, and extract anchor point features and interpolated image features;

[0011] S4: Construct the loss using sample features, anchor features, and interpolated image features, and update the generator and discriminator;

[0012] S5: Using the updated generator and discriminator, input random noise to generate an image.

[0013] Furthermore, step S2 specifically includes the following steps:

[0014] S201: Randomly select a set of images from the image dataset as the training set, input the training set into the specified convolutional layer of the discriminator, and extract training features;

[0015] S202: Project the extracted training features onto the pre-shape space to obtain data features;

[0016] S203: Construct geodesic curve functions based on data characteristics;

[0017] S204: Generate a set of weights based on the Dirichlet distribution, iterate the geodesic curve function based on the weights to obtain the intermediate feature vector, and obtain the geodesic surface function, i.e. the sample features, when the number of iterations is equal to the number of images in the training set;

[0018] S205: Repeat S204, generating a new set of weights each time to obtain multiple sets of sample features.

[0019] Furthermore, the expression for the geodesic curve function is:

[0020]

[0021] In the formula, Let be a curve function, where τ1 and τ2 represent the start and end characteristic vectors of the geodesic curve, the parameter s controls the position of the generated vector on the geodesic curve, and d(τ1,τ2) represents the geodesic distance between τ1 and τ2.

[0022] Furthermore, step S3 specifically includes the following steps:

[0023] S301: For each set of sample features obtained in step S205, a set of random noise is obtained by sampling through Gaussian distribution. Based on the weights corresponding to the sample features and the random noise, the anchor noise is calculated.

[0024] S302: By performing linear interpolation on random noise sampled from two adjacent Gaussian distributions, a set of interpolated noise is obtained;

[0025] S303: Input the anchor noise and interpolation noise sets into the generator to obtain the anchor image and the interpolation image. Then, input the anchor image into the specified convolutional layer of the discriminator to obtain the interpolation image features.

[0026] S304: Project the interpolated image features onto the pre-shape space to obtain anchor point features.

[0027] Furthermore, step S4 specifically includes the following steps:

[0028] S401: Calculate the cosine similarity between the vector of the sampled feature at the first position and the vector of the neighborhood of the first position;

[0029] S402: Calculate the cosine similarity between the vector of the anchor point feature at the first position and the vector of the neighborhood of the first position;

[0030] S403: Traverse all corresponding positions in the space of the first position neighborhood to obtain the autocorrelation matrices of the sampled features and anchor features at the first position respectively;

[0031] S404: Construct autocorrelation consistency loss based on the autocorrelation matrix of sampling features and anchor features;

[0032] S405: Construct a non-saturating loss based on the interpolated image to supervise the interpolated image;

[0033] S406: Construct the first multidimensional vector, the dimension of which is equal to the number of elements in the interpolation feature set;

[0034] S407: Construct a second multidimensional vector, the dimension of which is equal to the number of elements in the interpolation feature set;

[0035] S408: Construct distance constraint loss between interpolated images;

[0036] S409: Construct the adversarial loss of the generator, and obtain the optimization function of the generator by interpolating the distance constraint loss between images and the adversarial loss;

[0037] S410: Construct the adversarial loss of the discriminator, and construct the optimization function of the discriminator;

[0038] S411: Update the parameters of the generator and discriminator based on the optimization functions of the generator and discriminator until the loss converges.

[0039] Furthermore, the expression for autocorrelation consistency loss is:

[0040]

[0041] In the formula, L sl1 (·) represents the smooth-l1 loss function. The cosine similarity of the sampled features. Let ω be the cosine similarity of the anchor features, and ω ~ Dir be the Dirichlet distribution weights. The training set consists of image data, where z ~ p(z) represents random noise sampled using a Gaussian distribution. This indicates the calculation of expectation.

[0042] Furthermore, the expression for calculating the unsaturated loss is:

[0043]

[0044] In the formula, L inp For unsaturated loss, Z inp (z′1,z′ k ) is the interpolation noise set. For interpolated image features, log(·) is the logarithm to base 10, z′1, z′ k ~p(z) represents interpolation noise, This indicates the calculation of expectation.

[0045] Furthermore, the expression for the first multidimensional vector is:

[0046]

[0047] In the formula, V inp [·] represents the first multidimensional vector. For the anchor point image, z′ i represents the i-th interpolation noise, and k is the dimension of the vector.

[0048] Furthermore, the expression for the second multidimensional vector is:

[0049] V q [1:k-1]=1, V q [k] = k-1

[0050] In the formula, V q [·] represents the second multidimensional vector, and k is the dimension of the vector.

[0051] Furthermore, the expression for calculating the distance constraint loss between images is as follows:

[0052] L dl =L kl (V inp V q )

[0053] In the formula, L dl For the distance constraint loss between images, L kl For KL divergence loss, V inp V is the first multidimensional vector. q It is the second multidimensional vector.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1) This invention obtains sampling features by mapping the training features of the original image in a pre-shape space, calculates anchor features and interpolated image features based on the sampling features, and uses them to train the generator of the discriminator, thereby obtaining an image generation model that can output images without the need for additional information for training, and the generated images are of high quality.

[0056] 2) In the context of data scarcity, this invention combines shape space theory to effectively depict the feature distribution space, thereby achieving efficient image generation model training. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0058] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0059] like Figure 1 As shown, this invention is an image generation method based on shape space theory, comprising the following steps:

[0060] S1: Input image dataset and random noise, construct image generation model and pre-shape space. Image generation model includes generator and discriminator;

[0061] S2: Randomly select a set of images from the image dataset, input them into the discriminator to obtain training features, project the training features to obtain data features, construct a geodesic surface in the pre-shape space based on the data features, and obtain sample features from the geodesic surface according to the randomly generated sampling weights.

[0062] S3: Generate random noise through Gaussian distribution sampling, calculate anchor point features through random noise and sample features, construct anchor point images and interpolated images, and extract anchor point features and interpolated image features;

[0063] S4: Construct the loss using sample features, anchor features, and interpolated image features, and update the generator and discriminator;

[0064] S5: Using the updated generator and discriminator, input random noise to generate an image.

[0065] Example 1

[0066] This invention relates to an image generation method based on shape space theory, comprising the following steps:

[0067] S1: Input image dataset and random noise, construct image generation model and pre-shape space. Image generation model includes generator and discriminator;

[0068] S2: Randomly select a set of images from the image dataset, input them into the discriminator to obtain training features, project the training features to obtain data features, construct a geodesic surface in the pre-shape space based on the data features, and obtain sample features from the geodesic surface according to randomly generated sampling weights; specifically including the following steps:

[0069] S201: Randomly select a set of images from the image dataset as the training set x, including n images, and input the training set x into the discriminator. In the specified convolutional layer l, the training features are extracted, denoted as... c represents the number of channels, h represents the height, and w represents the width;

[0070] S202: Extracted training features Projecting onto the pre-shape space yields the data features τ.

[0071] S203: Construct geodesic curve functions based on data characteristics; the expression for the geodesic curve function is:

[0072]

[0073] In the formula, Let be a curve function, where τ1 and τ2 represent the start and end characteristic vectors of the geodesic curve, the parameter s controls the position of the generated vector on the geodesic curve, and d(τ1,τ2) represents the geodesic distance between τ1 and τ2.

[0074] S204: Generate a set of weights ω based on the Dirichlet distribution. Based on the weighted iterative geodesic curve function, the intermediate eigenvector is obtained, expressed as:

[0075]

[0076] In the formula, μ j Let be the j-th intermediate vector.

[0077] When the number of iterations equals the number of images in the training set, the geodesic surface function is obtained. That is, sample characteristics;

[0078] S205: Repeat S204, generating a new set of weights each time to obtain multiple sets of sample features.

[0079] S3: Generate random noise through Gaussian distribution sampling, calculate anchor point features using the random noise and sample features, construct anchor point images and interpolated images, and extract anchor point features and interpolated image features; specifically including the following steps:

[0080] S301: For each set of sample features obtained in step S205, a set of random noise {z} is obtained by sampling using a Gaussian distribution. i For i∈[1,n]}, calculate the anchor noise based on the weights and random noise corresponding to the features of the sample.

[0081] S302: By performing linear interpolation on random noise sampled from two adjacent Gaussian distributions, a set of interpolated noise is obtained. z′1,z′2,...,z′ k This is the interpolation result;

[0082] S303: Anchor point noise and interpolation noise set Z inp Input the generator to obtain the anchor point image. and interpolated images Then input the anchor point image into the discriminator. In the specified convolutional layer l, the interpolated image features are obtained.

[0083] S304: Interpolate image features Projecting onto the pre-shape space yields anchor point features. c represents the number of channels, h represents the height, and w represents the width.

[0084] S4: Construct the loss using sample features, anchor features, and interpolated image features, and update the generator and discriminator; specifically including the following steps:

[0085] S401: Calculate sampling features The vector at the first position (u,v) The vector of the first position neighborhood (a,b) cosine similarity The calculation expression is:

[0086]

[0087] In the formula, <·,·> denotes the dot product, and ||·|| denotes the Euclidean norm;

[0088] S402: Calculate anchor point features The vector at the first position (u,v) The vector of the first position neighborhood (a,b) cosine similarity The calculation expression is:

[0089]

[0090] In the formula, <·,·> denotes the dot product, and ||·|| denotes the Euclidean norm;

[0091] S403: Traverse all corresponding positions in space of the first location's neighborhood (a, b) to obtain the sampling features. and anchor point features The autocorrelation matrices at the first position (u,v) are respectively denoted as... and

[0092] S404: Construct the autocorrelation consistency loss based on the autocorrelation matrices of the sampling features and anchor features, with the following expression:

[0093]

[0094] In the formula, L sl1 (·) represents the smooth-l1 loss function. The cosine similarity of the sampled features. Let ω be the cosine similarity of the anchor features, and ω ~ Dir be the Dirichlet distribution weights. The training set consists of image data, where z ~ p(z) represents random noise sampled using a Gaussian distribution. This indicates the calculation of expectation.

[0095] S405: Based on the interpolated image An unsaturated loss is constructed to supervise the interpolated image; the expression for calculating the unsaturated loss is:

[0096]

[0097] In the formula, L inp For unsaturated loss, Z inp (z′1,z′ k ) is the interpolation noise set. For interpolated image features, log(·) is the logarithm to base 10, z′1, z′ k ~p(z) represents interpolation noise, This indicates the calculation of expectation.

[0098] S406: Construct the first multidimensional vector, expressed as:

[0099]

[0100] In the formula, V inp [·] represents the first multidimensional vector. For the anchor point image, z′ i represents the i-th interpolation noise, k is the dimension of the vector, and the dimension of the vector is equal to the number of elements in the interpolation feature set;

[0101] S407: Construct the second multidimensional vector, expressed as:

[0102] V q [1:k-1]=1, V q [k] = k-1

[0103] In the formula, V q [·] represents the second multidimensional vector, k is the dimension of the vector, and the dimension of the vector is equal to the number of elements in the interpolation feature set.

[0104] S408: Construct the distance constraint loss between interpolated images L dl =L kl (V inp V q ), where L kl Indicates the KL divergence loss;

[0105] S409: Adversarial Loss of Constructing Generators The generator's optimization function is obtained by combining the interpolation image distance constraint loss and the adversarial loss. Where λ1 and λ2 are both fixed hyperparameters;

[0106] S410: Adversarial Loss of Constructing a Discriminator Constructing the optimization function of the discriminator Where λ3 are all fixed hyperparameters;

[0107] S411: Based on the generator's optimization function And the optimization function of the discriminator Update generator and discriminator The parameters are adjusted until the loss converges.

[0108] S5: Through the updated generator and discriminator Input random noise and generate an image.

[0109] The method of this invention is applicable to datasets of various data types, and the generator and discriminator can be conventionally used neural networks.

[0110] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An image generation method based on shape space theory, characterized in that, Includes the following steps: S1: Input image dataset and random noise, construct image generation model and pre-shape space. Image generation model includes generator and discriminator; S2: Randomly select a set of images from the image dataset, input them into the discriminator to obtain training features, project the training features to obtain data features, construct a geodesic surface in the pre-shape space based on the data features, and obtain sample features from the geodesic surface according to the randomly generated sampling weights. S3: Generate random noise through Gaussian distribution sampling, calculate anchor point features through random noise and sample features, construct anchor point images and interpolated images, and extract anchor point features and interpolated image features; S4: Construct the loss using sample features, anchor features, and interpolated image features, and update the generator and discriminator; S5: Using the updated generator and discriminator, input random noise to generate an image; Step S2 specifically includes the following steps: S201: Randomly select a set of images from the image dataset as the training set, input the training set into the specified convolutional layer of the discriminator, and extract training features; S202: Project the extracted training features onto the pre-shape space to obtain data features; S203: Construct geodesic curve functions based on data characteristics; S204: Generate a set of weights based on the Dirichlet distribution, iterate the geodesic curve function based on the weights to obtain the intermediate feature vector, and obtain the geodesic surface function, i.e. the sample features, when the number of iterations is equal to the number of images in the training set; S205: Repeat S204, generating a new set of weights each time to obtain multiple sets of sample features; Step S3 specifically includes the following steps: S301: For each set of sample features obtained in step S205, a set of random noise is obtained by sampling through Gaussian distribution. Based on the weights corresponding to the sample features and the random noise, the anchor noise is calculated. S302: By performing linear interpolation on random noise sampled from two adjacent Gaussian distributions, a set of interpolated noise is obtained; S303: Input the anchor noise and interpolation noise sets into the generator to obtain the anchor image and the interpolation image. Then, input the anchor image into the specified convolutional layer of the discriminator to obtain the interpolation image features. S304: Project the interpolated image features onto the pre-shape space to obtain anchor point features.

2. The image generation method based on shape space theory according to claim 1, characterized in that, The expression for the geodesic curve function is: In the formula, It is a curve function. The eigenvectors representing the start and end points of a geodesic curve, with parameters... Control the position of the generated vector on the geodesic curve. express The geodesic distance between them.

3. The image generation method based on shape space theory according to claim 1, characterized in that, Step S4 specifically includes the following steps: S401: Calculate the cosine similarity between the vector of the sampled feature at the first position and the vector of the neighborhood of the first position; S402: Calculate the cosine similarity between the vector of the anchor point feature at the first position and the vector of the neighborhood of the first position; S403: Traverse all corresponding positions in the space of the first position neighborhood to obtain the autocorrelation matrices of the sampled features and anchor features at the first position respectively; S404: Construct autocorrelation consistency loss based on the autocorrelation matrix of sampling features and anchor features; S405: Construct a non-saturating loss based on the interpolated image to supervise the interpolated image; S406: Construct the first multidimensional vector, the dimension of which is equal to the number of elements in the interpolation feature set; S407: Construct a second multidimensional vector, the dimension of which is equal to the number of elements in the interpolation feature set; S408: Construct distance constraint loss between interpolated images; S409: Construct the adversarial loss of the generator, and obtain the optimization function of the generator by interpolating the distance constraint loss between images and the adversarial loss; S410: Construct the adversarial loss of the discriminator, and construct the optimization function of the discriminator; S411: Update the parameters of the generator and discriminator based on the optimization functions of the generator and discriminator until the loss converges.

4. The image generation method based on shape space theory according to claim 3, characterized in that, The expression for the autocorrelation consistency loss is: In the formula, Indicates smooth- loss function The cosine similarity of the sampled features. Cosine similarity of anchor point features. For the Dirichlet distribution weights, For the training set image data, The random noise is obtained by sampling through a Gaussian distribution. This indicates the calculation of expectation.

5. The image generation method based on shape space theory according to claim 3, characterized in that, The expression for calculating the unsaturated loss is as follows: In the formula, This is a non-saturated loss. For interpolation noise set, For interpolated image features, It is a logarithm to the base 10. This represents interpolation noise. This indicates the calculation of expectation.

6. The image generation method based on shape space theory according to claim 3, characterized in that, The expression for the first multidimensional vector is: ， ， ； In the formula, For the first multidimensional vector, For anchor point images, represents the i-th interpolation noise, and k is the dimension of the vector.

7. The image generation method based on shape space theory according to claim 3, characterized in that, The expression for the second multidimensional vector is: ， In the formula, Let k be the second multidimensional vector, where k is the dimension of the vector.

8. The image generation method based on shape space theory according to claim 3, characterized in that, The expression for calculating the distance constraint loss between images is as follows: In the formula, For the distance constraint loss between images, For KL divergence loss, For the first multidimensional vector, It is the second multidimensional vector.

Images