Small sample generation method and system based on generative adversarial network, computer equipment and storage medium

Abstract

The invention discloses a small sample generation method and system based on a generative adversarial network, computer equipment and a storage medium, and the method comprises the steps: inputting a sample image into a pre-trained generator, and generating a sample image through the pre-trained generator; the number of the output sample images is greater than that of the input instance images; the training process of the pre-training generator is composed of internal circulation and external circulation; training in the internal loop, and updating the global weight in the external loop; the generation process is also composed of internal circulation and external circulation; in the outer loop, the meta learner guides weight adjustment of the generator according to learning experience in the training process, and sample images are generated in the inner loop.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and image generation, and specifically relates to a small sample generation method, system, device and storage medium based on a generative adversarial network. Background Art

[0002] The development speed of artificial intelligence is remarkable, but compared with the human brain, it still has a significant gap in efficiently applying existing knowledge to new tasks. In response to the problem of image sample generation, scholars have proposed a method called Few-Shot Learning (FSL). In the field of few-shot learning, scholars have proposed a variety of meta-learning theories. Among them, Reptile is a commonly used meta-learning algorithm that learns how to quickly adapt to new tasks by quickly adjusting model parameters between multiple tasks.

[0003] However, the existing Reptile algorithm has complex network structure, low computational efficiency and is prone to mode collapse. Mode collapse (Catastrophic Forgetting) is a problem in neural network training, which means that when a neural network learns a new task, it may forget the previously learned tasks. Summary of the invention

[0004] Purpose of the invention: In order to solve the problems of complex network structure, low computational efficiency, limited performance and easy mode collapse in existing algorithms when generating small samples, the present invention proposes a small sample generation method, system, device and storage medium based on a generative adversarial network. This method can generate more realistic data samples through a generator even when data samples are scarce.

[0005] Technical solution: A small sample generation method based on a generative adversarial network, including:

[0006] The sample image is input into the pre-trained generator, and the pre-trained generator generates sample images; the number of generated sample images is greater than the number of input sample images;

[0007] The pre-trained generator is obtained according to the following training process:

[0008] Build a framework including generator, discriminator, encoder and meta-learner;

[0009] The training process consists of an inner loop and an outer loop;

[0010] In the inner loop, the encoder encodes the real sample into a potential representation, and the generator generates a new sample based on the two random noises and the potential representation; the discriminator discriminates the new sample generated by the generator, obtains the discrimination result, and calculates the loss function based on the discrimination result; the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions; the optimization process is repeated until the output of the generator is stable;

[0011] In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator with stable output on the validation set, and passes the adjusted weights of the generator, discriminator, and encoder to the generator, discriminator, and encoder; the inner loop is re-executed until the performance of the generator with stable output on the validation set meets expectations.

[0012] Furthermore, the pre-trained generator generates sample images according to the following generation process:

[0013] The generation process consists of an inner loop and an outer loop;

[0014] A pre-trained generator is a generator that has completed the training process;

[0015] In the inner loop, the pre-trained generator generates sample images based on random noise and the latent representation obtained by encoding the example images by the encoder that has completed the training process;

[0016] In the outer loop, the meta-learner guides the weight adjustment of the generator based on the learning experience during the training process.

[0017] Furthermore, during the training process, the loss functions involved include: the loss function of the discriminator, the loss function of the generator, the loss function of the encoder, and the mode-seeking regularization term, expressed as:

[0018]

[0019] In the formula, represents the loss function of the discriminator, represents the loss function of the generator, represents the loss function of the encoder, λ E represents the encoder penalty term, λ ms represents the penalty term of the mode-seeking regularization term, represents the mode seeking regularization term, expressed as:

[0020]

[0021] Where d(·,·) represents the L1 norm distance metric, which is used to calculate the distance between two vectors, z1 and z2 represent two random noises, and G1(z1,r) and G1(z2,r) are new samples generated by the generator based on the two random noises and the latent representation r, respectively.

[0022] Furthermore, the loss function of the discriminator is It is expressed as:

[0023]

[0024] In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

[0025] Furthermore, the loss function of the generator is It is expressed as:

[0026]

[0027] In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

[0028] Furthermore, the loss function of the encoder is It is expressed as:

[0029]

[0030] In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, y1 and y2 represent images generated by the generator based on two random noise vectors, x i represents a sample randomly selected from the image input to the encoder, and I represents the total number of images input to the encoder.

[0031] Furthermore, in the inner loop, the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions. The specific operations include:

[0032] The local minimum of the discriminator weight is obtained by minimizing the discriminator loss function, which is expressed as:

[0033]

[0034] In the formula, τ represents the task τ, k represents the number of iterations, Represents the loss function value of the discriminator for the task τ; the local minimum value of the generator's weight is obtained by minimizing the generator's loss function, which is expressed as:

[0035]

[0036] In the formula, Represents the loss function value of the generator for task τ;

[0037] The local minimum of the encoder weights is obtained by minimizing the encoder loss function, expressed as:

[0038]

[0039] In the formula, Represents the loss function value of the encoder for the task τ.

[0040] Furthermore, in the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator on the validation set obtained after the inner loop training, including:

[0041] In the outer loop, the weight of the discriminator Φ D , the weight of the generator Φ G and the encoder weights Φ E They are updated by the following formula:

[0042]

[0043] Where W Dτ , W Gτ , W Eτ They are the initial weights of the discriminator for task τ, the initial weights of the generator for task τ, and the initial weights of the encoder for task τ, respectively.

[0044] The present invention discloses a small sample generation system based on a generative adversarial network, comprising:

[0045] A pre-trained generator for generating sample images based on example images; the number of sample images outputted is greater than the number of instance images inputted;

[0046] The pre-trained generator is obtained according to the following training process:

[0047] Build a framework including generator, discriminator, encoder and meta-learner;

[0048] The training process consists of an inner loop and an outer loop;

[0049] In the inner loop, the encoder encodes the real sample into a potential representation, and the generator generates a new sample based on the two random noises and the potential representation; the discriminator discriminates the new sample generated by the generator, obtains the discrimination result, and calculates the loss function based on the discrimination result; the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions; the optimization process is repeated until the output of the generator is stable;

[0050] In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator with stable output on the validation set, and passes the adjusted weights of the generator, discriminator, and encoder to the generator, discriminator, and encoder; the inner loop is re-executed until the performance of the generator with stable output on the validation set meets expectations.

[0051] Furthermore, the pre-trained generator generates sample images according to the following generation process:

[0052] The generation process consists of an inner loop and an outer loop;

[0053] A pre-trained generator is a generator that has completed the training process;

[0054] In the inner loop, the pre-trained generator generates sample images based on random noise and the latent representation obtained by encoding the example images by the encoder that has completed the training process;

[0055] In the outer loop, the meta-learner guides the weight adjustment of the generator based on the learning experience during the training process.

[0056] Furthermore, during the training process, the loss functions involved include: the loss function of the discriminator, the loss function of the generator, the loss function of the encoder, and the mode-seeking regularization term, expressed as:

[0057]

[0058] In the formula, represents the loss function of the discriminator, represents the loss function of the generator, represents the loss function of the encoder, λ E represents the encoder penalty term, λ ms represents the penalty term of the mode-seeking regularization term, represents the mode seeking regularization term, expressed as:

[0059]

[0060] Where d(·,·) represents the L1 norm distance metric, which is used to calculate the distance between two vectors, z1 and z2 represent two random noises, and G1(z1,r) and G1(z2,r) are new samples generated by the generator based on the two random noises and the latent representation r, respectively.

[0061] Furthermore, the loss function of the discriminator is It is expressed as:

[0062]

[0063]

[0064] In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

[0065] Furthermore, the loss function of the generator is It is expressed as:

[0066]

[0067] In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

[0068] Furthermore, the loss function of the encoder is It is expressed as:

[0069]

[0070] In the formula, y 1i represents a sample randomly drawn from y1, y 2irepresents a sample randomly drawn from y2, y1 and y2 represent images generated by the generator based on two random noise vectors, x i represents a sample randomly selected from the image input to the encoder, and I represents the total number of images input to the encoder.

[0071] Furthermore, in the inner loop, the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions. The specific operations include:

[0072] The local minimum of the discriminator weight is obtained by minimizing the discriminator loss function, which is expressed as:

[0073]

[0074] In the formula, τ represents the task τ, k represents the number of iterations, Represents the loss function value of the discriminator for the task τ; the local minimum value of the generator's weight is obtained by minimizing the generator's loss function, which is expressed as:

[0075]

[0076] In the formula, Represents the loss function value of the generator for task τ;

[0077] The local minimum of the encoder weights is obtained by minimizing the encoder loss function, expressed as:

[0078]

[0079] In the formula, Represents the loss function value of the encoder for the task τ.

[0080] Furthermore, in the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator on the validation set obtained after the inner loop training, including:

[0081] In the outer loop, the weight of the discriminator Φ D , the weight of the generator Φ G and the encoder weights Φ E They are updated by the following formula:

[0082]

[0083] Where W Dτ , W Gτ , W Eτ They are the initial weights of the discriminator for task τ, the initial weights of the generator for task τ, and the initial weights of the encoder for task τ, respectively.

[0084] The present invention discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a small sample generation method based on a generative adversarial network disclosed above are implemented.

[0085] The present invention discloses a storage medium, which stores a small sample generation program. When the small sample generation program is executed by at least one processor, the steps of the above-disclosed small sample generation method based on a generative adversarial network are implemented.

[0086] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0087] (1) The method of the present invention generates data for simulating large-scale data distribution through adversarial training, that is, two networks compete with each other, that is, Generative Adversarial Network (GANs), which is a deep learning architecture composed of two neural networks: generator G and discriminator D; Generative Adversarial Network can extract key features from a few examples to perform generation tasks, so it can be used in meta-learning of small sample image generation; The present invention proposes a new prediction framework, which can utilize the key feature capabilities of meta-learning in small sample scenarios to generate diverse real images with fewer training parameters and prevent mode collapse; Compared with the Reptile algorithm, the network structure is simpler and the computational efficiency is higher; At the same time, an innovative objective function is adopted to improve the diversity of generated images; By increasing the number of generator iterations and the use of random noise vectors, the mode collapse problem is effectively alleviated;

[0088] (2) During the generation process, the method of the present invention uses the parameters obtained in the training phase to adapt to new tasks, so that it can learn from a small amount of data and apply the learned knowledge to new tasks; in addition, through optimization of four objective functions, it can converge quickly even in the case of a small amount of data, and has good generalization ability. It is foreseeable that it can be applied to a variety of data sets and has strong generalization ability and performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A flowchart of a small sample generation method based on a generative adversarial network proposed by the present invention;

[0090] Figure 2 Schematic diagram of the training phase principle in an embodiment of the present invention;

[0091] Figure 3 It is a schematic diagram of the principle of the generation stage in an embodiment of the present invention. DETAILED DESCRIPTION

[0092] The present invention discloses a small sample generation method, system, device and storage medium based on a generative adversarial network, which aims to quickly learn to generate reasonable output images by using only a small number of image examples. The method utilizes the generator and discriminator of the generative adversarial network, combined with fast adaptive meta-learning technology, to achieve efficient learning and generation of a small amount of sample data, solving the problem of limited performance of traditional methods under small sample data. For high-dimensional small sample data sets, an unsupervised meta-learning model framework designed by the present invention is used. The unsupervised meta-learning model framework aims to automatically discover the intrinsic structure and pattern of data from a large amount of unlabeled data through unsupervised learning, and use these learned structures and patterns to generate new data with similar characteristics in the generation stage.

[0093] The unsupervised meta-learning model framework is divided into two main stages, the first stage is the training stage, and the second stage is the generation stage. In the training stage, the original image data is sampled and assigned to the training task τ i and the test task τ j Before starting processing, the training task τ i and the test task τ j The batch size of is the same as that of j Not included in the training task τ i The unsupervised meta-learning model framework is able to quickly learn and adapt across multiple tasks. Compared with traditional methods based on small sample learning, it has stronger cross-task generalization capabilities.

[0094] The present invention also optimizes different loss functions in the inner loop and the outer loop, and the unsupervised meta-learning model framework can more effectively learn from a small amount of data and apply it to new tasks. The training process and the generation process are both composed of an inner loop and an outer loop. A generative adversarial network-based encoder is specially added to the inner loop. The encoder converts the original image data into a feature vector, inputs two random noises and the feature vector from the encoder into the generator, and generates a new image. This design enables the generator to adapt to different tasks more flexibly, depending not only on random noise, but also on the features extracted from the original image. During the generation process, the unsupervised meta-learning model framework uses the weights of the discriminator, generator, and encoder obtained in the training phase to adapt to the test task τ j , so that the model can learn from a small amount of data and apply the learned knowledge to new tasks. Both the training process and the generation process are trained through the inner loop and the global weights are updated through the outer loop.

[0095] In order to make the objectives, technical solutions and advantages of the present invention clearer, the small sample generation method, system, device and storage medium based on the generative adversarial network of the present invention will be further described in combination with the drawings and embodiments of the present invention.

[0096] Embodiment 1:

[0097] This embodiment discloses a small sample generation method based on a generative adversarial network. Figures 1 to 3 As shown in the figure, it is mainly divided into two stages. The first stage is the training stage, and the second stage is the generation stage. Both the training process and the generation process are composed of an inner loop and an outer loop. Training is performed through the inner loop, and the weight parameters are updated through the outer loop. In the generation process, the weights of the generator and encoder obtained in the training stage are used to adapt to new test tasks, so that they can learn from a small amount of data and apply the learned knowledge to new tasks, which helps to generate richer and more diverse images while improving the generalization ability of the model.

[0098] The small sample generation method of this embodiment involves a generator, a discriminator, an encoder and a meta-learner. The generator is responsible for generating samples based on random noise, the discriminator is responsible for judging whether the generated samples are real, the encoder is responsible for encoding the real samples into potential representations, and the meta-learner is responsible for guiding the training process of the entire model. This embodiment introduces the GAN structure to form an adversarial relationship between the generator and the discriminator. The generator strives to generate more realistic samples to deceive the discriminator, while the discriminator strives to distinguish between real samples and generated samples. This adversarial relationship helps the generator learn richer data distributions and patterns.

[0099] Specifically, the training process consists of an inner loop and an outer loop. The training process is now further explained.

[0100] (1) Inner loop: In the inner loop, the encoder first encodes the real sample into a latent representation, and then the generator generates new samples based on the random noise and the latent representation. Next, the discriminator discriminates the generated new samples and calculates the loss function based on the discrimination results. Finally, the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions. Now, combined with Figure 2 To illustrate the inner loop process, the encoder E receives the training task τ iThe generator G then uses two random noise vectors (z1 and z2) and the feature vector r provided by the encoder E as input to generate two images y1 and y2. At the same time, the discriminator D will distinguish between the input x and the two images y1 and y2 generated by the generator G. In this embodiment, two random noise vectors are used in combination with the feature vector provided by the encoder. This approach increases the input diversity of the generator, helps to generate richer and more diverse images, and improves the generalization ability of the model. The inner loop is trained until the output of the generator is stable and then paused;

[0101] (2) Outer loop: In the outer loop, the meta-learner updates the weights of the discriminator D, generator G, and encoder E according to specific rules based on the performance of the current generator on the validation set to adapt to the new training task.

[0102] During the training process, this embodiment constructs a loss function related to the meta-learning training process to quickly converge the model and improve the generalization ability of the model.

[0103] For each task τ, I images are sampled from the input x and output y, and the loss functions of the discriminator and generator can be constructed as:

[0104]

[0105] in:

[0106]

[0107] Among them, i represents the index of I sampled images, C(x) is used to measure the authenticity of the independent variable x. If the value of C(x) is negative, it means that the input image is more like an artificially generated "pseudo image"; if the value of C(x) is positive, it means that the input image looks more like a real image.

[0108] Considering that the output is generated based on two random noise vectors z1 and z2, the loss functions of the discriminator and generator are improved to the following form:

[0109] The loss function of the discriminator can be improved as follows:

[0110]

[0111] In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x irepresents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

[0112] The loss function of the generator can be improved as follows:

[0113]

[0114] The encoder loss function is constructed as follows:

[0115]

[0116] For each task τ, the weight W of the discriminator is initialized before starting training. D , the weight of the generator W G and the encoder weights W E In the inner loop, by minimizing the loss function of the discriminator The loss function of the generator And the loss function of the encoder To obtain the local minimum of the weights:

[0117]

[0118] In the outer loop, the weight of the discriminator Φ D , the weight of the generator Φ G and the encoder weights Φ E will be updated by the following formula:

[0119]

[0120] From the formula, we can see that by minimizing the difference between the initial global parameters and the weights obtained by the inner loop optimization, the meta-learning model can effectively update its parameters during the update process.

[0121] In summary, the overall loss function of this model can be expressed as:

[0122]

[0123] in:

[0124]

[0125] represents the pattern-seeking regularization term, where d(·,·) is the L1 norm distance metric used to calculate the distance between two vectors. Ideally, when the distance between random vectors z1 and z2 is greater, the distance between the generated images G1(z1,r) and G1(z2,r) should also be greater, so that the generator can better capture different patterns and produce richer and more diverse results when generating images. The pattern-seeking regularization achieves this goal by maximizing the ratio of the spacing of the generated images to the spacing of the random vectors. This embodiment will and are set as hyperparameters and are all set to 1. By introducing the mode-seeking regularization term and maximizing the ratio of the spacing of the generated images to the spacing of the random vectors, the generator module can be encouraged to capture different patterns as much as possible, which helps to generate more diverse and high-quality images.

[0126] The generation process also consists of an inner loop and an outer loop. The generation process is now further explained.

[0127] (1) Inner loop: The generator generates new samples based on random noise and latent representation (encoded by the encoder). The generated samples can be directly used for subsequent tasks or as a means of data enhancement.

[0128] (2) The outer-loop meta-learner guides the weight adjustment of the generator based on the learning experience in the training phase.

[0129] In the generation process, the parameters of the discriminator D, generator G, and encoder E obtained in the training phase are used to adapt to the new test task τ j , allowing the model to learn from small amounts of data and apply what it has learned to new tasks.

[0130] The encoder of this embodiment can encode real samples into potential representations, which can capture the intrinsic structure and pattern of the data. During the generation process, the generator can use these potential representations to generate new samples with similar characteristics to the original data. This design enables the generator to adapt to different tasks more flexibly, not only depending on random noise, but also on features extracted from the original image. At the same time, the encoder can also be used as a regularization method to prevent the generator from overfitting.

[0131] Both the training process and the generation process are trained through the inner loop and the global weights are updated through the outer loop.

[0132] This embodiment uses the parameters obtained in the training phase to adapt to new tasks, so that it can learn from a small amount of data and apply the learned knowledge to new tasks; in addition, by optimizing the four loss functions, it can converge quickly even with a small amount of data and has good generalization ability, and it is foreseeable that it can be applied to a variety of data sets.

[0133] Embodiment 2:

[0134] Based on Example 1, this example proposes a small sample generation system based on a generative adversarial network, including:

[0135] A pre-trained generator for generating sample images based on example images; the number of sample images outputted is greater than the number of instance images inputted;

[0136] The pre-trained generator is obtained according to the following training process:

[0137] Build a framework including generator, discriminator, encoder and meta-learner;

[0138] The training process consists of an inner loop and an outer loop;

[0139] In the inner loop, the encoder encodes the real sample into a potential representation, and the generator generates a new sample based on the two random noises and the potential representation; the discriminator discriminates the new sample generated by the generator, obtains the discrimination result, and calculates the loss function based on the discrimination result; the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions; the optimization process is repeated until the output of the generator is stable;

[0140] In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator with stable output on the validation set, and passes the adjusted weights of the generator, discriminator, and encoder to the generator, discriminator, and encoder; the inner loop is re-executed until the performance of the generator with stable output on the validation set meets expectations;

[0141] The pre-trained generator generates sample images according to the following generation process:

[0142] The generation process consists of an inner loop and an outer loop;

[0143] A pre-trained generator is a generator that has completed the training process;

[0144] In the inner loop, the pre-trained generator generates sample images based on random noise and the latent representation obtained by encoding the example images by the encoder that has completed the training process;

[0145] In the outer loop, the meta-learner guides the weight adjustment of the generator based on the learning experience during the training process.

[0146] This embodiment uses the parameters obtained in the training phase to adapt to new tasks, so that it can learn from a small amount of data and apply the learned knowledge to new tasks; in addition, it can converge quickly even with a small amount of data and has good generalization ability, and it is foreseeable that it can be applied to a variety of data sets.

[0147] Embodiment 3:

[0148] This embodiment discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps disclosed in any of the above embodiments are implemented.

[0149] Embodiment 4:

[0150] This embodiment discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps disclosed in any of the above embodiments are implemented.

[0151] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0152] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A small sample generation method based on a generative adversarial network, characterized by: include: Input the sample image into the pre-trained generator, and the pre-trained generator generates a sample image; The number of generated sample images is greater than the number of input sample images; The pre-trained generator is obtained according to the following training process: Build a framework including generator, discriminator, encoder and meta-learner; The training process consists of an inner loop and an outer loop; In the inner loop, the encoder encodes the real sample into a potential representation, and the generator generates a new sample based on the two random noises and the potential representation; the discriminator discriminates the new sample generated by the generator, obtains the discrimination result, and calculates the loss function based on the discrimination result; the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions; the optimization process is repeated until the output of the generator is stable; In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator with stable output on the validation set, and passes the adjusted weights of the generator, discriminator, and encoder to the generator, discriminator, and encoder; the inner loop is re-executed until the performance of the generator with stable output on the validation set meets expectations.

2. The small sample generation method based on a generative adversarial network according to claim 1, characterized in that: The pre-trained generator generates sample images according to the following generation process: The generation process consists of an inner loop and an outer loop; A pre-trained generator is a generator that has completed the training process; In the inner loop, the pre-trained generator generates sample images based on random noise and the latent representation obtained by encoding the example images by the encoder that has completed the training process; In the outer loop, the meta-learner guides the weight adjustment of the generator based on the learning experience during the training process.

3. The small sample generation method based on a generative adversarial network according to claim 1, characterized in that: During the training process, the loss functions involved include: the loss function of the discriminator, the loss function of the generator, the loss function of the encoder, and the mode-seeking regularization term, expressed as: In the formula, represents the loss function of the discriminator, represents the loss function of the generator, represents the loss function of the encoder, λ E represents the encoder penalty term, λ ms represents the penalty term of the mode-seeking regularization term, represents the mode seeking regularization term, expressed as: Where d(·,·) represents the L1 norm distance metric, which is used to calculate the distance between two vectors, z1 and z2 represent two random noises, and G1(z1,r) and G1(z2,r) are new samples generated by the generator based on the two random noises and the latent representation r, respectively.

4. The small sample generation method based on a generative adversarial network according to claim 3, characterized in that: Loss function of the discriminator It is expressed as: In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

5. The method for generating small samples based on a generative adversarial network according to claim 3, characterized in that: The loss function of the generator It is expressed as: In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

6. The method for generating small samples based on a generative adversarial network according to claim 3, characterized in that: The loss function of the encoder It is expressed as: In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, y1 and y2 represent images generated by the generator based on two random noise vectors, x i represents a sample randomly selected from the image input to the encoder, and I represents the total number of images input to the encoder.

7. The method for generating small samples based on a generative adversarial network according to claim 3, characterized in that: In the inner loop, the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions. The specific operations include: The local minimum of the discriminator weight is obtained by minimizing the discriminator loss function, which is expressed as: In the formula, τ represents the task τ, k represents the number of iterations, Represents the loss function value of the discriminator for task τ; The local minimum of the generator's weight is obtained by minimizing the generator's loss function, which is expressed as: In the formula, represents the loss function value of the generator for task τ; The local minimum of the encoder weights is obtained by minimizing the encoder loss function, expressed as: In the formula, Represents the loss function value of the encoder for the task τ.

8. The method for generating small samples based on a generative adversarial network according to claim 7, characterized in that: In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder based on the performance of the generator on the validation set obtained after the inner loop training, including: In the outer loop, the weight of the discriminator Φ D , the weight of the generator Φ G and the encoder weights Φ E They are updated by the following formula: Where W Dτ , W Gτ , W Eτ They are the initial weights of the discriminator for task τ, the initial weights of the generator for task τ, and the initial weights of the encoder for task τ, respectively.

9. A small sample generation system based on a generative adversarial network, characterized in that: include: A pre-trained generator for generating sample images based on example images; The number of output sample images is greater than the number of input instance images; The pre-trained generator is obtained according to the following training process: Build a framework including generator, discriminator, encoder and meta-learner; The training process consists of an inner loop and an outer loop; In the inner loop, the encoder encodes the real sample into a potential representation, and the generator generates a new sample based on the two random noises and the potential representation; the discriminator discriminates the new sample generated by the generator, obtains the discrimination result, and calculates the loss function based on the discrimination result; the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions; the optimization process is repeated until the output of the generator is stable; In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder according to the performance of the generator with stable output on the validation set, and passes the adjusted weights of the generator, discriminator, and encoder to the generator, discriminator, and encoder; the inner loop is re-executed until the performance of the generator with stable output on the validation set meets expectations.

10. The small sample generation system based on generative adversarial network according to claim 9, characterized in that: The pre-trained generator generates sample images according to the following generation process: The generation process consists of an inner loop and an outer loop; A pre-trained generator is a generator that has completed the training process; In the inner loop, the pre-trained generator generates sample images based on random noise and the latent representation obtained by encoding the example images by the encoder that has completed the training process; In the outer loop, the meta-learner guides the weight adjustment of the generator based on the learning experience during the training process.

11. The small sample generation system based on generative adversarial network according to claim 9, characterized in that: During the training process, the loss functions involved include: the loss function of the discriminator, the loss function of the generator, the loss function of the encoder, and the mode-seeking regularization term, expressed as: In the formula, represents the loss function of the discriminator, represents the loss function of the generator, represents the loss function of the encoder, λ E represents the encoder penalty term, λ ms represents the penalty term of the mode-seeking regularization term, represents the mode seeking regularization term, expressed as: Where d(·,·) represents the L1 norm distance metric, which is used to calculate the distance between two vectors, z1 and z2 represent two random noises, and G1(z1,r) and G1(z2,r) are new samples generated by the generator based on the two random noises and the latent representation r, respectively.

12. The small sample generation system based on generative adversarial network according to claim 11, characterized in that: Loss function of the discriminator It is expressed as: In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

13. The small sample generation system based on generative adversarial network according to claim 11, characterized in that: The loss function of the generator It is expressed as: In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, x represents the image input to the encoder, x i represents a sample randomly drawn from x, y1 and y2 represent images generated by the generator based on two random noise vectors; D1(·) represents the discriminator; i represents the image index; represents mathematical expectation.

14. The small sample generation system based on generative adversarial network according to claim 11, characterized in that: The loss function of the encoder It is expressed as: In the formula, y 1i represents a sample randomly drawn from y1, y 2i represents a sample randomly drawn from y2, y1 and y2 represent images generated by the generator based on two random noise vectors, x i represents a sample randomly selected from the image input to the encoder, and I represents the total number of images input to the encoder.

15. The small sample generation system based on generative adversarial network according to claim 11, characterized in that: In the inner loop, the weights of the generator, discriminator, and encoder are updated by optimizing their respective loss functions. The specific operations include: The local minimum of the discriminator weight is obtained by minimizing the discriminator loss function, which is expressed as: In the formula, τ represents the task τ, k represents the number of iterations, Represents the loss function value of the discriminator for task τ; The local minimum of the generator's weight is obtained by minimizing the generator's loss function, which is expressed as: In the formula, represents the loss function value of the generator for task τ; The local minimum of the encoder weights is obtained by minimizing the encoder loss function, expressed as: In the formula, Represents the loss function value of the encoder for the task τ.

16. The small sample generation system based on generative adversarial network according to claim 15, characterized in that: In the outer loop, the meta-learner adjusts the weights of the generator, discriminator, and encoder based on the performance of the generator on the validation set obtained after the inner loop training, including: In the outer loop, the weight of the discriminator Φ D , the weight of the generator Φ G and the encoder weights Φ E They are updated by the following formula: Where W Dτ , W Gτ , W Eτ They are the initial weights of the discriminator for task τ, the initial weights of the generator for task τ, and the initial weights of the encoder for task τ, respectively.

17. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of a method for generating small samples based on a generative adversarial network as described in any one of claims 1 to 8 are implemented.

18. A storage medium, characterized in that: The storage medium stores a small sample generation program, which, when executed by at least one processor, implements the steps of a small sample generation method based on a generative adversarial network as described in any one of claims 1 to 8.

Images