A method for image data augmentation based on single instance distribution sampling

Abstract

The present invention provides an image data augmentation method based on single instance distributed sampling, which supplements data for most categories with less data in a database in a targeted manner, fundamentally alleviating the problem of poor tail category recognition performance caused by data imbalance; based on the learned distributed sampling of a single instance sample, the database is infinitely expanded, and the expanded sample is highly similar in semantics to the original instance sample, but the image detail information is different, so that the database covers all possible image data as much as possible, effectively enriches the diversity of the database, greatly improves the generalization performance of the model when dealing with various unknown data, enhances the stability of the model, and solves the key problems in the actual application of the model to a certain extent; there is no need to label the data, only a small-scale collection of the original data is required to expand the database into a large-scale training database, and all problems faced in data collection are fully solved quickly, accurately and efficiently.

Description

Technical Field

[0001] The present invention relates to the general field of image data processing or generation technology, and in particular to an image data augmentation method based on single instance distribution sampling. Background Art

[0002] In the process of building an artificial intelligence recognition system, a large amount of target data needs to be collected and accurately labeled by field experts or relevant labeling personnel to build a huge database.

[0003] However, in practical applications, because common category data is easy to collect and has a high frequency of occurrence, and rare category data has a low frequency of occurrence and is difficult to collect, the constructed database has a serious long-tail effect. For example, in natural scenarios, a few categories have a large number of samples, while most categories only collect a small amount of data, and the number of different categories in the database is seriously unbalanced. In the medical field, normal samples account for the vast majority of the database samples, while abnormal samples with high diagnostic value account for only a very small part, and the database is full of large amounts of worthless data. This situation causes the model prediction results to naturally bias towards categories with a larger number, seriously affecting the recognition performance of high-value, small-number categories. At the same time, in practical applications, the algorithm will encounter a large number of samples that are not covered by the training database, causing the model to face serious generalization problems, which in turn affects its practical application value.

[0004] Existing methods generally collect data as much as possible and build training databases. Low-value minority category samples occupy the majority of the database, resulting in extreme data imbalance in the training database; the training database data is incomplete and does not cover all possible data, and the collected data lacks diversity; a deep learning recognition model based on supervised learning is built, and the deep neural network directly performs feature extraction and automatic classification.

[0005] However, the database constructed by the existing methods contains a large amount of low-value training data, which leads to an extremely unbalanced database. As a result, the model recognition results naturally tend to favor categories with a larger number, which seriously affects the model's ability to recognize high-value, rare, and abnormal categories. When the constructed database has a narrow coverage, incomplete data, and lacks diversity, it is easy to cause the model's recognition performance for unknown samples to decline and the model's generalization to decrease, affecting the model's practical application value. Moreover, data collection requires a long period of time, and unknown obstacles may be encountered during data collection (such as rare categories that are hard to come by), which seriously affects data collection efficiency. Summary of the invention

[0006] The present invention aims to solve the problem of image data sample coverage, and provides an image data augmentation method based on single instance distribution sampling. The method performs sampling based on the distribution obeyed by a single instance, quickly supplements rare category data at the sample level, alleviates the inhibitory effect of the head category on the tail category, and ensures the diversity of generated data, improves the generalization performance of the model, and avoids the model from overfitting the tail category data. At the same time, target data can be generated according to specific category requirements, and rare category samples can be further supplemented on a large scale and in a targeted manner, so as to quickly build a comprehensive, complete, and diversified training database, fundamentally solve the negative impact of the data level on model training, and greatly improve model performance.

[0007] The present invention provides an image data augmentation method based on single instance distribution sampling, comprising the following steps:

[0008] S1. constructing an initial database, wherein the initial database includes natural image data and / or medical image data;

[0009] S2. Construct an instance feature extraction model. The instance feature extraction model extracts the feature vector of the image block of the single instance to be analyzed, performs data enhancement to obtain the image pair before and after enhancement, and uses the image pair before and after enhancement as the positive sample pair to obtain the positive sample image set. A specified number of images are randomly extracted from the initial database to extract the feature vector to obtain the negative sample image set. The loss function is used to extract the feature vector of the image block of the single instance to be analyzed, and the image pair before and after enhancement is used as the positive sample pair to obtain the positive sample image set. Reduce the distance between positive sample pairs, increase the distance between the current sample and other negative samples, and finally obtain the unique features of a single instance;

[0010] S3, modeling each single instance-specific feature to obtain an instance feature distribution learning model, based on the instance feature distribution learning model, performing distribution estimation of all single instance-specific features at the feature level to obtain the feature distribution obeyed by each instance, and sampling based on the feature distribution obeyed by each instance to obtain the specific features of the single instance;

[0011] S4. Construct a feature decoding image generation model, input the specific features of a single instance into the feature decoding image generation model to obtain an image that belongs to the same category as the single instance but has different content; or the extracted instance features can be combined with different instance features to output an image related to the combined instance features; the image generated by the feature decoding image generation model is expanded into the database to obtain an expanded database, and an image data augmentation method based on single instance distribution sampling is completed.

[0012] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred embodiment, in step S1, medical images are acquired by electron microscope, phase contrast microscope and optical microscope and data is collected by microscope camera to obtain medical image data;

[0013] Natural image data is obtained by collecting natural images through a camera or a web crawler and cleaning the collected images.

[0014] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred embodiment, in step S2, the instance feature extraction model is a self-attention network model;

[0015] Step S2 includes the following steps:

[0016] S21, constructing an instance feature extraction model, wherein the instance feature extraction model divides the single instance image to be analyzed into image blocks, each of which has a size of 16*16, so that each image block is represented by a feature vector;

[0017] S22, performing data enhancement on the single instance image to be analyzed and dividing it into 16*16 image blocks to obtain enhanced image blocks, wherein the data enhancement methods include rotation, flipping, image deformation, contrast and brightness enhancement, extracting feature vectors of the enhanced image blocks and combining them with feature vectors of the pre-enhancement image blocks to obtain pre-enhancement-enhancement image pairs as positive sample pairs, and all positive sample pairs constitute a positive sample image set;

[0018] S23, the instance feature extraction model randomly extracts a specified number of images from the initial database as negative sample images, divides the negative sample images into image blocks, each image block has a size of 16*16, and makes each image block represented by a feature vector to obtain a negative sample image set;

[0019] S24. Loss function for:

[0020] ;

[0021] Among them, q is the feature vector of the image block before enhancement, is the enhanced image block feature vector, is the feature vector of the negative sample image patch, A hyperparameter to control the similarity;

[0022] S24. By optimizing the loss function Reduce the distance between positive sample pairs and increase the distance between the current sample and other negative samples. When the loss function After reaching the target value, a single instance-specific feature is obtained, and the single instance-specific feature is a feature vector of the image block obtained after optimization.

[0023] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred embodiment, in step S3, the instance feature distribution learning model is an encoding-decoding structure of a self-attention network;

[0024] When training the instance feature distribution learning model, the instance feature distribution learning model assumes that the single instance-specific features after noise addition obey the Gaussian distribution, and adds noise to the single instance-specific features step by step until the single instance-specific features completely obey the standard normal distribution, and the single instance-specific features become Gaussian noise; then the single instance-specific features that have become Gaussian noise are denoised step by step to gradually restore the instance features before noise addition;

[0025] The instance feature distribution learning model learns the specific distribution of the unique features of a single instance during the denoising and denoising process and obtains the specific features of the single instance by sampling based on the specific distribution;

[0026] When using the instance feature distribution learning model for reasoning, the instance feature distribution learning model denoises the unique features of a single instance step by step to obtain the specific features of the single instance.

[0027] The image data augmentation method based on single instance distribution sampling described in the present invention, as a preferred embodiment, the training method of the instance feature distribution learning model comprises the following steps:

[0028] S31, modeling each unique feature of a single instance to obtain an instance feature distribution learning model, and performing noise processing on the unique feature of the single instance by the instance feature distribution learning model;

[0029] The noise adding method is: for the input single instance unique feature sample Gradually add mean 0 and variance The Gaussian noise makes the conditional distribution becomes a Gaussian distribution, in the last T time, Completely becomes zero mean, variance is I The multivariate standard normal distribution of I is the identity matrix;

[0030] S32, performing denoising operations on the added noise step by step based on the multivariate standard normal distribution, gradually restoring the original single instance features and generating an image through a decoder;

[0031] The denoising method is: based on T The multivariate standard Gaussian distribution at each moment gradually denoises the added noise. The conditional distribution formed after each denoising step is a multivariate Gaussian distribution, and finally the single instance feature before denoising is obtained. In the denoising process, each step obtains The specific parameters of the Gaussian distribution that follows can be randomly sampled from the distribution represented in the last step to obtain the same Similar single instance features;

[0032] S33, the denoising-denoising model learns the true distribution of each instance through the loss function Loss and samples under the true distribution to obtain the specific features of a single instance;

[0033] When using the instance feature distribution learning model for reasoning, only step S32 is used to perform step-by-step denoising to obtain the specific features of a single instance.

[0034] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred embodiment, in step S31, the noise adding process is:

[0035] ;

[0036] in, For a given Distribution The distribution of compliance, N is a normal distribution, For the t -1 specific sample at time, For the t A specific sample of the moment, For the t time The variance of the distribution followed.

[0037] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred embodiment, in step S32, the denoising process is:

[0038] ;

[0039] in, for exist The conditional distribution under N is a normal distribution, =1- ， 。

[0040] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred embodiment, in step S33,

[0041] ;

[0042] in, For distribution and distribution of KL Divergence, For the distribution The expectations below, for In a given The predicted distribution under for In a given The predicted distribution under .

[0043] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred mode, in step S4, the feature decoding image generation model matches the specific features of the single instance with the features in the feature vector vocabulary, finds the most similar feature vector in the vocabulary, and replaces all image blocks of the single instance to be analyzed with the feature vector in the vocabulary; then, according to the language model rules, the feature decoding image generation model gives the initial feature vector and the corresponding feature vector and gradually predicts the next correct feature vector:

[0044] ;

[0045] in, is the joint distribution of the single instance image patches to be analyzed,

[0046] For the t The edge distribution of image patches; prediction t When calculating the conditional distribution of time, we need to know t=t -1 time has arrived t =1; finally, the complete predicted feature vector of the single instance image is obtained;

[0047] The predicted feature vector is input into the image decoding module of the feature decoding image generation model to be decoded into the corresponding real image, and then cleaned and entered into the database;

[0048] During decoding, the mean square error loss function is used to constrain the image generated by the feature decoding image decoding module to be as consistent as possible with the single instance image;

[0049] ;

[0050] in, M is the total number of images, For the generated image, is the original input image.

[0051] The image data augmentation method based on single instance distribution sampling described in the present invention is, as a preferred mode, in step S4, when the feature vectors of different instances are weightedly fused, a real image containing features of two instances is obtained to generate a directional image:

[0052] ;

[0053] in, is the weighted weight, , is the weighted instance feature, , are the characteristics of different instances respectively;

[0054] In step S4, the image may also be generated using the single instance specific features obtained by denoising the random features sampled from the normal distribution using the instance feature distribution learning model.

[0055] The present invention has the following advantages:

[0056] (1) The present invention supplements data for most categories with less data in the database, fundamentally alleviating the problem of poor recognition performance of tail categories caused by data imbalance, improving the overall recognition performance of the model, and making up for the shortcomings of the model, so that the model has higher application value.

[0057] (2) The present invention infinitely expands the database based on the learned distribution sampling of a single instance sample, and the expanded samples are highly semantically similar to the original instance samples, but the image detail information is different, so that the database covers all possible image data as much as possible, effectively enriching the diversity of the database, greatly improving the generalization performance of the model when dealing with various unknown data, enhancing the stability of the model, and to a certain extent solving the key problems in the actual application of the model.

[0058] (3) The present invention does not require any labeling of the data. It only needs to collect the original data on a small scale to expand the database into a large-scale training database with complete categories, diverse types, rich data and huge quantity, so as to quickly, accurately and efficiently solve all problems faced in data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flowchart of an image data augmentation method based on single instance distribution sampling;

[0060] Figure 2 This is an algorithm training flowchart for an image data augmentation method based on single instance distribution sampling;

[0061] Figure 3 An overview of the initial database for an image data augmentation method based on single instance distribution sampling;

[0062] Figure 4 A schematic diagram of denoising and denoising of an image data augmentation method based on single instance distribution sampling;

[0063] Figure 5 Model generation for an image data augmentation method based on single instance distribution sampling Figure 1 ;

[0064] Figure 6 Model generation for an image data augmentation method based on single instance distribution sampling Figure 2 ;

[0065] Figure 7 Model generation for an image data augmentation method based on single instance distribution sampling Figure 3 ;

[0066] Figure 8 A graph is generated for an embodiment of an image data augmentation method based on single instance distribution sampling. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Example 1

[0068] like Figure 1~2 As shown, an image data augmentation method based on single instance distribution sampling is mainly composed of four steps.

[0069] 1. If Figure 3 As shown, build the initial database

[0070] Different types of data have different collection methods. For medical image data, images can be acquired with the help of electron microscopes, phase contrast microscopes, and optical microscopes, and data can be collected using microscope cameras. For natural image data, data can be collected using ordinary cameras or web crawlers, and the collected images can be cleaned and stored.

[0071] 2. Instance feature extraction

[0072] Based on the "initial database", an "instance feature extraction model" is constructed to extract unique features from all instances in the database. The instance features are optimized using the mutual information maximization strategy of a single instance to extract the most accurate features of the instance.

[0073] 3. Instance feature distribution awareness

[0074] Model the features of each instance obtained in 2, learn the specific distribution of each instance feature based on the "noise addition-denoising" model, and use this distribution for sampling.

[0075] 4. Image Generation

[0076] Based on the feature distribution of each instance obtained in 3, a specific feature corresponding to the instance is sampled from the distribution, and the feature is input into the image generation module to obtain an image that belongs to the same category as the instance but has different content. The extracted instance features can also be combined with different instance features to output images related to the combined instance features in a directional manner.

[0077] The present invention performs rapid data augmentation based on “single instance distribution sampling” to build a comprehensive, complete and diversified training database.

[0078] In order to train the instance feature extraction model, the instance feature distribution perception model and the image generation model, it is necessary to first collect data preliminarily and build an "initial database". Then, the instance feature extraction model is trained based on the "initial database". The features of a single instance are obtained based on the instance feature extraction model. The feature is input into the "instance feature distribution perception model" and further optimized to obtain the specific distribution of a single instance feature. The specific distribution of a single instance feature is sampled to obtain the sampled instance feature, and the instance feature is input into the "image generation model" for feature decoding to obtain the target image. At the same time, different instance features can be fused to obtain images related to the fused instance features. The obtained image is further cleaned and added to the database to build a large-scale training database. The specific process is as follows:

[0079] (1) Constructing the initial database

[0080] First, the present invention collects relevant data, collects all collectible data as much as possible, covers samples that may be encountered in actual application scenarios, builds an "initial database", enriches the diversity of the database, and lays a solid foundation for obtaining a better feature extraction model.

[0081] (2) Construction of instance feature extraction module

[0082] The feature extraction scheme based on supervised learning tends to extract common features related to categories, but ignores the unique features of a single instance. In order to extract the unique features of a single instance, the present invention adopts an optimization strategy of maximizing the mutual information of instances based on the single instance classification method to build an unsupervised single instance feature extraction model.

[0083] The present invention uses a self-attention network to encode features of a single instance. The network divides a single instance image into image blocks, each of which is 16*16 in size, and each image block is represented by a feature vector. In order to construct an instance-level classification task, the present invention performs strong data enhancement on each image to obtain a pre-enhancement-post-enhancement image pair as a positive sample image set. At the same time, 60,000 images are randomly selected from the training database as a negative sample image set to construct an optimization goal of maximizing instance-level mutual information. The specific optimization goals are as follows:

[0084] ;

[0085] in, represents the loss function of the model, q Represents the characteristics of the current instance, Represents the enhanced features of the current instance, Represents the characteristics of all negative sample instances, and r is a hyperparameter that controls the size of similarity.

[0086] This loss function will gradually reduce the distance between the positive sample pairs before and after enhancement, and gradually increase the distance between the current sample and other negative samples, ultimately obtaining unique features of a single instance, reducing the similarity of features between different instances of the same category, and achieving the goal of instance feature extraction.

[0087] The unique features of a single instance are a concept proposed in relation to supervised learning tasks. The features learned in supervised learning tasks are the "common features" of a certain class of samples, and the unique features of a single sample in that class will be ignored. Common features contain less detailed information and are not suitable for generating tasks.

[0088] The unique features of a single instance contain the unique information of a single sample, and the generated image has more details and is more realistic.

[0089] (3) Instance feature distribution awareness

[0090] All the data collected by the present invention each obeys a certain distribution. If we can infer the specific distribution that a single data obeys and sample from the distribution, we can supplement the data of rare categories with small amounts of data and increase the diversity of the data. However, it is very difficult to estimate the data distribution directly. Based on the "noise-denoising" model, we estimate the distribution of all sample instances at the feature level, use the estimated distribution to sample instance features, decode the sampled features into images, and expand the database.

[0091] The denoising-denoising model assumes that the instance features after denoising obey the Gaussian distribution, and denoises the instance features step by step until the instance features completely obey the standard normal distribution, at which point the instance features completely become Gaussian noise. Then, the instance features that have become Gaussian noise are denoised step by step, gradually restoring the instance features before denoising. In this process, the model learns the specific distribution of the instance features, and samples according to the distribution to obtain the specific features required for the next level of denoising. The specific process is as follows:

[0092] First, based on the instance feature obtained in 2, the feature is denoised. The denoising method is as follows:

[0093] ;

[0094] in, Indicates a given When distributed, Obey the distribution. N represents a normal distribution, Indicates t The specific sample at time -1 (the sample after the instance feature is added with noise, that is, the instance feature with noise added), Indicates t time The variance of the distribution that follows, I Represents the unit matrix. The above formula represents the specific sample of the input , gradually adding mean 0 and variance Gaussian noise, so that the conditional distribution becomes a Gaussian distribution, in the last T time, Completely becomes zero mean, variance is I The multivariate standard normal distribution of .

[0095] Then, based on the multivariate standard normal distribution, the added noise is denoised step by step, the original single instance features are gradually restored and the image is generated through the decoder. The specific operations are as follows:

[0096] ;

[0097] in, express exist The conditional distribution under N represents a normal distribution, =1- ， 。 The above form is based on TThe multivariate standard Gaussian distribution at each moment is used to gradually denoise the added noise. The conditional distribution formed after each denoising step is a multivariate Gaussian distribution, and finally the single instance feature before denoising is obtained. In the denoising process, each step obtains The specific parameters of the Gaussian distribution can be randomly sampled from the distribution represented in the last step to obtain a large number of Similar single instance characteristics.

[0098] The present invention learns the true distribution of each instance through the following loss function:

[0099] ;

[0100] in, Loss represents the loss function to be optimized, Representation distribution and distribution of KL Divergence, Indicates that the distribution By optimizing the above loss function, the model gradually learns the true distribution of a single instance, and then can sample under this distribution.

[0101] The schematic diagram of noise addition and denoising is as follows Figure 4 shown.

[0102] (4) Image generation

[0103] Based on the single instance feature obtained by sampling in step 3, the feature can be decoded to generate an image related to the information contained in the feature. The image generation module is a decoder based on the "self-attention network architecture".

[0104] During the image generation phase, the

[0105] Image feature extraction -> noise addition -> denoising -> language model feature prediction -> feature decoding -> generate image.

[0106] In the inference phase, there are three cases:

[0107] a. Sampling random features from normal distribution -> denoising -> language model feature prediction -> feature decoding -> generating images.

[0108] b. The number of images that need to be increased in a targeted manner -> feature extraction (mapped to feature vectors) -> denoising -> language model feature prediction -> feature decoding -> image generation.

[0109] c. The number of images A that need to be increased in a targeted manner -> feature extraction (mapped to feature vectors) -> the number of images B that need to be increased in a targeted manner -> feature extraction (mapped to feature vectors) -> feature fusion -> denoising -> language model feature prediction -> feature decoding -> generate image.

[0110] The generated images are real images generated by AI.

[0111] The specific method is as follows: First, a single instance image is divided into image blocks, each of which is 16*16 in size. Each divided image block is mapped into a feature vector through a self-attention network. The feature vector is matched with the features in the "feature vector vocabulary", and the most similar feature vector in the vocabulary is found. All image blocks are replaced with the feature vectors in the vocabulary. There is a public database for feature word vector tables in the industry, and existing word vector tables can be used directly. Or use methods such as vq-vae and vq-gan to train word vector tables suitable for local databases. Then, according to the "language model" rules, the initial feature vector and the feature vector corresponding to the instance are given, and the model gradually predicts the next correct feature vector based on the initial given feature vector.

[0112] The language model predicts the next word that should appear based on the known text in order. For example, for the sentence "machine learning is amazing", given the word "machine", it predicts "machine".

[0113] Then given "machine", predict "learning", given "machine learning", predict "learning". After the introduction of the image field, the above words or single characters are replaced by each image block. That is, based on the known image blocks, predict the next image block that may appear and is related to the known image blocks.

[0114] The specific prediction process is as follows:

[0115] ;

[0116] in, represents the joint distribution of a single instance image patch, Indicates t The edge distribution of the image patch. t When calculating the conditional distribution of time, we need to know t=t -1 time has arrived t = 1. And so on, we finally get the complete feature vector of a single instance image.

[0117] Then, the obtained feature vector is input into the image decoding module, the instance feature is decoded into the corresponding real image, and then stored in the warehouse after cleaning. The present invention uses the mean square error loss function to constrain the decoding process so that the image generated by the decoding model is as consistent as possible with the input image. The specific optimization goals are as follows:

[0118] ;

[0119] Where M is the total number of images, For the generated image, is the original input image. Through the above target optimization, the generated image gradually approaches the original image, and finally the two are almost the same.

[0120] When the feature vectors of different instances are weighted and fused, a real image containing the features of both instances is obtained, which realizes directional image generation and effectively supplements the data. The specific weighting method is as follows:

[0121] ;

[0122] in, is the weighted weight, , is the weighted instance feature, , They represent the characteristics of different instances.

[0123] Through the above steps, the scarce tail categories can be targetedly augmented through image generation, and images can be generated by sampling from instance distribution on a large scale, which greatly enriches the diversity of the database and improves the generalization ability and practical application value of downstream models.

[0124] like Figures 5 to 8 The images generated after model augmentation.

[0125] The specific embodiments are as follows:

[0126] Assume that the application scenario is to "expand the data of a certain form of birds". The specific flow chart is as follows:

[0127] 1. Select the bird category to be amplified and give a sample image.

[0128] 2. Input the given sample image into the feature extraction module to obtain the extracted sample features.

[0129] 3. Input the sample features into the feature decoding model to generate images, obtain generated images similar to the given examples, and supplement such images in a targeted manner and store them in the database.

[0130] The image after directional supplementation from the left single instance is as follows Figure 8 shown.

[0131] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for image data augmentation based on single instance distribution sampling, characterized in that: The following steps are involved: S1. constructing an initial database, wherein the initial database includes natural image data and / or medical image data; S2, constructing an instance feature extraction model, wherein the instance feature extraction model extracts the feature vector of the image block of the single instance to be analyzed, performs data enhancement to obtain a pre-enhancement-post-enhancement image pair, the pre-enhancement-post-enhancement image pair is used as a positive sample pair to obtain a positive sample image set, randomly extracts a specified number of images from the initial database to extract feature vectors to obtain a negative sample image set, and uses a loss function to extract feature vectors. Reduce the distance between the positive sample pairs, increase the distance between the current sample and other negative samples, and finally obtain the unique features of a single instance; S3, modeling each of the single instance-specific features to obtain an instance feature distribution learning model, performing distribution estimation on all the single instance-specific features at the feature level based on the instance feature distribution learning model to obtain a feature distribution obeyed by each instance, and sampling based on the feature distribution obeyed by each instance to obtain a single instance specific feature; S4. Construct a feature decoding image generation model, input the specific features of the single instance into the feature decoding image generation model to obtain an image that belongs to the same category as the single instance but has different content; or the extracted instance features can be combined with different instance features to output an image related to the combined instance features; the image generated by the feature decoding image generation model is expanded into the database to obtain an expanded database, and an image data augmentation method based on single instance distribution sampling is completed.

2. The image data augmentation method based on single instance distribution sampling according to claim 1, characterized in that: In step S1, the medical image data is obtained by acquiring medical images through an electron microscope, a phase contrast microscope and an optical microscope and collecting data using a microscope camera; The natural image data is obtained by collecting natural images through a camera or a web crawler and cleaning the collected images.

3. The image data augmentation method based on single instance distribution sampling according to claim 1, characterized in that: In step S2, the instance feature extraction model is a self-attention network model; Step S2 includes the following steps: S21, constructing the instance feature extraction model, wherein the instance feature extraction model divides the single instance image to be analyzed into image blocks, each image block has a size of 16×16, and each image block is represented by a feature vector; S22, performing data enhancement on the single instance image to be analyzed and dividing it into 16×16 image blocks to obtain enhanced image blocks, wherein the data enhancement method includes rotation, flipping, image deformation, contrast and brightness enhancement, extracting a feature vector of the enhanced image block and combining it with a feature vector of the pre-enhancement image block to obtain the pre-enhancement-enhancement image pair as the positive sample pair, and all the positive sample pairs constitute the positive sample image set; S23, the instance feature extraction model randomly extracts a specified number of images from the initial database as negative sample images, divides the negative sample images into image blocks, each image block has a size of 16×16, and makes each image block represented by a feature vector to obtain the negative sample image set; S24. Loss function for: ； Among them, q is the feature vector of the image block before enhancement, is the enhanced image block feature vector, is the feature vector of the negative sample image patch, and τ is a hyperparameter that controls the similarity; S24, by optimizing the loss function Reduce the distance between the positive sample pairs and increase the distance between the current sample and other negative samples. When the loss function After reaching the target value, the single instance-specific feature is obtained, and the single instance-specific feature is the image block feature vector obtained after optimization.

4. The image data augmentation method based on single instance distribution sampling according to claim 1, characterized in that: In step S3, the instance feature distribution learning model is an encoding-decoding structure of a self-attention network; When training the instance feature distribution learning model, the instance feature distribution learning model assumes that the single instance-specific feature after noise addition obeys a Gaussian distribution, and adds noise to the single instance-specific feature step by step until the single instance-specific feature completely obeys a standard normal distribution, and the single instance-specific feature becomes Gaussian noise; then the single instance-specific feature that has become Gaussian noise is denoised step by step, and the instance feature before noise addition is gradually restored; The instance feature distribution learning model learns the specific distribution of the unique features of the single instance during the denoising and denoising process and obtains the specific features of the single instance by sampling according to the specific distribution; When the instance feature distribution learning model is used for reasoning, the instance feature distribution learning model denoises the unique features of a single instance step by step to obtain the specific features of the single instance.

5. The image data augmentation method based on single instance distribution sampling according to claim 4, characterized in that: The training method of the instance feature distribution learning model comprises the following steps: S31, modeling each of the single instance-specific features to obtain the instance feature distribution learning model, and performing noise processing on the single instance-specific features by the instance feature distribution learning model; The noise adding method is: for the input single instance unique feature sample Gradually add mean 0 and variance The Gaussian noise makes the conditional distribution becomes a Gaussian distribution, in the last T time, Completely becomes zero mean, variance is I The multivariate standard normal distribution of I is the identity matrix; S32, performing denoising operations on the added noise step by step based on the multivariate standard normal distribution, gradually restoring the original single instance features and generating an image through a decoder; The denoising method is: based on T The multivariate standard Gaussian distribution at each moment gradually denoises the added noise. The conditional distribution formed after each denoising step is a multivariate Gaussian distribution, and finally the single instance feature before denoising is obtained; in the denoising process, each step obtains The specific parameters of the Gaussian distribution that follows can be randomly sampled from the distribution represented in the last step to obtain the same Similar single instance features; S33, the denoising-denoising model learns the true distribution obeyed by each instance through the loss function Loss and performs sampling under the true distribution to obtain the specific features of the single instance; When the instance feature distribution learning model is used for reasoning, only step S32 is used for step-by-step denoising to obtain the specific features of the single instance.

6. The image data augmentation method based on single instance distribution sampling according to claim 5, characterized in that: In step S31, the process of adding noise is as follows: ； in, For a given Distribution The distribution of N is a normal distribution, For the t -1 specific sample at time, For the t A specific sample of the moment, For the t time The variance of the distribution that follows.

7. The image data augmentation method based on single instance distribution sampling according to claim 6, characterized in that: In step S32, the denoising process is as follows: ； in, for exist The conditional distribution under N is a normal distribution, =1- ， 。 8. The image data augmentation method based on single instance distribution sampling according to claim 7, characterized in that: In step S33, ； in, For distribution and distribution of KL Divergence, For the distribution The expectations below, for In a given The predicted distribution under for In a given The predicted distribution under .

9. The image data augmentation method based on single instance distribution sampling according to claim 1, characterized in that: In step S4, the feature decoding image generation model matches the specific features of the single instance with the features in the feature vector vocabulary, finds the most similar feature vector in the vocabulary, and replaces all image blocks of the single instance to be analyzed with the feature vector in the vocabulary; then, according to the language model rules, the feature decoding image generation model gives the initial feature vector and the corresponding feature vector and gradually predicts the next correct feature vector: ； in, is the joint distribution of the single instance image patches to be analyzed, For the t The edge distribution of image patches; prediction t When calculating the conditional distribution of time, we need to know t=t -1 time has arrived t =1; finally, the complete predicted feature vector of the single instance image is obtained; Input the predicted feature vector into the image decoding module of the feature decoding image generation model, decode it into the corresponding real image, and then enter it into the database after cleaning; The mean square error loss function is used for constraints during decoding; ； in, M is the total number of images, For the generated image, is the original input image.

10. The image data augmentation method based on single instance distribution sampling according to claim 1, characterized in that: In step S4, when the feature vectors of different instances are weightedly fused, a real image containing features of both instances is obtained to generate a directional image: ； in, is the weighted weight, , is the weighted instance feature, , are the characteristics of different instances respectively; In step S4, the image may also be generated using a single instance specific feature obtained by denoising a random feature sampled from a normal distribution using the instance feature distribution learning model.

Images