Data set distillation method based on representative sample feature distribution matching
Through the feature distribution matching method based on representative samples, the high deviation problem of the existing data set distillation algorithm when the number of iterations increases is solved, and the high-quality synthetic data set generation and cross-model generalization capabilities are realized on large data sets. Improves training stability and efficiency.
Patent Information
- Application Number
- CN202410724130.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-05
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-06-05
AI Technical Summary
Existing dataset distillation algorithms introduce high bias when the number of iterations increases, resulting in reduced performance of synthetic datasets and inability to scale to large datasets or high compression rates. The use of random sampling in existing robust methods may lead to matching target bias. and noise.
Using the feature distribution matching method based on representative samples, representative samples are selected through the k-means algorithm, data interpolation and differential twin enhancement are performed, feature distribution matching loss and discriminant loss are calculated, and network parameters are optimized to generate synthetic samples with strong interpretability. .
It realizes a high-quality synthetic dataset with good generalization ability across models, is suitable for large datasets, and accelerates the training process, improving training stability and efficiency.
Smart Images

Figure CN118887486A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of data set distillation, and in particular relates to a data set distillation method based on representative sample feature distribution matching. Background Art
[0002] At present, deep learning is largely due to the amount of data for training deep neural networks. However, the massive amount of data increases the burden of storage and transmission, and further generates a cumbersome model training process. In order to alleviate these shortcomings, it is undoubtedly a good choice to obtain a dataset with high information density through synthesis. The dataset distillation algorithm obtains a high-quality synthetic dataset to replace the original dataset while retaining a sufficiently small sample size in each class. The purpose is to accelerate network training, reduce data storage, and avoid potential data privacy issues during transmission.
[0003] The existing dataset distillation algorithms mainly include:
[0004] (1) Based on gradient matching algorithm
[0005] The gradient matching-based algorithm updates the synthetic dataset by minimizing the gradient matching loss at each training step. However, as the number of iterations increases, this method will introduce high bias. The synthetic dataset learned through gradient matching, especially the large gradient samples, will have a lot of bias. The performance will degrade when crossing models.
[0006] (2) Algorithm based on trajectory matching
[0007] In order to overcome the problem of deviation accumulation caused by gradient matching, the trajectory matching algorithm proposes to use expert models and student models to learn the real data set and synthetic data set respectively, and continuously fit the training trajectories of real samples and synthetic samples during the training process. Although this method can obtain high-quality synthetic samples, it is quite sensitive to hyperparameters. It is only applicable to small data sets, cannot be expanded to large data sets, and is not applicable to large compression rates.
[0008] (3) Regularization-based algorithms
[0009] Regularization-based algorithms use some regularization methods to guide the synthesis process, and enhance the interpretability of images by learning the hidden patterns of images and controlling the diversity of synthetic data. A pre-trained decoder can be used to obtain the encoding of the latent vector, and then the low-dimensional compressed encoding can be restored to the image pixel space. However, this algorithm is difficult to train due to the large number of model parameters, and has high requirements for hardware equipment. It cannot be extended to large classification datasets such as ImageNet.
[0010] The invention is named "A robust dataset distillation method and system" and the application publication number is "CN 115761414A". The invention proposes a robust dataset distillation method. First,
[0011] Random sampling is performed in the real data set to obtain an initialized synthetic data set; then random sampling is performed in the real data set to obtain an initialized real data sampling set; adversarial perturbations are added to the initialized real data sampling set to obtain a perturbed sample sampling set; the feature representations of the perturbed sample sampling set and the synthetic data set are respectively calculated, and the difference between the feature representations is used as the loss function; the synthetic data set is optimized and updated according to the back propagation of the loss function, the optimization process is continuously iterated until the loss function converges, and the current distilled data set is saved as the final robust distilled data set.
[0012] This method uses a random sampling initialization method, which may lead to deviations in matching targets and introduce some noise and large gradient samples. Summary of the invention
[0013] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a data set distillation method based on representative sample feature distribution matching. The method first selects representative samples for initialization, and then uses a parameterized differential twin enhancement strategy to perform data enhancement on these representative samples. Finally, by optimizing the feature distribution matching loss and discrimination loss between the real sample and synthetic sample levels, a synthetic sample with strong interpretability is obtained. This method can greatly accelerate the entire training process, and the entire training process is relatively stable. It has a wide range of applicable data sets, obtains high-quality synthetic samples, and has good generalization capabilities across models.
[0014] In order to achieve the above object, the technical solution adopted by the present invention is:
[0015] A data set distillation method based on representative sample feature distribution matching includes the following steps:
[0016] Step 1: Use the k-means algorithm to select representative real samples and synthetic samples for initialization;
[0017] The real samples refer to the images in the original dataset, while the synthetic samples refer to the representative images generated by the original dataset after the compression algorithm. In the first round, the representative synthetic samples are the cluster centers obtained by using the k-means algorithm based on the set IPC (Image Per Class) of the real dataset. Subsequently, k-means will be initialized based on the synthetic samples obtained by the previous round of optimization.
[0018] Step 2: performing parameterization processing on the representative real samples and synthetic samples through data interpolation;
[0019] Step 3: Use the differential twin enhancement strategy to perform data enhancement on the parameterized real samples and synthetic samples;
[0020] Step 4: Input the real samples and synthetic samples after data augmentation into the model, calculate the feature distribution between the real sample and synthetic sample levels, and use the difference between the two as the feature distribution matching loss function;
[0021] Step 5: Use the prototype features of the synthetic samples to classify the real samples and calculate the correlation between them as the sample discrimination loss;
[0022] Step 6: Weight the losses obtained in step 4 and step 5 to obtain the final optimized loss function, and continuously iterate and optimize the network parameters and synthetic samples until the loss function converges, and finally obtain a synthetic sample with strong interpretability. The specific process of initialization of step 1 is:
[0023] Step 1.1: According to the set mini-batch size K, as the K value of the k-means clustering algorithm, divide the sub-clusters in each class of the real sample to obtain representative samples in each class, and use the sample centers of the divided K sub-clusters as the initialization samples.
[0024] The k-means clustering algorithm is based on the Euclidean distance metric, and the formula is as follows:
[0025]
[0026] Here represents the subcluster c i The center of x i represents each sample, M represents the total number of samples, and J(c,μ) represents the minimum Euclidean distance of all samples in subcluster c to the subcluster center μ;
[0027] In step 1.2, according to the set IPC (Image Per Class) value, the k-means clustering algorithm is also used to initialize the synthetic samples. The real samples of each class are divided into IPC subclusters, and the centers of these subclusters are used as initialization samples.
[0028] The specific process of the parameterization process in step 2 is as follows:
[0029] Step 2.1: For the representative real samples and synthetic samples obtained in step 1, the bilinear upsampling method is used to partition them. Considering the correlation between adjacent pixels of the image, two linear interpolations are performed in the X-axis direction and the Y-axis direction. The linear interpolation formula is:
[0030]
[0031] Here (X0, Y0), (X1, Y1) are known points;
[0032] Step 2.2 performs a size transformation operation on the representative real samples and synthetic samples after partitioning obtained in step 2.1, and restores them to the original sample size, so that the synthetic samples are optimized from multiple local minima.
[0033] The specific process of the differential twin enhancement strategy in step 3 is as follows:
[0034] For the real samples and synthetic samples after parameterization in step 2, the same common data enhancement strategy is used for cropping, rotation, and color jittering. On this basis, cutout and cutmix data enhancement strategies are added. Here, the entire differential twin enhancement module must be differentiable to ensure that the synthetic samples can be back-propagated and updated, which can be expressed by the formula:
[0035]
[0036] Here S represents the synthetic sample, A(·) represents the sampled enhancement strategy, D(·) represents the matching target between the real sample and the synthetic sample, and L θ (·) represents the loss function of training the network;
[0037] Finally, we get the processed real sample T k and synthetic sample S k , k represents the category.
[0038] The specific process of calculating the feature distribution matching loss in step 4 is as follows:
[0039] Step 4.1 For the real sample T of the same class k obtained in step 3 k and synthetic sample S k Input into the L-layer network;
[0040] Step 4.2 Based on step 4.1, obtain the real samples T k and synthetic sample S k The feature vectors of the remaining intermediate layers except the last output layer are expressed by the formula:
[0041]
[0042] Here, and Represent the feature vectors of the real sample and synthetic sample of the kth class respectively, Represents the feature vector of the real sample of the kth class in the i-th layer of the network, Represents the feature vector of the k-th class of synthetic samples in the i-th layer of the network. θ () indicates training network;
[0043] Step 4.3: Since the batch sizes are different and cannot be compared, the prototype features of the real samples and synthetic samples are calculated based on the feature vectors obtained in step 4.2, and the average feature vector of each class is calculated as the prototype feature. C′=C×H×W, C′ represents the size of the corresponding feature map;
[0044] Step 4.4 uses the mean square error to calculate the feature distribution matching loss of the prototype features of the real sample and the synthetic sample obtained in step 4.3 in the middle layer. The formula is as follows:
[0045]
[0046] Here, K represents the number of categories in the dataset, and L represents the total number of layers in the training network. Represents the prototype features of the real sample of the kth class in the lth layer of the network, Represents the prototype features of the synthetic samples of the kth category in the lth layer of the network.
[0047] The specific process of calculating the discrimination loss in step 5 is as follows:
[0048] Step 5.1 uses the prototype features of the synthetic samples of this class in the last layer of the training network as a classifier to classify all real samples in this class, and uses the result of the inner product to measure the correlation between samples, which is expressed by the formula:
[0049]
[0050] Here N′=N×K, I contains the logits values of N′×K real samples, N represents the number of real samples, K represents the number of categories in the data set, Represents the feature vector value of the real sample in this class at the last layer of the network, that is, the logits value. Represents the prototype features of synthetic samples of this class;
[0051] Step 5.2: Based on the inner product value obtained in step 5.1, calculate the softmax value of each real sample in the class and get the predicted classification probability value, which is expressed as follows:
[0052] p i =softmax(I i )
[0053] Step 5.3: Based on the classification probability value obtained in step 5.2, use the cross entropy loss function to obtain the discriminant loss, which is expressed as:
[0054]
[0055] The specific process of iterative optimization of network parameters and synthetic samples in step 6 is as follows:
[0056] Step 6.1 weights the feature distribution matching loss and discrimination loss obtained in steps 4.4 and 5.3 to obtain the final loss function, which is expressed as:
[0057] L=L m +αL c
[0058] Here α is a weighting factor;
[0059] Step 6.2 uses the output logits value of the last network layer of the synthetic sample of each class obtained in step 5.1, and uses the cross entropy loss function to iteratively update the network parameter θ, which is expressed by the formula:
[0060]
[0061] Here M represents the number of synthetic samples, m i Represents the probability value after comparing the synthetic sample with the true label;
[0062] Step 6.3 uses the loss function obtained in step 6.1 to iteratively update the synthetic samples, and finally obtains synthetic samples with strong interpretability, which can be expressed as:
[0063]
[0064] A dataset distillation system based on representative sample feature distribution matching, comprising:
[0065] Initialization clustering module: Use the k-means clustering algorithm to generate sub-clusters, and use the sub-cluster centers as the initialization of real samples and synthetic samples;
[0066] Data enhancement module: bilinear interpolation is used to upsample the image to achieve diversity of image information, and differential twin enhancement strategy is used to enhance the image;
[0067] Feature distribution matching module: The mean square error of the feature vectors of real samples and synthetic samples in the middle layer is used as the feature distribution matching loss;
[0068] Discriminant loss module: The prototype features of the synthetic samples in the last network layer are used as classifiers to classify the real samples, obtain the discriminant loss, and finally obtain synthetic samples with strong interpretability. First, the real samples and synthetic samples are initialized by the initialization clustering module to obtain the initialized samples; then the data enhancement module generates diversified samples to reduce overfitting; during the network training process, the feature distribution of the synthetic samples is made close to the feature distribution of the real samples through the feature distribution matching module and the discriminant loss module, and finally obtain synthetic samples with strong interpretability. A data set distillation device based on representative sample feature distribution matching, including:
[0069] Memory: used to store a computer program for implementing the data set distillation method based on representative sample feature distribution matching;
[0070] A processor is used to implement the data set distillation method based on representative sample feature distribution matching when executing the computer program.
[0071] A computer-readable storage medium, comprising:
[0072] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement a data set distillation method based on representative sample feature distribution matching.
[0073] Beneficial effects of the present invention:
[0074] The present invention adopts the k-means strategy of initializing real samples and synthetic samples. The selected samples can cover the feature distribution of the entire class as evenly as possible, thereby achieving the purpose of stable training and rapid optimization of synthetic samples.
[0075] The bilinear interpolation parameterization image strategy adopted by the present invention takes into account the correlation between adjacent pixels of the image and can optimize the synthesized samples from multiple local minimum values.
[0076] The differential twin enhancement strategy adopted by the present invention cannot be directly randomly enhanced due to the different sample numbers of real samples and synthetic samples. Considering the problem of back-propagation optimization of synthetic samples, the data enhancement operation must be differentiable, which realizes the effective enhancement of samples.
[0077] The feature distribution matching strategy adopted in the present invention optimizes the synthetic samples from a hierarchical perspective, so that the feature distribution of the real samples can be captured, the features can be aligned across different scales, and the feature distribution of the synthetic samples can be further made close to the feature distribution of the real samples.
[0078] The discriminative loss strategy adopted by the present invention improves the discriminative ability of synthetic samples and makes their prototype features more practical. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] Figure 1 The present invention is a flow chart of the method.
[0080] Figure 2 This is a module diagram of feature distribution matching loss and discrimination loss of the present invention.
[0081] Figure 3 This is a comparison chart of the classification accuracy of the present invention using different initialization strategies on the CIFAR10 dataset.
[0082] Figure 4 This is the final partially compressed image obtained by the present invention on the CIFAR100 dataset when IPC=10. DETAILED DESCRIPTION
[0083] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0084] See also Figure 1 , a dataset distillation method based on representative sample feature distribution matching, comprising the following steps:
[0085] Step 1: Use the k-means algorithm to select representative real samples and synthetic samples for initialization, so that the selected samples can cover the feature distribution of the entire class as evenly as possible, achieving the purpose of stable training and rapid optimization of synthetic samples.
[0086] The real samples refer to the images in the original dataset, while the synthetic samples refer to the representative images generated by the original dataset after the compression algorithm. In the first round, the representative synthetic samples are the cluster centers obtained by using the k-means algorithm based on the set IPC (Image Per Class) of the real dataset. Subsequently, k-means will be initialized based on the synthetic samples obtained by the previous round of optimization.
[0087] The datasets used are MNIST, FashionMNIST, CIFAR10, CIFAR100 and TinyImageNet. The K value of each class of real samples is set to 128, and the number of saved samples of synthetic samples in each class is set to IPC = 1 / 10 / 50.
[0088] Step 1.1: Divide sub-clusters in each class of the real sample according to the set mini-batch size K as the K value of the k-means clustering algorithm. Obtain representative samples in each class, and use the sample centers of the divided K sub-clusters as initialization samples.
[0089] The k-means clustering algorithm is based on the Euclidean distance metric, and the formula is as follows:
[0090]
[0091] Here represents the center of the subcluster, x i represents each sample, M represents the total number of samples, and J(c,μ) represents the minimum Euclidean distance of all samples in subcluster c to the subcluster center μ.
[0092] Step 1.2: According to the set IPC (Image Per Class) value, the synthetic samples are initialized using the k-means clustering algorithm. The real samples of each class are divided into IPC subclusters, and the centers of these subclusters are used as initialization samples.
[0093] Step 2: The representative real samples and synthetic samples obtained in step 1 are parameterized by data interpolation. The correlation between adjacent pixels of the image is considered, so that the synthetic samples can be optimized from multiple local minima.
[0094] Step 2.1: Use bilinear upsampling to partition the representative real samples and synthetic samples after clustering obtained in step 1, and perform linear interpolation twice in the X-axis direction and the Y-axis direction. The formula for linear interpolation is:
[0095]
[0096] Here (X0, Y0), (X1, Y1) are known points.
[0097] Step 2.2 performs a resize operation on the partitioned representative real samples and synthetic samples obtained in step 2.1 to restore them to the original sample size.
[0098] Step 3: Use the differential twin enhancement strategy to perform data enhancement on the parameterized real samples and synthetic samples obtained in step 2.
[0099] Make real samples and synthetic samples more interpretable and increase data diversity.
[0100] For the processed real samples and synthetic samples obtained in step 2, the same common data enhancement strategies are used, such as cropping, rotation, and color jittering. On this basis, cutout and cutmix data enhancement strategies are added. Here, considering the problem of backpropagation optimization of synthetic samples, the entire differential twin enhancement module must be differentiable to ensure that the synthetic samples can be backpropagated and updated. Expressed in formula:
[0101]
[0102] Here S represents the synthetic sample, A(·) represents the sampled enhancement strategy, D(·) represents the matching target between the real sample and the synthetic sample, and L θ (·) represents the loss function of training the network.
[0103] See also Figure 2 , Step 4: Input the real samples and synthetic samples obtained in step 3 into the model.
[0104] The feature distribution between the real sample and the synthetic sample level is calculated, and the difference between the two is used as the feature distribution matching loss function. In this way, the synthetic samples can be optimized from a hierarchical perspective, the feature distribution of the real sample can be captured, the features can be aligned across different scales, and the feature distribution of the synthetic sample can be further made close to the feature distribution of the real sample.
[0105] The training model uses a three-layer ConvNet architecture. Then, the network is trained from scratch using these synthetic datasets and evaluated on real test datasets.
[0106] The training network structure consists of three convolutional blocks, each of which consists of a 3×3 convolutional layer with 128 kernels, followed by instance normalization, ReLU activation layer, and 2×2 average pooling layer with a step size of 2. Due to the large input volume of the dataset, a fourth convolutional block is added when training on the TinyImageNet dataset. The initial learning rate is 0.01, the weight decay is 5e-4, and the momentum value is 0.9. The outer loop is trained for 2000 iterations to optimize the synthetic images, and the inner loop is trained for 300 iterations to optimize the network parameters.
[0107] Step 4.1 For the real sample T of the same class k obtained in step 3 k and synthetic sample S k Input to the L-layer network.
[0108] Step 4.2 Based on step 4.1, obtain the real samples T k and synthetic sample S k The feature vectors of the remaining intermediate layers except the last output layer are expressed by the formula:
[0109]
[0110]
[0111] Here, and Represent the feature vectors of the real sample and synthetic sample of the kth class respectively, Represents the feature vector of the real sample of the kth class in the i-th layer of the network, represents the feature vector of the k-th class of synthetic samples in the i-th layer of the network, φ θ () indicates training network.
[0112] Step 4.3 Since the batch sizes are different and cannot be compared, the prototype features of the real samples and synthetic samples are calculated based on the feature vectors obtained in 4.2, and the average feature vector of each class is calculated as the prototype feature. C′=C×H×W, where C′ represents the size of the corresponding feature map.
[0113] Step 4.4 uses the mean square error to calculate the feature distribution matching loss of the prototype features of the real sample and the synthetic sample obtained in step 4.3 in the middle layer. The formula is as follows:
[0114]
[0115] Here, K represents the number of categories in the dataset, and L represents the total number of layers in the training network. Represents the prototype features of the real sample of the kth class in the lth layer of the network, Represents the prototype features of the synthetic samples of the kth category in the lth layer of the network.
[0116] See also Figure 2 , Step 5: Use the prototype features of the synthetic samples to classify the real samples and calculate the correlation between them as the sample discrimination loss.
[0117] Step 5.1 uses the prototype features of the synthetic samples of this class in the last layer of the training network as a classifier to classify all real samples in this class, and uses the result of the inner product to measure the correlation between samples, which is expressed by the formula:
[0118]
[0119] Here N′=N×K, I contains the logits values of N′×K real samples, N represents the number of real samples, K represents the number of categories in the data set, Represents the feature vector value of the real sample in this class at the last layer of the network, that is, the logits value. Represents the prototype features of synthetic samples of this class;
[0120] Step 5.2: Based on the inner product value obtained in step 5.1, calculate the softmax value of each real sample in the class and get the predicted classification probability value, which is expressed as follows:
[0121] p i =softmax(I i )
[0122] Step 5.3: Based on the classification probability value obtained in step 5.2, use the cross entropy loss function to obtain the discriminant loss, which is expressed as:
[0123]
[0124] Step 6: Weight the losses obtained in step 4 and step 5 to obtain the final optimized loss function, and iterate and optimize the network parameters and synthetic samples until the loss function converges. Finally, a synthetic sample with strong interpretability is obtained, where the weighting factor α=1.
[0125] Step 6.1 weights the feature distribution matching loss and discrimination loss obtained in steps 4.4 and 5.3 to obtain the final loss function, which is expressed as:
[0126] L=L m +αL c
[0127] Here α is a weighting factor.
[0128] Step 6.2 uses the output logits value of the last network layer of the synthetic sample of each class obtained in step 5.1, and uses the cross entropy loss function to iteratively update the network parameter θ, which is expressed by the formula:
[0129]
[0130] Here M represents the number of synthetic samples, m i Represents the probability value after comparing the synthetic sample with the true label.
[0131] Step 6.3 uses the loss function obtained in step 6.1 to iteratively update the synthetic samples, and finally obtains synthetic samples with strong interpretability, which can be expressed as:
[0132]
[0133] To evaluate the performance of synthetic images, our method (Ours) is compared with three core set selection methods, namely Random, Herding, and K-Center. It is also compared with five advanced dataset distillation methods, namely DC, DM, IDM, MTT, and DiM. The experimental results are shown in Table 1.
[0134] From the results in Table 1, we can see that although the performance on MNIST and FashionMNIST is slightly lower than DiM, it is improved on the CIFAR10 dataset. In IPC=1, the accuracy is improved by 1.3%. In addition, the performance of DiM will drop significantly on larger datasets, but the method of this application (Ours) can be applied to larger datasets and can still maintain performance on the TinyImageNet dataset, which has a wide applicability.
[0135] Table 1 Classification accuracy on multiple datasets
[0136]
[0137] To test the generalization of synthetic datasets, the performance results of training ConvNet-3 on the CIFAR10 dataset are given in Table 2. Synthetic data is learned using one architecture (C) and then evaluated on another architecture (T), by training a model from scratch and testing it on real test data. We evaluate several neural architectures, including ConvNet, AlexNet, VGG-11, ResNet10, ResNet18, DenseNet-121. Batch normalization is used in all network structures.
[0138] It can be seen that the method of this application (Ours) has good generalization and can improve by about 10% on multiple test models. In particular, when the test model is ResNet10, the accuracy can be improved by about 20% compared with MTT.
[0139] Table 2 Cross-model generalization evaluation
[0140]
[0141] See also Figure 3 , is a comparison chart of classification accuracy using different initialization strategies on the CIFAR10 dataset. By comparing several different sampling methods, the effectiveness of representative sample selection is verified, namely Random, Central, and Boundary. By simply using the samples in the sub-cluster as the initialization of the synthetic image, it is verified that the performance is significantly better than random initialization. A total of 800 rounds of experiments were iterated on the CIFAR10 dataset, where IPC=10 was set. When the number of training iterations is around 200, the method of this application (Ours) achieves better performance than random sampling and significantly improves the training efficiency. As the number of training iterations continues to increase, the performance will further improve.
[0142] See also Figure 4, which is the final partially compressed image obtained by the present invention on the CIFAR100 dataset when IPC=10. The method of the present application (Ours) introduces more classification features and diversity to the synthetic image.
[0143] A data set distillation device based on representative sample feature distribution matching, comprising:
[0144] Memory: used to store a computer program for implementing the data set distillation method based on representative sample feature distribution matching;
[0145] A processor is used to implement the data set distillation method based on representative sample feature distribution matching when executing the computer program.
[0146] A computer-readable storage medium, comprising:
[0147] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement a data set distillation method based on representative sample feature distribution matching.
Claims
1. A dataset distillation method based on representative sample feature distribution matching, It is characterized in that The following steps are involved: Step 1: Use the k-means algorithm to select representative real samples and synthetic samples for initialization; Step 2: performing parameterization processing on the representative real samples and synthetic samples through data interpolation; Step 3: Use the differential twin enhancement strategy to perform data enhancement on the parameterized real samples and synthetic samples; Step 4: Input the real samples and synthetic samples after data augmentation into the model, calculate the feature distribution between the real sample and synthetic sample levels, and use the difference between the two as the feature distribution matching loss function; Step 5: Use the prototype features of the synthetic samples to classify the real samples and calculate the correlation between them as the sample discrimination loss; Step 6: Weight the losses obtained in step 4 and step 5 to obtain the final optimized loss function. Continuously iterate and optimize the network parameters and synthetic samples until the loss function converges, and finally obtain a synthetic sample with strong interpretability.
2. A data set distillation method based on representative sample feature distribution matching according to claim 1, It is characterized in that The specific process of initialization of step 1 is as follows: Step 1.1: According to the set mini-batch size K, as the K value of the k-means clustering algorithm, divide sub-clusters in each class of the real sample to obtain representative samples in each class, and use the sample centers of the divided K sub-clusters as the initialization samples; The k-means clustering algorithm is based on the Euclidean distance metric, and the formula is as follows: represents the subcluster c i The center of x i represents each sample, M represents the total number of samples, and J(c,c) represents the minimum Euclidean distance sum of all samples in subcluster c to the subcluster center μ; In step 1.2, according to the set IPC (Image Per Class) value, the k-means clustering algorithm is also used to initialize the synthetic samples. The real samples of each class are divided into IPC subclusters, and the centers of these subclusters are used as initialization samples.
3. The method for data set distillation based on representative sample feature distribution matching according to claim 2, It is characterized in that The specific process of the parameterization process in step 2 is as follows: Step 2.1: For the representative real samples and synthetic samples after clustering obtained in step 1, use the bilinear upsampling method to partition them, and perform two linear interpolations in the X-axis direction and the Y-axis direction. The linear interpolation formula is: (X 0 ,Y 0 ),(X 1 ,Y 1 ) is a known point; Step 2.2 performs a size transformation operation on the representative real samples and synthetic samples after partitioning obtained in step 2.1, and restores them to the original sample size, so that the synthetic samples are optimized from multiple local minima.
4. The method for data set distillation based on representative sample feature distribution matching according to claim 3, It is characterized in that The specific process of the differential twin enhancement strategy in step 3 is as follows: For the real samples and synthetic samples after parameter processing in step 2, the same common data enhancement strategy is used, such as cropping, rotation, color jitter, and cutout and cutmix data enhancement strategies are added, which can be expressed by the formula: S represents the synthetic sample, A(·) represents the sampled enhancement strategy, D(·) represents the matching target between the real sample and the synthetic sample, and L θ (·) represents the loss function of training the network; Finally, we get the processed real sample T k and synthetic sample S k , k represents the category.
5. The method for data set distillation based on representative sample feature distribution matching according to claim 4, It is characterized in that The specific process of calculating the feature distribution matching loss in step 4 is as follows: Step 4.1 For the real sample T of the same class k obtained in step 3 k and synthetic sample S k Input into the L-layer network; Step 4.2 Based on step 4.1, obtain the real samples T k and synthetic sample S k The feature vectors of the remaining intermediate layers except the last output layer are expressed by the formula: and Represent the feature vectors of the real sample and synthetic sample of the kth class respectively, Represents the feature vector of the real sample of the kth class in the i-th layer of the network, represents the feature vector of the k-th class of synthetic samples in the i-th layer of the network, φ θ () indicates training network; Step 4.3 calculates the prototype features of the real samples and synthetic samples based on the feature vectors obtained in step 4.2, and calculates the average feature vector of each class as the prototype feature. C′ represents the size of the corresponding feature map; Step 4.4 uses the mean square error to calculate the feature distribution matching loss of the prototype features of the real sample and the synthetic sample obtained in step 4.3 in the middle layer. The formula is as follows: K represents the number of categories in the data set, L represents the total number of layers in the training network, Represents the prototype features of the real sample of the kth class in the lth layer of the network, Represents the prototype features of the synthetic samples of the kth category in the lth layer of the network.
6. The method for data set distillation based on representative sample feature distribution matching according to claim 5, It is characterized in that The specific process of calculating the discrimination loss in step 5 is as follows: Step 5.1 uses the prototype features of the synthetic samples of this class in the last layer of the training network as a classifier to classify all real samples in this class, and uses the result of the inner product to measure the correlation between samples, which is expressed by the formula: I contains the logits values of N′×K real samples, where N represents the number of real samples and K represents the number of categories in the data set. Represents the feature vector value of the real sample in this class at the last layer of the network, that is, the logits value. Represents the prototype features of synthetic samples of this class; Step 5.2: Based on the inner product value obtained in step 5.1, calculate the softmax value of each real sample in the class and get the predicted classification probability value, which is expressed as follows: p i =softmax(I i ) Step 5.3: Based on the classification probability value obtained in step 5.2, use the cross entropy loss function to obtain the discriminant loss, which is expressed as:
7. The method for data set distillation based on representative sample feature distribution matching according to claim 6, It is characterized in that The specific process of iterative optimization of network parameters and synthetic samples in step 6 is as follows: Step 6.1 weights the feature distribution matching loss and discrimination loss obtained in steps 4.4 and 5.3 to obtain the final loss function, which is expressed as: L=L m +αL c α is a weighting factor; Step 6.2 uses the output logits value of the last network layer of the synthetic sample of each class obtained in step 5.1, and uses the cross entropy loss function to iteratively update the network parameter θ, which is expressed by the formula: M represents the number of synthetic samples, m i Represents the probability value after comparing the synthetic sample with the true label; Step 6.3 uses the loss function obtained in step 6.1 to iteratively update the synthetic samples, and finally obtains a synthetic sample with strong interpretability, which can be expressed as:
8. A data set distillation system based on representative sample feature distribution matching for implementing the method according to any one of claims 1 to 7, It is characterized in that include: Initialization clustering module: Use the k-means clustering algorithm to generate sub-clusters, and use the sub-cluster centers as the initialization of real samples and synthetic samples; Data enhancement module: bilinear interpolation is used to upsample the image to achieve diversity of image information, and differential twin enhancement strategy is used to enhance the image; Feature distribution matching module: The mean square error of the feature vectors of real samples and synthetic samples in the middle layer is used as the feature distribution matching loss; Discriminant loss module: The prototype features of the synthetic samples in the last network layer are used as classifiers to classify real samples, obtain the discriminant loss, and finally obtain synthetic samples with strong interpretability.
9. A dataset distillation device based on representative sample feature distribution matching, It is characterized in that include: Memory: used to store a computer program for implementing a data set distillation method based on representative sample feature distribution matching as described in any one of claims 1 to 7; A processor, used to implement a data set distillation method based on representative sample feature distribution matching as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, It is characterized in that include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement any one of claims 1-7, a data set distillation method based on representative sample feature distribution matching.
Citation Information
Patent Citations
Robust data set distillation method and system
CN115761414A
Deep forged image detection and model training method and device, equipment and medium
CN112598643A
AdvGAN-based adversarial sample generation method
CN115510986A
Speech synthesis method and device based on artificial intelligence, computer equipment and medium
CN116580690A
Enhancement processing method and device for image classification, equipment and medium
CN116740432A
Cited By
Synthetic image data set distillation method and system based on curvature guidance
CN122049407A
Data set distillation method based on long-range parameter trajectory matching and trainable synthesis
CN122264025A
