Diversified semantic distribution matching data compression method and system

Abstract

The invention relates to a diversified semantic distribution matching data compression method and system. The method comprises the following steps: pre-training a group of models on an original data set to obtain a pre-trained model; each category in the original data set is modeled as Gaussian distribution by semantic embedding extracted by a pre-training model, a category center represents inherent features of the category, and a covariance matrix represents semantic changes of instances in the category; performing class center alignment and covariance matrix alignment on the synthesized data set and the original data set; optimizing the synthetic data set by using a plurality of pre-training models, and storing a historical class center of the synthetic data set optimized by a previous pre-training model through a memory bank; and aligning the class center of the synthetic data set optimized by the current pre-training model with a historical class center to obtain a semantic diversified synthetic data set. According to the method, a framework based on distribution matching is followed, and the semantic information is fully considered from the perspective of the semantic information, so that the semantic diversity of the compressed synthetic instance is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a diverse semantic distribution matching data compression method and system. Background Art

[0002] In the era of big data, the exponential growth of data volume not only puts tremendous pressure on storage, but also increases the significant overhead of model training. Therefore, how to compress large-scale data into a compact and information-rich subset to maintain similar performance while facilitating the training procedure has become an important issue that needs to be solved. To this end, an intuitive approach is to select representative instances from the original dataset as the core set. However, directly discarding most of the instances during sample selection may lead to the loss of key underlying information, which is crucial for model training. Fortunately, data compression technology has been proposed as a solution to compress the original large-scale dataset into a smaller synthetic dataset while ensuring that the model trained on the synthetic dataset can exhibit comparable performance to that trained on the original dataset.

[0003] Existing dataset compression methods can generally be divided into two categories: optimization-based methods and distribution matching-based methods. (1) Optimization-based methods: This method adopts a two-level optimization process, in which the model is first updated with the original dataset, and then the synthetic dataset is optimized by minimizing the distance to achieve a good match between the original data and the synthetic data. (2) Distribution matching-based methods: This method improves training efficiency by eliminating model updates and directly optimizes the synthetic data by using the corresponding class centers calculated by the randomly initialized model on the synthetic dataset and the original dataset.

[0004] The disadvantages of the prior art are as follows:

[0005] 1) Optimization-based methods: Optimization-based methods integrate the information learned from the original dataset into the synthetic dataset during model training. To achieve this goal, it is necessary to perform iterative two-layer optimization of the model and the synthetic dataset, and simulate the model training process through meta-learning to generate synthetic instances. Under this framework, a variety of methods have been proposed to optimize synthetic instances, including matching training trajectories, constructing sample features using data hallucination networks, and selecting original sample features through clustering. Although these methods have made some progress in improving the performance of synthetic data, they still need to adopt a two-layer optimization paradigm to alternately update the model and synthetic data, which makes the optimization more difficult and time-consuming.

[0006] 2) Distribution matching-based methods: Distribution matching-based methods use the distribution of synthetic datasets to align with the distribution of real datasets in feature space, thereby avoiding model updates and eliminating the complexity of double-layer optimization. However, class center alignment only in feature space ignores the semantic diversity of the original dataset and may lead to oversimplification of synthetic data. Summary of the invention

[0007] The present invention aims at the above problems and provides a diverse semantic distribution matching data compression method and system. The present invention follows a distribution matching-based framework and fully considers semantic information from the perspective of semantic information to enhance the semantic diversity of compressed synthetic instances.

[0008] The technical solution adopted by the present invention is as follows:

[0009] A diverse semantic distribution matching data compression method includes the following steps:

[0010] Pre-train a set of models on the original data set to obtain a pre-trained model;

[0011] Each category in the original dataset is modeled as a Gaussian distribution using the semantic embedding extracted by the pre-trained model, where the class center represents the inherent characteristics of the category and the covariance matrix represents the semantic changes of instances in the category;

[0012] Align the class centers and covariance matrices of the synthetic dataset with the original dataset;

[0013] Optimize the synthetic dataset using multiple pre-trained models, and store the class center of the synthetic dataset optimized by the previous pre-trained model, i.e., the historical class center, through the memory bank;

[0014] The class centers of the synthetic dataset optimized by the current pre-trained model are aligned with the historical class centers to obtain a semantically diverse synthetic dataset.

[0015] Furthermore, for the original dataset Where (x i ,y i ) represents the original data sample, y i ∈L={l1,l2,…,l C}, L is the set of categories, C represents the number of categories, Represents the number of samples of the original data, and compresses IPC instances for each category l c , get the synthetic dataset Among them (s j ,y j ) represents a synthetic data sample, The number of samples representing the synthetic data; the synthetic dataset is trained using randomly selected instances from the original dataset. Initialize; use feature extractor The embedding function g extracts features for each instance in the synthetic dataset and the original dataset, and the extracted features are used and Representation, and then use the extracted features to calculate the class center μ S and μ T ,as well as and The covariance matrix Cov for each category in S and Cov T .

[0016] Furthermore, the cluster center alignment is to use the Euclidean distance to The class center μ T,c and The class center μ S,c Alignment, the class center alignment loss function used is:

[0017]

[0018] in, Indicated in and From the same category c A batch of data extracted from .

[0019] Furthermore, the covariance matrix alignment is used to capture semantic diversity, and the covariance matrix alignment loss function adopted is:

[0020]

[0021] Among them, (m,n) represents the position of the covariance matrix, Cov T,c (m,n) represents each element of the covariance matrix of each batch of raw data, Cov S,c (m,n) represents each element of the covariance matrix of each batch of synthetic data.

[0022] Furthermore, the class center of the synthetic data set optimized by the current pre-training model is aligned with the historical class center, and the historical class center alignment loss function used is:

[0023]

[0024] in Represents a historical center.

[0025] Furthermore, the following steps are used to update the historical class center in the memory library:

[0026] First, use the feature extractor For synthetic data Calculate the cluster center after the tth optimization

[0027]

[0028] in Represents the category l after t optimizations c The synthetic data represents the optimized synthetic data instance;

[0029] Then update the history class center in the memory:

[0030]

[0031] where α is the smoothing factor.

[0032] Furthermore, the class center alignment loss function Covariance matrix alignment loss function And the historical class center alignment loss function Combined together, we get the total loss function, which is the diversified semantic distribution matching loss function.

[0033] A diverse semantic distribution matching data compression system, comprising:

[0034] A pre-training module is used to pre-train a set of models on the original data set to obtain a pre-trained model;

[0035] A feature extraction module is used to model each category in the original dataset as a Gaussian distribution using the semantic embedding extracted by the pre-trained model, where the class center represents the inherent characteristics of the category and the covariance matrix represents the semantic changes of instances in the category;

[0036] The class center and covariance matrix alignment module is used to align the class centers and covariance matrices of the synthetic dataset with the original dataset;

[0037] The historical class center alignment module is used to optimize the synthetic dataset using multiple pre-trained models. The class center of the synthetic dataset optimized by the previous pre-trained model, namely the historical class center, is stored in the memory bank, and the class center of the synthetic dataset optimized by the current pre-trained model is aligned with the historical class center to obtain a semantically diverse synthetic dataset.

[0038] The beneficial effects of the present invention are as follows: the present invention proposes a diversified semantic distribution matching data compression method, which significantly enhances the semantic diversity of synthetic data by aligning features from the perspective of semantic distribution. In order to ensure the stability of the process of optimizing synthetic data, the present invention deeply analyzes the differences in feature space distribution among different models, and aligns the class center of the current synthetic data with the historically optimized synthetic data. This alignment strategy not only improves the expressive power of the features, but also effectively reduces the overfitting of specific models, thereby improving the overall performance. The present invention conducted a series of detailed experiments to verify the effectiveness of the present invention on image and speech datasets. The results show that by implementing this method, the model exhibits state-of-the-art performance and excellent generalization capabilities in a variety of scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a diagram of the diversified semantic distribution matching framework.

[0040] Figure 2 Feature distribution and matching loss on the same synthetic data under six pre-trained models.

[0041] Figure 3 The performance on synthetic data with different numbers of pre-training rounds (left) and different numbers of pre-trained models (right). Evaluation is performed on CIFAR-10 with IPC=50.

[0042] Figure 4 This is the impact analysis of λ1 and λ2 on CIFAR-10 at IPC=50.

[0043] Figure 5 It is the data distribution of original images and synthetic images of the same category using DM and DSDM on CIFAR-10, where IPC = 10 / 50. The "green dots" represent real data, while the "red stars" represent synthetic data.

[0044] Figure 6 The synthetic image visualization results of DM and DSDM generated with IPC=10 on CIFAR-10, where: (a) synthetic image of DM; (b) synthetic image of DSDM. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below through specific embodiments and drawings.

[0046] The present invention proposes a diversified semantic distribution matching data compression method, which aims to obtain semantically diverse synthetic data. In order to obtain accurate semantic information from the original data, a set of models are first pre-trained on the original data set. Then, each category in the original data set is modeled as a Gaussian distribution using the semantic embedding extracted by the pre-trained model, where the class center represents the inherent characteristics of the category and the covariance matrix represents the semantic changes of the instances in the category. Subsequently, the class center and covariance matrix of the corresponding synthetic data are aligned to make the distribution of the synthetic data similar to that of the original data. By additionally aligning the covariance matrix, the synthetic data will be dispersed in the space of the original data, thereby ensuring the performance of the synthetic data in the evaluation task. In addition, since the optimized synthetic data is optimized by multiple pre-trained models to generalize to different models, the training process will experience large fluctuations. Therefore, a memory bank is introduced to store the historical class centers of the synthetic data optimized by the previous pre-trained model. Finally, the synthetic data of the current pre-trained model is aligned with the historical class centers to ensure that the synthetic data can generalize well to various models.

[0047] The data compression method of the diversified semantic distribution matching (DSDM) proposed in the present invention is as follows: Figure 1 As shown. In order to obtain accurate instance features, multiple models are trained using the original dataset. For the tth iteration, a feature subnetwork is initialized As a feature extractor, we instantiate a model by randomly selecting one of these pre-trained models. Instantiation means assigning weight values ​​to the feature extractor. Then, we extract the weights from the original dataset. and synthetic datasets Derive instance features. The extracted features are then used to calculate the class center μ S and μ T ,as well as and The covariance matrix Cov for each category in S and Cov T The class center contains the inherent features, while the covariance matrix captures the semantic diversity of all features in each class. The class center and covariance matrix of each class are then matched. However, The feature distribution of will show significant differences during the iteration process. In order to alleviate the instability caused by these differences, the historical class center h of the synthetic data is (t-1) The class center μ of the current synthetic data S Alignment. In order to preserve the historical overall class centers of each pre-trained model, a memory bank is built to store the historical class centers. Finally, the total loss is summed up to optimize the synthetic data.

[0048] (I) Basic framework of data compression based on distribution matching:

[0049] Data compression is the process of converting large training datasets into Compressed into a small synthetic dataset (x i ,y i ) represents the original data sample, Represents the number of samples of the original data, (s j ,y j ) represents a synthetic data sample, represents the number of samples of synthetic data, where The model trained on Typically, we use Or initialize the synthetic dataset with randomly selected original instances from random noise Then, by minimizing the information loss between the synthesized sample and the original sample Optimize as shown in formula (1):

[0050]

[0051] Where D represents a distance metric, such as mean square error (MSE), The representation method of the matched information of the original data, Indicates the representation of the information that the synthetic data is matched.

[0052] The distribution matching-based method is to generate synthetic data with training performance close to the original data by directly matching the class center of the synthetic data set with the class center of the original data set, as shown in formula (2):

[0053]

[0054] Where θ F The feature subnetwork parameterized by θ is instantiated by the model without output layer, and g is θ F Parameterized embedding function, θ represents the initialized model, P θ represents the weight probability of the model, Represents the expected probability of the model's weights.

[0055] (II) Diversified semantic distribution matching:

[0056] In deep neural networks, different instances belonging to the same category often undergo subtle semantic transformations in their deeper feature representations. Given that covariance matrices imply interrelationships and variances between features, they provide a rich understanding of semantic features. The present invention considers that features with unique semantic transformations can be represented using covariance matrices of specific categories. Therefore, the present invention derives semantic directions from the estimated covariance matrix, assuming a class-centered normal distribution. The goal of the present invention is to simplify matching each category The covariance matrix of the generated features is close to The covariance matrix (i.e. ).

[0057] Prior distribution matching uses an initialized neural network to measure the original dataset and synthetic datasets The maximum mean difference (MMD) between them. However, aligning the embeddings with a randomly initialized neural network will lead to inaccurate embeddings because the neural network knows nothing about the current task. Only accurate embeddings can capture effective semantic features. To alleviate this problem, the present invention adopts the use of The strategy of pre-training multiple models. In each iteration t, one of these pre-trained models is randomly selected and its last layer (excluding the output layer) is used as the feature extractor

[0058] For large-scale original datasets where y i ∈L={l1,l2,…,l C}, L is the set of categories, C is the number of categories, and IPC (Instances Per Class) instances are compressed for each category. c , that is, for the synthetic dataset It is usually initialized with randomly selected instances from the original dataset. In order to effectively capture the original dataset The semantic features of To obtain instance embedding. Use the embedding function g to generate embeddings for each instance in the synthetic dataset and the original dataset, and the generated features are used and Indicates that

[0059] Given the inherent differences in the directions of semantic features between classes, we should first and Therefore, the Euclidean distance is used to align the inherent semantic features of each class. The class center (denoted as μ T，c )and The class center (denoted as μ S，c ) alignment. The loss function for this type of center alignment is defined as follows:

[0060]

[0061] in Indicated in and From the same category c A batch of data extracted from .

[0062] When aligning class centers, we align the covariance matrices to capture semantic diversity. The goal of semantic diversity alignment is It can be expressed as:

[0063]

[0064]

[0065] Among them, (m,n) represents the position of the covariance matrix, Cov T,c (m,n) represents each element of the covariance matrix of each batch of raw data, Cov S,c (m,n) represents each element of the covariance matrix of each batch of synthetic data.

[0066] (III) Alignment with historical centers:

[0067] In order to improve the model generalization ability of synthetic data in different evaluation tasks, the present invention uses multiple pre-trained models to capture various semantic features. However, there are significant differences in the feature distribution of the same type of synthetic data in different pre-trained models. Figure 2 In (a), the feature distribution of category data extracted from six pre-trained models is visualized. It is obvious that the data extracted by different pre-trained models have different distributions. Figure 2 (b) shows the matching loss between synthetic data and original data under these six pre-training models. It can be seen that the distance between the same synthetic data and the same original data in different pre-training models varies greatly, resulting in severe fluctuations in the training loss of data compression when the feature extraction model changes.

[0068] In order to improve the stability of the performance of synthetic data on multiple pre-trained models, the present invention introduces a memory bank to store the historical class centers of synthetic data optimized by previous pre-trained models. Then, the synthetic data of the current model is compared with the corresponding historical class centers. Alignment. Therefore, the historical class center alignment loss is calculated as:

[0069]

[0070] To store and update the class centers for historical class center alignment, we first use the corresponding feature extractor For synthetic data Calculate the cluster center after the tth optimization

[0071]

[0072] in Represents the category l after t optimizations c The synthetic data Represents an optimized synthetic data instance.

[0073] Next, update the history class center in the memory library:

[0074]

[0075] where α is a smoothing factor, usually set to 0.99.

[0076] (IV) Overall loss and training algorithm:

[0077] In short, minimize the sum of the above losses, including the class center alignment loss Semantic Diversity Alignment Loss and historical class center alignment loss Combining these loss terms, the diversified semantic distribution matching loss of the present invention can be expressed as:

[0078]

[0079] The pseudo code of the DSDM method of the present invention is shown in Algorithm 1. In addition, the partitioning and expansion of the initialization are used to increase the The number of representations extracted from is increased without incurring additional storage cost. Specifically, using a factor parameter l, each synthesized instance is decomposed into l×l mini-examples, which are then upsampled to their original size during training. This approach effectively maximizes The same enhancement technique is integrated into the work of

[17] to further exploit its advantages in adjusting the distribution to enhance diversity.

[0080]

[0081] (V) Key points of the present invention:

[0082] 1. In this paper, a new dataset compression method is proposed, called Diversified Semantic Distribution Matching (DSDM), which explicitly aims to capture the semantic diversity characteristics of the original dataset. DSDM contains three carefully designed losses: class center alignment, semantic diversity alignment, and historical class center alignment. The class center alignment and semantic diversity alignment modules are used to capture the distribution of the original dataset. Historical class center alignment makes DSDM perform more stably during training.

[0083] 2. Experimental results on five image datasets and one speech dataset show that the DSDM of the present invention is superior to the existing technology and has good generalization on different model architectures.

[0084] (VI) Effects of the present invention:

[0085] 1) Dataset: In order to evaluate the effectiveness of the dataset compression method of the present invention, experiments were conducted on five image datasets and one speech dataset. The five image datasets are: MNIST, Fashion-MNIST (FMNIST), SVHN, CIFAR-10 and CIFAR-100. Specifically, MNIST is a black and white dataset consisting of 60,000 training images and 10,000 test images from 10 different categories. SVHN is a color dataset consisting of 73,257 bits and 26,032 bits for training and testing, respectively. CIFAR-10 / 100 consists of 50,000 training images and 10,000 test images from 10 and 100 different classes, respectively. For the speech dataset, the Mini-Speech-Commands dataset is used, which consists of 8,000 one-second audio clips representing 8 command categories.

[0086] 2) Experimental setup: A 3-layer convolutional network (ConvNet-3) and a 4-layer convolutional network (ConvNet-4) were used for the image and speech datasets, respectively. M = 10 pre-trained models were initialized, and each model was trained for E = 20 rounds. T = 10K matching optimization iterations were performed. For the weight parameters, λ1 = 50 and λ2 = 0.2 were set. In addition, the optimized learning rate η was set for the synthetic examples. DSDM = 0.1, and the factor parameter l was fixed to 2 in all data sets.

[0087] 3) Evaluation Metrics: The Top-1 test accuracy of the original dataset test set is used as the evaluation metric to evaluate the performance of the network trained on the synthetic dataset. The evaluation task was trained 1500 times. Five experiments were conducted and the mean and standard deviation of the results were reported.

[0088] 4) Results on image datasets: The performance of the proposed DSDM and competitors on five image datasets are shown in Tables 1-5. The following conclusions can be drawn from these results: First, DSDM consistently outperforms the current state-of-the-art, especially when the IPC in the synthetic datasets is set to 10 and 50. In particular, the proposed method outperforms the second-best method M3D by 3.0% and 5.9% on CIFAR-10, respectively. Second, training with the original dataset (Whole) represents the upper limit of dataset compression performance. It is observed that on SVHN, the difference from the upper limit is only 4.1% when the compression ratio is 0.7%, while on CIFAR-100, the difference from the upper limit decreases slightly by 2.2% when the compression ratio is 10%. These results suggest that the proposed method has the potential to achieve lossless compression by maintaining semantic diversity. Third, when IPC=1, the improvement of DSDM is limited. This is because the covariance matrix of only one synthetic instance cannot be estimated. However, the proposed DSDM still outperforms most other methods because DSDM aligns features from multiple models to stabilize training.

[0089] 5) Results of speech dataset: Each speech data is preprocessed into a magnitude spectrogram of size 64×64. For factor parameter l, each spectrogram is decomposed into 2 mini samples along the time axis. Table 2 shows the test accuracy of the mini speech commands. As shown in Table 6, it can be seen that the DSDM of the present invention also greatly outperforms the baseline method, confirming its effectiveness in the speech field. This means that the method of the present invention is useful in both image and speech fields.

[0090] 6) Cross-architecture evaluation: For compressed synthetic data, a more important question is whether they can generalize well to different networks in the evaluation task. In Table 7, the performance of the synthetic dataset of the present invention (optimized on ConvNet-3) on the evaluation tasks trained with ConvNet-3, ResNet-10 and DenseNet-121 is shown. It is worth noting that the DSDM of the present invention not only outperforms other methods on a specific optimized model (ConvNet-3), but also shows significant performance improvements on other unseen network architectures (ResNet-10 and DenseNet-121). This robust cross-architecture generalization shows that the synthetic dataset of the present invention has enhanced semantic diversity and integrated information from different pre-models, and can generalize well to different backbones of the evaluation task.

[0091] 7) Experimental analysis:

[0092] (1) Ablation Experiment: To evaluate the effectiveness of each component in DSDM, an ablation experiment was performed on CIFAR-10, including the class center alignment loss. Pre-trained model (PM), semantic diversity alignment loss and historical class center alignment loss The results are shown in Table 8, and the following conclusions can be drawn: (1) When IPC = 10 and IPC = 50, the class center alignment using the pre-trained model is enhanced by 2.9% and 1.4%, respectively. This shows that accurate instance embedding can be obtained using the pre-trained model, thereby generating accurate intrinsic semantic features for each class. (2) The semantic diversity alignment method proposed in the present invention is improved by 2.4% and 4.9%, respectively. This emphasizes the effectiveness of the covariance matrix of the present invention in capturing semantic diversity. It is worth noting that as the IPC increases (i.e., IPC = 50), the obtained synthetic instances are more diverse, resulting in greater improvement. (3) The historical class center alignment emphasizes the ability to maintain the semantic diversity of feature distribution between different models, which can further improve the performance. In summary, the three designed loss functions are complementary and jointly improve the overall performance.

[0093] (2) The number of pre-training epochs E directly affects the accuracy of instance embeddings extracted by the pre-trained model (further affecting the class center and semantic diversity of synthetic data). To show their effect, ConvNets on CIFAR-10 are pre-trained with E in {1, 5, 10, 15, 20, 25, 30}, and the DSDM of the present invention is performed using these pre-trained models at IPC = 50. Figure 3 As shown on the left, as E increases, the performance gradually improves until the best performance is achieved at E = 20. After that, the effect of increasing E on the performance is not significant, which indicates that the DSDM of the present invention only needs to increase a few pre-training E to achieve better performance.

[0094] (3) Impact of the number of pre-trained models: More pre-trained models mean that the synthetic data can be better generalized to the evaluation task. However, more pre-trained models also take up more training time. Therefore, DSDM is performed with different numbers of models M∈{1,5,10,20} and pre-training epochs E=20. Figure 3 On the right, the performance on synthetic data is shown. When M is set to 1 and 5, the performance is slightly worse. This shows that the number of models has a significant impact on the performance on synthetic data. Nevertheless, the DSDM of the present invention achieves the best performance using only 10 pre-trained models, and consumes little storage and computation.

[0095] (4) Effect of λ1: Parameter λ1 is a weighted parameter for the degree of improvement in semantic diversity. Figure 4The effect of λ1 on the performance of CIFAR-10 synthetic data when IPC=50 is shown. It can be found that when λ1 ranges from 10 to 50, the performance will be improved, while the effect of increasing the range of λ1 from 50 to 100 is almost negligible. So λ1 is set to 50. In addition, although λ1=10, it is observed that the DSDM of the present invention improves by 2.3% over the suboptimal M3D result, which further illustrates the importance of semantic diversity matching in data compression.

[0096] (5) Effect of λ2: Similarly, Figure 4 The influence of the weighting parameter λ2 on the performance of the historical class center is observed. When λ2=0.2, the DSDM performance of the present invention reaches the optimal value. As λ2 increases, the performance decreases, which may be due to the fact that the class center of the historical model is very different from the current model, causing the current model to deviate from the class center of the original data.

[0097] 8) Visualization Analysis:

[0098] (1) Distribution visualization: In order to qualitatively illustrate the effectiveness of the DSDM of the present invention in modeling the distribution of the original dataset, t-SNE is used to visualize the synthetic data of the same category with IPC=10 / 50 obtained by DM and DSDM on CIFAR-10. Figure 5 As can be seen from the figure, the synthetic data obtained by DSDM is more dispersed than the data obtained by DM.

[0099] (2) Synthetic images: In order to evaluate whether the proposed method captures the semantic diversity of the original dataset, Figure 6 The synthetic data of DM and DSDM on CIFAR-10 are visualized in Figure 2. Each class contains 10 images. Two key conclusions are drawn from the observations: (1) Compared with DM, the synthetic data of DSDM of the present invention has greater differences, proving that the covariance matrix alignment of the present invention captures the semantic diversity of the original dataset. (2) The synthetic data of DSDM of the present invention has stronger discrimination between each class, indicating that the class center of the pre-trained model effectively preserves the inherent characteristics of the class.

[0100] Table 1 Top-1 accuracy of the test model trained with synthetic images on the MINST dataset

[0101]

[0102] Table 2 Top-1 accuracy of the test model trained with synthetic images on the FashionMNIST dataset

[0103]

[0104] Table 3 Top-1 accuracy of the test model trained with synthetic images on the SVHN dataset

[0105]

[0106] Table 4 Top-1 accuracy of the test model trained with synthetic images on the CIFAR-10 dataset

[0107]

[0108] Table 5 Top-1 accuracy of the test model trained with synthetic images on the CIFAR-100 dataset

[0109]

[0110]

[0111] Table 6 Top-1 accuracy of speech datasets for ConvNet-4 trained on synthetic spectrograms

[0112]

[0113] Table 7. Cross-architecture generalization performance (%) on CIFAR-10. Synthetic images are compressed by ConvNet-3 and evaluated using other architectures.

[0114]

[0115] Table 8 Ablation experiment analysis of various components with IPC = 10 / 50 on CIFAR10

[0116]

[0117] Another embodiment of the present invention provides a diverse semantic distribution matching data compression system, which includes:

[0118] A pre-training module is used to pre-train a set of models on the original data set to obtain a pre-trained model;

[0119] A feature extraction module is used to model each category in the original dataset as a Gaussian distribution using the semantic embedding extracted by the pre-trained model, where the class center represents the inherent characteristics of the category and the covariance matrix represents the semantic changes of instances in the category;

[0120] The class center and covariance matrix alignment module is used to align the class centers and covariance matrices of the synthetic dataset with the original dataset;

[0121] The historical class center alignment module is used to optimize the synthetic dataset using multiple pre-trained models. The class center of the synthetic dataset optimized by the previous pre-trained model, namely the historical class center, is stored in the memory bank, and the class center of the synthetic dataset optimized by the current pre-trained model is aligned with the historical class center to obtain a semantically diverse synthetic dataset.

[0122] The specific implementation process of each module refers to the above description of the method of the present invention.

[0123] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0124] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0125] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A diverse semantic distribution matching data compression method, characterized in that: The following steps are involved: Pre-train a set of models on the original data set to obtain a pre-trained model; Each category in the original dataset is modeled as a Gaussian distribution using the semantic embedding extracted by the pre-trained model, where the class center represents the inherent characteristics of the category and the covariance matrix represents the semantic changes of instances in the category; Align the class centers and covariance matrices of the synthetic dataset with the original dataset; Optimize the synthetic dataset using multiple pre-trained models, and store the class center of the synthetic dataset optimized by the previous pre-trained model, i.e., the historical class center, through the memory bank; The class centers of the synthetic dataset optimized by the current pre-trained model are aligned with the historical class centers to obtain a semantically diverse synthetic dataset.

2. The method according to claim 1, characterized in that For the original dataset Where (x i ,y i ) represents the original data sample, y i ∈L={l1,l2,…,l C }, L is the set of categories, C represents the number of categories, Represents the number of samples of the original data, and compresses IPC instances for each category l c , get the synthetic dataset Among them (s j ,y j ) represents a synthetic data sample, The number of samples representing the synthetic data; the synthetic dataset is trained using randomly selected instances from the original dataset. Initialize; use feature extractor The embedding function g extracts features for each instance in the synthetic dataset and the original dataset, and the extracted features are used and Representation, and then use the extracted features to calculate the class center μ S and μ T ,as well as and The covariance matrix Cov for each category in S and Cov T .

3. The method according to claim 2, characterized in that The cluster center alignment is achieved by using the Euclidean distance The class center μ T,c and The class center μ S,c Alignment, the class center alignment loss function used is: in, Indicated in and From the same category c A batch of data extracted from .

4. The method according to claim 3, characterized in that The covariance matrix alignment is used to capture semantic diversity, and the covariance matrix alignment loss function used is: Among them, (m,n) represents the position of the covariance matrix, Cov T,c (m,n) represents each element of the covariance matrix of each batch of raw data, Cov S,c (m,n) represents each element of the covariance matrix of each batch of synthetic data.

5. The method according to claim 4, characterized in that The class center of the synthetic data set optimized by the current pre-training model is aligned with the historical class center, and the historical class center alignment loss function used is: in Represents a historical center.

6. The method according to claim 5, characterized in that To update the history class center in the memory, follow these steps: First, use the feature extractor For synthetic data Calculate the cluster center after the tth optimization in Represents the category l after t optimizations c The synthetic data represents the optimized synthetic data instance; Then update the history class center in the memory: where α is the smoothing factor.

7. The method according to claim 1, characterized in that Aligning class centers to loss function Covariance matrix alignment loss function And the historical class center alignment loss function Combined together, we get the total loss function, which is the diversified semantic distribution matching loss function.

8. A diverse semantic distribution matching data compression system, characterized in that: include: A pre-training module is used to pre-train a set of models on the original data set to obtain a pre-trained model; A feature extraction module is used to model each category in the original dataset as a Gaussian distribution using the semantic embedding extracted by the pre-trained model, where the class center represents the inherent characteristics of the category and the covariance matrix represents the semantic changes of instances in the category; The class center and covariance matrix alignment module is used to align the class centers and covariance matrices of the synthetic dataset with the original dataset; The historical class center alignment module is used to optimize the synthetic dataset using multiple pre-trained models. The class center of the synthetic dataset optimized by the previous pre-trained model, namely the historical class center, is stored in the memory bank, and the class center of the synthetic dataset optimized by the current pre-trained model is aligned with the historical class center to obtain a semantically diverse synthetic dataset.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images