Data set distillation method based on boundary perception diffusion model

Abstract

The disclosure provides a data set distillation method based on a boundary perception diffusion model, comprising: pre-training an initial expert model using an original training set, fitting a decision boundary of the initial expert model to a true boundary by adjusting a weight parameter; generating a multi-condition fine-tuning loss reflecting different categories using a class probability predicted by the trained expert model as a confidence weight, and guiding fine-tuning of the diffusion model through minimization of a weighted sum of the multi-condition fine-tuning loss; performing multi-condition sampling using the fine-tuned diffusion model, and guiding the image generation of the diffusion model to approach the decision boundary region between the two categories by introducing labels to weight and average the noise prediction results of the target class and adjacent competing classes according to a mixing coefficient; and dynamically mixing boundary discrimination samples generated through multi-condition sampling and intra-class representative samples generated through single-condition sampling according to a preset ratio to construct a lightweight synthetic data set as a final result of data set distillation.

Description

Technical Field

[0001] This disclosure relates to the fields of machine learning and data processing, and more specifically, to a dataset distillation method based on a boundary-aware diffusion model. Background Technology

[0002] In the field of modern machine learning, the success of deep models largely depends on the driving force of large-scale datasets. However, with the dramatic increase in the number of model parameters and the improvement in image resolution, massive amounts of data pose serious challenges to data storage, transmission, and model training efficiency. To alleviate this problem, the dataset distillation technique has emerged. Its core goal is to construct a concise synthetic dataset, enabling models trained on this dataset to achieve performance similar to those trained on the original large-scale data.

[0003] Recently, methods have attempted to distill datasets using generative techniques such as diffusion models. However, existing diffusion distillation methods primarily focus on generating samples with intra-class representativeness, i.e., capturing the core semantic features of each class. While these methods have made progress in generating high-quality images, they often neglect the explicit preservation of decision boundaries, which are crucial for the model's ability to accurately distinguish between different classes. Diffusion models are typically constrained by a single class label condition during the generation process, leading to a significant deviation between the implicit decision boundaries induced by the synthetic dataset and the boundaries of the real dataset, thus limiting the classification performance of the distilled dataset in downstream tasks. Although some improved methods introduce auxiliary classifiers to guide the generation process, they still struggle to address the boundary ambiguity caused by the high semantic similarity between classes. Summary of the Invention

[0004] This disclosure aims to address the problem of decision boundary information loss in existing dataset distillation methods. By introducing a boundary-aware framework, highly informative samples located near the decision boundary are explicitly synthesized, thereby aligning the decision boundary between the synthetic dataset and the real dataset. To address the aforementioned technical problem, this disclosure provides a dataset distillation algorithm based on a boundary-aware diffusion model with multi-condition fine-tuning and sampling.

[0005] In general, a dataset distillation method based on a boundary-aware diffusion model is provided. The method includes: pre-training an initial expert model using the original training set; adjusting the weight parameters of the initial expert model to make its decision boundary fit the true boundary; using the class probabilities predicted by the trained expert model as confidence weights to generate a multi-conditional fine-tuning loss reflecting different classes; guiding the fine-tuning of the diffusion model by minimizing the weighted sum of the multi-conditional fine-tuning losses, thereby distilling the discriminative knowledge of the trained expert model into the data generation distribution of the diffusion model; performing multi-conditional sampling using the fine-tuned diffusion model, introducing labels for the target class and adjacent competing classes, and weighting the noise prediction results of the target class and adjacent competing classes according to a mixing coefficient to guide the image generation of the diffusion model towards the decision boundary region between the two classes; dynamically mixing the boundary discrimination samples generated by multi-conditional sampling with the intra-class representative samples generated by single-conditional sampling according to a preset ratio to construct a lightweight synthetic dataset as the final result of the dataset distillation.

[0006] According to an embodiment, adjusting the weight parameters of the initial expert model includes: using stochastic gradient descent to update the model weight parameters during the training process of the initial expert model, wherein stochastic gradient descent calculates the gradient of the loss function with respect to the weight parameters, updates the weight parameters in the opposite direction of the gradient, gradually reduces the value of the loss function, and iteratively optimizes it on the original training set until the value of the loss function converges or reaches a preset number of training rounds to obtain the optimal weight parameters.

[0007] According to the embodiment, the loss function uses the cross-entropy loss function to measure the difference between the expert model's prediction results and the true labels. By minimizing this loss function, the probability distribution output by the model approximates the true label distribution.

[0008] According to an embodiment, the process of guiding the fine-tuning of the diffusion model using the confidence weights output by the trained expert model includes: obtaining the multi-condition prediction probability distribution of the original training set through the trained expert model while keeping the weight parameters in the trained expert model unchanged, using the prediction probability distribution as the confidence weights, and performing a weighted summation of the single-condition denoising scores of the diffusion model based on the confidence weights to construct a weighted sum of the multi-condition fine-tuning loss.

[0009] According to an embodiment, the distance between the generated sample and the decision boundary is controlled by adjusting the mixing coefficient to control the mixing ratio of the target class and the adjacent competing class.

[0010] According to an embodiment, the boundary discrimination samples generated by the multi-condition sampling include samples located at the decision boundary between the target class and adjacent competing classes.

[0011] According to the embodiment, the preset ratio for dynamically mixing the boundary discrimination samples generated by multi-condition sampling with the intra-class representative samples generated by single-condition sampling is 1:1.

[0012] In another general aspect, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the dataset distillation method based on a boundary-aware diffusion model as described above.

[0013] In another general aspect, a computer device is provided, the computer device comprising: a processor; and a memory storing a computer program that, when executed by the processor, implements the dataset distillation method based on a boundary-aware diffusion model as described above.

[0014] According to the dataset distillation method based on the boundary-aware diffusion model disclosed herein, through multi-condition diffusion fine-tuning and multi-condition sampling, the discriminative boundary features of the real dataset are effectively distilled into synthetic samples from both the generation distribution modeling and sampling processes, achieving accurate alignment between the decision boundaries of the synthetic dataset and the real dataset. Furthermore, the dataset distillation method based on the boundary-aware diffusion model disclosed herein effectively balances the intra-class representative semantics and boundary discriminative information of the synthetic samples, significantly improving the generation quality of lightweight synthetic datasets, the classification accuracy of downstream models, and the generalization ability across different network architectures. Attached Figure Description

[0015] The above and other aspects, features and advantages of this disclosure will become more clearly understood from the following detailed embodiments, taken in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of a dataset distillation method based on a boundary-aware diffusion model according to an embodiment.

[0016] Figure 2 This is a block diagram of a dataset distillation method based on a boundary-aware diffusion model according to an embodiment. Detailed Implementation

[0017] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, after understanding the disclosure of this application, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will be readily apparent. For example, the order of operations described herein is merely illustrative and is not limited to the order set forth herein; rather, changes that will be readily understood after understanding the disclosure of this application are possible, except for operations that must occur in a specific order. Furthermore, for clarity and brevity, descriptions of features known after understanding the disclosure of this application may be omitted.

[0018] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus and / or systems described herein, many of which will become clear upon understanding the disclosure of this application.

[0019] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0021] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0022] This disclosure proposes a dataset distillation method based on a boundary-aware diffusion model, utilizing diffusion generation technology. It combines multi-condition diffusion fine-tuning and multi-condition sampling strategies to effectively obtain the relative generation probability density between different categories through multi-condition learning, and learns a highly discriminative visual representation of the boundary to accurately capture the decision structure of real datasets. First, the diffusion model is fine-tuned based on the classification confidence of the expert model, distilling discriminative knowledge into the generation distribution. Then, based on the fine-tuned generation distribution, multi-condition linear combination guides the model to synthesize highly informative boundary samples in the category boundary region. By co-modeling representative samples and boundary samples, the association discriminative information between different categories is effectively explored, thereby achieving alignment of the decision boundary between the synthetic dataset and the real dataset. This enables the model to effectively handle problems such as high category similarity and ambiguous boundary localization in complex scenarios.

[0023] Dataset distillation is a technique that uses algorithms to extract key information from a large-scale original training set to generate a small but highly representative synthetic dataset. Its core goal is to train a model using a very small amount of "distilled data" instead of the complete dataset. Unlike traditional knowledge distillation, which focuses on model compression, dataset distillation focuses on the compression and optimization of the data itself, and is considered a form of "distillation at the data level."

[0024] Dataset distillation typically involves using a pre-trained "teacher model" (trained on the complete dataset) to generate synthetic data through a backpropagation process. This synthetic data enables the "student model" to learn quickly and approximate the teacher's performance. The teacher model generates high-quality "soft labels" (such as probability distributions or thought chains) by reasoning about the data. The student model uses these labels as supervision during training, learning the decision-making logic rather than simply replicating the results. This approach allows the smaller model to capture deep relationships between data points, improving generalization ability.

[0025] Dataset distillation using diffusion models transfers information from high-quality datasets to smaller, more efficient datasets, aiming to preserve the core features and distribution of the original data while reducing storage and computational overhead. This method combines the advantages of diffusion models in generating high-quality samples by learning the latent representations of the original data to filter or reconstruct representative subsets.

[0026] When performing dataset distillation using a diffusion model, the diffusion model generates data samples from noise through a stepwise denoising process, learning the complex distribution of the data during its training. The distillation process includes: using a teacher model to encode the original training set using the diffusion model, extracting the latent feature vector or representation of each sample; selecting key samples based on these representations using methods such as clustering, diversity measures, or information gain; and retraining a lightweight model using the selected samples with a student model to achieve efficient inference.

[0027] In knowledge distillation, the decision region refers to the area that the model divides in the feature space for each category. A sample is predicted to belong to the corresponding category based on which region it falls into. These regions are separated by the model's decision boundaries, collectively forming the classifier's overall judgment logic. The decision boundary is the dividing line between different decision regions. Explicitly preserving the decision boundary is to maintain the teacher model's method of dividing the decision region, ensuring that the student model's classification "landscape" in the feature space is as consistent as possible with the teacher's.

[0028] In this disclosure, "Expert Models" refers to deep learning models that excel in specific tasks or domains. For example, the ResNet family of networks is a type of deep residual network widely used in image recognition tasks. ResNet addresses the vanishing gradient problem during deep network training by introducing skip connections, allowing the network to be trained deeper and thus improving model performance. The core idea of ​​the ResNet family of networks is to alleviate the vanishing and exploding gradient problems during deep network training through residual blocks, allowing information to be directly passed from one layer to subsequent layers, thereby enabling the network to learn more complex feature representations.

[0029] To enable those skilled in the art to better understand this disclosure, specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart of a dataset distillation method based on a boundary-aware diffusion model according to an embodiment.

[0031] Reference Figure 1 The dataset distillation method based on the boundary-aware diffusion model according to the embodiments may include: In step S101, the initial expert model is pre-trained using the original training set, and the decision boundary of the initial expert model is made to fit the true boundary by adjusting the weight parameters of the initial expert model.

[0032] According to the embodiment, adjusting the weight parameters of the initial expert model includes: using stochastic gradient descent to update the model weight parameters during the training process of the initial expert model. Stochastic gradient descent calculates the gradient of the loss function relative to the weight parameters and updates the weight parameters in the opposite direction of the gradient, gradually decreasing the value of the loss function and iteratively optimizing on the original training set until the value of the loss function converges or a preset number of training epochs is reached to obtain the optimal weight parameters. Adjusting the weight parameters of the expert model allows the decision regions of the real dataset to be explicitly preserved.

[0033] According to the embodiment, the loss function uses the cross-entropy loss function to measure the difference between the expert model's prediction results and the true labels. By minimizing this loss function, the probability distribution of the model output approximates the true label distribution.

[0034] In step S102, the class probabilities predicted by the trained expert model are used as confidence weights to generate multi-condition fine-tuning losses that reflect different classes. The fine-tuning of the diffusion model is guided by minimizing the weighted sum of the multi-condition fine-tuning losses, thereby distilling the discriminative knowledge of the trained expert model into the data generation distribution of the diffusion model.

[0035] According to the embodiment, guiding the generation process of the diffusion model using the confidence weights output by the trained expert model includes: obtaining the multi-condition prediction probability distribution of the original training set through the trained expert model while keeping the weight parameters in the trained expert model unchanged, using the prediction probability distribution as the confidence weights, and performing a weighted summation of the single-condition denoising scores of the diffusion model based on the confidence weights to construct a weighted sum of the multi-condition fine-tuning loss.

[0036] The generation likelihood of a diffusion model satisfies the relative discriminative constraints between categories. In other words, during the generation process, by designing specific loss functions or constraint terms, the generated samples maintain the relative distance relationship between categories in the feature space. This is typically achieved by adding a constraint term representing the relative relationship between categories to the loss function, or by incorporating discriminative knowledge provided by expert models through knowledge distillation to integrate the relative discriminative relationships between categories into the generation process. For example, for categories A and B, the constraint mechanism ensures that samples of category A maintain an appropriate distance from samples of category B in the feature space. By adjusting the noise prediction process of the diffusion model, the likelihood probability distribution of the generated samples can reflect the relative importance between categories.

[0037] In step S103, multi-condition sampling is performed using a fine-tuned diffusion model. By introducing labels for the target class and adjacent competing classes, and weighting the noise prediction results of the target class and adjacent competing classes according to the mixing coefficient, the image generation of the diffusion model is guided to move closer to the decision boundary region where the two classes meet.

[0038] According to an embodiment, the distance between the generated samples and the decision boundary is controlled by adjusting the mixing coefficient to control the mixing ratio of the target class and adjacent competing classes. This adjustment enables boundary alignment during the synthesis process.

[0039] According to an embodiment, the boundary discrimination samples generated by multi-condition sampling include samples located at the decision boundary between the target class and adjacent competing classes.

[0040] In step S104, the boundary discrimination samples generated by multi-condition sampling and the intra-class representative samples generated by single-condition sampling are dynamically mixed according to a preset ratio to construct a lightweight synthetic dataset as the final result of dataset distillation.

[0041] According to the embodiment, the preset ratio for dynamically mixing the boundary discrimination samples generated by multi-condition sampling with the intra-class representative samples generated by single-condition sampling is 1:1.

[0042] The dataset distillation method based on the boundary-aware diffusion model according to the embodiment can maximize the improvement of model performance with the smallest data size. The final synthetic dataset improves the ability of the synthetic dataset to reconstruct the decision boundary of the real dataset by integrating the representative information of a single condition and the discriminative information of multiple conditions.

[0043] The learning rate of the stochastic gradient descent optimization algorithm disclosed herein can be set to 0.001, the batch size to 64, and the number of training epochs to 300. After the model calculates the gradient, the magnitude of the parameter update is equal to the learning rate multiplied by the gradient value. A learning rate of 0.001 is a relatively small value, meaning that the magnitude of each parameter update is small, ensuring the stability of the training process and avoiding oscillations or divergence caused by excessively large step sizes. A smaller learning rate helps the model converge to a more accurate optimal solution, but may lead to slower convergence. The batch size refers to the number of samples used in each training iteration. In this example, 64 samples are used in each training iteration to calculate the gradient and update the model parameters. A batch size of 64 means that the model adjusts its parameters based on the average gradient information of 64 samples during each update. This setting strikes a balance between computational efficiency and convergence stability, avoiding both using a single sample (which may introduce large variance) and using all the data (which may consume excessive memory). The number of training epochs refers to the number of times the entire training dataset is fully traversed. 300 rounds means the model will traverse the entire training set 300 times, updating the model parameters with each iteration. The number of training rounds directly affects the sufficiency of the model's training; 300 rounds ensures the model has enough time to learn patterns and rules in the data, thus achieving better performance. This configuration ensures the model maintains good convergence stability during training while fully learning data features, making it a common and effective setting in deep learning training. However, this disclosure is not limited to this, and adjustments can be made to the above settings.

[0044] Figure 2 This is a block diagram of a dataset distillation method based on a boundary-aware diffusion model according to an embodiment.

[0045] Reference Figure 2 This paper illustrates a specific example of a dataset distillation method based on a boundary-aware diffusion model. By fine-tuning the generation distribution of the diffusion model, the diffusion model can inherit the classification information in the real dataset, and then effectively generate boundary synthetic samples through multi-condition guidance during the sampling stage.

[0046] In block 201, an expert model is trained using a real large-scale dataset, corresponding to step S101 above.

[0047] Using real large-scale datasets as the original training set This involves training a specific expert model to make its decision boundary fit the true boundary of the dataset. The expert model can be ResNet18, ReNet-101, or ResNet50, etc. For the input samples... , label y, expert model Category K, the prediction result is represented as Using traditional cross-entropy loss. The expert model is trained using the following loss function: (Equation 1) (1) in, ( ) is an indicator function, where 1 is true when y=k and 0 otherwise. The training process uses the stochastic gradient descent optimization algorithm on the training set. The training yielded Used for subsequent fine-tuning of the diffusion model.

[0048] In block 202, during the fine-tuning of the diffusion model, the class probabilities predicted by the trained expert model are used as confidence weights to calculate the multi-condition fine-tuning diffusion loss under different label class conditions for the same real image. The multi-condition fine-tuning optimization objective is obtained by weighting the loss with the classification distribution of the expert model, which corresponds to step S102 above.

[0049] In obtaining a pre-trained expert model Then, with its weights fixed, only the classification probability distribution it generates is used to guide the diffusion model. Fine-tuning training of the diffusion model. A parameter-efficient architecture (e.g., DiT combined with DiffFit) is chosen as the generator to achieve efficient distillation of large-scale datasets. For any given real-world image input... First, use expert models Predict its classification distribution : (2) This distribution reflects the expert model's influence at a macro level. The multi-condition prediction probability distribution of each category is used as the prediction confidence level, and this prediction probability distribution is used as the confidence level weight. In actual fine-tuning, to ensure the discriminative nature of the samples, the top categories with the highest predicted probabilities by the expert model are usually selected. Subsequent computations are performed on 2 (e.g., K=2) categories. The diffusion model modulates the original image through a forward noise-adding process. Gradually convert to Noisy samples at time points In the reverse denoising process, the diffusion model By class tag To conditionally predict the noise injected during the forward process Diffusion optimization objective under single-label conditions (i.e., the denoising score matching loss) is expressed as Equation 3 below: (3) This represents the injected standard Gaussian noise. The expected operation is represented by the loss function, which enables the diffusion model to capture the conditional generation distribution of a specific class by minimizing the mean square error between the predicted noise and the actual noise. .

[0050] To effectively distill the discriminative knowledge of the expert model into the generative distribution of the diffusion model, this disclosure uses the probability distribution predicted by the expert model as weights to perform a weighted summation of the single-conditional loss for the same sample under different class conditions, thus obtaining the multi-conditional fine-tuning loss. As shown in equation 4: (4) in, For class-balanced datasets, it is usually assumed that the prior distribution is categorical. By minimizing this loss The diffusion model not only learns how to generate samples of a specific category, but also captures the relative discriminative strength between categories, thus providing a basis for the subsequent synthesis of boundary samples with rich discriminative information.

[0051] In block 203, when sampling using the fine-tuned diffusion model, multiple label conditions are used to perform multi-condition sampling using the fine-tuned diffusion model, and single label conditions are used to perform single-condition sampling using the generated diffusion model. This guides the generation of typical samples and boundary samples within the category. The two types of samples are combined to form a synthetic dataset, corresponding to step S103 above.

[0052] For each synthetic sample to be generated, first sample a random noise from a standard Gaussian distribution. As the initial variable, specify the target category label. And a competing category label Competitive category tags Typically, categories adjacent to or easily confused with the target category in the feature space are selected to construct the boundary environment. Subsequently, random noise is gradually transformed into image samples through a back-diffusion process of a diffusion model.

[0053] Specifically, the backdiffusion process starts at time step t=T and iterates step by step until t=1. At each time step in the backdiffusion denoising process of the diffusion model... Using the fine-tuned model Target noise prediction and competing noise prediction are performed separately for two categories of conditions. Target noise prediction This can be expressed as formula 5 below: (5) Competition-type noise prediction This can be expressed as formula 6 below: (6) Construct boundary-aware sampling direction and utilize mixing coefficients The degree to which the denoising process is biased towards the target class is adjusted to control the distance between the generated samples and the true decision boundary. Combined with classifier-free guidance (CFG), the final denoising direction is determined. It is a linear combination of the scores of unconditional prediction, target class prediction, and competition class prediction, as shown in Equation 7 below: (7) in, To guide the scaling factor, This indicates unconditional input. Adjustments can be made to... Equation 7 above can guide image generation to move closer to the blurred area at the boundary between the two types.

[0054] At each time step t, the model accepts the current noise sample. The corresponding time step t and a label And predict the noise in the current sample. At each time step, the reverse update formula in Equation 8 is applied. Updated to : (8) In the formula, Let be the noise scheduling coefficient at step t. This represents the cumulative product term. The final result is... This is the synthesized sample we need.

[0055] According to the algorithm disclosed herein, the time step can be set to T=50, and the cumulative product term can be set as shown in Equation 9: (9) To balance the "representativeness" and "discriminativeness" of the samples, the final synthetic dataset consists of two parts: representative samples within each class and boundary discrimination samples. The representative samples within each class are obtained through single-condition sampling (i.e.,...). The samples are generated using the multi-condition sampling method described above (e.g., ...). These samples are used to learn the core semantics. Boundary discrimination samples are generated using the multi-condition sampling method described above. This is used to accurately fit the decision boundary. In practice, these two types of samples can be mixed in a 1:1 ratio to form the final distillation dataset.

[0056] This disclosure uses a pre-trained expert model on a large-scale real-world dataset to explicitly characterize the decision boundaries of the real dataset. By using the classification probability predictions output by the expert model as confidence weights, the diffusion model is fine-tuned under multiple conditions, distilling discriminative knowledge into the conditional generation distribution of the diffusion model. During the sampling phase, competing class labels are introduced, and the noise prediction directions of the target class and competing classes are linearly combined using mixing coefficients to guide the generation of boundary synthetic samples located near the decision boundary. These boundary synthetic samples are then proportionally mixed with representative in-class samples to construct the final lightweight synthetic dataset. This achieves precise alignment of the decision boundaries between the synthetic dataset and the real dataset, significantly improving the model classification accuracy and cross-architecture generalization ability of the synthetic dataset.

[0057] An exemplary embodiment of the present disclosure also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to execute a dataset distillation method based on a boundary-aware diffusion model according to the present disclosure. The computer-readable recording medium is any data storage device capable of storing data read by a computer system. Examples of computer-readable recording media include: read-only memory, random access memory, read-only optical disc, magnetic tape, floppy disk, optical data storage device, and carrier waves (such as data transmission via the Internet through wired or wireless transmission paths).

[0058] An exemplary embodiment of the present disclosure also provides a computer device. The computer device includes a processor and a memory. The memory stores a computer program. The computer program is executed by the processor, causing the processor to perform a computer program for a dataset distillation method based on a boundary-aware diffusion model according to the present disclosure.

[0059] Therefore, the exemplary embodiments of this disclosure can be implemented as methods in a computer or a non-transitory computer-readable medium storing computer-executable instructions. In the exemplary embodiments, when executed by a processor, the computer-readable instructions can perform a method according to at least one aspect of this disclosure.

[0060] Furthermore, the methods according to exemplary embodiments of this disclosure can be implemented in the form of program instructions that can be executed by various computer devices and recorded on a computer-readable medium.

[0061] Computer-readable media may include program instructions, data files, data structures, etc., individually or in combination. Program instructions recorded on a computer-readable medium may be specifically designed and configured for the exemplary embodiments of this disclosure, or may be known and available to those skilled in the art of computer software. Computer-readable recording media may include hardware devices configured to store and execute program instructions. For example, computer-readable recording media may be or include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as optical-magnetic disks; ROM; RAM; flash memory; etc. Program instructions may include not only machine language code generated by a compiler, but also high-level language code executable by a computer through an interpreter, etc.

[0062] While this disclosure includes specific examples, it will be apparent to those skilled in the art that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be obtained if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner and / or if components in the described system, architecture, apparatus, or circuit are replaced or supplemented by other components or their equivalents. Therefore, the scope of this disclosure is not limited by the specific embodiments but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents shall be construed as included in this disclosure.

Claims

1. A dataset distillation method based on a boundary-aware diffusion model, characterized in that, The dataset distillation method includes: The initial expert model is pre-trained using the original training set, and the decision boundary of the initial expert model is made to fit the true boundary by adjusting the weight parameters of the initial expert model. The class probabilities predicted by the trained expert model are used as confidence weights to generate multi-condition fine-tuning losses that reflect different classes. The fine-tuning of the diffusion model is guided by minimizing the weighted sum of the multi-condition fine-tuning losses, thereby distilling the discriminative knowledge of the trained expert model into the data generation distribution of the diffusion model. Multi-condition sampling is performed using a fine-tuned diffusion model. By introducing labels for the target class and adjacent competing classes, and weighting the noise prediction results of the target class and adjacent competing classes according to the mixing coefficient, the image generation of the diffusion model is guided to move closer to the decision boundary region between the two classes. The boundary discrimination samples generated by multi-condition sampling and the intra-class representative samples generated by single-condition sampling are dynamically mixed according to a preset ratio to construct a lightweight synthetic dataset as the final result of dataset distillation.

2. The dataset distillation method according to claim 1, characterized in that, The process of using the confidence weights output by the trained expert model to guide the fine-tuning of the diffusion model includes: obtaining the multi-condition prediction probability distribution of the original training set through the trained expert model while keeping the weight parameters in the trained expert model unchanged, using the prediction probability distribution as the confidence weight, and performing a weighted summation of the single-condition denoising score of the diffusion model based on the confidence weight to construct a weighted sum of the multi-condition fine-tuning loss.

3. The dataset distillation method according to claim 1, characterized in that, The distance between the generated samples and the decision boundary is controlled by adjusting the mixing coefficient to control the mixing ratio of the target class and adjacent competing classes.

4. The dataset distillation method according to claim 1, characterized in that, The boundary discrimination samples generated by the multi-condition sampling include samples located at the decision boundary between the target class and adjacent competing classes.

5. The dataset distillation method according to claim 1, characterized in that, The preset ratio for dynamically mixing boundary discrimination samples generated through multi-condition sampling with intra-class representative samples generated through single-condition sampling is 1:

1.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the dataset distillation method based on the boundary-aware diffusion model as described in any one of claims 1 to 5.

7. A computer device, characterized in that, The computer device includes: processor; A memory storing a computer program that, when executed by a processor, implements the dataset distillation method based on a boundary-aware diffusion model as described in any one of claims 1 to 5.

Images