A time-step adaptive diffusion model post-training quantization method

Abstract

This invention belongs to the field of diffusion model quantization and proposes a time-step adaptive post-training quantization method for diffusion models, including: Step S1: Obtaining a pre-trained full-precision diffusion model and determining the sampler and sampling strategy; Step S2: Generating and sampling images based on the full-precision diffusion model obtained in Step S1 and the sampler to obtain a validation set for post-training quantization; Step S3: Establishing a diffusion model and initializing the quantization parameters of the diffusion model; Step S4: Reconstructing and training the diffusion model; Step S5: Evaluating the performance of the reconstructed and trained diffusion model. This invention effectively solves the problems of existing quantization methods ignoring the differences in activation value ranges between channels, not considering the impact of iterative inference on quantization reconstruction, and the performance collapse of the quantized model at low quantization bit widths. It fully leverages the advantages of time-step-aware post-training quantization methods, resulting in a quantized diffusion model with superior performance and robustness.

Description

Technical Field

[0001] This invention belongs to the field of diffusion model quantization, and particularly relates to a time-step adaptive diffusion model quantization method after training. Background Technology

[0002] The main idea behind diffusion-based image generation is to first continuously add noise to an image. After a series of noise-adding processes, the image eventually becomes pure Gaussian noise. Then, a neural network learns the inverse process of this process, gradually removing noise to ultimately generate an image from the random noise. This process requires multiple inference operations on a complex neural network, resulting in significant memory and time overhead.

[0003] Neural network quantization methods use a mapping transformation to represent weights and data stored in high-precision floating-point numbers (usually 32-bit floating-point numbers) with low-precision data (usually 8 bits or lower fixed-point numbers), thereby compressing model storage space and accelerating inference. The principle behind quantization is that for deep neural networks represented by 32-bit floating-point numbers, there is significant redundancy in the numerical range, with many values ​​not being used in the calculations. Simultaneously, many edge devices have limited storage capacity and require optimization for low-precision data operations; therefore, using low-precision neural networks for inference offers a significant speedup effect.

[0004] Quantization inevitably leads to the loss of previously trained information in neural networks, resulting in performance degradation. Therefore, it's necessary to adjust the quantized network parameters to recover some of the lost performance. Depending on whether the entire network needs retraining, quantization methods can be categorized into quantization-aware training methods and post-training quantization methods. Quantization-aware training simulates the quantization process by inserting pseudo-quantized nodes into network layers during training, thus measuring the error caused by quantization. Post-training quantization typically doesn't train all network weights and only requires a small dataset as a validation set to adjust or retrain the quantization parameters of weights and activation values. While quantization-aware training methods often achieve accuracy similar to full-precision models, they suffer from high training overhead and reliance on complete datasets in practical applications. Therefore, post-training quantization is usually the preferred method for practical model deployment.

[0005] Because diffusion models require inference at different time steps on the same network, and the distribution of activation values ​​within the same layer often varies significantly, coupled with the iterative nature of inference leading to deviation errors in quantization and inference phases—characteristics absent in traditional neural networks—the direct application of general quantization methods to diffusion generative models results in substantial quantization errors. Currently, numerous researchers have addressed the quantization compression problem of diffusion models, but none have comprehensively considered the differences in activation values ​​across time steps and channels. Furthermore, existing methods largely employ quantization frameworks designed for general models, where the reconstruction of model blocks deviates from the iterative inference process of diffusion models. This ultimately leads to deviation errors in the input data, preventing the model from achieving optimal performance. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a time-step adaptive post-training quantization method for diffusion models. This method mainly involves fine-grained initialization in the time domain and a quantization reconstruction method to eliminate deviation errors. It can effectively solve the problems of existing quantization methods ignoring the differences in activation value ranges between channels, not considering the specific impact of iterative inference on quantization reconstruction, and the performance collapse of the quantized model at low quantization bit widths. It fully leverages the advantages of time-step aware post-training quantization methods, enabling the quantized diffusion model to have better performance and robustness.

[0007] The specific technical solution of the present invention is as follows:

[0008] A time-step adaptive diffusion model quantization method after training includes the following steps:

[0009] Step S1: Obtain the pre-trained full-precision diffusion model and determine the sampler and sampling strategy;

[0010] Step S2: Based on the full-precision diffusion model obtained in step S1 and the sampler, generate an image and sample it to obtain a quantized verification set after training;

[0011] Step S3: Establish a quantization diffusion model and initialize the quantization parameters of the diffusion model;

[0012] Step S4: Reconstruct and train the diffusion model;

[0013] Step S5: Evaluate the performance of the reconstructed diffusion model.

[0014] Preferably, the method for obtaining the pre-trained full-precision diffusion model in step S1 is as follows:

[0015] Build a diffusion model and pre-train it using an existing dataset; or directly use an open-source, full-precision diffusion model available online.

[0016] Preferably, the diffusion model is a U-Net network, and the sampler is a DDIM sampler.

[0017] Preferably, the existing dataset is the CIFAR-10 dataset, the LSUN-Bedrooms dataset, or the ImageNet-1K dataset.

[0018] Preferably, step S3 specifically includes:

[0019] For convolutional layers, the distribution of input activation values ​​for all intermediate layers is obtained by using the inference results of the diffusion model on the validation set; the maximum value of all activation value distributions is calculated on each input channel, and the reparameterization coefficients are calculated, and then the reparameterization operation is performed;

[0020] For the softmax layer, the distribution of activation values ​​after the softmax layer in all self-attention layers is statistically analyzed. Maximum likelihood estimation is used to determine whether it has the property of power-law distribution, and then a uniform quantizer or a logarithmic quantizer is selected accordingly.

[0021] Preferably, the reparameterization coefficients in step S3 are determined as follows:

[0022] a) A strategy of grouping based on sampling time steps is adopted, with each sampling time step as the center for uniform grouping. A uniform quantizer is set at each time step of all convolutional layers except the softmax layer, and the quantization parameters are set individually.

[0023] b) For each sampling time step, calculate the maximum value of the distribution of all activation values ​​on each input channel, and use a weighted average method to calculate the reparameterization coefficient.

[0024] Preferably, the uniform quantizer is represented as:

[0025]

[0026] x quant =Φ(x int ,0,2 bit-1 )

[0027] x dequant =(x quant -z g )*s g

[0028] Among them, s g and z g Let represent the quantization parameters at the g-th sampling time step, Φ(·,·,·) represent rounding and truncation operations respectively, bit represents the quantization bit width, and x int Integer values ​​are represented by x, and full-precision values ​​are represented by x. quant x represents the quantized value. dequant This represents the value after dequantization.

[0029] Preferably, the quantizer is represented as:

[0030]

[0031] x quant =Φ(x int ,0,2 bit-1 )

[0032]

[0033] in, Φ(·,·,·) represent rounding and truncation operations respectively, bit represents the quantization bit width, s represents the scaling factor, and x int Integer values ​​are represented by x, and full-precision values ​​are represented by x. quant x represents the quantized value. dequant This represents the value after dequantization.

[0034] Preferably, step S4 specifically includes:

[0035] Step S4-1: Train the diffusion model after initializing the quantization parameters using the verification set obtained in step S2;

[0036] Step S4-2: Generate a new validation set using the trained current diffusion model and collector;

[0037] Step S4-3: Train the current diffusion model using the new verification set;

[0038] Step S4-4: Determine whether the current diffusion model has reached the convergence condition. If yes, complete the reconstruction training; otherwise, return to step S4-2.

[0039] Preferably, step S5 specifically includes:

[0040] The diffusion model obtained in step S4 is used to generate an image using the sampler in step S1. The performance index is calculated and compared with the full-precision diffusion model to determine the performance of the diffusion model.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. The present invention provides a time-step adaptive diffusion model post-training quantization method, which proposes a time-step aware channel reparameterization method, overcoming the shortcomings of existing algorithms in being aware of channel activation values ​​of diffusion models, and enhancing the adaptability of diffusion models to a wider range of input data.

[0043] 2. The present invention provides a time-step adaptive diffusion model training post-quantization method that can dynamically and adaptively select a uniform quantizer or a logarithmic quantizer according to different activation value distribution characteristics, thereby improving the performance of the diffusion model under full quantization settings.

[0044] 3. The present invention provides a time-step adaptive diffusion model post-training quantization method to address the data bias problem in the post-training quantization reconstruction process, thereby filling the gap in existing reconstruction algorithms that do not address the characteristics of diffusion models and further improving model performance. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly introduced below. The features and advantages of the present invention can be more clearly understood by referring to the accompanying drawings. The accompanying drawings are schematic and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a time-step adaptive diffusion model training post-quantization method according to the present invention.

[0047] Figure 2 This is a flowchart illustrating the implementation of time-step-aware channel reparameterization and dynamic adaptive quantizer in this invention.

[0048] Figure 3 This is a flowchart illustrating the data deviation error reduction method in this invention. Detailed Implementation

[0049] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0051] like Figure 1As shown, the time-step adaptive diffusion model quantization method proposed in this invention mainly includes the following steps:

[0052] Step S1: Obtain the pre-trained full-precision diffusion model and determine the sampler and sampling strategy.

[0053] Depending on the specific needs, the full-precision diffusion model can be obtained in two ways:

[0054] (1) Establish a diffusion model and pre-train it using existing datasets; the diffusion model is a U-Net model, and the existing datasets include the CIFAR-10 dataset, the LSUN-Bedrooms dataset, and the ImageNet-1K dataset. The sampler is the DDIM (Denoising Diffusion Implicit Models) sampler.

[0055] (2) Directly use the open-source full-precision diffusion model weights on the Internet, including obtaining the link from the corresponding repository on Github or directly obtaining the full-precision diffusion model structure and weights from open-source diffusion model libraries such as Huggingface.

[0056] Step S2: Generate an image using the full-precision diffusion model and obtain a quantized verification set after training through a sampler.

[0057] Step S3: Establish a diffusion model and initialize the quantization parameters of the diffusion model. For example... Figure 2 As shown, two initialization methods are used:

[0058] (1) For convolutional layers, a strategy of grouping based on sampling time steps is adopted, with each sampling time step as the center for uniform grouping, and quantization parameters are set separately.

[0059] Assume the inference time steps of the diffusion model are determined as {t0, t1, ..., t2}. T The reasoning formula for the uniform quantizer after grouping is as follows:

[0060]

[0061] x qiant =Φ(x int ,0,2 bit-1 )

[0062] x dequant =(x quant -z g )*s g

[0063] Among them, s g and z gLet represent the quantization parameters at the g-th sampling time step, Φ(·,·,·) represent rounding and truncation operations respectively, bit represents the quantization bit width, and x int Integer values ​​are represented by x, and full-precision values ​​are represented by x. quant x represents the quantized value. dequant This represents the value after dequantization.

[0064] Then, a reparameterization method was employed for a single sampling time step. Assume that at a single sampling time step t, the input value of the convolutional layer is X. t ∈R B*C*H*W The main approach to reparameterization is to find a channel-wise transform r. t ∈R C , so that in X t After dividing by the corresponding coefficient, the maximum value range of the activation value for each channel is equal. Taking the c-th channel as an example:

[0065]

[0066] in, This represents the activation value of the c-th channel at the t-th sampling time step after reparameterization. This represents the activation value of the c-th channel at the t-th sampling time step. This represents the reparameterization coefficient for the c-th channel at the t-th sampling time step.

[0067] The specific approach is to scale the maximum value of each channel to the same range for each channel:

[0068]

[0069] Where max(X) t :,c () represents the maximum activation value of the c-th channel. This represents the target range. In applications involving reparameterization of a single sampling time step, this is relevant. The value of will not have a significant impact on the final result. For example, the mean of the maximum activation values ​​of all channels can be chosen as the target for reparameterization balancing. While transforming the activation values, the weights of the convolutional layer are transformed as follows:

[0070]

[0071] in, This represents all the kernel weights corresponding to the c-th channel of the convolutional layer. Since the convolution operation is essentially a linear transformation, the output of this convolutional layer remains unchanged after the above reparameterization scaling operation.

[0072] Combining the strategy of grouping by multiple sampling time steps, the first step is to select... Set it to the minimum value among the maximum values ​​of each channel, and select the method as follows:

[0073]

[0074] Among them, X t :,d This represents the activation value of the d-th channel at the t-th sampling time step, where D represents the number of input channel dimensions.

[0075] Meanwhile, to ensure that large activation values ​​are constrained, a weighted average method is used to calculate the reparameterization coefficients. The weights are chosen to be the proportion of the maximum activation value at that sampling time step to the total number of sampling time steps. The specific calculation formula is as follows:

[0076]

[0077] in, The table shows the uniform reparameterization coefficients on the i-th channel. X represents the reparameterization coefficient for the i-th channel at the t-th sampling time step. t :,i Let represent the activation value of the i-th channel at the t-th sampling time step, and max(·) represents finding the maximum value. This weight ensures that the transformation coefficients are given more attention at sampling time steps with larger activation values, thus further reducing the overall channel size. After introducing the above weighted averaging method, the sampling time step-aware channel reparameterization method is formally established. The final reparameterization scaling formula is:

[0078]

[0079] (2) For convolutional layers, statistically analyze the distribution of activation values ​​after the Softmax layer in all self-attention layers, use maximum likelihood estimation to determine whether they have the property of power-law distribution, and select a uniform quantizer or a logarithmic quantizer accordingly.

[0080] The quantizer is represented as:

[0081]

[0082] x quant =Φ(x int ,0,2 bit-1 )

[0083]

[0084] in, Φ(·,·,·) represent rounding and truncation operations, respectively, bit represents the quantization bit width, and s represents the scaling factor. A characteristic of quantizers is that they allocate more quantization bits near 0 and less quantization bits for larger values; this characteristic perfectly matches the properties of a long-tailed distribution.

[0085] Based on this, the power-law probability density function that best fits the current distribution and its corresponding confidence level can be fitted using the maximum likelihood estimation method, thus allowing selection between a uniform quantizer and a logarithmic quantizer. The final quantizer selection method is as follows:

[0086]

[0087] Where R represents the log-likelihood ratio of the distribution to the power-law distribution and other distributions, and threshold is used as a hyperparameter to determine whether the activation value of the layer is determined by a log quantizer or a uniform quantizer.

[0088] Step S4: Reconstruct and train the diffusion model.

[0089] like Figure 3 As shown, by introducing data from the quantized model as a validation set, the model can focus on the data distribution during the actual inference process, thereby reducing errors. The specific method is as follows:

[0090] (1) The diffusion model after quantization parameter initialization is trained using the verification set obtained in step S2;

[0091] (2) A new validation set is generated using the trained current diffusion model and collector. The new validation set is equivalent to the actual input activation distribution at each sampling time step obtained by the diffusion model strictly according to the inference sequence, so this part of the data is completely aligned with the actual inference situation.

[0092] (3) The new verification set is used to train the current diffusion model. During the training process, the activation value distribution of the diffusion model during actual inference in (2) is learned, which contains some activation data information of the diffusion model during inference.

[0093] (4) Determine whether the current diffusion model has reached the convergence condition. If yes, complete the reconstruction training; otherwise, return to (2). During the multiple reconstruction training processes, the data generated by the current diffusion model and the data distribution of the quantized final diffusion model in the inference stage at each stage show a gradual alignment trend.

[0094] Step S5: Evaluate the performance of the reconstructed diffusion model.

[0095] The diffusion model obtained in step S4 is used to generate an image using the sampler in step S1. The performance index is calculated and compared with the full-precision diffusion model to determine the performance of the diffusion model.

[0096] Example 1

[0097] 1. Obtain the pre-trained full-precision diffusion model and determine the sampler and sampling strategy.

[0098] 1.1 This example selects the U-Net model and the DDIM (Denoising Diffusion Implicit Models) sampler as the models for quantization. Three different U-Net models are used: the first U-Net model is pre-trained on the CIFAR-10 dataset, the second U-Net model is pre-trained on the LSUN-Bedrooms dataset, and the third U-Net model is pre-trained on the ImageNet-1K dataset.

[0099] 1.2 The CIFAR-10 dataset contains 60,000 32×32 pixel color images, divided into 10 classes with 6,000 images per class. This dataset mainly covers common objects in daily life, such as airplanes, cars, cats, and dogs. The LSUN-Bedrooms dataset contains over 400,000 bedroom images, providing rich samples for training and testing; the image resolution is typically 256×256 pixels. The ImageNet-1K dataset includes images in 1000 categories, with 1.28 million training images and 50,000 validation images. It covers not only common categories in daily life, such as different animals and objects, but also finer-grained classifications within the same category, such as different cats.

[0100] 1.3 For the model pre-trained on the CIFAR-10 dataset, during the inference phase, the sampling steps of the DDIM sampler are set to 100, noise is generated using the squaring method, and the variance control parameter η is set to 0. For the model pre-trained on LSUN-bedrooms, the DDIM sampling steps are 200, noise is generated using the squaring method, and the variance control parameter η is set to 0. For the model pre-trained on the ImageNet-1K dataset, the DDIM sampler is also used as the solver, the sampling time steps are set to 20, and the variance control parameter η is set to 0. This model uses a classifier-free method to generate conditional control images, and the corresponding coefficient is set to 3.0.

[0101] 2. Use the full-precision model for sampling to generate a post-trained quantized validation set.

[0102] For the model pre-trained on the CIFAR-10 dataset, a full-precision model was used to sample 256 images at each of the 100 inference time steps to form a quantized validation set. For the model pre-trained on LSUN-Bedrooms, 128 images were sampled at each of the 200 inference steps. For the model pre-trained on the ImageNet-1k dataset, a full-precision model was used to generate one image for each of the 1000 classes as a validation set, including information from all 20 time steps.

[0103] 3. Establish a diffusion model and initialize its quantization parameters. In the quantization phase of the diffusion model, the scaling factor *s* and zero point *z* for weight and activation quantization at each layer are first determined using an initial hyperparameter method. This method searches for the 0.999, 0.9999, and 0.99999 quantiles of the corresponding parameter distribution and selects the set with the smallest mean squared error as the initialization result. Subsequently, for all convolutional layer inputs, a time-step-aware channel reparameterization method is used to statistically calculate the scaling factor *r* for all channels, truncating all coefficients greater than 100 to prevent excessive weight amplification. This is then used to reparameterize the corresponding activation values ​​and weights. Simultaneously, maximum likelihood estimation is performed on all post-Softmax activation value distributions, with a threshold set to 0 for this step.

[0104] 4. Quantization Parameter Reconstruction Training. Mean squared error is used to constrain the training of the diffusion model. Backpropagation is used to update and optimize the network parameter weights until the model's loss region converges. The reconstruction process uses the BRECQ method for block reconstruction training only on the weights, with 20,000 training epochs. During optimization, the learning rate is set to 1e-3, and the Adam optimizer is used. The batch size is set to 8. In each iteration, validation sets from 8 different sampling time steps are mixed for reconstruction, with 20,000 reconstruction epochs. In the initial 4,000 steps, the β of the Adaround reconstruction is set to 0, gradually increasing linearly to a final value of 20 in subsequent steps. Then, using the reconstructed diffusion model, 25,600 validation images are generated across all sampling time steps, evenly distributed. The newly collected validation sets are used for 10,000 reconstruction epochs, and this process is repeated twice to obtain the final diffusion model. In this example, both model training and evaluation are performed on the PyTorch platform. The model was trained on a single NVIDIA GTX3080TI GPU (11GB), and the final report generates the best-performing quantized model.

[0105] 5. Performance Evaluation of the Diffusion Model. Using the quantized diffusion model, 50,000 images were generated on the corresponding dataset, and the FID, IS, or sFID metrics were calculated between the model and the original images in the dataset. These metrics were then compared with the performance metrics of the full-precision model. Smaller FID and sFID metrics, and larger IS metrics, indicate better model performance. In subsequent descriptions, WxAy represents quantizing the model weights to x bits and the activation values ​​to y bits. On the CIFAR-10 dataset, the FID and IS of the full-precision model were 4.14 and 9.12, respectively. The performance of the model quantized according to this invention was as follows: FID and IS were 4.07 and 9.08 on W8A8, 4.34 and 9.03 on W4A8, 4.32 and 9.08 on W6A6, and 5.72 and 8.74 on W4A4. On the LSUN-Bedrooms dataset, the FID and sFID of the full-precision model were 2.98 and 7.09, respectively. The performance of the quantized model according to this invention is as follows: FID and sFID are 3.22 and 7.33 on W8A8, 3.67 and 7.70 on W4A8, and 3.51 and 7.58 on W4A32. On the ImageNet-1K dataset, the FID, IS, and sFID of the full-precision model are 10.91, 235.64, and 7.67, respectively. The performance of the diffusion model quantized according to this embodiment is as follows: FID, IS, and sFID are 10.03, 241.25, and 7.49 on W8A8, 9.81, 239.38, and 7.69 on W4A8, and 10.17, 232.49, and 7.53 on W4A32. From the above results, it can be seen that the diffusion model quantized according to this embodiment can maintain excellent performance with a low quantization bit width, while also having the advantage of being able to perform fixed-point inference on hardware.

[0106] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0107] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0108] In this invention, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A time-step adaptive diffusion model quantization method after training, characterized in that, Includes the following steps: Step S1: Obtain the pre-trained full-precision diffusion model and determine the sampler and sampling strategy; Step S2: Based on the full-precision diffusion model obtained in step S1 and the sampler, generate an image and sample it to obtain a quantized verification set after training; Step S3: Establish a quantization diffusion model and initialize the quantization parameters for the model; Step S4: Reconstruct and train the quantization diffusion model; Step S5: Evaluate the performance of the reconstructed and trained quantized diffusion model; Step S3 specifically includes: For convolutional layers, the input activation value distribution of all intermediate layers is obtained by using the inference results of the quantization diffusion model on the validation set; the maximum value of all activation value distributions is calculated on each input channel, and the reparameterization coefficients are calculated, and then the reparameterization operation is performed; For the softmax layer, the distribution of activation values ​​after the softmax layer in all self-attention layers is statistically analyzed. Maximum likelihood estimation is used to determine whether it has the property of power-law distribution, and then a uniform quantizer or a logarithmic quantizer is selected accordingly. The reparameterization coefficients in step S3 are determined as follows: a) A strategy of grouping based on sampling time steps is adopted, with each sampling time step as the center for uniform grouping. A uniform quantizer is set at each time step of all convolutional layers except the softmax layer, and the quantization parameters are set individually. b) For each sampling time step, calculate the maximum value of the distribution of all activation values ​​on each input channel, and use a weighted average method to calculate the reparameterization coefficient; Step S4 specifically includes: Step S4-1: Train the quantization diffusion model after initializing the quantization parameters using the verification set obtained in step S2; Step S4-2: Generate a new verification set using the trained current quantization diffusion model and collector; Step S4-3: Train the current quantization diffusion model using the new verification set; Step S4-4: Determine whether the current quantization diffusion model has reached the convergence condition. If yes, complete the reconstruction training; otherwise, return to step S4-2.

2. The time-step adaptive diffusion model training post-quantization method according to claim 1, characterized in that, The method for obtaining the pre-trained full-precision diffusion model in step S1 is as follows: Build a diffusion model and pre-train it using an existing dataset; or directly use an open-source, full-precision diffusion model available online.

3. The time-step adaptive diffusion model training post-quantization method according to claim 2, characterized in that, The diffusion model is a U-Net network, and the sampler is a DDIM sampler.

4. The time-step adaptive diffusion model training post-quantization method according to claim 2, characterized in that, The existing datasets are the CIFAR-10 dataset, the LSUN-Bedrooms dataset, or the ImageNet-1K dataset.

5. The time-step adaptive diffusion model training post-quantization method according to claim 1, characterized in that, The uniform quantizer is represented as follows: in, and Let represent the quantization parameters at the g-th sampling time step, and These represent rounding and truncation operations, respectively. Indicates the quantization bit width. Integer values ​​are represented by x, and full-precision values ​​are represented by 'x'. This represents the quantized value. This represents the value after dequantization.

6. The time-step adaptive diffusion model training post-quantization method according to claim 1, characterized in that, The quantizer is represented as: in, and These represent rounding and truncation operations, respectively. This indicates the quantization bit width, and s represents the scaling factor. Integer values ​​are represented by x, and full-precision values ​​are represented by 'x'. This represents the quantized value. This represents the value after dequantization.

7. The time-step adaptive diffusion model training post-quantization method according to claim 1, characterized in that, Step S5 specifically includes: The quantized diffusion model obtained in step S4 is used to generate an image using the sampler in step S1. The performance index is calculated and compared with the full-precision diffusion model to determine the performance of the quantized diffusion model.

Images