Diffusion model sampling and distillation method for advertising image material generation

Abstract

This application relates to the field of computer vision in machine learning technology, and more particularly to a diffusion model sampling and distillation method for generating advertising image material. The method comprises loading a pre-trained diffusion model and executing a complete diffusion sampling process, collecting complete potential feature trajectories through multi-step denoising inference, performing broken line simplification analysis on the collected potential feature trajectories to generate a set of key time steps, and then, based on the set of key time steps and the guidance of a teacher diffusion model, performing segmented progressive distillation training to construct a student-based lightweight generation model specifically for the advertising field. This application can achieve both high efficiency and high quality in generating advertising material, while generating professional advertising material in a very small number of sampling steps.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a diffusion model sampling and distillation method for generating advertising image materials. Background Art

[0002] With the rapid development of digital marketing and e-commerce, the production of advertising creatives is shifting from traditional manual design to automated generation based on deep learning. Diffusion models, due to their excellent generation quality and diversity, have become a research hotspot in image synthesis, demonstrating great potential in advertising creativity, scene construction, and creative iteration. In the field of advertising image creative generation, a variety of diffusion-based acceleration and optimization methods have been developed, including denoising diffusion probabilistic models, denoising diffusion implicit models, ordinary differential equation solver methods, knowledge distillation acceleration techniques, random time step selection techniques, and large-scale general model techniques. These techniques are currently the most common technical approaches.

[0003] In practice, existing technologies often fail to balance image quality while increasing sampling speed, and lack in-depth optimization for the visual characteristics of the advertising field. In order to resolve the contradiction between sampling efficiency and generation quality, and to improve the professional performance of advertising materials, two new solutions are proposed. One is an accelerated advertising image generation system based on the denoising diffusion implicit model (DDIM), which applies DDIM technology to accelerate the generation process of advertising materials. DDIM modifies the sampling process of the standard denoising diffusion probability model (DDPM) to allow some time steps to be skipped during the generation process, reducing the process that originally required thousands of iterations to dozens of steps. The other is a domain-specific diffusion model distillation system based on the low-rank adaptation method (LoRA): This solution uses low-rank adaptation technology to perform lightweight fine-tuning on pre-trained large diffusion models to adapt to the image generation needs of specific fields, while reducing the sampling steps through model distillation.

[0004] However, the two solutions mentioned above each have their own limitations in practice: although the DDIM-based accelerated generation system greatly compresses the number of iteration steps to dozens of steps, its uniform skipping strategy does not distinguish between the differences in the contributions of each stage to image details and textures, resulting in a decay in the expression of key structures and tiny features. While the LoRA-based domain distillation system has improved professional performance, it relies on a pre-set fixed number of sampling steps and lacks the ability to dynamically respond to key sampling moments in different advertising content, making it difficult to balance batch generation efficiency and overall visual coherence. If the two are simply superimposed, it is difficult to simultaneously ensure a high degree of detail restoration and domain style consistency within a limited sampling budget. Therefore, how to balance the high efficiency and high quality of advertising material generation while generating professional advertising materials within a very small number of sampling steps is a current challenge. Summary of the Invention

[0005] This application provides a diffusion model sampling and distillation method for generating advertising image materials. This method can achieve both high efficiency and high quality in generating advertising materials while generating professional advertising materials in a very small number of sampling steps. This application provides the following technical solutions:

[0006] In a first aspect, the present application provides a diffusion model sampling and distillation method for generating advertising image materials, the method comprising:

[0007] Load the pre-trained diffusion model and perform a complete diffusion sampling process to collect the complete potential feature trajectory through multi-step denoising inference;

[0008] Performing a broken line simplification analysis on the collected potential feature trajectories to generate a set of key time steps;

[0009] Based on the key time step set and the guidance of the teacher diffusion model, sampling segmented progressive distillation training is performed to construct a student lightweight generation model dedicated to the advertising field.

[0010] In a specific implementation scheme, loading the pre-trained diffusion model and performing a complete diffusion sampling process to collect a complete potential feature trajectory through multi-step denoising inference includes:

[0011] Load the trained diffusion model and set the total time steps of the diffusion process , construct the initial latent variables As the initial input of the diffusion process, Represents the initial noise state of the image and is defined as follows:

[0012] ;

[0013] in, and denote the spatial height and width of the potential feature map, respectively, is the number of channels, represents the set of all real numbers;

[0014] According to the time step Decrease to Multi-step denoising inference is performed in the order of , enter the current latent variable And the corresponding time step information is fed into the diffusion model, and the updated latent variables are output through the forward propagation of the neural network , after each step of inference is completed, save the potential features of the current time step , record the potential variables of all time steps in sequence, forming a line of length The potential feature trajectory .

[0015] In a specific implementation scheme, performing a broken line simplification analysis on the collected potential feature trajectories includes:

[0016] For each spatial position in the potential feature trajectory and each channel , extract its The corresponding eigenvalue sequence , which is considered as a set of two-dimensional points , where the horizontal axis is the time step , the vertical coordinate is the corresponding eigenvalue, forming multiple broken lines;

[0017] Apply the RDP algorithm to calculate the perpendicular distance between the straight line between the endpoints of each polyline and each midpoint. The calculation formula of the perpendicular distance is as follows:

[0018] ;

[0019] in, 、 、 are the two-dimensional coordinate vectors of the starting point, middle point and end point of the polyline respectively, represents the cross product of two-dimensional vectors, Represents the modulus of a vector;

[0020] If the maximum vertical distance exceeds the preset threshold, the corresponding time step is retained, and the two ends of the polyline are recursively simplified until the maximum vertical distance of all points does not exceed the preset threshold.

[0021] In a specific embodiment, the performing broken line simplification analysis on the collected potential feature trajectories further includes:

[0022] For each spatial position in the potential feature trajectory and each channel In the latent feature trajectory The sequence of changes in As the analysis object, the corresponding time series polyline is constructed. Each polyline can be represented as a point set:

[0023] ;

[0024] For the starting and ending points of a polyline 、 , and any point in between , and its DNRD distance is defined as follows:

[0025] ;

[0026] in, , represents the feature difference between the frames before and after the midpoint; , represents the standard time step interval; , indicating the overall characteristic difference between the first and last points; , indicating the time span between the beginning and the end;

[0027] If the maximum DNRD distance exceeds the preset threshold, the corresponding time step is retained, and the two ends of the polyline are recursively simplified until the maximum DNRD distance of all points does not exceed the preset threshold.

[0028] In a specific embodiment, generating a set of key time steps includes:

[0029] After completing the polyline simplification, count the frequency of occurrence of all retained time steps in all polylines ;

[0030] Setting frequency thresholds , ;

[0031] Choose to satisfy The time steps constitute the key time step set , ,in, is the maximum value of frequency, .

[0032] In a specific implementation scheme, the sampling segmented progressive distillation training based on the key time step set and the guidance of the teacher diffusion model to construct a student lightweight generative model dedicated to the advertising field includes:

[0033] The time interval The key time step set in Rearrange in chronological order and construct a series of time interval pairs , where each pair satisfy , represents a transition from a noise state Towards the target state denoising transition;

[0034] In distillation training, the teacher model provides intermediate feature pairs , student model Receive the teacher model at time step Features , and predict its corresponding time step The initial noise representation ;

[0035] Introducing scoring auxiliary network , the training objective of the scoring network is to minimize the following weighted mean square error loss function:

[0036] ;

[0037] in, and Represents the teacher model at time step 、 potential representation of Represents the data pairs generated by the teacher model The mathematical expectation of and Represents the time step 、 The noise scheduling coefficient of Represents the student model at time step , receiving input The estimated noise or gradient of the posterior output; is the student model parameter gradient;

[0038] Weight function The definition is as follows:

[0039] ;

[0040] in, Represents the time step The frequency of being selected as a key node in the entire feature trajectory polyline simplification process.

[0041] In a specific embodiment, the training process of the student model includes:

[0042] First, the model structure and parameters are initialized. The teacher model is a complete diffusion model with fixed parameters. The student model Using a simplified neural network architecture, the scoring auxiliary network is used to learn the true gradient pattern on the generative path of the teacher model;

[0043] Then sample training pairs, and randomly sample a pair of time periods from the key time step set in each round of training , get the corresponding feature pair ; Using the scoring auxiliary network to calculate the true gradient direction, construct a loss function, and guide the update of the student model parameters;

[0044] The parameters of the student model and the parameters of the scoring auxiliary network are distilled and trained through the loss function. After the distillation training is completed, the parameters containing only Learning lightweight generative models.

[0045] In a second aspect, the present application provides a diffusion model sampling and distillation system for generating advertising image materials, which adopts the following technical solutions:

[0046] A diffusion model sampling and distillation system for generating advertising image materials, comprising:

[0047] The trajectory sampling module is used to load the pre-trained diffusion model and perform the complete diffusion sampling process to collect the complete potential feature trajectory through multi-step denoising inference;

[0048] A time step screening module is used to perform a broken line simplification analysis on the collected potential feature trajectories to generate a set of key time steps;

[0049] The model distillation module is used to construct a student lightweight generative model dedicated to the advertising field by sampling segmented progressive distillation training based on the key time step set and the guidance of the teacher diffusion model.

[0050] In a third aspect, the present application provides an electronic device comprising a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement a diffusion model sampling and distillation method for generating advertising image materials as described in the first aspect.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores a program, and when the program is executed by a processor, it is used to implement a diffusion model sampling and distillation method for generating advertising image materials as described in the first aspect.

[0052] In summary, the beneficial effects of this application include at least:

[0053] (1) By extracting a set of key time steps from the complete diffusion sampling path and designing a jump denoising path based on it, inference and training are performed only at these key nodes, effectively avoiding the problem of a large number of redundant time steps in traditional diffusion models. During the inference phase, the student model only needs to perform sampling at a very small number of key time points, thereby significantly reducing the number of steps and computation required to generate an image. During the training phase, the jump-type segmented distillation strategy avoids supervision of the continuous full trajectory, significantly reducing the complexity of sample preparation, gradient calculation, and network update. Overall, compared with the conventional diffusion model, this solution achieves an order of magnitude acceleration effect, providing solid support for the deployment of applications in advertising scenarios that require extremely high generation response speed.

[0054] (2) In order to avoid image quality degradation caused by jump sampling, this application introduces a scoring auxiliary network as a gradient tuner of the teacher model knowledge in distillation training, fitting the true denoising direction in the cross-segment mapping process, compensating for the deviation problem under the Gaussian approximation, and ensuring that the student model can accurately capture the potential feature evolution law in a long span of time. In addition, by modeling the frequency weighting of key time steps, the loss function further focuses on learning at places where the image structure changes drastically, enhancing the student model's ability in semantic consistency and detail expression. Therefore, even if the generation path is significantly compressed, the output advertising image still has rich details and a stable structure, without blurring, deformation, or other quality loss phenomena.

[0055] (3) The constructed distillation process not only pursues model lightweight and inference speed, but also specifically combines the special needs of the advertising industry for visual expression style, content composition and semantic consistency. By using the complete diffusion model as a teacher to guide professional image trajectories, the student model learns semantically meaningful latent representation mappings at critical moments, thereby having stronger advertising image expression capabilities. At the same time, the student model adopts a modular compression design with extremely low deployment resource usage, which facilitates rapid launch on industrial-grade advertising delivery platforms. Overall, this technical solution successfully achieved the migration of professional advertising images from "heavy model + slow generation" to "light model + fast output", fully meeting the balance requirements between real-time generation, low computing power deployment and content quality.

[0056] By loading a pre-trained diffusion model, full-time-step denoising inference is performed from random noise to clear images, and the latent variables at each step are recorded to construct a potential feature evolution trajectory. The temporal features of each pixel channel in the trajectory are treated as broken lines, and the nodes that best reflect the changes in image structure and details are extracted. The key time step set is then screened based on the frequency of node occurrence. Using the true latent representation of the teacher diffusion model at key time steps as supervision, a scoring auxiliary network is designed to correct long-span approximation deviations, and a lightweight student model is trained to accurately complete the denoising mapping from noise to image with only key jumps. The resulting student model relies on key time step extraction and segmented distillation within a very small number of sampling steps to effectively skip redundant intermediate iterations, significantly reducing the amount of inference and training computations while ensuring the consistency of details and semantics with the help of the scoring network. Ultimately, this achieves a significant improvement in generation efficiency while maintaining the visual quality of professional advertising materials.

[0057] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement it in accordance with the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1It is a flowchart of the diffusion model sampling and distillation method for generating advertising image materials in an embodiment of the present application.

[0059] Figure 2 This is a structural block diagram of a diffusion model sampling and distillation system for generating advertising image materials in an embodiment of the present application.

[0060] Figure 3 It is a block diagram of an electronic device for diffusion model sampling and distillation for generating advertising image materials in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0062] Optionally, the present application uses the diffusion model sampling and distillation method for generating advertising image materials provided in various embodiments as an example for explanation in an electronic device, where the electronic device is a terminal or a server. The terminal may be a computer, a tablet computer, etc. This embodiment does not limit the type of electronic device.

[0063] Reference Figure 1 , is a flow chart of a diffusion model sampling and distillation method for generating advertising image materials provided by an embodiment of the present application. The method includes at least the following steps:

[0064] Step S101: load the pre-trained diffusion model and perform a complete diffusion sampling process to collect complete potential feature trajectories through multi-step denoising inference.

[0065] In step S101, the goal is to obtain a latent feature trajectory that describes the entire image generation process. This trajectory reflects the evolution of the diffusion model from the initial high-noise state to the completed image generation state and serves as the basis for subsequent key time step screening and model distillation. A pretrained diffusion model is loaded and, starting from the initial noisy state, the complete sampling process is performed sequentially in time steps, recording the intermediate latent feature representation of the current time step at each step.

[0066] Specifically, first load a diffusion model that has been trained on a large-scale image dataset. After loading, set the total time steps of the diffusion process. , is a positive integer, indicating that the total number of The initial latent variables are then constructed. , which is usually a multidimensional tensor randomly sampled from a standard normal distribution and serves as the initial input to the diffusion process. This latent variable represents the initial noise state of the image and is defined as follows:

[0067] ;

[0068] in, and denote the spatial height and width of the potential feature map, respectively, is the number of channels, represents the set of all real numbers, and the specific value is determined according to the model design. Next, according to the time step from Decrease to In the order of , multi-step denoising reasoning is performed to gradually restore the image potential representation. At each time step , enter the current latent variable And the corresponding time step information is fed into the diffusion model, and the updated latent variables are output through the forward propagation of the neural network , thus achieving the gradual elimination of noise and the gradual restoration of image semantic details. After each step of reasoning is completed, the potential features of the current time step are saved , record the potential variables of all time steps in sequence, thus forming a line of length The potential feature trajectory This trajectory fully reflects the generation process from the pure noise state to the final clear image latent representation, providing detailed time series data for subsequent analysis. By comparing and analyzing the various latent features in the trajectory, we can evaluate the role and contribution of each time step in denoising and image detail reconstruction.

[0069] Step S102: Perform broken line simplification analysis on the collected potential feature trajectories to generate a set of key time steps.

[0070] In practice, the goal of step S102 is to filter out key time steps that significantly impact image generation details from the complete latent feature trajectory. This allows subsequent sampling and distillation processes to perform computational updates only on these key time steps, reducing computational effort and improving efficiency. By treating the temporal features of each pixel position and channel dimension in the latent feature trajectory as a broken line and analyzing it using the RDP algorithm, the key nodes, or key time steps, that preserve changes in image detail are identified.

[0071] In step S102, first, for each spatial position in the potential feature trajectory and each channel , extract its The corresponding eigenvalue sequence , which is considered as a set of two-dimensional points , where the horizontal axis is the time step , with the corresponding eigenvalue on the ordinate, thus forming multiple broken lines that reflect the temporal evolution of the pixel channel's features. Each broken line is then simplified using the RDP algorithm. This algorithm determines which time steps contribute significantly to shape change by calculating the perpendicular distances between the line's endpoints and each midpoint.

[0072] The vertical distance is calculated as follows:

[0073] ;

[0074] in, 、 、 are the two-dimensional coordinate vectors of the starting point, middle point and end point of the polyline respectively, represents the cross product of two-dimensional vectors, Represents the modulus of the vector. If the maximum vertical distance exceeds the preset threshold, the corresponding time step is retained and the two ends of the polyline are recursively simplified until the maximum vertical distance of all points does not exceed the preset threshold.

[0075] After completing the polyline simplification of all pixels and channels, count the frequency of occurrence of all retained time steps in all polylines , the frequency reflects the importance of the corresponding time step in the overall potential feature change. The higher the frequency, the greater the contribution of the time step to the restoration of image details. Based on this, the frequency threshold is set , , choose to meet The time steps constitute the key time step set , ,in, is the maximum value of frequency, The time steps in this set serve as key time points that only require full sampling and updating in the future, reducing computational overhead while ensuring image quality. This step fully utilizes the broken line simplification algorithm to effectively filter temporal features, ensuring that the time points that best reflect the important details of the image generation process are retained, providing accurate and efficient training data for the distillation process.

[0076] In addition, as a preferred embodiment, in another feasible embodiment, the broken line simplification method is improved by introducing an asymmetric distance measurement method based on the normalized change rate, which can effectively improve the robustness and difference recognition ability of the key time step extraction.

[0077] Specifically, each pixel position and each channel In the latent feature trajectory The sequence of changes in As the analysis object, construct the corresponding time series polyline. Each polyline can be represented as a point set:

[0078] ;

[0079] Unlike traditional RDP methods, this implementation introduces the Directional Normalized Change Rate Distance (DNRD) as a metric for determining node retention importance. This metric not only considers the degree of deviation of the midpoint from the overall polyline shape but also incorporates directional information and the normalized difference between adjacent slopes, enhancing the ability to sensitively identify potential asymmetric feature changes.

[0080] For the starting and ending points of a polyline 、 , and any point in between , and its DNRD distance is defined as follows:

[0081] ;

[0082] in, , represents the feature difference between the frames before and after the midpoint; , represents the standard time step interval; , indicating the overall characteristic difference between the first and last points; , representing the time span from the beginning to the end. The numerator in this formula measures the deviation between the local rate of change and the overall rate of change, while the denominator normalizes the degree of deviation to prevent amplification when the overall change amplitude is small. If the maximum DNRD distance exceeds the preset threshold, the corresponding time step is retained, and the simplification process is recursively repeated at both ends of the polyline until the maximum DNRD distance of all points does not exceed the preset threshold. Finally, after processing all polylines, a subset of key time steps for each feature change sequence is obtained.

[0083] The above-mentioned DNRD distance measurement formula accurately measures the degree of deviation of the intermediate point from the overall trend by introducing the difference between the local change rate and the global change rate; and by adding a normalization term to the denominator, it effectively prevents the misjudgment of key points due to numerical amplification when the overall change amplitude is small, thereby enhancing the sensitivity to nonlinear mutations and local discontinuous changes. This design not only takes into account the amplitude of feature changes, but also incorporates directionality and trend consistency, avoiding the limitations of traditional RDP algorithms that only judge the importance of points based on vertical geometric offsets. In particular, it exhibits stronger robustness and selectivity when faced with non-uniform speed, fluctuations or drastic changes in time series features. Therefore, while retaining the validity of information and reducing redundant calculation points, the DNRD method improves the ability to accurately identify key time steps during the simplification process, making it more suitable for simplified scenarios with complex potential feature trajectories.

[0084] After step S102, the present application also verifies the fidelity boundary of the key time step compression strategy and establishes a theoretical basis for error controllability. The purpose is to theoretically verify the key time step set selected based on the broken line simplification method. Specifically, it is necessary to establish an error propagation model to characterize how the difference between the simplified latent feature trajectory and the original complete trajectory propagates during the diffusion sampling process while retaining a finite number of critical time steps, and to evaluate the impact of this difference on the quality of the final image generation.

[0085] Specifically, during the polyline simplification process, the upper limit of the local approximation error of each pixel channel trajectory is set to Although the error is locally controlled on each trajectory, its gradual propagation in the sampling path may cause structural deviations in the global image. Therefore, the inverse update function of the diffusion model is introduced , and assume that it satisfies the Lipschitz continuity condition on the latent space:

[0086] ;

[0087] in, is the Lipschitz constant, and Represents any two potential states. Based on the above conditions, if at each sampling time step The maximum difference between the simplified latent variable and the original latent variable is , then this error in the reverse sampling path is at most proportional to Therefore, in the latent space of size And when the norm is used to measure the error, the upper bound of the difference is as follows:

[0088] ;

[0089] Furthermore, by arrive The reverse expansion process is summarized and analyzed, and it is proved that the upper bound of the error of the final image is ; That is, the overall error shows a linear growth with the degree of compression of the key time step and is controlled by three factors: the simplification threshold , sampling steps and the Lipschitz coefficient of the inverse function of the diffusion model This theoretical result demonstrates that, given a reasonable choice of error threshold and set of critical time steps, the worst-case generation bias caused by compression of the entire sampling path is bounded and predictable. This error bound provides a theoretical basis for interpretability and reliability in subsequent sampling updates performed only on the set of critical time steps.

[0090] Step S103: Based on the key time step set and the guidance of the teacher diffusion model, sampling segmented progressive distillation training is performed to build a student lightweight generation model dedicated to the advertising field.

[0091] In step S103, the target is based on the obtained key time step set , formally launched the distillation training process, leveraging a high-performance teacher diffusion model to guide a smaller, more lightweight student model to learn jump-based denoising mappings in the latent space, thereby building a lightweight generative model that can be efficiently deployed in advertising image generation scenarios. Through the knowledge supervision provided by the teacher model, the gradient guidance of the scoring auxiliary network, and the structural compression of key time steps, the overall sampling process is significantly streamlined while maintaining stable generation quality.

[0092] Specifically, first the original time interval The key time step set in Rearrange in chronological order and construct a series of time interval pairs , where each pair satisfy , represents a transition from a noise state Towards the target state In distillation training, the teacher model provides the intermediate feature pairs , student model The learning goal is: the student model receives the teacher model at time step Features , and predict its corresponding time step The initial noise representation , to simulate a long-span denoising mapping process.

[0093] It should be noted that the teacher model in this application is a full-scale diffusion model that has been pre-trained on a large-scale advertising image dataset. This model has strong image generation capabilities, and its diffusion denoising process serves as an ideal reference path for the latent space trajectory, providing accurate intermediate feature pairs and gradient directions for the distillation stage. The student model is a lightweight generative model that has been structurally optimized through parameter compression and path reconstruction. Its infrastructure can be a compressed version of U-Net or other lightweight encoding-decoding generative networks, which have the ability to generate corresponding images from the latent state of key time steps. The student model is initially initialized by distilling the teacher model to obtain parameter initialization, or it can start from an unsupervised pre-training state and complete cross-time period learning through the subsequent scoring auxiliary network guidance.

[0094] However, due to There is usually a large time span between the two. If we simply rely on Gaussian reparameterization to approximate the loss, it will often cause misleading predictions. Therefore, this step introduces a scoring auxiliary network , its training goal is to fit the true gradient direction of the teacher model during long-span migration, providing the student model with a more stable training signal that is closer to the true distribution. The training goal of the scoring network is to minimize the following weighted mean square error loss function:

[0095] ;

[0096] in, and Represents the teacher model at time step 、 potential representation of Represents the data pairs generated by the teacher model The mathematical expectation of the above, that is, the average of the entire training process based on the teacher output; and Represents the time step 、 The noise scheduling coefficient, the cumulative term in the fixed noise scheduling function, controls the noise amplitude at each time step; Represents the student model at time step , receiving input The estimated noise or gradient of the posterior output; is the student model parameter gradient.

[0097] Weight function The definition is as follows:

[0098] ;

[0099] in, Represents the time step The frequency of being selected as a key node in the whole feature trajectory polyline simplification process. This frequency value reflects the time step The importance of frequency weighting to the global image generation quality. By adding this frequency weighting mechanism, the loss function can adaptively focus on those time periods that contribute significantly to the semantic changes of the image, thereby improving the efficiency and effectiveness of distillation learning.

[0100] The training process of the student model is as follows:

[0101] First, the model structure and parameters are initialized. The teacher model is a complete diffusion model with fixed parameters. The student model A simplified neural network architecture, such as a deeply compressed U-Net, is used to construct a cross-segment denoising map with a small number of parameters; a scoring auxiliary network is used to learn the true gradient pattern on the teacher model generation path. Training pairs are then sampled, and each round of training randomly samples a pair of time periods from the key time step set. , get the corresponding feature pair ; Use the scoring auxiliary network to calculate the true gradient direction, construct the loss function, and guide the update of the student model parameters. Then the parameters of the student model and the parameters of the scoring auxiliary network are jointly optimized through the above loss function for distillation training. After the distillation training is completed, the output contains only The student lightweight generation model is used for deployment; in the inference phase, sampling jumps are only performed at key time steps, significantly reducing the number of generation steps and computational costs; combined with the characteristics of the advertising image generation task, the lightweight model can quickly output high-quality images, meeting the dual requirements of efficiency and quality in actual business.

[0102] Compared with the traditional distillation method that only relies on the static Gaussian approximation path, by training only on key time steps, the student model can skip the intermediate redundant process, greatly improving the sampling speed and inference efficiency. The introduction of the scoring auxiliary network effectively avoids the quality loss caused by the Gaussian approximation deviation, so that the student model maintains accuracy in long-span predictions. The frequency factor in the loss function guides the model to pay more attention to the moments when the actual image structure changes drastically, improving the model's ability to maintain the semantic structure and local details of the image. Finally, through the above distillation training process, the student model is finally obtained. , and exported as a lightweight inference version. In actual use, it is only called on the key time step set , you can complete the efficient generation from random noise or text conditional encoding to the final advertising creative.

[0103] The key time-step-driven segmented distillation method proposed in this step significantly reduces the number of sampling steps while maintaining generation quality. A scoring-assisted network corrects the gradient deviation of long-span denoising paths, and combined with a frequency-based importance weighting mechanism, enables the student model to accurately learn jump-like mapping relationships in the latent space. The resulting lightweight diffusion generative model boasts efficient inference and semantic fidelity, making it particularly suitable for applications such as advertising image generation, where both speed and quality are paramount.

[0104] In summary, this application loads a pretrained diffusion model, performs full-timestep denoising inference from random noise to a clear image, and records the latent variables at each step to construct a latent feature evolution trajectory. The temporal features of each pixel channel in the trajectory are treated as broken lines, and the nodes that best reflect changes in image structure and detail are extracted. A set of key timesteps is then selected based on the frequency of node occurrence. Using the true latent representation of the teacher diffusion model at key timesteps as supervision, a scoring auxiliary network is designed to correct long-span approximation biases. A lightweight student model is trained to accurately complete the denoising mapping from noise to image while requiring only key jumps. The resulting student model, relying on key timestep extraction and segmented distillation, effectively skips redundant intermediate iterations within a very small number of sampling steps. This significantly reduces the computational effort for inference and training, while also ensuring detail and semantic consistency through the scoring network. Ultimately, this significantly improves generation efficiency while maintaining the visual quality of professional advertising creatives.

[0105] Figure 2 This is a structural block diagram of a diffusion model sampling and distillation system for generating advertising image materials provided by an embodiment of the present application. The system includes at least the following modules:

[0106] The trajectory sampling module is used to load the pre-trained diffusion model and perform the complete diffusion sampling process to collect the complete potential feature trajectory through multi-step denoising inference;

[0107] The time step screening module is used to perform line simplification analysis on the collected potential feature trajectories and generate a set of key time steps;

[0108] The model distillation module is used to build a student-based lightweight generative model specifically for the advertising field through sampling segmented progressive distillation training based on a set of key time steps and the guidance of a teacher diffusion model.

[0109] For relevant details, please refer to the above method embodiment.

[0110] Figure 3 4 is a block diagram of an electronic device provided in one embodiment of the present application. The device includes at least a processor 401 and a memory 402.

[0111] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0112] Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 402 is used to store at least one instruction, which is executed by processor 401 to implement the diffusion model sampling and distillation method for generating advertising image materials provided in the method embodiment of the present application.

[0113] In some embodiments, the electronic device may optionally include a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Illustratively, the peripheral devices include, but are not limited to, radio frequency circuitry, a touchscreen display, audio circuitry, and a power supply.

[0114] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0115] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored. The program is loaded and executed by a processor to implement the diffusion model sampling and distillation method for generating advertising image materials in the above method embodiment.

[0116] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, in which a program is stored. The program is loaded and executed by a processor to implement the diffusion model sampling and distillation method for generating advertising image materials in the above-mentioned method embodiment.

[0117] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A diffusion model sampling and distillation method for generating advertising image materials, characterized in that: The method comprises: Load the pre-trained diffusion model and perform the complete diffusion sampling process to collect the complete potential feature trajectory through multi-step denoising inference, including: loading the trained diffusion model, setting the total time steps of the diffusion process , construct the initial latent variables As the initial input of the diffusion process, Represents the initial noise state of the image and is defined as follows: ; in, and denote the spatial height and width of the potential feature map, respectively, is the number of channels, Represents the set of all real numbers; according to the time step from Decrease to Multi-step denoising inference is performed in the order of , enter the current latent variable And the corresponding time step information is fed into the diffusion model, and the updated latent variables are output through the forward propagation of the neural network , after each step of inference is completed, save the potential features of the current time step , record the potential variables of all time steps in sequence, forming a line of length The potential feature trajectory ; Perform line simplification analysis on the collected potential feature trajectories to generate a set of key time steps, including: for each spatial position in the potential feature trajectory and each channel , extract its The corresponding eigenvalue sequence , which is considered as a set of two-dimensional points , where the horizontal axis is the time step , the vertical coordinate is the corresponding eigenvalue, forming multiple broken lines; Apply the RDP algorithm to calculate the perpendicular distance between the straight line between the endpoints of each polyline and each midpoint. The calculation formula of the perpendicular distance is as follows: ； in, 、 、 are the two-dimensional coordinate vectors of the starting point, middle point and end point of the polyline respectively, represents the cross product of two-dimensional vectors, Represents the modulus of a vector; If the maximum vertical distance exceeds the preset threshold, the corresponding time step is retained and the two ends of the polyline are recursively simplified until the maximum vertical distance of all points does not exceed the preset threshold; Alternatively, performing a simplified broken line analysis on the collected potential feature trajectories further includes: For each spatial position in the potential feature trajectory and each channel In the latent feature trajectory The sequence of changes in As the analysis object, the corresponding time series polyline is constructed. Each polyline can be represented as a point set: ； For the starting and ending points of a polyline 、 , and any point in between , and its DNRD distance is defined as follows: ; in, , represents the feature difference between the frames before and after the midpoint; - , represents the standard time step interval; , indicating the overall characteristic difference between the first and last points; , indicating the time span between the beginning and the end; If the maximum DNRD distance exceeds the preset threshold, the corresponding time step is retained, and the two ends of the polyline are recursively simplified until the maximum DNRD distance of all points does not exceed the preset threshold; After completing the polyline simplification, count the frequency of occurrence of all retained time steps in all polylines ; Set frequency threshold , ; Choose to meet The time steps constitute the key time step set , ,in, is the maximum value of frequency, ; Based on the key time step set and the guidance of the teacher diffusion model, piecewise progressive distillation training is used to build a student lightweight generative model dedicated to the advertising field, including: The time interval The key time step set in Rearrange in chronological order and construct a series of time interval pairs , where each pair satisfy , represents a transition from a noise state Towards the target state denoising transition; In distillation training, the teacher model provides intermediate feature pairs , student model Receive the teacher model at time step Features , and predict its corresponding time step The initial noise representation ; Introducing scoring auxiliary network , the training objective of the scoring network is to minimize the following weighted mean square error loss function: ； in, and Represents the teacher model at time step 、 potential representation, Represents the data pairs generated by the teacher model The mathematical expectation of and Represents the time step 、 The noise scheduling coefficient of Represents the student model at time step , receiving input The estimated noise or gradient of the posterior output; is the student model parameter gradient; Weight function The definition is as follows: ； in, Represents the time step The frequency of being selected as a key node in the entire feature trajectory polyline simplification process.

2. The diffusion model sampling and distillation method for generating advertising image materials according to claim 1, characterized in that: The training process of the student model includes: First, the model structure and parameters are initialized. The teacher model is a complete diffusion model with fixed parameters. The student model Using a simplified neural network architecture, the scoring auxiliary network is used to learn the true gradient pattern on the generative path of the teacher model; Then sample training pairs, and randomly sample a pair of time periods from the key time step set in each round of training , get the corresponding feature pair ; Using the scoring auxiliary network to calculate the true gradient direction, construct a loss function, and guide the update of the student model parameters; The parameters of the student model and the parameters of the scoring auxiliary network are distilled and trained through the loss function. After the distillation training is completed, the parameters containing only Learning lightweight generative models.

3. A diffusion model sampling and distillation system for generating advertising image materials, characterized in that: include: The trajectory sampling module is used to load the pre-trained diffusion model and perform the complete diffusion sampling process, collecting the complete potential feature trajectory through multi-step denoising inference, including: loading the trained diffusion model, setting the total time steps of the diffusion process , construct the initial latent variables As the initial input of the diffusion process, Represents the initial noise state of the image and is defined as follows: ; in, and denote the spatial height and width of the potential feature map, respectively, is the number of channels, Represents the set of all real numbers; according to the time step from Decrease to Multi-step denoising inference is performed in the order of , enter the current latent variable And the corresponding time step information is fed into the diffusion model, and the updated latent variables are output through the forward propagation of the neural network , after each step of inference is completed, save the potential features of the current time step , record the potential variables of all time steps in sequence, forming a line of length The potential feature trajectory ; The time step screening module is used to perform line simplification analysis on the collected potential feature trajectories to generate a set of key time steps, including: for each spatial position in the potential feature trajectory and each channel , extract its The corresponding eigenvalue sequence , which is considered as a set of two-dimensional points , where the horizontal axis is the time step , the vertical coordinate is the corresponding eigenvalue, forming multiple broken lines; Apply the RDP algorithm to calculate the perpendicular distance between the straight line between the endpoints of each polyline and each midpoint. The calculation formula of the perpendicular distance is as follows: ； in, 、 、 are the two-dimensional coordinate vectors of the starting point, middle point and end point of the polyline respectively, represents the cross product of two-dimensional vectors, Represents the modulus of a vector; If the maximum vertical distance exceeds the preset threshold, the corresponding time step is retained and the two ends of the polyline are recursively simplified until the maximum vertical distance of all points does not exceed the preset threshold; Alternatively, performing a simplified broken line analysis on the collected potential feature trajectories further includes: For each spatial position in the potential feature trajectory and each channel In the latent feature trajectory The sequence of changes in As the analysis object, the corresponding time series polyline is constructed. Each polyline can be represented as a point set: ； For the starting and ending points of a polyline 、 , and any point in between , and its DNRD distance is defined as follows: ; in, , represents the feature difference between the frames before and after the midpoint; - , represents the standard time step interval; , indicating the overall characteristic difference between the first and last points; , indicating the time span between the beginning and the end; If the maximum DNRD distance exceeds the preset threshold, the corresponding time step is retained, and the two ends of the polyline are recursively simplified until the maximum DNRD distance of all points does not exceed the preset threshold; After completing the polyline simplification, count the frequency of occurrence of all retained time steps in all polylines ; Set frequency threshold , ; Choose to meet The time steps constitute the key time step set , ,in, is the maximum value of frequency, ; The model distillation module is used to build a student-based lightweight generative model specifically for the advertising field using piecewise progressive distillation training based on the key time step set and the guidance of the teacher diffusion model, including: The time interval The key time step set in Rearrange in chronological order and construct a series of time interval pairs , where each pair satisfy , represents a transition from a noise state Towards the target state denoising transition; In distillation training, the teacher model provides intermediate feature pairs , student model Receive the teacher model at time step Features , and predict its corresponding time step The initial noise representation ; Introducing scoring auxiliary network , the training objective of the scoring network is to minimize the following weighted mean square error loss function: ； in, and Represents the teacher model at time step 、 potential representation, Represents the data pairs generated by the teacher model The mathematical expectation of and Represents the time step 、 The noise scheduling coefficient of Represents the student model at time step , receiving input The estimated noise or gradient of the posterior output; is the student model parameter gradient; Weight function The definition is as follows: ； in, Represents the time step The frequency of being selected as a key node in the entire feature trajectory polyline simplification process.

4. An electronic device, characterized in that: The device includes a processor and a memory; the memory stores a program, and the program is loaded and executed by the processor to implement a diffusion model sampling and distillation method for generating advertising image materials as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that The storage medium stores a program, which, when executed by a processor, is used to implement a diffusion model sampling and distillation method for generating advertising image materials as described in any one of claims 1 to 2.

Images