Diffusion model sampling and distilling method for advertisement image material generation

Abstract

The invention relates to the computer vision direction in the technical field of machine learning, in particular to a diffusion model sampling and distilling method for advertisement image material generation, which comprises the following steps of: loading a pre-training diffusion model, executing a complete diffusion sampling process, and collecting a complete potential characteristic track through multi-step denoising reasoning; broken line simplification analysis is carried out on the collected potential feature tracks, and a key time step set is generated; and based on the key time step set and the guidance of a teacher diffusion model, sampling segmented progressive distillation training, and constructing a student lightweight generation model special for the advertisement field. According to the method, high efficiency and high quality of advertisement material generation can be considered, and specialized advertisement materials can be generated in few sampling steps.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a diffusion model sampling and distillation method for advertising image material generation. Background Art

[0002] With the rapid development of digital marketing and e-commerce, the production mode of advertising materials is shifting from traditional manual design to deep learning-based automated generation. Due to its excellent generation quality and diversity, the diffusion model has become a research hotspot in the field of image synthesis, showing great potential in advertising creativity, scene construction, and material iteration. In the field of advertising image material generation, there are already various acceleration and optimization methods based on the diffusion model, including denoising diffusion probabilistic models, denoising diffusion implicit models, ordinary differential equation solver methods, knowledge distillation acceleration techniques, random time step selection techniques, and large general model techniques, etc. The above-mentioned techniques are currently the most common technical means.

[0003] In implementation, the existing technologies often cannot balance image quality while improving the sampling speed, and lack in-depth optimization for the visual characteristics of the advertising field. To solve the contradiction between sampling efficiency and generation quality, and to improve the professional performance of advertising materials, two new solutions are proposed. One is an accelerated generation system for advertising images based on the denoising diffusion implicit model (DDIM), which applies DDIM technology to accelerate the generation process of advertising materials. DDIM modifies the sampling process of the standard denoising diffusion probabilistic model (DDPM), allowing some time steps to be skipped during the generation process, reducing the process that originally required thousands of iterations to dozens of steps. The other is a domain-specific diffusion model distillation system based on the low-rank adaptation method (LoRA): This solution uses low-rank adaptation technology to perform lightweight fine-tuning on a pre-trained large diffusion model to meet the image generation requirements of a specific domain, and at the same time reduces the sampling steps through model distillation.

[0004] However, the above two solutions have their limitations in practice: Although the accelerated generation system based on DDIM significantly compresses the number of iteration steps to dozens of steps, its uniform step-skipping strategy does not distinguish the contribution differences of each stage to image details and textures, resulting in attenuation of the expression of key structures and tiny features. And the domain distillation system based on LoRA, although it has improved in professional performance, relies on a preset fixed number of sampling steps, lacks the dynamic response ability to key sampling moments in different advertising contents, and is difficult to balance batch generation efficiency and overall visual coherence. If the two are simply superimposed, it is also difficult to ensure high-detail restoration and domain style consistency under a limited sampling budget. Therefore, how to balance the high efficiency and high quality of advertising material generation while generating professional advertising materials in very few sampling steps is a difficult problem currently faced. Summary of the Invention

[0005] This application provides a diffusion model sampling and distillation method for advertising image material generation, which can generate professional advertising materials within very few sampling steps while taking into account both high efficiency and high quality in advertising material generation. This application provides the following technical solutions: In a first aspect, this application provides a diffusion model sampling and distillation method for advertising image material generation, and the method includes: Loading a pre-trained diffusion model and performing a complete diffusion sampling process, and collecting a complete latent feature trajectory through multi-step denoising inference; Performing polyline simplification analysis on the collected latent feature trajectory to generate a set of key time steps; Based on the set of key time steps and the guidance of the teacher diffusion model, sampling for segmented progressive distillation training to construct a student lightweight generation model dedicated to the advertising field.

[0006] In a specific feasible implementation, the loading of the pre-trained diffusion model and performing a complete diffusion sampling process, and collecting a complete latent feature trajectory through multi-step denoising inference includes: Loading the trained diffusion model, setting the total number of time steps of the diffusion process , constructing an initial latent variable as the starting input of the diffusion process, representing the initial noise state of the image, defined as follows: ; wherein, and respectively represent the spatial height and width of the latent feature map, is the number of channels, represents the set of all real numbers; Performing multi-step denoising inference in the order of decreasing time steps from to . In each time step , inputting the current latent variable and the corresponding time step information into the diffusion model, and through the forward propagation of the neural network, outputting the updated latent variable . After each step of inference is completed, saving the latent feature of the current time step, and successively recording the latent variables of all time steps to form a latent feature trajectory with a length of .

[0007] In a specific feasible implementation, the performing polyline simplification analysis on the collected latent feature trajectory includes: For each spatial position in the latent feature trajectory and each channel Extract its eigenvalue sequences at all time steps and regard them as a set of two-dimensional point sets where the abscissa is the time step and the ordinate is the corresponding eigenvalue, forming multiple broken lines; Apply the RDP algorithm to each broken line, calculate the perpendicular distance between the straight line connecting the endpoints of the broken line and each intermediate point. The calculation formula for the perpendicular distance is as follows: ; where , , are the two-dimensional coordinate vectors of the starting point, intermediate point, and ending point of the broken line respectively, represents the cross product of two-dimensional vectors, represents the modulus of the vector; If the maximum perpendicular distance exceeds the preset threshold, retain the corresponding time step points and recursively simplify both ends of the broken line until the maximum perpendicular distance of all points does not exceed the preset threshold.

[0008] In a specific feasible implementation, the simplifying analysis of broken lines for the collected potential feature trajectories further includes: For each spatial position and each channel in the potential feature trajectories regard the change sequence as the analysis object, construct the corresponding time-series broken lines, and each broken line can be represented as a point set: ; For the start and end points , ,and any intermediate point of a certain broken line, its DNRD distance is defined as follows: ; where represents the feature difference between the frames before and after the intermediate point; represents the standard time step interval; represents the overall feature difference between the start and end points; represents the time span between the start and end points; If the maximum DNRD distance exceeds the preset threshold, retain the corresponding time step points and recursively simplify both ends of the broken line until the maximum DNRD distance of all points does not exceed the preset threshold.

[0009] In a specific feasible implementation, the generation of the set of key time steps includes: After completing the polyline simplification, count the occurrence frequencies of all the retained time steps in all polylines. ; Set the frequency threshold. , ; Select the time steps that satisfy to form a set of key time steps , , where is the maximum value of the frequency, .

[0010] In a specific feasible implementation, the sampling segmented progressive distillation training based on the set of key time steps and the guidance of the teacher diffusion model to construct a student lightweight generation model dedicated to the advertising field includes: Rearrange the set of key time steps in the time interval in chronological order and construct a series of time interval pairs , where each pair satisfies , representing a denoising transition from the noise state to the target state ; In the distillation training, the teacher model provides intermediate feature pairs , and the student model receives the features of the teacher model at the time step , and predicts the initial noise representation at its corresponding time step ; Introduce a scoring auxiliary network , and the training objective of the scoring network is to minimize the following weighted mean square error loss function: ; where, and respectively represent the latent representations of the teacher model at the time steps , ; represents the mathematical expectation on the data pair generated by the teacher model; and respectively represent the noise scheduling coefficients at the time steps , ; represents the estimated noise or gradient output by the student model at the time step after receiving the input ; is the student model parameter Gradient; Weight function Is defined as follows: ; Wherein, Represents the time step The frequency of being selected as a key node during the simplification of all feature trajectory polylines.

[0011] In a specific implementable embodiment, the training process of the student model includes: First, initialize the model structure and parameters. The teacher model is a complete diffusion model with fixed parameters. The student model Adopts a simplified neural network architecture. The scoring auxiliary network is used to learn the true gradient pattern on the generation path of the teacher model; Subsequently, sample training pairs. Randomly sample a pair of time periods from the set of key time steps in each round of training , and obtain the corresponding feature pairs ; Calculate the true gradient direction using the scoring auxiliary network, construct a loss function, and guide the update of the student model parameters; The parameters of the student model and the parameters of the scoring auxiliary network are distilled and trained through the loss function. After the distillation training is completed, export a learning lightweight generation model that only contains .

[0012] In a second aspect, the present application provides a diffusion model sampling and distillation system for advertising image material generation, adopting the following technical solutions: A diffusion model sampling and distillation system for advertising image material generation, comprising: A trajectory sampling module, configured to load a pre-trained diffusion model and execute a complete diffusion sampling process, and collect a complete latent feature trajectory through multi-step denoising inference; A time step screening module, configured to perform polyline simplification analysis on the collected latent feature trajectory to generate a set of key time steps; A model distillation module, configured to perform sampling and segmented progressive distillation training based on the set of key time steps and the guidance of the teacher diffusion model, and construct a student lightweight generation model dedicated to the advertising field.

[0013] In a third aspect, the present application provides an electronic device, the device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a diffusion model sampling and distillation method for advertising image material generation as described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores a program, and when the program is executed by a processor, the program is used to implement a diffusion model sampling and distillation method for generating advertising image materials as described in the first aspect.

[0015] In summary, the beneficial effects of this application include at least: (1) By extracting a set of key time steps from the complete diffusion sampling path and designing a jump denoising path based on it, reasoning and training are performed only at these key nodes, effectively avoiding the problem of a large number of redundant time steps in traditional diffusion models. In the reasoning stage, the student model only needs to perform sampling at a very small number of key time points, which greatly reduces the number of steps and calculations required to generate an image. In the training stage, the jump-type segmented distillation strategy avoids supervision of the continuous full trajectory and significantly reduces the complexity of sample preparation, gradient calculation, and network update. Overall, compared with the conventional diffusion model, this solution achieves an order of magnitude acceleration effect, providing solid support for the deployment of applications in advertising scenarios that require extremely high generation response speed.

[0016] (2) In order to avoid image quality degradation caused by jump sampling, this application introduces a scoring auxiliary network as a gradient tuner of the teacher model knowledge in distillation training, fits the true denoising direction in the cross-segment mapping process, compensates for the deviation problem under the Gaussian approximation, and ensures that the student model can accurately capture the potential feature evolution law in a long span of time. In addition, by frequency-weighted modeling of key time steps, the loss function further focuses on learning at places where the image structure changes dramatically, enhancing the student model's ability in semantic consistency and detail expression. Therefore, even if the generation path is significantly compressed, the output advertising image still has rich details and a stable structure, without blurring, deformation, or other quality loss phenomena.

[0017] (3) The constructed distillation process not only pursues model lightweight and inference speed, but also takes into account the special needs of the advertising industry for visual expression style, content composition and semantic consistency. By using the complete diffusion model as a teacher to guide the professional image trajectory, the student model learns semantically meaningful latent representation mappings at critical moments, thereby having stronger advertising image expression capabilities. At the same time, the student model adopts a modular compression design with extremely low deployment resource occupancy, which is convenient for rapid launch on industrial-grade advertising delivery platforms. Overall, this technical solution successfully achieved the migration of professional advertising images from "heavy model + slow generation" to "light model + fast output", fully meeting the balance requirements between real-time generation, low computing power deployment and content quality.

[0018] By loading a pre-trained diffusion model, perform full-time-step denoising inference from random noise to clear images, record the latent variables at each step, and construct the latent feature evolution trajectory; the temporal features of each pixel channel in the trajectory are regarded as broken lines, extract the nodes that can best reflect the changes in image structure and details, and filter out the set of key time steps based on the frequency of node occurrences; using the true latent representations of the teacher diffusion model at the key time step pairs as supervision, design a scoring auxiliary network to correct the long-span approximation bias, and train a lightweight student model to accurately complete the denoising mapping from noise to image with only key jumps. The finally obtained student model, within very few sampling steps, relies on key time step extraction and segmented distillation to effectively skip redundant intermediate iterations, greatly reducing both the inference and training computational amounts, and ensuring the consistency of details and semantics with the help of the scoring network. Finally, while maintaining the visual quality of professional advertising materials, the generation efficiency is significantly improved.

[0019] The above description is only an overview of the technical solution of this application. In order to be able to understand the technical means of this application more clearly and implement it in accordance with the content of the specification, the following takes the preferred embodiments of this application and combines with the drawings to describe in detail as follows. Brief Description of the Drawings

[0020] Figure 1 It is a schematic flowchart of the diffusion model sampling and distillation method for advertising image material generation in an embodiment of this application.

[0021] Figure 2 It is a structural block diagram of the diffusion model sampling and distillation system for advertising image material generation in an embodiment of this application.

[0022] Figure 3 It is a block diagram of an electronic device for the diffusion model sampling and distillation for advertising image material generation in an embodiment of this application. Detailed Description of the Embodiment

[0023] The following combines the drawings and embodiments to further describe the specific implementation manners of this application in detail. The following embodiments are used to illustrate this application, but do not limit the scope of this application.

[0024] Optionally, this application takes the diffusion model sampling and distillation method for advertising image material generation provided in each embodiment as an example for illustration in an electronic device. The electronic device is a terminal or a server. The terminal can be a computer, a tablet computer, etc. The type of the electronic device is not limited in this embodiment.

[0025] Refer to Figure 1 , which is a schematic flowchart of the diffusion model sampling and distillation method for advertising image material generation provided in an embodiment of this application. The method at least includes the following steps: Step S101: Load a pre-trained diffusion model and perform a full diffusion sampling process to collect a complete latent feature trajectory through multi-step denoising inference.

[0026] In step S101, the goal is to obtain a latent feature trajectory that describes the entire process of image generation. This trajectory is used to reflect the evolution of the diffusion model from a high-noise initial state to the completed image generation state, and serves as the basis for subsequent key time step screening and model distillation. Load a pre-trained diffusion model and start from the initial noise state, perform a complete sampling process in chronological order of time steps, and record the intermediate latent feature representation at the current time step at each step.

[0027] Specifically, first load a diffusion model that has been trained on a large-scale image dataset. After loading, set the total number of time steps in the diffusion process , as a positive integer, indicating that the process is divided into sampling stages from the initial state to the final state. Subsequently, construct an initial latent variable , which is usually a multi-dimensional tensor randomly sampled from a standard normal distribution and serves as the starting input for the diffusion process. This latent variable represents the initial noise state of the image and is defined as follows: ; where and represent the spatial height and width of the latent feature map respectively, is the number of channels, represents the set of all real numbers, and the specific values are determined according to the model design. Next, perform multi-step denoising inference in descending order of time steps from to to gradually restore the image latent representation. At each time step , input the current latent variable and the corresponding time step information into the diffusion model. Through the forward propagation of the neural network, output the updated latent variable , thereby gradually eliminating the noise and gradually restoring the semantic details of the image. After each step of inference is completed, save the latent feature at the current time step, and record the latent variables at all time steps in sequence, thus forming a latent feature trajectory of length . This trajectory fully reflects the generation process from the pure noise state to the final clear image latent representation, providing detailed time series data for subsequent analysis. By comparing and analyzing the latent features in the trajectory, the role and contribution of each time step in denoising and image detail reconstruction can be evaluated.

[0028] Step S102: Perform polyline simplification analysis on the collected potential feature trajectories to generate a set of key time steps.

[0029] In implementation, the goal of step S102 is to screen out the key time steps from the obtained complete potential feature trajectories that have a significant impact on the details of image generation. Thus, in the subsequent sampling and distillation processes, calculations and updates are only performed for these key time steps, achieving the purpose of reducing the computational amount and improving efficiency. By regarding the temporal features of each pixel position and channel dimension in the potential feature trajectories as a polyline, and then using the RDP algorithm to analyze the polyline, the key nodes that retain the details of the image change, that is, the key time steps, are identified.

[0030] In step S102, first, for each spatial position and each channel in the potential feature trajectories, extract the sequence of eigenvalue corresponding to all time steps , regard it as a set of two-dimensional points , where the abscissa is the time step , and the ordinate is the corresponding eigenvalue, thus forming multiple polylines to reflect the changing trend of the pixel channel features over time. Then, for each polyline, apply the RDP algorithm for simplification. This algorithm determines which time steps contribute significantly to the shape change by calculating the perpendicular distance between the straight line connecting the endpoints of the polyline and each intermediate point. The calculation formula for the perpendicular distance is as follows:

[0031] ; ; where , , are the two-dimensional coordinate vectors of the starting point, intermediate point, and ending point of the polyline respectively, represents the cross product of two-dimensional vectors, represents the magnitude of the vector. If the maximum perpendicular distance exceeds the preset threshold, the corresponding time step points are retained, and the two ends of the polyline are recursively simplified until the maximum perpendicular distance of all points does not exceed the preset threshold.

[0032] After completing the polyline simplification of all pixels and channels, count the occurrence frequency of all the retained time step points in all polylines , and this frequency reflects the importance of the corresponding time step in the overall potential feature change. The higher the frequency, the greater the contribution of this time step to the restoration of image details. Based on this, set a frequency threshold , , select the time steps that satisfy to form a set of key time steps , , where is the maximum value of the frequency, . The time steps in this set are used as the key time points that only need to be updated by complete sampling subsequently, achieving the effect of reducing the computational overhead while ensuring the image generation quality. This step makes full use of the line simplification algorithm to effectively screen the temporal features, ensuring that the time points that can best reflect the important detail changes in the image generation process are retained, providing accurate and efficient training data for the distillation link.

[0033] In addition, preferably, in another feasible embodiment, an improvement is made on the line simplification method, introducing an asymmetric distance metric based on the normalized change rate, which can effectively improve the robustness and differential recognition ability of key time step extraction.

[0034] Specifically, still taking each pixel position and each channel in the latent feature trajectory as the change sequence for analysis, a corresponding temporal line is constructed. Each line can be represented as a point set: ; Different from the traditional RDP method, in this embodiment, a direction-normalized change rate distance (DNRD) is introduced as an important indicator for judging the importance of node retention. This metric method not only considers the deviation degree of the intermediate point from the overall line shape, but also introduces the direction information and the normalized difference of adjacent slopes, enhancing the sensitive recognition ability of potential asymmetric feature changes.

[0035] For the start and end points , , and any intermediate point of a certain line, its DNRD distance is defined as follows: ; Among them, , represents the feature difference between the frames before and after the intermediate point; , represents the standard time step interval; , represents the overall feature difference between the start and end points; , represents the time span between the start and end. The numerator in this formula measures the deviation between the local change rate and the overall change rate, and the denominator normalizes the degree of deviation to prevent abnormal amplification when the overall change amplitude is small. If the maximum DNRD distance exceeds the preset threshold, the corresponding time step point is retained, and the two ends of the line are recursively simplified until the maximum DNRD distance of all points does not exceed the preset threshold. Finally, after processing all lines, the key time step subset of each feature change sequence is obtained.

[0036] The above DNRD distance metric formula accurately measures the deviation degree of the intermediate point from the overall trend by introducing the difference between the local change rate and the global change rate; and by adding a normalization term to the denominator, it effectively prevents misjudgment of key points caused by numerical amplification when the overall change amplitude is small, thus enhancing the sensitivity to non-linear mutations and local discontinuous changes. This design not only considers the amplitude of feature changes, but also incorporates directionality and trend consistency, avoiding the limitations brought by the traditional RDP algorithm's judgment of point importance only based on vertical geometric offset, especially showing stronger robustness and selectivity when facing non-uniform, fluctuating or drastic changes in time series features. Therefore, while retaining the effectiveness of information and reducing redundant calculation points, the DNRD method improves the ability to accurately identify key time steps in the simplification process, and is more suitable for the simplification scenario of complex potential feature trajectories.

[0037] After step S102, the present application also verifies the fidelity boundary of the key time step compression strategy and establishes a theoretical basis for error controllability. The purpose is to verify the feasibility of the set of key time steps selected based on the polyline simplification method at the theoretical level. Specifically, an error propagation model needs to be established to describe how the difference between the simplified potential feature trajectory and the original complete trajectory propagates during the diffusion sampling process when retaining a limited number of key time steps, and to evaluate the impact degree of this difference on the final image generation quality.

[0038] Specifically, during the polyline simplification process, the upper limit of the local approximation error of each pixel channel trajectory is set to . Although this error is locally controlled on each trajectory, its gradual propagation in the sampling path may cause structural deviations in the global image. Therefore, the reverse update function of the diffusion model is introduced, and it is assumed that it satisfies the Lipschitz continuity condition in the latent space: ; where is the Lipschitz constant, and represent any two latent states. Based on the above conditions, if at each sampling time step , the maximum difference between the simplified latent variable and the original latent variable is , then this error is amplified at most in proportion to in the reverse sampling path. Therefore, when the size of the latent space is and the error is measured using the norm, there is the following upper bound of the difference: ; Furthermore, by integrating the diffusion path from to Conduct an inductive analysis of the reverse expansion process to prove that the upper bound of the error of the final image is ; that is, the overall error shows a linear growth related to the compression degree of the key time steps, and is controlled by three factors: the simplification threshold , the number of sampling steps , and the Lipschitz coefficient of the reverse function of the diffusion model. This theoretical result shows that, on the premise of reasonably selecting the error threshold and the set of key time steps, the generation bias caused by the compression behavior of the entire sampling path is bounded and predictable in the worst case. This error bound provides a theoretical basis for interpretability and reliability for subsequent sampling updates only on the set of key time steps.

[0039] Step S103: Based on the set of key time steps and the guidance of the teacher diffusion model, sample and conduct progressive distillation training to construct a lightweight student generation model dedicated to the advertising field.

[0040] In step S103, the goal is to formally carry out the distillation training process based on the obtained set of key time steps . With the help of the high-performance teacher diffusion model, guide a student model with fewer parameters and a lighter structure to learn the jump denoising mapping in the latent space, so as to construct a lightweight generation model that can be efficiently deployed in the advertising image generation scenario. Through the knowledge supervision provided by the teacher model, the gradient guidance of the scoring auxiliary network, and the structure compression of the key time steps, the overall sampling process is significantly streamlined and the generation quality is stably maintained.

[0041] Specifically, first rearrange the set of key time steps in the original time interval in chronological order, and construct a series of time interval segments , where each pair satisfies , representing a denoising transition from the noise state to the target state . In the distillation training, the teacher model provides intermediate feature pairs , and the learning objective of the student model is: the student model receives the feature of the teacher model at time step , and predicts the initial noise representation at its corresponding time step to simulate a long-span denoising mapping process.

[0042] It should be noted that the teacher model in this application is a full-scale diffusion model that has been pre-trained on a large-scale advertising image dataset. This model has powerful image generation capabilities. Its diffusion denoising process serves as an ideal reference path for the latent space trajectory, providing accurate intermediate feature pairs and gradient directions for the distillation stage. The student model is a lightweight generation model optimized in terms of parameter compression and path reconstruction in terms of structure. Its basic architecture can be a compressed version of U-Net or other lightweight encoding-decoding type generation networks, and it has the ability to generate corresponding images from the latent state at key time steps. The student model is initialized with parameters distilled from the teacher model at the beginning, or it can also start from an unsupervised pre-training state and complete cross-time learning through the guidance of the subsequent scoring auxiliary network.

[0043] However, since there is usually a large time span between , if only relying on Gaussian reparameterization to approximately calculate the loss, it often causes prediction misleading. Therefore, this step introduces a scoring auxiliary network ; where and respectively represent the latent representations of the teacher model at time steps , . denotes the mathematical expectation taken over the data pairs generated by the teacher model, that is, averaging over the entire training process based on the teacher's output; and respectively represent the noise scheduling coefficients at time steps , , which are the cumulative terms in the fixed noise scheduling function and control the noise amplitude at each time step; represents the estimated noise or gradient output by the student model at time step after receiving the input ; is the gradient of the student model parameters .

[0044] The weight function is defined as follows: ; where represents the frequency of the time step being selected as a key node during the simplification of the entire feature trajectory polyline. This frequency value reflects the time step The importance for the global image generation quality. By adding this frequency weighting mechanism, the loss function can adaptively focus on those time periods that contribute significantly to the semantic changes in the image, thereby improving the efficiency and effectiveness of distillation learning.

[0045] The training process of the student model is as follows: First, initialize the model structure and parameters. The teacher model is a complete diffusion model with fixed parameters; the student model adopts a simplified neural network architecture, such as a deeply compressed U-Net, to construct a cross-segment denoising mapping with a small number of parameters; the scoring auxiliary network is used to learn the true gradient pattern on the generation path of the teacher model. Subsequently, sample training pairs. In each round of training, randomly sample a pair of time periods from the set of key time steps , and obtain the corresponding feature pairs ; use the scoring auxiliary network to calculate the true gradient direction, construct a loss function, and guide the update of the student model parameters. Then, the parameters of the student model and the parameters of the scoring auxiliary network are jointly optimized through the above loss function for distillation training. After the distillation training is completed, export the student lightweight generation model containing only for deployment; in the inference stage, only perform sampling jumps at key time steps, significantly reducing the number of generation steps and computational costs; combined with the characteristics of the advertising image generation task, this lightweight model can quickly output high-quality images, meeting the dual requirements of efficiency and quality in actual business.

[0046] Compared with traditional distillation methods that only rely on static Gaussian approximation paths, by training only at key time steps, the student model can skip intermediate redundant processes, greatly improving the sampling speed and inference efficiency. Introducing the scoring auxiliary network effectively avoids the quality loss caused by Gaussian approximation deviation, enabling the student model to maintain accuracy in long-span predictions. The frequency factor in the loss function guides the model to pay more attention to the moments when the actual image structure changes violently, improving the model's ability to maintain the image semantic structure and local details. Finally, through the above distillation training process, the student model is finally obtained, and exported as a lightweight inference version. In actual use, only call on the set of key time steps to complete the efficient generation from random noise or text conditional encoding to the final advertising material.

[0047] The key time step-driven segmented distillation method proposed in this step significantly reduces the number of sampling steps while maintaining the generation quality, corrects the gradient deviation of the long-span denoising path through the scoring auxiliary network, and combines the frequency-based importance weighting mechanism, enabling the student model to accurately learn the jump mapping relationship in the latent space. The finally constructed lightweight diffusion generation model has the advantages of efficient inference and semantic fidelity, and is especially suitable for application scenarios that emphasize both generation speed and quality, such as advertising image generation.

[0048] In summary, this application loads a pre-trained diffusion model, performs full-time-step denoising inference from random noise to clear images, records the latent variables at each step, and constructs a latent feature evolution trajectory. The temporal features of each pixel channel in the trajectory are regarded as broken lines, the nodes that can best reflect the changes in image structure and details are extracted, and a set of key time steps is selected based on the node occurrence frequency. Using the true latent representations of the teacher diffusion model at the key time step pairs as supervision, a scoring auxiliary network is designed to correct the long-span approximation deviation. The lightweight student model is trained to accurately complete the denoising mapping from noise to image with only key jumps. The finally obtained student model, within very few sampling steps, relies on key time step extraction and segmented distillation to effectively skip redundant intermediate iterations, greatly reducing both the inference and training computational amounts, and ensuring the consistency of details and semantics with the help of the scoring network. Finally, while maintaining the visual quality of professional advertising materials, the generation efficiency is greatly improved.

[0049] Figure 2 FIG. is a structural block diagram of a diffusion model sampling and distillation system for generating advertising image materials provided by an embodiment of this application. The system at least includes the following modules: A trajectory sampling module, configured to load a pre-trained diffusion model and perform a complete diffusion sampling process, and collect a complete latent feature trajectory through multi-step denoising inference; A time step screening module, configured to perform broken line simplification analysis on the collected latent feature trajectory to generate a set of key time steps; A model distillation module, configured to perform segmented progressive distillation training based on the set of key time steps and the guidance of the teacher diffusion model to construct a lightweight student generation model dedicated to the advertising field.

[0050] For relevant details, refer to the above method embodiment.

[0051] Figure 3 FIG. is a block diagram of an electronic device provided by an embodiment of this application. The device at least includes a processor 401 and a memory 402.

[0052] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0053] The memory 402 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 402 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 401 to implement the diffusion model sampling and distillation method for advertising image material generation provided in the method embodiments of the present application.

[0054] In some embodiments, the electronic device may optionally further include: a peripheral device interface and at least one peripheral device. The processor 401, the memory 402, and the peripheral device interface may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface through a bus, signal lines, or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply, etc.

[0055] Of course, the electronic device may also include fewer or more components, and this embodiment does not limit this.

[0056] Optionally, the present application further provides a computer-readable storage medium, and a program is stored in the computer-readable storage medium, and the program is loaded and executed by the processor to implement the diffusion model sampling and distillation method for advertising image material generation in the above method embodiments.

[0057] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium. A program is stored in the computer-readable storage medium and is loaded and executed by a processor to implement the diffusion model sampling and distillation method for advertising image material generation in the above method embodiments.

[0058] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0059] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A diffusion model sampling and distillation method for advertising image material generation, characterized in that, The method includes: Loading a pre-trained diffusion model and performing a complete diffusion sampling process, and collecting a complete latent feature trajectory through multi-step denoising inference; Performing polyline simplification analysis on the collected latent feature trajectory to generate a set of key time steps; Based on the set of key time steps and the guidance of the teacher diffusion model, sampling segmental progressive distillation training to construct a student lightweight generation model dedicated to the advertising field.

2. The diffusion model sampling and distillation method for advertising image material generation according to claim 1, wherein The loading of the pre-trained diffusion model and performing a complete diffusion sampling process, and collecting a complete latent feature trajectory through multi-step denoising inference includes: Load the trained diffusion model and set the total number of time steps for the diffusion process , construct the initial latent variable as the starting input for the diffusion process, representing the initial noise state of the image, is defined as follows: ； Among them, and respectively represent the spatial height and width of the potential feature map, is the number of channels, represents the set of all real numbers; Execute multi-step denoising inference in the order of decreasing from to . In each time step , input the current latent variable and the corresponding time step information into the diffusion model. Through the forward propagation of the neural network, output the updated latent variable . After each step of inference is completed, save the latent features of the current time step . Record the latent variables of all time steps in sequence to form a latent feature trajectory with a length of .

3. The diffusion model sampling and distillation method for advertising image material generation according to claim 2, characterized in that, The performing of polyline simplification analysis on the collected latent feature trajectory includes: For each spatial position in the potential feature trajectory And each channel , extract its The corresponding eigenvalue sequence , as a set of two-dimensional points , where the horizontal axis is the time step , the ordinate is the corresponding eigenvalue, forming multiple broken lines; Applying the RDP algorithm to each polyline to calculate the perpendicular distance between the straight line between the endpoints of the polyline and each intermediate point. The calculation formula of the perpendicular distance is as follows: ； Among them, , , are two-dimensional coordinate vectors of the starting point, middle point, and ending point of the broken line respectively, represents the cross product of two-dimensional vectors, represents the modulus of the vector; If the maximum perpendicular distance exceeds a preset threshold, the corresponding time step points are retained, and the two ends of the polyline are recursively simplified until the maximum perpendicular distance of all points does not exceed the preset threshold.

4. The diffusion model sampling and distillation method for advertising image material generation according to claim 3, characterized in that The performing of polyline simplification analysis on the collected latent feature trajectory further includes: For each spatial position in the latent feature trajectory and each channel in the latent feature trajectory the sequence of changes is taken as the object of analysis, and the corresponding time-series broken line is constructed. Each broken line can be expressed as a set of points: ； For the start and end points of a certain polyline , , and any arbitrary point in between , its DNRD distance is defined as follows: ； Among them, , representing the feature difference between the frames before and after the midpoint; , representing the standard time step interval; , representing the overall feature difference between the first and last points; , representing the time span between the first and last points; If the maximum DNRD distance exceeds a preset threshold, the corresponding time step points are retained, and the two ends of the polyline are recursively simplified until the maximum DNRD distance of all points does not exceed the preset threshold.

5. The diffusion model sampling and distillation method for advertising image material generation according to claim 4, wherein, The generating of the set of key time steps includes: After the polyline simplification is completed, count the occurrence frequencies of all the retained time steps in all polylines ; Set the frequency threshold , ; Select the time steps that satisfy to form a set of key time steps , , where is the maximum value of the frequency, .

6. The diffusion model sampling and distillation method for advertising image material generation according to claim 2, characterized in that, The based on the set of key time steps and the guidance of the teacher diffusion model, sampling segmental progressive distillation training to construct a student lightweight generation model dedicated to the advertising field includes: The set of key time steps in the time interval is rearranged in chronological order and a series of time interval segments are constructed such that each pair satisfies and represents a denoising transition from the noise state to the target state; In distillation training, the teacher model provides intermediate feature pairs , the student model receives the features of the teacher model at time step and predicts the initial noise representation at its corresponding time step ; ;​ Introduce a scoring auxiliary network , and the training objective of the scoring network is to minimize the following weighted mean square error loss function: ； Among them, and respectively represent the latent representations of the teacher model at time steps and ; represents the mathematical expectation on the data pair generated by the teacher model; and respectively represent the noise scheduling coefficients at time steps and ; represents the estimated noise or gradient output by the student model at time step after receiving the input ; is the gradient of the student model parameter ; Weight function is defined as follows: ； Among them, represents the time step The frequency of being selected as a key node during the simplification process of the entire feature trajectory polyline.

7. The diffusion model sampling and distillation method for advertising image material generation according to claim 6, characterized in that, The training process of the student model includes: First, initialize the model structure and parameters. The teacher model is a complete diffusion model with fixed parameters, and the student model adopts a simplified neural network architecture. The scoring auxiliary network is used to learn the true gradient pattern on the generation path of the teacher model; Subsequently, sample training pairs, and randomly sample a pair of time periods from the set of key time steps in each round of training , and obtain the corresponding feature pair ; use the scoring auxiliary network to calculate the true gradient direction, construct a loss function, and guide the update of the student model parameters; The parameters of the student model and the parameters of the scoring auxiliary network are distilled and trained through a loss function. After the distillation training is completed, a lightweight learning generation model containing only is exported.

8. A diffusion model sampling and distillation system for advertising image material generation, characterized in that, Including: A trajectory sampling module for loading a pre-trained diffusion model and performing a complete diffusion sampling process, and collecting a complete latent feature trajectory through multi-step denoising inference; A time step screening module for performing polyline simplification analysis on the collected latent feature trajectory to generate a set of key time steps; A model distillation module for, based on the set of key time steps and the guidance of the teacher diffusion model, sampling segmental progressive distillation training to construct a student lightweight generation model dedicated to the advertising field.

9. An electronic device, characterized in that, The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a diffusion model sampling and distillation method for advertising image material generation according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored in the storage medium, and when the program is executed by a processor, it is used to implement a diffusion model sampling and distillation method for advertising image material generation according to any one of claims 1 to 7.

Images