Training method, device, equipment, medium and program of generation model

Abstract

This application provides a method, apparatus, device, medium, and program for training a generative model. This application relates to the field of generative model training technology. The method includes: responding to a generation command from a user terminal, acquiring a first image generated by the generative model to be trained based on a first time step information and a second image generated by the model to be trained based on a second time step information and a generation command; obtaining a boundary alignment loss based on the first and second images; and iteratively training the generative model to be trained based on the boundary alignment loss until the boundary alignment loss meets a preset condition, thereby obtaining a target generative model. This method aims to improve the generation efficiency of the generative model.

Description

Technical Field

[0001] This application relates to the field of generative model training, and more particularly to a method, apparatus, device, medium, and program for training a generative model. Background Technology

[0002] Generative models based on probabilistic flow ordinary differential equations (PF-ODE), such as diffusion models and flow matching, have become the mainstream technical paradigm in the field of image and video generation. The core idea of ​​these models is to synthesize samples by simulating the reverse evolution process from simple noise distributions to complex data distributions. However, to ensure high fidelity of the generated samples, these models typically rely on numerical solvers (such as the Euler method or DDIM) for dozens or even hundreds of iterative calculations. This serial sampling mechanism leads to significant inference latency, making it difficult to apply to scenarios with strict real-time requirements.

[0003] To address the aforementioned technical challenges, "distillation" has become a mainstream acceleration solution. Its core principle is to train a lightweight student model to mimic the multi-step behavior of a large teacher model, achieving high-quality generation in a very small number of steps (e.g., 1 to 4 steps). At the implementation level, the most representative solutions include Consistency Models (CMs / LCMs) and standard LoRA (Low-Rank Adaptation) techniques. Consistency Models attempt to force the learning of a function that directly maps any point on the trajectory to the starting point, but this strong constraint often leads to the loss of high-frequency details in the generated image, resulting in a blurry or overly smooth appearance. Standard LoRA, in current applications, primarily focuses on adjusting the specific features of the generated content; its design does not address the optimization of the model's internal "sampling dynamics," thus it cannot be directly used to accelerate inference. Therefore, existing technologies suffer from the technical problem of low generation efficiency in generative models. Summary of the Invention

[0004] This application provides generative model training methods, apparatus, devices, media, and programs to achieve the technical effect of improving the generation efficiency of generative models.

[0005] In a first aspect, embodiments of this application provide a method for training a generative model, comprising:

[0006] In response to the user's generation command, the system acquires the first image generated by the model to be trained based on the first time step information and the second image generated by the model to be generated based on the second time step information and the generation command, respectively; wherein, the time step information is used to characterize the current time scale when generating the image based on the generation command, and the second time step information is less than the first time step information;

[0007] The boundary alignment loss is obtained based on the first and second images;

[0008] The training model is iteratively trained based on the boundary alignment loss until the boundary alignment loss meets the preset conditions, thus obtaining the target generative model.

[0009] In one possible implementation, before acquiring the first image generated by the model to be generated from the first time step information based on the generation instruction and the second image generated by the model to be generated from the second time step information based on the generation instruction, the process includes:

[0010] Obtain the target image generated based on the generation instructions;

[0011] Obtain the total time step information for generating the target image; wherein, the total time step information is used to characterize the total time steps used to generate the target image based on the generation instructions;

[0012] The first time step information is determined based on the total time step information; wherein, the first time step information is less than or equal to the total time step information;

[0013] The second time step information is determined based on the first time step information; wherein the second time step information is located between the first time step information and the total time step information.

[0014] In one possible implementation, determining the second time step information based on the first time step information includes:

[0015] Obtain feature information from the first image;

[0016] The intervals of the first time step information and the total time step information are divided according to the feature information, and the intervals are divided into high noise intervals and low noise intervals.

[0017] Prioritize determining the second time step information in the high-noise region.

[0018] In one possible implementation, the boundary alignment loss is obtained based on the first image and the second image, including:

[0019] Obtain the original image from the user's generated command;

[0020] The first tensor distance between the first image and the original image, and the second tensor distance between the second image and the original image are determined respectively; wherein, the tensor distance is used to characterize the difference information between the two images;

[0021] The boundary alignment loss is obtained based on the first tensor distance and the second tensor distance.

[0022] In one possible implementation, a trajectory low-rank adaptive module is inserted into the generative model to be trained. The trajectory low-rank adaptive module is used to optimize parameters during iterative training.

[0023] In one possible implementation, before the boundary alignment loss satisfies a preset condition and the target generation model is obtained, the following steps are also included:

[0024] In each training iteration, the total loss is obtained based on the boundary alignment loss and the loss of the generative model to be trained during training.

[0025] After the boundary alignment loss meets the preset condition, it is determined whether the total loss is less than or equal to the preset threshold. If so, the target generation model is obtained.

[0026] If not, continue iteratively training the untrained generative model until the boundary alignment loss meets the preset conditions and the total loss is less than or equal to the preset threshold.

[0027] Secondly, embodiments of this application provide a training apparatus for a generative model, comprising:

[0028] The first processing module is used to respond to the generation command from the user terminal and acquire the first image generated by the model to be trained based on the generation command from the first time step information and the second image generated by the model to be generated based on the generation command from the second time step information; wherein, the time step information is used to characterize the current time scale when generating the image based on the generation command, and the second time step information is less than the first time step information;

[0029] The second processing module is used to obtain the boundary alignment loss based on the first image and the second image;

[0030] The third processing module is used to iteratively train the generative model to be trained based on the boundary alignment loss until the boundary alignment loss meets the preset conditions, thus obtaining the target generative model.

[0031] In one possible implementation, the first processing module is further configured to:

[0032] Obtain the target image generated based on the generation instructions;

[0033] Obtain the total time step information for generating the target image; wherein, the total time step information is used to characterize the total time steps used to generate the target image based on the generation instructions;

[0034] The first time step information is determined based on the total time step information; wherein, the first time step information is less than or equal to the total time step information;

[0035] The second time step information is determined based on the first time step information; wherein the second time step information is located between the first time step information and the total time step information.

[0036] In one possible implementation, the first processing module is further configured to:

[0037] Obtain feature information from the first image;

[0038] The intervals of the first time step information and the total time step information are divided according to the feature information, and the intervals are divided into high noise intervals and low noise intervals.

[0039] Prioritize determining the second time step information in the high-noise region.

[0040] In one possible implementation, the second processing module is further configured to:

[0041] Obtain the original image from the user's generated command;

[0042] The first tensor distance between the first image and the original image, and the second tensor distance between the second image and the original image are determined respectively; wherein, the tensor distance is used to characterize the difference information between the two images;

[0043] The boundary alignment loss is obtained based on the first tensor distance and the second tensor distance.

[0044] In one possible implementation, a trajectory low-rank adaptive module is inserted into the generative model to be trained. The trajectory low-rank adaptive module is used to optimize parameters during iterative training.

[0045] In one possible implementation, the third processing module is further configured to:

[0046] In each training iteration, the total loss is obtained based on the boundary alignment loss and the loss of the generative model to be trained during training.

[0047] After the boundary alignment loss meets the preset condition, it is determined whether the total loss is less than or equal to the preset threshold. If so, the target generation model is obtained.

[0048] If not, continue iteratively training the untrained generative model until the boundary alignment loss meets the preset conditions and the total loss is less than or equal to the preset threshold.

[0049] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0050] The memory stores computer-executed instructions;

[0051] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0052] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0053] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0054] The generative model training method, apparatus, device, medium, and program provided in this application first respond to the user's generation command and drive the generative model to be trained to generate images based on the information of the first time step and the information of the second time step, thereby obtaining a first image and a second image. A boundary alignment loss is calculated based on the first and second images. By iteratively updating the parameters of the generative model to be trained according to the boundary alignment loss, training is completed and a target generative model is obtained when the model meets a preset convergence condition. By forcing the first image generated in the first time step to align with the corresponding second image generated in the second time step, which is usually of higher quality, the boundary alignment loss effectively guides the model to learn how to more accurately predict or retain the final necessary details in the early stages of incomplete information. This provides a followable and effective technical path for achieving a high-quality and high-efficiency generative model. Through the constraint of boundary alignment, it ensures that the acceleration process does not sacrifice the fidelity of the generated content, thereby fundamentally solving the problem between generation speed and generation quality. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0056] Figure 1 A flowchart illustrating the training method for the generative model provided in this application. Figure 1 ;

[0057] Figure 2 A flowchart illustrating the training method for the generative model provided in this application. Figure 2 ;

[0058] Figure 3 A schematic diagram of the structure of the training device for the generative model provided in this application;

[0059] Figure 4 A hardware schematic diagram of the training device for the generative model provided in this application.

[0060] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of methods and approaches consistent with some aspects of this application as detailed in the appended claims.

[0062] In existing technologies, to improve the generation efficiency and effectiveness of generative models, a lightweight student model is typically constructed. This student model then mimics the behavior of a pre-trained, powerful teacher model through knowledge transfer. The goal is to compress the generation process, which requires hundreds of steps in the teacher model, into a very small number of steps (e.g., 1 to 4 steps) in the student model. In terms of specific technical approaches, two representative implementation methods have emerged:

[0063] One approach is the consistency model. The core of this scheme is imposing a very strong constraint: the model must be able to directly predict the starting point of the trajectory (i.e., clean data) regardless of where it starts from (i.e., any noise level). This "one-step" design is logically straightforward, aiming to achieve the ultimate single-step generation. However, its technical problem lies in the fact that this strong constraint ignores the inherent gradualism of the generation process itself. In high-noise regions, the semantic information contained in the data is extremely sparse. Forcing the model to directly predict the final details exceeds its reasonable learning capacity, causing the model to tend to output an "average," "safe" prediction result. This leads to a significant loss of high-frequency details, manifesting as an overly smooth overall generated image, blurred textures, and a lack of sharpness, severely affecting the visual fidelity of the generated image.

[0064] The second approach involves fine-tuning using standard LoRA techniques. This method involves introducing low-rank adapters into a pre-trained model, fine-tuning these newly added, minimally parameterized modules to adapt to the new task. However, in current practice, LoRA applications focus on "semantic fine-tuning" or "style transfer," i.e., changing the theme, style, or specific objects of the model-generated content. Its optimization goal is content-level feature transformation, without addressing the model's own evolutionary path from noise to data. Therefore, although the LoRA module itself is lightweight, standard usage cannot change the model's inherently slow sampling process, nor can it reduce the number of generation steps, thus failing to address the fundamental problem of improving inference speed.

[0065] In summary, while existing technologies aim to accelerate performance, their implementation methods all have inherent flaws: consistency models sacrifice generation quality for speed, while standard LoRA is simply unusable for acceleration. A deeper technical problem lies in the fact that these methods fail to effectively decouple and synergistically optimize "generation quality" and "generation efficiency." Furthermore, many acceleration strategies lack a solid theoretical foundation, relying on empirical loss function design and failing to rigorously demonstrate mathematically why their methods can guarantee quality while accelerating performance. This results in a lack of systematic guidance for technical optimization, making it difficult to fundamentally solve the efficiency dilemma faced by generative models in demanding application scenarios.

[0066] This invention aims to address the core problems of high-frequency detail loss and model semantic knowledge destruction in existing generative model acceleration techniques. Its solution is not a simple patching of the existing "distillation" framework, but rather the construction of a complete and innovative system encompassing theory, practice, methodology, and apparatus.

[0067] Traditional methods, which require models to directly predict clean data from noisy states, suffer from a strong constraint that is the root cause of image blurring and smoothing. To address this, this application introduces the theory of "error bound tightening," mathematically proving that using intermediate states on the generated trajectory as monitoring signals results in an error upper bound that is superior to directly predicting the endpoint. This provides a solid theoretical foundation for subsequent method design, transforming the optimization process from empirical exploration to a path guided by clear mathematical objectives.

[0068] Guided by theory, this application designs a training mechanism. The specific idea of ​​this mechanism is to proactively construct a more learnable optimization objective for the model during training. By randomly sampling an earlier, less noisy intermediate state point and using the model itself to generate a high-quality intermediate reference image, a distant, blurry ultimate target is transformed into a nearby, clear alignment target. The model is forced to approximate its output in less noisy states in detail with its output in noisier states. This essentially involves progressively "transferring" high-frequency details that are difficult to learn from the "clear" interval to the "blurry" interval, effectively avoiding the loss of detail caused by direct prediction.

[0069] To implement the above approach efficiently and flexibly, this application redefines the LoRA trajectory technique by proposing a "Low-Rank Adaptive Trajectory (LoRA) Module." This module is no longer used to learn new semantics or styles, but is specifically designed as a "trajectory optimizer." Its sole task is to learn how to geometrically correct the dynamic path of the generated model, i.e., "straighten" the trajectory. By integrating this module into a pre-trained base model and freezing the base model parameters, this application ensures that the model's original powerful generation capabilities are not compromised while simultaneously improving generation speed. Finally, an iterative optimization of the LoRA trajectory parameters is driven by a comprehensive objective function that integrates the base reconstruction loss and the boundary alignment loss, ensuring that while the model generates data rapidly, its output quality converges to a preset high standard.

[0070] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0071] Figure 1 A flowchart illustrating the training method for the generative model provided in this application. Figure 1 ,like Figure 1 As shown, the method includes:

[0072] S101. In response to the generation command from the user terminal, acquire the first image generated by the generator model to be trained based on the generation command from the first time step information and the second image generated by the generator model to be trained based on the generation command from the second time step information.

[0073] In this embodiment, the time step information is used to characterize the current time scale when generating the image based on the generation instruction, and the second time step information is less than the first time step information. Upon receiving the generation instruction from the user terminal, the model to be trained is driven to perform two forward computations. First, the model is set to generate at a first time step with more noise and less complete information, resulting in a first image with blurred details but a preliminary structure. Then, the model is set to generate at a second time step with less noise and richer information (this time step is earlier than the first time step), resulting in a relatively clear second image with richer details. This step creates a set of comparative samples with intrinsic quality differences for the model, concretizing the abstract generation process into image pairs with different levels of completion under the same instruction, providing a direct and structured data foundation for subsequent fine-tuning optimization.

[0074] S102. Obtain the boundary alignment loss based on the first image and the second image.

[0075] In this embodiment, after obtaining the first and second images, a certain difference metric between them is calculated to obtain the boundary alignment loss. This loss function is designed so that the first image does not directly reach the quality of the final perfect image, but rather aims to align with the higher-quality second image. By forcing the model to output a higher quality in the later, less noisy stages of generation, especially during periods of higher noise, the model is guided to learn how to more effectively preserve and reconstruct high-frequency details crucial to the final quality under information-constrained conditions.

[0076] S103. Iteratively train the generative model to be trained according to the boundary alignment loss until the boundary alignment loss meets the preset conditions to obtain the target generative model.

[0077] In this embodiment, the calculated boundary alignment loss is used as the core training signal, and the parameters of the generative model to be trained are iteratively adjusted through the backpropagation algorithm. Each iteration aims to reduce the quality gap between the first and second images. This process continues until the boundary alignment loss converges to below a preset threshold, indicating that the model has learned to generate images with quality close to that of later time steps in earlier time steps. At this point, training terminates, and the optimized target generative model is obtained. This step encodes the boundary alignment constraints into the model's parameters through continuous gradient optimization, ultimately producing a reinforced model. The core improvement of this model lies in the fact that its single-step or a few-step generation output can contain detailed information that would otherwise require more steps to accumulate, thereby significantly improving image fidelity while maintaining generation speed, achieving an effective balance between efficiency and quality.

[0078] The generative model training method provided in this application first responds to a user-side generation command, driving the generative model to be trained to execute two generation processes: one based on a first time step to generate a first image, and the other based on a second time step to generate a second image, wherein the generation process corresponding to the second time step information leads the first time step information in terms of time scale. Subsequently, using the boundary alignment loss between the first and second images as the optimization objective, the model parameters are iteratively adjusted to minimize this loss. Finally, when the boundary alignment loss reaches a preset convergence criterion, the trained target generative model is obtained. This significantly enhances the model's performance in a single or a few inference steps, effectively improving the fidelity and detail richness of the generative model's output without significantly increasing inference computational complexity. It provides an effective solution to the long-standing technical contradiction between generation efficiency and generation quality, making high-performance applications of generative models possible in resource-constrained environments.

[0079] Figure 2 A flowchart illustrating the training method for the generative model provided in this application. Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the examples, the training method for the generative model is described in detail, which includes:

[0080] S201. In response to the generation command from the user terminal, obtain the target image generated based on the generation command; obtain the total time step information of the generated target image; determine the first time step information based on the total time step information.

[0081] In this embodiment, a high-quality target image is first required. This is typically achieved by calling a mature and powerful pre-trained generative model (such as Stable Diffusion) to process the user's generation command (e.g., "a cat under the stars") with a sufficient number of sampling steps (e.g., 50 steps). This process ensures that the target image itself has rich details and high fidelity, serving as a reliable benchmark for subsequent evaluation of the generation quality. Subsequently, the total number of steps required to generate this target image is recorded, i.e., the total time step information (e.g., T=50). This value defines the scale of the complete generation process from pure noise to the final image.

[0082] After obtaining the total number of time steps, the key operation is to determine the first time step information based on this total number of steps. Here, "determining" doesn't mean fixing it to a single value, but rather randomly sampling a time step from the interval [0,T] (e.g., [0,50]) during training as the "first time step information" (e.g., t=35). This random sampling mechanism is crucial; it ensures that the model is adequately trained at different stages of the entire generation path (whether early, middle, or late), thus avoiding situations where the model only excels at handling specific noise levels and enhancing its robustness and generalization ability. Ultimately, this determined "first time step information" (t=35) will be used in subsequent steps to generate a relatively coarse first image that needs optimization.

[0083] This step, by explicitly defining the total time step information, constrains the training process within a standardized timeframe consistent with the actual generation process, preventing a disconnect between the training and inference phases. Dynamically and randomly determining the first time step information is equivalent to setting diverse training scenarios covering varying levels of difficulty for the model, forcing it to learn the ability to recover details from incomplete information of varying degrees. This lays a solid foundation for ultimately training a robust model that can consistently output high-quality results under diverse noise conditions.

[0084] S202. Obtain feature information of the first image; divide the intervals of the first time step information and the total time step information according to the feature information, and divide the intervals into high noise intervals and low noise intervals; preferentially determine the second time step information in the high noise interval.

[0085] In this embodiment, the first image of the generated result corresponding to the first time step information (e.g., t=35) determined in step S201 is first analyzed in real time to extract its key feature information. This feature information may include, but is not limited to, the frequency distribution of the image, the overall entropy value, or the deep feature map extracted by the pre-trained network. Based on these quantified feature indicators, the interval from the first time step information (t=35) to the total time step information (T=50) (i.e., [35, 50]) is intelligently divided into a "high noise interval" and a "low noise interval". For example, through analysis, it is found that between time step 35 and 40, the signal-to-noise ratio of the image is extremely low, the contour and texture are very blurry, and the uncertainty is high. Therefore, this sub-interval is marked as a high noise interval. Between time step 40 and 50, the main structure and most details of the image have stabilized, and the signal-to-noise ratio has significantly improved. Therefore, it is marked as a low noise interval.

[0086] After dividing the regions, a key strategy is implemented: prioritize random sampling within the high-noise region to determine the information k for the second time step. This means there is a higher probability of selecting a time point (e.g., k=38) from the high-noise region (e.g., [35,40]) rather than from the low-noise region. The underlying logic of this strategy is that the high-noise region is the stage where information is most uncertain during the generation process, making accurate predictions most difficult for the model, and it is also the region where traditional methods are most likely to lose high-frequency details. Therefore, prioritizing the distribution of training supervision signals (i.e., images from the clearer k-time step) into this high-noise region can most effectively guide the model to learn how to perform tasks in the early, information-scarce stages, and also lay a correct foundation for subsequent detail generation.

[0087] This step employs a dynamic region partitioning strategy based on image features, adaptively identifying and focusing on high-noise regions along the generation path for supervision. This ensures the model receives timely feedback where guidance is most needed, significantly enhancing its detail recovery and preservation capabilities under challenging conditions. This data-driven, biased sampling mechanism allows the entire training process to converge to a better solution more efficiently. The resulting model exhibits stronger robustness and higher output quality when dealing with generation tasks starting from high noise levels, effectively mitigating the oversmoothing problem commonly found in accelerated models.

[0088] S203. Obtain the first image generated by the generator model to be trained based on the first time step information and the second image generated by the generator model to be trained based on the second time step information and the generation instruction.

[0089] In this embodiment, this step utilizes the generative model to be trained, whose parameters already include the base model and the LoRA trajectory module being optimized, to perform two independent forward inference processes. In the first inference, the model is set to the relatively high-noise "first time step information" (e.g., t=35) determined in step S201, and a user command (e.g., "a clear photo of a cat") is input. Based on this, the model generates a preliminary first image with somewhat blurred details. Immediately following, a second inference is performed. This time, the model is set to the relatively low-noise "second time step information" (e.g., k=38) determined in step S202, and the exact same user command is input. Based on this, the model generates a clearer second image with richer details. These two generation processes share the same model weights and input commands; the only variable is the time step. This ensures that the second image should theoretically be a further, higher-quality, and natural continuation of the first image along the generation path.

[0090] This step, by fixing the model and instructions and varying only the time step, results in a first and second image that correspond in content. The difference between them stems purely from the depth of the generation process, not from deviations in model capability or semantic understanding. This highly controllable sample pair construction method allows the boundary alignment loss calculated in subsequent steps to accurately measure the quality gap between generation with "large step size" (from t to 0) and generation with "small step size" (from k to 0), thus providing a very clear and effective supervision signal for model parameter optimization. This step transforms the abstract concept of trajectory optimization into a concrete and operable data comparison task, serving as a crucial bridge connecting the time step sampling strategy with the final model performance improvement.

[0091] S204. Obtain the original image of the user-end generated instruction; determine the first tensor distance between the first image and the original image, and the second tensor distance between the second image and the original image; obtain the boundary alignment loss based on the first tensor distance and the second tensor distance.

[0092] In this embodiment, tensor distance is used to characterize the difference information between two images. This step requires obtaining an original image as a quality assessment benchmark. This is typically a high-fidelity result obtained during the preprocessing stage by a high-performance pre-trained model (such as a teacher model) processing the user's generation instruction (such as "a clear picture of a cat") with a large number of sampling steps (e.g., 50 steps). This image represents the ideal quality achievable under the current task. Subsequently, the first image and the second image, generated in step S203, representing the two images generated by the model from different starting points under the current parameters, are compared with this original image at the pixel level or feature level.

[0093] This comparison is achieved by calculating "tensor distances." For example, the absolute difference (L1 distance) or squared difference (L2 distance) of corresponding pixel values ​​between the first image and the original image is calculated pixel by pixel. All differences are then summed or averaged to obtain a scalar value, namely the first tensor distance. Similarly, the second tensor distance between the second image and the original image is calculated. These two distance values ​​quantify the difference between the model's generated results at time steps t and k and the ideal target, respectively. Finally, a "boundary alignment loss" is constructed based on these two tensor distances. The core logic is to calculate a difference function between the first and second tensor distances. This loss function only produces a significant gradient when the model's first tensor distance differs from its second tensor distance.

[0094] This step introduces the original image as an objective benchmark and compares it with the model's output at different generation stages. The boundary alignment loss function can keenly capture the performance gap between "large stride" generation (directly from t to the endpoint) and "small stride" generation (from k to the endpoint). This design forces the model to strive to make its output quality approach the level it can achieve under better conditions (small stride) when performing fast, large stride inference, thereby effectively "distilling" or "pushing forward" high-quality detail information from the later stages of the generated trajectory to the earlier stages. Ultimately, this allows the model to significantly improve the detail fidelity and overall quality of the output image while maintaining fast generation, successfully mitigating the performance degradation caused by reducing the number of sampling steps.

[0095] S205. Iteratively train the generative model to be trained based on the boundary alignment loss. In each training iteration, obtain the total loss based on the boundary alignment loss and the loss of the generative model to be trained during training. After the boundary alignment loss meets the preset condition, determine whether the total loss is less than or equal to the preset threshold. If yes, obtain the target generative model. If no, continue iteratively training the generative model to be trained until the boundary alignment loss meets the preset condition and the total loss is less than or equal to the preset threshold.

[0096] In this embodiment, the design principle of this step is to adopt a two-stage, comprehensive convergence judgment strategy to ensure that the final model not only performs well in the specific "boundary alignment" task, but also maintains a high level of overall generation capability.

[0097] Specifically, the generative model enters an iterative process. In each iteration, instead of updating the model solely based on the core metric—boundary alignment loss—it also considers a more fundamental and general training loss, such as a reconstruction loss that measures the basic similarity between the generated and original images. The boundary alignment loss and this basic training loss are weighted and summed to obtain a global total loss, which is then used as the basis for backpropagation and parameter optimization. The significance of this approach is that it prevents the model from deviating from the most fundamental learning objective of the generative model—the correctness of the generated content in terms of overall structure and semantics—in order to excessively satisfy the constraints of boundary alignment.

[0098] After training has continued for a period of time, the first step is to check whether the boundary alignment loss has decreased and stabilized below a preset acceptable threshold, indicating that the model has initially learned how to straighten its generated trajectory. However, this step does not immediately terminate training at this point; instead, it checks whether the global total loss has also decreased to a lower preset threshold. If the total loss has not yet reached the target, training will continue even if the boundary alignment loss has converged.

[0099] This step effectively avoids the risk of the model getting trapped in local optima, i.e., excelling only at boundary alignment but suffering a decline in basic generation quality. This dual-protection mechanism compels the model to maintain a baseline of generation quality while optimizing its sampling dynamics, ultimately producing a practical model that not only performs exceptionally well in accelerated scenarios but also possesses robust and reliable generation capabilities. This enables the target generation model obtained by this invention to truly meet the complex and ever-changing demands of real-world applications, achieving a high-quality balance between acceleration and generation quality.

[0100] One possible implementation involves inserting a trajectory low-rank adaptive module into the generative model to be trained. This low-rank adaptive module optimizes the parameters during iterative training. First, a pre-trained, high-performance base generative model (e.g., a complete Stable Diffusion model) is prepared. The structure and parameters of this base model are fixed. Internally, instead of modifying any existing layers or connections, an additional, trainable "trajectory low-rank adaptive module" is inserted. This module typically exists as a lightweight, low-rank matrix that can be cleverly integrated into key layers of the base model (such as attention mechanisms or convolutional layers) to interact with the base model's parameters.

[0101] Throughout the training process (as in steps S201-S205), all the original parameters of the base model are frozen, meaning that these parameters remain unchanged during backpropagation and parameter updates. All gradient calculations and parameter optimizations are performed only for this newly inserted "trajectory low-rank adaptive module." The parameters of this module are randomly initialized at the beginning of training, and then it learns how to fine-tune the behavior of the base model by optimizing objectives such as boundary alignment loss.

[0102] Figure 3 A schematic diagram of the structure of the training device for the generative model provided in this application is shown below. Figure 4 As shown, the training device 30 for the generative model provided in this embodiment includes:

[0103] The first processing module 301 is used to respond to the generation command from the user terminal and acquire the first image generated by the model to be trained based on the generation command from the first time step information and the second image generated by the model to be trained based on the generation command from the second time step information; wherein, the time step information is used to characterize the current time scale when generating the image based on the generation command, and the second time step information is less than the first time step information.

[0104] The second processing module 302 is used to obtain the boundary alignment loss based on the first image and the second image;

[0105] The third processing module 303 is used to iteratively train the generative model to be trained based on the boundary alignment loss until the boundary alignment loss meets the preset conditions, thereby obtaining the target generative model.

[0106] In one possible implementation, the first processing module 301 is further configured to:

[0107] Obtain the target image generated based on the generation instructions;

[0108] Obtain the total time step information for generating the target image; wherein, the total time step information is used to characterize the total time steps used to generate the target image based on the generation instructions;

[0109] The first time step information is determined based on the total time step information; wherein, the first time step information is less than or equal to the total time step information;

[0110] The second time step information is determined based on the first time step information; wherein the second time step information is located between the first time step information and the total time step information.

[0111] In one possible implementation, the first processing module 301 is further configured to:

[0112] Obtain feature information from the first image;

[0113] The intervals of the first time step information and the total time step information are divided according to the feature information, and the intervals are divided into high noise intervals and low noise intervals.

[0114] Prioritize determining the second time step information in the high-noise region.

[0115] In one possible implementation, the second processing module 302 is further configured to:

[0116] Obtain the original image from the user's generated command;

[0117] The first tensor distance between the first image and the original image, and the second tensor distance between the second image and the original image are determined respectively; wherein, the tensor distance is used to characterize the difference information between the two images;

[0118] The boundary alignment loss is obtained based on the first tensor distance and the second tensor distance.

[0119] In one possible implementation, a trajectory low-rank adaptive module is inserted into the generative model to be trained. The trajectory low-rank adaptive module is used to optimize parameters during iterative training.

[0120] In one possible implementation, the third processing module 303 is further configured to:

[0121] In each training iteration, the total loss is obtained based on the boundary alignment loss and the loss of the generative model to be trained during training.

[0122] After the boundary alignment loss meets the preset condition, it is determined whether the total loss is less than or equal to the preset threshold. If so, the target generation model is obtained.

[0123] If not, continue iteratively training the untrained generative model until the boundary alignment loss meets the preset conditions and the total loss is less than or equal to the preset threshold.

[0124] The training device for the generative model provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0125] Figure 4 A hardware schematic diagram of the training device for the generative model provided in this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0126] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0127] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0128] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0129] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0130] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0131] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0132] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0133] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0134] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0135] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, methods, or units, and may be electrical, mechanical, or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0138] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0140] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for training a generative model, characterized in that, include: In response to the user's generation command, the system acquires a first image generated by the model to be trained based on the generation command from the first time step information and a second image generated by the model to be generated based on the generation command from the second time step information; wherein, the time step information is used to characterize the current time scale when generating the image based on the generation command, and the second time step information is less than the first time step information; The boundary alignment loss is obtained based on the first image and the second image; The target generative model is iteratively trained based on the boundary alignment loss until the boundary alignment loss meets the preset conditions, thereby obtaining the target generative model.

2. The method according to claim 1, characterized in that, Before acquiring the first image generated by the model to be generated from the first time step information based on the generation instruction and the second image generated by the model to be generated from the second time step information based on the generation instruction, the process includes: Obtain the target image generated based on the generation instructions; Obtain the total time step information for generating the target image; wherein, the total time step information is used to characterize the total time steps used to generate the target image based on the generation instruction; First time step information is determined based on the total time step information; wherein, the first time step information is less than or equal to the total time step information; The second time step information is determined based on the first time step information; wherein the second time step information is located within the range of the first time step information and the total time step information.

3. The method according to claim 2, characterized in that, Determining the second time step information based on the first time step information includes: Obtain the feature information of the first image; Based on the feature information, the intervals of the first time step information and the total time step information are divided into high noise intervals and low noise intervals. The second time step information is determined preferentially in the high-noise region.

4. The method according to claim 1, characterized in that, The step of obtaining the boundary alignment loss based on the first image and the second image includes: Obtain the original image from which the generation command for the user terminal was generated; A first tensor distance between the first image and the original image, and a second tensor distance between the second image and the original image are determined respectively; wherein, the tensor distance is used to characterize the difference information between the two images; The boundary alignment loss is obtained based on the first tensor distance and the second tensor distance.

5. The method according to claim 1, characterized in that, A low-rank trajectory adaptive module is inserted into the generative model to be trained. The low-rank trajectory adaptive module is used to optimize parameters during the iterative training.

6. The method according to any one of claims 1-5, characterized in that, Before the boundary alignment loss satisfies the preset condition and the target generation model is obtained, the following steps are also included: In each training iteration, the total loss is obtained based on the boundary alignment loss and the loss of the generator model to be trained during training. After the boundary alignment loss meets the preset condition, it is determined whether the total loss is less than or equal to the preset threshold. If so, the target generation model is obtained. If not, the training of the model to be trained continues iteratively until the boundary alignment loss meets the preset conditions and the total loss is less than or equal to the preset threshold.

7. A training device for a generative model, characterized in that, include: The first processing module is configured to respond to a generation command from the user terminal by acquiring a first image generated by the model to be trained based on the generation command from a first time step information and a second image generated by the model to be generated based on the generation command from a second time step information; wherein, the time step information is used to characterize the current time scale when generating the image based on the generation command, and the second time step information is less than the first time step information; The second processing module is used to obtain the boundary alignment loss based on the first image and the second image; The third processing module is used to iteratively train the generator model to be trained based on the boundary alignment loss until the boundary alignment loss meets the preset conditions, thereby obtaining the target generator model.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images