Anti-homogenization training method and device for diffusion model

Abstract

The present application relates to the technical field of image generation, in particular to a method and device for anti-homogenization training of a diffusion model, wherein the method comprises: obtaining a group of images generated by the same prompt word, and constructing a distance matrix of the group of images; performing spectral clustering on the distance matrix to divide the group of images into different semantic clusters; adaptively assigning an exploration reward according to the size of each semantic cluster to obtain a plurality of semantic clusters after the exploration reward; and inputting the plurality of semantic clusters after the exploration reward into the diffusion model after structure perception regularization to obtain a group of images after generalization. Thus, the mode collapse and other problems caused by the existing group relative strategy optimization method in image generation are solved.

Description

Technical Field

[0001] This invention relates to the field of image generation technology, and in particular to an anti-homogenization training method and apparatus for diffusion models. Background Technology

[0002] Image generation techniques based on reinforcement learning (RL), particularly those based on the GRPO (Group Relative Policy Optimization) paradigm, primarily focus on improving the quality of generated images by optimizing the policy model to better align with human aesthetic preferences. Their technical solutions are mainly reflected in two aspects: Adaptation and Improvement of Policy Optimization Algorithms: Researchers have adapted RL (Reinforcement Learning) algorithms (such as Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO)) successfully applied to Large Language Models (LLMs) to diffusion and flow models for preference alignment and task-specific optimization. This trend has extended to flow-based models. Flow-GRPO, a group relative policy optimization framework based on flow matching, and DanceGRPO, a group relative policy optimization framework in the field of visual generation, integrate GRPO-style policy updates into flow matching, transforming deterministic Ordinary Differential Equation (ODE) sampling into Stochastic Differential Equation (SDE) formulas to introduce exploration noise. MixGRPO proposes a hybrid ODE-SDE sampling strategy to improve training efficiency. Flow-CPS addresses the noise inconsistency problem in SDE sampling by introducing a noise consistency scheme to achieve more accurate reward estimation. To address the challenge of reward sparsity in multi-step trajectories, TempFlow-GRPO goes beyond simply assigning a single global reward. BranchGRPO introduces a tree-like branching mechanism that allows multiple trajectories to share a prefix and branch off in intermediate steps, enabling dense, hierarchical reward fusion.

[0003] Preliminary explorations into pattern collapse: Existing research on preventing pattern collapse in generative models (especially LLMs) can be broadly categorized into two types: (a) Sample / token selection: For example, DivPO selects preferred sample pairs online, using "high-quality and rare" generated outputs as positive examples and "common but low-quality" results as negative examples, improving diversity while maintaining generation quality. Cui et al. addressed the entropy collapse problem by pruning action probabilities and applying KL regularization to tokens with high covariance. (b) Integrating multiple reward signals: For example, CPO modularly injects multiple creative dimensions into the preference optimization objective. In the field of image generation, Astolfi et al. used Pareto front comparison models and found that the new models had stronger consistency and realism but insufficient diversity; DiADM decoupled diversity and quality by introducing a diversity-aware module with pseudo-unconditional feature inputs; Ding et al. utilized human judgment of similarity, combined with latent spatial projection and contrastive learning to progressively infer diversity indicators.

[0004] While the aforementioned existing technologies have made efforts to improve image generation quality or explore diversity to some extent, such as Figure 1 As shown, the generated results tend to be homogenized (nearly identical character appearances, highly similar perspectives, etc.), limiting the model's application in creative scenarios. Pattern collapse has two main causes: (1) The short-sightedness of reward modeling: such as Figure 2 As shown in the left figure, existing GRPO methods heavily rely on single-sample quality as a reward signal. The reward model assigns an independent score to each image without considering the distributional relationships between generated samples under the same cue word. This makes the generative model "short-sighted" and "conservative," tending to overfit to a small subset of high-reward patterns while suppressing the exploration of creative or unconventional outputs. From a dynamic perspective, this triggers a self-reinforcing selection process, ultimately resulting in only the dominant high-reward patterns surviving, driving the model towards a unimodal distribution and causing a collapse in visual diversity.

[0005] (2) Mismatch between regularization and generation dynamics: such as Figure 2 As shown in the right figure, the uniform KL regularization introduced by existing methods fails to account for the uneven impact of the diffusion process itself on diversity. Research indicates that the early stages of the denoising trajectory (the first third of the steps) have a decisive influence on the diversity of the final output (accounting for approximately 66% of the total diversity change), while the later stages primarily refine visual quality. However, the standard KL penalty term, due to the variance term in its form, is insufficiently constrained in the early stages where diversity is crucial (high variance, weak penalty), while being over-constrained in the later refinement stages (low variance, strong penalty). This structural mismatch accelerates pattern extinction and limits the achievable quality-diversity tradeoff.

[0006] In summary, most existing GRPO-based methods primarily focus on improving the efficiency of policy optimization, neglecting the mode collapse problem. Even though some works (such as DivPO and CPO) have begun to address diversity, they either primarily target LLMs or introduce significant complexity when applied to image generation (such as the multi-stage training of DiADM and the method of Ding et al.), failing to propose an efficient and integrated solution from the fundamental perspectives of reward modeling and generative dynamics. Summary of the Invention

[0007] This invention provides a method and apparatus for anti-homogenization training of diffusion models to solve problems such as mode collapse caused by existing group relative strategy optimization methods in image generation.

[0008] A first aspect of the present invention provides a method for anti-homogenization training of a diffusion model, comprising the following steps: Obtain a set of images generated from the same prompt word, and construct a distance matrix for the set of images; Spectral clustering is performed on the distance matrix to divide the set of images into different semantic clusters; Exploration rewards are adaptively allocated based on the size of each semantic cluster to obtain semantic clusters with multiple exploration rewards; The semantic clusters obtained after the multiple exploration rewards are input into the structure-aware regularized diffusion model to obtain a set of generalized images.

[0009] Optionally, the step of obtaining a set of images generated from the same prompt word and constructing a distance matrix for the set of images includes: Obtain a set of images generated from the same prompt word; Calculate the perceptual distance between each pair of images in the set of images to form the distance matrix.

[0010] Optionally, performing spectral clustering on the distance matrix to divide the set of images into different semantic clusters includes: Construct an affinity matrix based on the distance matrix; Calculate the normalized Laplace matrix based on the affinity matrix; Spectral clustering is performed on the normalized Laplacian matrix to divide the set of images into different semantic clusters.

[0011] Optionally, the step of inputting the semantic clusters after the multiple exploration rewards into the structure-aware regularized diffusion model to obtain a generalized set of images includes: The early and late denoising steps of the initial diffusion model are determined based on the time threshold. The early denoising step is subject to Wasserstein distance constraints, and the later steps are deregulated to obtain the structure-aware regularized diffusion model. The semantic clusters obtained after the multiple exploration rewards are input into the structure-aware regularized diffusion model to obtain the generalized set of images.

[0012] A second aspect of the present invention provides an anti-homogenization training device for a diffusion model, comprising: The acquisition and construction module is used to acquire a set of images generated by the same prompt word and construct a distance matrix of the set of images; The clustering and partitioning module is used to perform spectral clustering on the distance matrix to divide the set of images into different semantic clusters; The reward allocation module is used to adaptively allocate exploration rewards based on the size of each semantic cluster, so as to obtain semantic clusters with multiple exploration rewards; The regularization module is used to input the semantic clusters after the multiple exploration rewards into the structure-aware regularized diffusion model to obtain a set of generalized images.

[0013] Optionally, the acquisition and construction module includes: The acquisition unit is used to acquire a set of images generated by the same prompt word; The first calculation unit is used to calculate the perceptual distance between each pair of the set of images to form the distance matrix.

[0014] Optionally, the clustering and partitioning module includes: Construction unit, used to construct affinity matrix based on distance matrix; The second calculation unit is used to calculate the normalized Laplace matrix based on the affinity matrix. A partitioning unit is used to perform spectral clustering on the normalized Laplacian matrix to divide the set of images into different semantic clusters.

[0015] Optionally, the regularization module includes: The determination unit is used to determine the early and late denoising steps of the initial diffusion model based on a time threshold. A constraint unit is used to apply Wasserstein distance constraints to the early denoising steps and to remove regularization constraints to the later steps, so as to obtain the structure-aware regularized diffusion model. The generalization unit is used to input the multiple semantic clusters after exploration rewards into the structure-aware regularized diffusion model to obtain a set of generalized images.

[0016] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the anti-homogenization training method for a diffusion model as described in the above embodiments.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for anti-homogenization training of a diffusion model.

[0018] The proposed anti-homogenization training method and apparatus for diffusion models in this invention employs a distribution-level reward mechanism that can perceive and incentivize the semantic diversity of generated samples, fundamentally altering the optimization dynamics and encouraging the model to explore novel visual patterns. It also designs a regularization method that matches the structure of the diffusion process, allocating constraint resources primarily to the critical early stages of diversity formation while relaxing restrictions in later refinement stages to ensure both image quality and reward optimization efficiency. By overcoming the trade-offs of existing technologies, this invention seeks to significantly enhance the semantic diversity of generated results while maintaining or even improving image quality, establishing a new and superior Pareto front for image generation based on the GRPO algorithm.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 The diagram illustrates the pattern collapse, where (a) is a high-quality image generated by the image generation model trained on GRPO that is prone to pattern collapse (such as similar faces or single shooting angles in the generated images), (b) is a high-quality and diverse image generated by the image generation model of the present invention, (c) is a comparison diagram of the training process of related technologies and the present invention, and (d) is a comparison diagram of the feature density of related technologies and the present invention. Figure 2 The diagram illustrates the causes of pattern collapse, where (a) shows the collapse of the policy model into the high-reward pattern due to single-sample reward modeling, and (b) shows that traditional regularization methods ignore the dominant role of early denoising in maintaining diversity. Figure 3 A flowchart illustrating an anti-homogeneity training method for a diffusion model according to an embodiment of the present invention; Figure 4This is a schematic diagram illustrating the specific execution of an anti-homogeneity training method for a diffusion model according to an embodiment of the present invention, wherein (a) is a schematic diagram of a distributed creativity reward mechanism based on semantic grouping, and (b) is a schematic diagram of a structure-aware regularization method; Figure 5 This is a schematic diagram comparing the effects of generating images according to an embodiment of the present invention and a baseline method, provided by an embodiment of the present invention. Figure 6 This is a schematic diagram of an ablation study on the Pareto frontier of quality and diversity for different modules and parameters according to an embodiment of the present invention, wherein (a) is the contribution of each module, (b) is the creativity reward coefficient, and (c) is the structure-aware regularization coefficient. Figure 7 According to an embodiment of the present invention, (a) is the effect of regularization and reward mechanisms on quality, and (b) is the effect of regularization and reward mechanisms on diversity; Figure 8 This is an example of sampling visualization during the training process provided according to an embodiment of the present invention; Figure 9 This is a block diagram of an anti-homogenization training device for a diffusion model according to an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

[0021] Explanation of reference numerals in the attached figures: 90 - Anti-homogenization training device for diffusion model, 901 - Acquisition and construction module, 902 - Clustering and partitioning module, 903 - Reward allocation module, 904 - Regularization module, 1001 - Memory, 1002 - Processor, 1003 - Communication interface. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0023] The following describes, with reference to the accompanying drawings, an anti-homogenization training method and apparatus for diffusion models according to embodiments of the present invention. Addressing the problem mentioned in the background art that focuses only on single-sample quality rewards and employs uniform KL regularization, leading to pattern collapse during image generation, the present invention provides an anti-homogenization training method for diffusion models. This method innovatively designs reward modeling and generation dynamics. In reward modeling, images generated from the same text description are spectral clustered, and then exploratory rewards are allocated based on the cluster size to incentivize the model to discover novel visual patterns. In the structure-aware regularization method, strong constraints are applied in the initial denoising stage to maintain sample diversity, and the constraints are gradually relaxed in subsequent stages, thereby improving the effectiveness of reward optimization.

[0024] Specifically, Figure 3 This is a flowchart illustrating an anti-homogenization training method for a diffusion model provided in an embodiment of the present invention.

[0025] like Figure 3 As shown, the anti-homogenization training method for the diffusion model includes the following steps: In step S301, a set of images generated by the same prompt word are obtained, and a distance matrix of the set of images is constructed.

[0026] In some embodiments, a set of images generated from the same prompt word is obtained, and a distance matrix of the set of images is constructed, including: Get a set of images generated from the same prompt word; Calculate the perceptual distance between each pair of images in a set of images to form a distance matrix.

[0027] like Figure 4 As shown in (a), in the actual execution process, a set of images generated by the same prompt word are acquired, and the DreamSim model is used to calculate the perceptual distance between each pair of images in the set to form a distance matrix. Among them, the DreamSim model can better align with human visual similarity judgment.

[0028] In step S302, spectral clustering is performed on the distance matrix to divide a set of images into different semantic clusters.

[0029] In some embodiments, spectral clustering is performed on the distance matrix to divide a set of images into different semantic clusters, including: Construct the affinity matrix based on the distance matrix; Calculate the normalized Laplace matrix based on the affinity matrix; Spectral clustering is performed on the normalized Laplacian matrix to divide a set of images into different semantic clusters.

[0030] like Figure 4As shown in (a), in the actual execution process, an affinity matrix is ​​constructed based on the distance matrix, and a normalized Laplacian matrix is ​​calculated based on the affinity matrix. Spectral clustering is then performed by calculating the normalized Laplacian matrix to divide the image into different semantic clusters, where each cluster represents a unique visual pattern.

[0031] Furthermore, embodiments of the present invention can also use other unsupervised clustering algorithms to replace spectral clustering to achieve semantic grouping of the generated images. For example, density-based clustering (such as DBSCAN) can automatically identify clusters of arbitrary shapes and can treat noise points (extremely rare patterns) as independent micro-clusters, thus giving them high rewards. Alternatively, hierarchical clustering can be used: different granular semantic groupings can be obtained by setting different distance thresholds, providing more flexible pattern partitioning. A similar effect can be achieved: these methods can also identify different visual pattern groups in the generated samples, thereby realizing the transformation from "single-sample reward" to "distribution-level reward" and adjusting the reward according to the "rarity" of the sample in the distribution.

[0032] In step S303, exploration rewards are adaptively allocated according to the size of each semantic cluster to obtain multiple semantic clusters with exploration rewards.

[0033] like Figure 4 As shown in (a), during actual execution, exploration rewards are adaptively allocated based on the size of each semantic cluster to obtain semantic clusters with multiple exploration rewards. For clusters with fewer samples (representing rare or novel patterns), samples within them will receive higher exploration rewards. ,in, Cluster size The total number of samples. The final reward for each semantic cluster. Original quality reward Weighted sum of exploration rewards ,Right now ,in, This is the weighting function.

[0034] This invention modifies the existing GRPO's singular and short-sighted reward signal, elevating the reward from the single-sample level to the distribution level. This proactively incentivizes the model to explore and maintain diverse visual patterns, directly counteracting the self-reinforcing selection dynamics that lead to pattern collapse. Furthermore, unlike related technologies that only use quality reward models (such as PickScore) for scoring, this invention introduces distribution-aware creativity rewards. It moves beyond viewing each sample in isolation, instead using cluster analysis to understand the semantic structure of the entire generated set and providing an "exploration bonus" for rare patterns, fundamentally changing the model's optimization objective.

[0035] In step S304, multiple semantic clusters after exploration rewards are input into the structure-aware regularized diffusion model to obtain a set of generalized images.

[0036] In some embodiments, multiple semantic clusters after exploration rewards are input into a structure-aware regularized diffusion model to obtain a generalized set of images, including: The early and late denoising steps of the initial diffusion model are determined based on the time threshold. Wasserstein distance constraints are applied to the early denoising steps, and regularization constraints are removed for the later steps to obtain a diffusion model with structure-aware regularization. Multiple semantic clusters after exploration rewards are input into a structure-aware regularized diffusion model to obtain a generalized set of images.

[0037] like Figure 4 As shown in (b), in actual implementation, this embodiment of the invention redesigned the regularization constraint for the characteristics of the diffusion model denoising process. Its core idea is to allocate the regularization "budget" according to the degree of influence of the denoising step on diversity.

[0038] Specifically, such as Figure 2 As shown in (b), the early denoising steps (approximately the first 1 / 3) have a decisive impact on the diversity of the final image (contributing approximately 66% of the diversity variation), while later steps primarily refine the quality. Therefore, this embodiment of the invention abandons the traditional uniform KL penalty and targets the critical early denoising steps ( ,in, Using a time threshold, a stronger Wasserstein distance constraint is adopted, and the variance denominator term in the KL divergence that has a weak constraint on early steps is removed, resulting in the following form:

[0039] in, The loss is the Wasserstein distance constraint. For the current model in The denoising results of the steps As a reference model The denoising results of the steps.

[0040] For later steps ( If the regularization constraint is completely removed, the model can be freely optimized to obtain higher quality images.

[0041] Thus, the structure-aware regularization of this invention ensures that strong constraints are applied during the critical period of diversity formation (early denoising) to maintain the breadth of semantic patterns; and that the constraints are removed during the quality optimization period (late denoising) to guarantee image fidelity. This achieves accurate and efficient allocation of the regularization "budget".

[0042] Furthermore, unlike related technologies that apply uniform KL divergence constraints throughout the denoising process, the embodiments of this invention employ stage-dependent, structure-aware regularization. It recognizes the uneven sensitivity to diversity at different stages of the diffusion process and makes dynamic adjustments accordingly, thus solving the mismatch problem between the original regularization and the generation dynamics.

[0043] Alternatively, different functions can be used in embodiments of the present invention to schedule the weight coefficients of the KL regularization term. It is not a simple step function with "strong constraints in the early stage and no constraints in the later stage". For example: Monotonically decreasing weighting coefficients: Design a weighting coefficient that changes with time step t Increasing and monotonically decreasing weight function For example, linear decay or exponential decay. This can also achieve the effect of applying strong constraints in the early stages and weakening the constraints in the later stages.

[0044] Content-based adaptive weights: dynamically adjust the regularization strength based on the variance or entropy of the current potential features, and strengthen constraints when feature uncertainty is high (usually in the early stages).

[0045] These proposals all follow the core insight that “early steps are critical to diversity” and exert greater influence at key stages to protect diversity by changing the allocation of regularization budgets.

[0046] The following five specific embodiments illustrate in detail the anti-homogenization training method for diffusion models proposed in this invention.

[0047] Example 1 compares the image generation effects of the anti-homogenization training method for diffusion models proposed in this invention with those of the baseline method, such as... Figure 5 As shown, in the qualitative experiment on diversity, the baseline method suffers from pattern collapse when generating main content (such as facial features, pose, font color, etc.), while the embodiment of the present invention achieves greater diversity and creativity while maintaining image quality and consistency with text description.

[0048] Example 2: The anti-homogenization training method for diffusion models proposed in this invention and the baseline method are respectively subjected to ablation studies on the Pareto frontier of quality and diversity for different modules and parameters, such as... Figure 6As shown, the contribution, creativity reward coefficient, and structure-aware regularization coefficient of each module in this embodiment of the invention are better than the baseline method.

[0049] Example 3 compares the rate of decrease in diversity during the training process of the anti-homogenization training method for diffusion models proposed in this invention with that of the baseline method, such as... Figure 7 As shown, the quality score of the implementation proposed in this embodiment is comparable to the baseline method, but the rate of decrease in diversity proposed in this embodiment is significantly slower.

[0050] Example 4 compares the sampling visualization during the training process of the anti-homogenization training method for the diffusion model proposed in this embodiment of the invention with that of the baseline method, such as... Figure 8 As shown, thanks to the reward mechanism for innovative samples in the embodiments of the present invention, the model can still generate diverse samples in the later stages of training, while the baseline method has difficulty in mining innovative samples.

[0051] Example 5, as shown in the experimental data in the table below, demonstrates that the method of this embodiment exhibits significant advantages across multiple backbone and reward model configurations. In the SD3.5-M / Pickscore combination, DreamSim diversity is improved by 18.8%, FID by 23.3%, BFID by a significant 184.2%, and SSIM structural similarity by 25.6%. Under the Flux.1-dev / Pickscore configuration, all metrics maintain steady growth, with DreamSim improving by 13.9% and FID by 9.1%. Particularly noteworthy is the SD3.5-M / HPSv3 setting, where BFID achieves a considerable improvement of 69.4%, while maintaining balanced improvements in other quality metrics. All experimental configurations confirm that our method, under similar quality conditions, comprehensively outperforms the Flow-GRPO benchmark in generating image diversity.

[0052]

[0053] In summary, the anti-homogenization training method for diffusion models proposed according to embodiments of the present invention has the following beneficial effects: (1) Solving the problem of pattern convergence caused by single-sample reward modeling: A distribution-level reward mechanism is designed to perceive and incentivize the semantic diversity of generated samples, fundamentally changing the optimization dynamics and encouraging the model to explore novel visual patterns rather than just using known high-scoring patterns. This solves the problem that the existing GRPO method relies on single-sample reward signals, causing the model to converge to a few safe patterns in pursuit of short-term high rewards. (2) Correcting the mismatch between regularization budget and diffusion generation dynamics: A regularization method matching the structure of the diffusion process is designed, which can allocate the constraint resources to the key early stage of diversity formation, while relaxing the restrictions in the later refinement stage to ensure image quality and reward optimization efficiency. This solves the problem that the existing uniform KL regularization fails to consider the decisive influence of the early denoising steps on diversity, resulting in insufficient constraints in the key stage and excessive constraints in the secondary stage. (3) Establish a better Pareto front for quality-diversity trade-off: By breaking through the trade-off bottleneck of existing technologies, we seek to significantly improve the semantic diversity of the generated results while maintaining or even improving image quality. This will establish a new and better Pareto front for image generation based on the GRPO algorithm, solving the problem that existing methods often fail to balance quality and diversity in GRPO training, and often sacrifice diversity to improve quality.

[0054] Next, referring to the accompanying drawings, an anti-homogenization training device for a diffusion model according to an embodiment of the present invention is described.

[0055] Figure 9 This is a block diagram illustrating an anti-homogenization training device for a diffusion model provided in an embodiment of the present invention.

[0056] like Figure 9 As shown, the anti-homogenization training device 90 for the diffusion model includes: an acquisition and construction module 901, a clustering and partitioning module 902, a reward allocation module 903, and a regularization module 904.

[0057] The acquisition and construction module 901 acquires a set of images generated from the same prompt word and constructs a distance matrix for the set of images. The clustering and partitioning module 902 performs spectral clustering on the distance matrix to divide the set of images into different semantic clusters. The reward allocation module 903 adaptively allocates exploration rewards based on the size of each semantic cluster to obtain multiple semantic clusters after exploration rewards. The regularization module 904 inputs the multiple semantic clusters after exploration rewards into a structure-aware regularized diffusion model to obtain a generalized set of images.

[0058] In some embodiments, the acquisition and construction module 901 includes: The acquisition unit is used to acquire a set of images generated from the same prompt word; The first computing unit is used to calculate the perceptual distance between each pair of images in a set of images to form a distance matrix.

[0059] In some embodiments, the clustering and partitioning module 902 includes: Building units are used to construct affinity matrices based on distance matrices; The second calculation unit is used to calculate the normalized Laplace matrix based on the affinity matrix; The partitioning unit is used to perform spectral clustering on the normalized Laplacian matrix to divide a set of images into different semantic clusters.

[0060] In some embodiments, the regularization module 904 includes: The determination unit is used to determine the early and late denoising steps of the initial diffusion model based on a time threshold. Constraint units are used to apply Wasserstein distance constraints to the early denoising steps and to remove regularization constraints to the later steps, so as to obtain a diffusion model with structure-aware regularization. The generalization unit is used to input multiple semantic clusters after exploration rewards into the structure-aware regularized diffusion model to obtain a generalized set of images.

[0061] It should be noted that the foregoing explanation of the anti-homogenization training method embodiment for the diffusion model also applies to the anti-homogenization training device for the diffusion model in this embodiment, and will not be repeated here.

[0062] The anti-homogeneity training device for diffusion models proposed according to embodiments of the present invention has the following beneficial effects: (1) Solving the problem of pattern convergence caused by single-sample reward modeling: A distribution-level reward mechanism is designed to perceive and incentivize the semantic diversity of generated samples, fundamentally changing the optimization dynamics and encouraging the model to explore novel visual patterns rather than just using known high-scoring patterns. This solves the problem that the existing GRPO method relies on single-sample reward signals, causing the model to converge to a few safe patterns in pursuit of short-term high rewards. (2) Correcting the mismatch between regularization budget and diffusion generation dynamics: A regularization method matching the structure of the diffusion process is designed, which can allocate the constraint resources to the key early stage of diversity formation, while relaxing the restrictions in the later refinement stage to ensure image quality and reward optimization efficiency. This solves the problem that the existing uniform KL regularization fails to consider the decisive influence of the early denoising steps on diversity, resulting in insufficient constraints in the key stage and excessive constraints in the secondary stage. (3) Establish a better Pareto front for quality-diversity trade-off: By breaking through the trade-off bottleneck of existing technologies, we seek to significantly improve the semantic diversity of the generated results while maintaining or even improving image quality. This will establish a new and better Pareto front for image generation based on the GRPO algorithm, solving the problem that existing methods often fail to balance quality and diversity in GRPO training, and often sacrifice diversity to improve quality.

[0063] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0064] The electronic device may include: a memory 1001, a processor 1002, and a computer program stored on the memory 1001 and capable of running on the processor 1002.

[0065] When the processor 1002 executes the program, it implements the anti-homogenization training method for the diffusion model provided in the above embodiments.

[0066] Furthermore, electronic devices also include: Communication interface 1003 is used for communication between memory 1001 and processor 1002.

[0067] The memory 1001 is used to store computer programs that can run on the processor 1002.

[0068] The memory 1001 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0069] If the memory 1001, processor 1002, and communication interface 1003 are implemented independently, then the communication interface 1003, memory 1001, and processor 1002 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0070] Optionally, in a specific implementation, if the memory 1001, processor 1002, and communication interface 1003 are integrated on a single chip, then the memory 1001, processor 1002, and communication interface 1003 can communicate with each other through an internal interface.

[0071] The processor 1002 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0072] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for anti-homogenization training of a diffusion model.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0074] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0075] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0076] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0077] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0078] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0079] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0080] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for anti-homogenization training of diffusion models, characterized in that, Includes the following steps: Obtain a set of images generated from the same prompt word, and construct a distance matrix for the set of images; Spectral clustering is performed on the distance matrix to divide the set of images into different semantic clusters; Exploration rewards are adaptively allocated based on the size of each semantic cluster to obtain semantic clusters with multiple exploration rewards; The semantic clusters obtained after the multiple exploration rewards are input into the structure-aware regularized diffusion model to obtain a set of generalized images.

2. The anti-homogenization training method for diffusion models according to claim 1, characterized in that, The step of obtaining a set of images generated from the same prompt word and constructing a distance matrix for the set of images includes: Obtain a set of images generated from the same prompt word; Calculate the perceptual distance between each pair of images in the set of images to form the distance matrix.

3. The anti-homogenization training method for diffusion models according to claim 1, characterized in that, The step of performing spectral clustering on the distance matrix to divide the set of images into different semantic clusters includes: Construct an affinity matrix based on the distance matrix; Calculate the normalized Laplace matrix based on the affinity matrix; Spectral clustering is performed on the normalized Laplacian matrix to divide the set of images into different semantic clusters.

4. The anti-homogenization training method for diffusion models according to claim 1, characterized in that, The step of inputting the semantic clusters after multiple exploration rewards into a structure-aware regularized diffusion model to obtain a generalized set of images includes: The early and late denoising steps of the initial diffusion model are determined based on the time threshold. The early denoising step is subject to Wasserstein distance constraints, and the later steps are deregulated to obtain the structure-aware regularized diffusion model. The semantic clusters obtained after the multiple exploration rewards are input into the structure-aware regularized diffusion model to obtain the generalized set of images.

5. A homogenization-resistant training device for a diffusion model, characterized in that, include: The acquisition and construction module is used to acquire a set of images generated by the same prompt word and construct a distance matrix of the set of images; The clustering and partitioning module is used to perform spectral clustering on the distance matrix to divide the set of images into different semantic clusters; The reward allocation module is used to adaptively allocate exploration rewards based on the size of each semantic cluster, so as to obtain semantic clusters with multiple exploration rewards; The regularization module is used to input the semantic clusters after the multiple exploration rewards into the structure-aware regularized diffusion model to obtain a set of generalized images.

6. The anti-homogenization training device for diffusion models according to claim 5, characterized in that, The acquisition and construction module includes: The acquisition unit is used to acquire a set of images generated by the same prompt word; The first computing unit is used to calculate the perceptual distance between each pair of the set of images to form the distance matrix.

7. The anti-homogenization training device for diffusion models according to claim 5, characterized in that, The clustering and partitioning module includes: Construction unit, used to construct affinity matrix based on distance matrix; The second calculation unit is used to calculate the normalized Laplace matrix based on the affinity matrix. A partitioning unit is used to perform spectral clustering on the normalized Laplacian matrix to divide the set of images into different semantic clusters.

8. The anti-homogenization training device for diffusion models according to claim 5, characterized in that, The regularization module includes: The determination unit is used to determine the early and late denoising steps of the initial diffusion model based on a time threshold. A constraint unit is used to apply Wasserstein distance constraints to the early denoising steps and to remove regularization constraints to the later steps, so as to obtain the structure-aware regularized diffusion model. The generalization unit is used to input the multiple semantic clusters after exploration rewards into the structure-aware regularized diffusion model to obtain the generalized set of images.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the anti-homogenization training method for a diffusion model as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the anti-homogeneity training method for diffusion models as described in any one of claims 1-4.

Images