A component-level preference alignment method in personalized design image generation

Abstract

The application discloses a component-level preference alignment method in personalized design image generation and belongs to the technical field of image processing. The training sample set constructed by the method comprises positive and negative reference images, so that a positive reference flow and a negative reference flow can be constructed, and a fusion module is inserted into the attention layer of the constructed model, so that the model can fuse the attention features of the positive reference flow and the negative reference flow. In the training process, the image generated by the model is more inclined to the positive reference image and the negative reference image is inhibited, so that the trained model can better match the preference of the user. Furthermore, the positive and negative reference images in the inference sample pair for constructing the training sample set are components, so that the trained model can better capture and align the creation intention of the fine-grained preference of the individual user through the preference component provided by the user, namely, the positive reference image and the negative reference image selected by the user.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a component-level preference alignment method in personalized design image generation. Background Technology

[0002] Creative work in fields such as conceptual design often relies on a discriminative feedback loop: positive feedback preserves successful elements, while negative feedback corrects errors. Traditional image synthesis models utilize cues and reference images as explicit templates for pixel-level feature transfer. While these methods excel at mimicking structure and texture, they fail to capture users' implicit and evolving preferences and lack the ability to capture and align negative preferences. Furthermore, these methods typically operate at the granular level of the entire image or conceptual subject, lacking component-level or local-level control over design goals, which is detrimental to the expression of fine-grained user preferences.

[0003] Patent application CN119478093A discloses a controllable image generation method based on semantic embedding and preference modeling, comprising the following steps: S1: Fine-tuning a large language model using a text-image pair dataset; wherein the text-image pair dataset includes several triples, each triple including an input image, instruction text, and a generated image; S2: Optimizing the instruction text embedding using the fine-tuned large language model and image reconstruction; S3: Fine-tuning a pre-trained image generation model using the optimized instruction text embedding; S4: Optimizing the fine-tuned pre-trained image generation model using a preference model; wherein the preference model is trained based on generated images labeled with ranking data; S5: Obtaining a generated image that matches the instruction text to be processed and the input image to be processed using the optimized pre-trained image generation model. Images generated by this invention are more in line with human aesthetics and values, and can stably generate high-quality images associated with the input image and instruction text under various noise conditions.

[0004] Personalized image generation research uses additional conceptual reference images as input to accurately reproduce specific styles or target entities in generated images to express positive preferences. Some existing methods have achieved combined generation based on multiple entity reference images, but significant challenges remain when dealing with multiple components of the same entity rather than multiple entities. Due to the closer spatial and semantic relationships between components, existing methods, such as those mentioned in the patent application, often suffer from issues like concept vanishing, concept leakage, and layout collapse. Furthermore, existing methods do not support the injection of negative references.

[0005] On the other hand, human preference alignment methods have been widely applied in the field of image generation. Direct Preference Optimization (DPO)-based methods support learning preference directions from the differences between manually labeled positive and negative sample pairs and training the generative model to align with these preferences. However, most existing research focuses on general human preferences such as aesthetic appeal and text fidelity, rather than modeling the personalized preference information provided by each user. Therefore, it is difficult to capture and align the fine-grained preferences and creative intentions of individual users in the iterative creation flow.

[0006] Based on the above research and its shortcomings, there is an urgent need to propose a human binary preference alignment method to improve the preference alignment quality in personalized image generation. Summary of the Invention

[0007] This invention provides a component-level preference alignment method for personalized design image generation, which can better capture and align the creative intent of individual users' fine-grained preferences.

[0008] This invention provides a component-level preference alignment method for personalized design image generation, comprising: Multiple original images are clustered, and the subject and its components are extracted from each original image of the same category to construct a subject-component sample pool. Positive and negative reference images are randomly assigned to predefined components through the subject-component sample pool to form inference sample pairs. The positive and negative reference images are all components within the subject-component sample pool. Multiple inference sample pairs are used to construct a sample set, which includes a training sample set and an initial validation sample set. A training model is constructed, which is an improved UNO model. The improved UNO model is formed by inserting a fusion module after the attention layer of the two-stream block and the single-stream block in the basic UNO model. A low-rank adaptation module is set in the attention layer. The prompt word, the noisy target image and the front reference image sequence are encoded to construct a positive reference stream, and the prompt word, the noisy target image and the negative reference image sequence are encoded to construct a negative reference stream. The positive reference stream and the negative reference stream are extracted separately through the attention layer. The results of the feature extraction are fused through the fusion module. The predicted image is obtained by using the improved UNO model. A personalized image generation model is obtained by training a low-rank adaptation module using a loss function based on the training sample set; the user-provided prompts, target images, and inference samples of each component selected by the user are encoded and input into the personalized image generation model to obtain the final predicted image.

[0009] Preferably, the feature extraction results are fused using a fusion module, including: The feature extraction results for the positive reference flow and the feature extraction results for the negative reference flow are weighted, summed, and then normalized. The feature extraction results for the negative reference flow are suppressed by assigning weights. The normalized result is then weighted and summed with the feature extraction results for the positive reference flow to obtain the fusion result.

[0010] Preferably, the positive reference image sequence is constructed from multiple positive reference images, and the negative reference image sequence is constructed from multiple corresponding negative reference images.

[0011] Preferably, when one or more frontal reference images are missing in the frontal reference image sequence, a null filling mechanism is introduced to explicitly set the attention intensity at the missing position to zero.

[0012] Preferably, a subject-component sample pool is constructed by extracting the subject and its components from each original image of the same category, including: Using the set regeneration prompt words, each original image is regenerated using the UNO method. Cluttered backgrounds and occluded components are removed to obtain the subject and its components. Then, the SAM3 model is used to extract the mask based on the predefined component names and segment the subject into multiple components to form a subject-component sample pool.

[0013] Preferably, inference sample pairs are randomly selected from the initial validation sample set, one or two components are randomly discarded from the selected inference sample pairs, and then put back into the initial validation sample set to construct a sparse validation sample set, which is used to validate the trained model.

[0014] Preferably, the loss function includes a preference alignment loss, which is obtained by summing the preference alignment losses of each component; The preference alignment loss of each component includes the preference optimization loss and anchoring loss of each component. The preference optimization loss of each component is composed of the positive sub-loss and the negative sub-loss of each component. The frontal sub-loss of each component is constructed by the norm of the difference between the original frontal target image of each component region and the prediction of each component region image by the improved UNO model, and the difference between the norm of the difference between the original frontal target image of each component region and the prediction of each component region image by the basic UNO model. The negative sub-loss for each component is constructed by the norm of the difference between the original negative target image and the component region image predicted by the basic UNO model, and the difference between the norm of the difference between the original negative target image and the component region image predicted by the improved UNO model.

[0015] Preferably, the anchoring loss of each component is obtained by taking the norm of the difference between the original target image of the front of each component and the region image of each component predicted by the improved UNO model.

[0016] Preferably, the loss function further includes a consistency loss, which is obtained by summing the positive consistency sub-loss and the negative consistency sub-loss of the background mask for each component; The frontal consistency sub-loss of the background mask for each component is obtained by taking the norm of the difference between the background regions corresponding to each component of the predicted image obtained by predicting the frontal target image using the improved UNO model and the basic UNO model. The negative consistency sub-loss of the background mask for each component is obtained by taking the norm of the difference between the background regions corresponding to each component of the predicted image obtained by predicting the negative target image using the improved UNO model and the basic UNO model.

[0017] Preferably, the loss function ,in, For preference alignment loss, the For consistency loss, For normalized time step parameters, This is a hyperparameter.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: The training sample set constructed in this invention includes positive and negative reference images, thereby enabling the construction of positive and negative reference flows. Furthermore, a fusion module is inserted into the attention layer of the constructed model, allowing the model to fuse the attention features of the positive and negative reference flows. During training, the images generated by the model tend to be more similar to the positive reference images while suppressing the negative reference images, thus enabling the trained model to better match user preferences.

[0019] Furthermore, the positive and negative reference images in the inference sample pairs used in the construction of the training sample set provided by this invention are components, enabling the trained model to better capture and align with the creative intent of individual users' fine-grained preferences through the user-provided preference components, namely the positive and negative reference images selected by the user. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a component-level preference alignment method in personalized design image generation, provided as a specific embodiment of the present invention; Figure 2 A flowchart illustrating a component-level preference alignment method for personalized design image generation, provided as a specific embodiment of the present invention; Figure 3A flowchart illustrating the application of the component-level preference alignment method in personalized design image generation, provided in a specific embodiment of the present invention. Detailed Implementation

[0021] To address the issues of lack of fine-grained preference expression and difficulty in modeling negative preferences in personalized concept design, this invention proposes a component-level preference alignment method for personalized image generation models.

[0022] This invention provides a component-level preference alignment method for personalized design image generation, such as... Figure 1 and Figure 2 As shown, it includes: (1) Constructing a training sample set: In a specific embodiment of the present invention, multiple original images are collected from multiple open-source image datasets, and the multiple original images are clustered according to image labels. Subjects and their components are extracted from each original image of the same category to construct a subject-component sample pool. Positive and negative reference images are randomly assigned to predefined components through the subject-component sample pool to form inference sample pairs. The positive and negative reference images are components within the subject-component sample pool. Multiple inference sample pairs construct an inference sample set, which includes the training sample set. It can be seen that the training sample set provided in the specific embodiment of the present invention is a component-level binary preference dataset. It can be understood that the meaning of the subject provided in the specific embodiment of the present invention is a complete semantic entity composed of multiple components, and the meaning of the component is a local structural unit that constitutes the subject.

[0023] Specifically, in order to ensure semantic diversity within the dataset, this embodiment of the invention collects original images from open-source image datasets such as Amazon Berkeley Objects (ABO) and clusters the collected original images according to image labels.

[0024] In the original images of the same category, specific embodiments of the present invention use the SAM3 (Segment Anything Model 3) model to extract masks based on predefined component names, thereby extracting multiple components from the main body of the original image and constructing a main body-component sample pool.

[0025] In a specific embodiment of the present invention, two components are randomly selected from the constructed subject-component sample pool as a positive reference image and a negative reference image. The positive reference image and the negative reference image are used to construct inference sample pairs. Multiple inference sample pairs are used to construct a sample set, which includes a training sample set and an initial verification sample set.

[0026] In a specific embodiment of the present invention, inference sample pairs are randomly selected from the initial validation sample set. One or two components are randomly discarded from the selected inference sample pairs. After random discarding, the selected inference samples contain only positive reference images or negative reference images, or neither positive nor negative reference images are present. This better reflects the contrast and sparsity of actual creative feedback. The samples are then put back into the initial validation sample set to construct a sparse validation sample set, which is used to validate the trained model.

[0027] In one specific embodiment, the original image provided by the present invention typically contains cluttered backgrounds and occluded components. The present invention uses a prompt word based on the basic UNO (Universal Customization Framework) model to remove the cluttered background and occlusions to obtain an original image including the subject and components. In one embodiment, the prompt word is "a panoramic image of a subject with [components] against a pure white background".

[0028] (2) Improve the basic UNO model to obtain the improved UNO model, i.e. the training model. The predicted image is obtained by improving the UNO model: The training model provided in the specific embodiment of the present invention is the improved UNO model. The basic UNO model disclosed in the prior art includes multiple single-stream blocks (image-text joint coding blocks) and dual-stream blocks (image-text independent coding blocks). The single-stream blocks and dual-stream blocks include attention layers. The improved UNO model provided in the specific embodiment of the present invention adds a fusion module after the attention layer. The fusion module can fuse the features of the positive reference stream and the negative reference stream. A low-rank adaptation module is also set in the attention layer. Specifically, the low-rank adaptation module is inserted into the Q, K, and V matrices of the attention layer. In a specific embodiment, the rank of LoRA is 256. In the training process of the specific embodiment of the present invention, only the low-rank adaptation module is trained, and the other parameters are frozen.

[0029] In a specific embodiment of this invention, a positive reference stream is constructed by encoding the prompt word, the noisy target image (i.e., the target noise map), and the negative reference map sequence. A negative reference stream is constructed by encoding the prompt word, the noisy target image (i.e., the target noise map), and the negative reference map sequence. Features are extracted from both the positive and negative reference streams separately through an attention layer. The feature extraction results are then fused using a fusion module, and the fused result replaces the features in the positive reference stream. The predicted image is obtained by improving the UNO model. Compared to the traditional CFG (Classifier-Free Guidance) method, which performs two-stream subtraction fusion at the output stage, the method provided in this embodiment directly fuses in the attention space, achieving more accurate and complex attribute alignment and suppressing feature leakage between components. Specifically, the prompt word is encoded using a text encoder, the target noise map and component-level positive and negative reference maps are encoded using an image encoder, and the negative and positive reference map sequences are component-level positive and negative reference maps.

[0030] During the validation phase, for the sparse validation sample set of the input, the specific embodiment of the present invention introduces a null padding mechanism. When a component reference is missing in any flow, the attention intensity of that flow at the corresponding position will be explicitly set to zero, and the model will only refer to another flow on that component.

[0031] In one specific embodiment, the present invention provides a method for fusing the feature extraction results through a fusion module, including: normalizing the feature extraction results for the positive reference flow and the feature extraction results for the negative reference flow after weighted summation, wherein the feature extraction results for the negative reference flow are suppressed by weight assignment, and the fusion result is obtained by weighted summation of the normalized result and the feature extraction results for the positive reference flow.

[0032] Specifically, the fusion results provided by specific embodiments of the present invention ,in and The results are the feature extraction results for the positive reference flow and the negative reference flow, respectively. For normalization operations, and As a hyperparameter, in one specific embodiment, , .

[0033] (3) A personalized image generation model is obtained by training a low-rank adaptation module based on the training sample set and using a loss function.

[0034] In order to adapt to the component-level features of human preferences in conceptual design scenarios and to solve the problem that the supervision signal comes from different parts of multiple images during training, this invention proposes a composite loss calculation method with spatial masking, which includes preference alignment loss and consistency loss. The preference alignment loss is obtained by summing the preference alignment losses of each component.

[0035] The specific embodiments of the present invention provide that the preference alignment loss of each component includes the preference optimization loss CompDPO of each component and the anchoring loss CompMSE of each component. The preference optimization loss CompDPO of each component is composed of the positive sub-loss and the negative sub-loss of each component.

[0036] The frontal sub-loss of each component provided in the specific embodiment of the present invention is constructed by the norm of the difference between the original frontal target image of each component region image and the prediction of each component region image by the improved UNO model, and the difference between the norm of the difference between the original frontal target image of each component region image and the prediction of each component region image by the basic UNO model for each component.

[0037] The negative sub-loss of each component provided in the specific embodiments of the present invention is constructed by using the norm of the difference between each component region image in the negative original target image and each component region image predicted by the basic UNO model, and the norm of the difference between each component region image in the negative original target image and each component region image predicted by the improved UNO model.

[0038] In a specific embodiment of the present invention, the anchoring loss of each component is obtained by taking the norm of the difference between the original target image facing forward and the predicted image obtained by the improved UNO model for each component.

[0039] Specifically, the preference alignment loss provided in specific embodiments of the present invention for:

[0040]

[0041]

[0042] in, For the first i The preference optimization loss of each component For the first i Anchoring loss of individual components N The number of components. The original target image is frontal. The first image is obtained by improving the UNO model from a sequence of noisy images, prompts, and reference images based on the original frontal target image. i The predicted image corresponding to each component for t The first moment i The noisy image of the original target image corresponding to each component. For the sequence of prompt words and reference images, For only in the first i The first component corresponds to the frontal original target image of the first component. i The operation of calculating the average pixel loss on each component The noisy image, prompt words, and reference image sequence based on the original frontal target image are obtained through the UNO model. i Predicted images of each component, The original target image is negative. The first noisy image, prompt words, and reference image sequence, obtained by improving the UNO model, is based on the original negative target image. i The predicted image corresponding to each component The first image is obtained by using a noisy image, cue words, and reference image sequence based on the negative original target image and the underlying UNO model. i The predicted image corresponding to each component For only in the first i The negative original target image corresponding to the component is the first i The operation of calculating the average pixel loss on each component and As a hyperparameter, in one specific embodiment, , It is understandable that the positive original target image is the desired outcome, i.e., the preferred image, while the negative original target image is the image to be suppressed and avoided as much as possible. Through the loss function described above, the trained model provided in the specific embodiment of the present invention can predict results that are closer to the positive original target image than those predicted by the base model, while avoiding the negative original target image.

[0043] The loss function provided in this specific embodiment of the invention also includes a consistency loss, which is used to limit the generation of background regions. In practice, optimizing only component regions may cause the model to drift in the background region or cause distortion of the global structure, resulting in phenomena such as component duplication or layout collapse. This specific embodiment of the invention uses regularization to ensure that the trainable model remains consistent with the frozen backbone model in regions outside the target component. This method retains the strong prior capabilities of the original model in terms of background cleanliness and global composition, while also supporting relatively free preference alignment adjustments for component regions.

[0044] The consistency loss provided in the specific embodiments of the present invention is obtained by summing the positive consistency sub-loss and the negative consistency sub-loss of the background mask of each component.

[0045] The frontal consistency sub-loss of the background mask for each component provided in the specific embodiment of the present invention is obtained by taking the norm of the difference between the background regions corresponding to each component of the predicted image obtained by predicting the frontal target image using the improved UNO model and the basic UNO model.

[0046] The negative consistency sub-loss of the background mask for each component provided in the specific embodiment of the present invention is obtained by taking the norm of the difference between the background regions corresponding to each component of the predicted image obtained by predicting the negative target image through the improved UNO model and the basic UNO model.

[0047] Specifically, the consistency loss provided in specific embodiments of the present invention for:

[0048] in, For the first i The background mask of the original image corresponding to each component. For the first i The background mask of the negative original image corresponding to each component.

[0049] In specific embodiments of the present invention, a time-step-related loss weighting strategy is followed to calculate the final loss: Among them are For normalized time step parameters, is a hyperparameter. This weighted strategy indicates that in the early stages of denoising, the model primarily focuses on establishing a global layout. Since specific target components are not yet spatially aligned with the mask, the model strictly adheres to consistency constraints to prevent structural collapse. In later stages, the model focuses on refining textures and details, thus assigning a higher relative weight to the preference alignment loss to accommodate preference data.

[0050] (4) The prompt words, target image and reasoning samples of each component selected by the user are encoded and then input into the personalized image generation model to obtain the final predicted image.

[0051] Specifically, such as Figure 3 As shown, multiple target images containing the subject are obtained through a text-based graph model based on the text prompts provided by the user. Inference sample pairs for each component are obtained based on the user's preference annotations for each component in the subject. These inference sample pairs include positive reference images and / or negative reference images, or neither positive nor negative reference images are included. The image encoder encodes the prompts provided by the user, the target images, and the inference sample pairs for each component selected by the user, and then inputs them into a personalized image generation model. Personalized image generation with component-level preference alignment is performed to obtain the final predicted image that satisfies the user's preferences.

Claims

1. A component-level preference alignment method for personalized design image generation, characterized in that, include: Multiple original images are clustered, and the subject and its components are extracted from each original image of the same category to construct a subject-component sample pool. Positive and negative reference images are randomly assigned to predefined components through the subject-component sample pool to form inference sample pairs. The positive and negative reference images are all components within the subject-component sample pool. Multiple inference sample pairs are used to construct a sample set, which includes a training sample set and an initial validation sample set. A training model is constructed, which is an improved UNO model. The improved UNO model is formed by inserting a fusion module after the attention layer of the two-stream block and the single-stream block in the basic UNO model. A low-rank adaptation module is set in the attention layer. The prompt word, the noisy target image and the front reference image sequence are encoded to construct a positive reference stream, and the prompt word, the noisy target image and the negative reference image sequence are encoded to construct a negative reference stream. The positive reference stream and the negative reference stream are extracted separately through the attention layer. The results of the feature extraction are fused through the fusion module. The predicted image is obtained by using the improved UNO model. A personalized image generation model is obtained by training a low-rank adaptation module using a loss function based on the training sample set; the user-provided prompts, target images, and inference samples of each component selected by the user are encoded and input into the personalized image generation model to obtain the final predicted image.

2. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, The feature extraction results are fused using the fusion module, including: The feature extraction results for the positive reference flow and the feature extraction results for the negative reference flow are weighted, summed, and then normalized. The feature extraction results for the negative reference flow are suppressed by assigning weights. The normalized result is then weighted and summed with the feature extraction results for the positive reference flow to obtain the fusion result.

3. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, The positive reference image sequence is constructed from multiple positive reference images, and the negative reference image sequence is constructed from multiple corresponding negative reference images.

4. The component-level preference alignment method in personalized design image generation according to claim 3, characterized in that, When one or more frontal reference images are missing in the frontal reference image sequence, a null padding mechanism is introduced to explicitly set the attention intensity at the missing positions to zero.

5. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, Subjects and their components are extracted from each original image of the same category to construct a subject-component sample pool, including: Using the set regeneration prompt words, each original image is regenerated using the UNO method. Cluttered backgrounds and occluded components are removed to obtain the subject and its components. Then, the SAM3 model is used to extract the mask based on the predefined component names and segment the subject into multiple components to form a subject-component sample pool.

6. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, Randomly select inference sample pairs from the initial validation sample set, randomly discard one or two components from the selected inference sample pairs, and then put them back into the initial validation sample set to construct a sparse validation sample set, which is used to validate the trained model.

7. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, The loss function includes a preference alignment loss, which is obtained by summing the preference alignment losses of each component. The preference alignment loss of each component includes the preference optimization loss and anchoring loss of each component. The preference optimization loss of each component is composed of the positive sub-loss and the negative sub-loss of each component. The frontal sub-loss of each component is constructed by the norm of the difference between the original frontal target image of each component region and the prediction of each component region image by the improved UNO model, and the difference between the norm of the difference between the original frontal target image of each component region and the prediction of each component region image by the basic UNO model. The negative sub-loss for each component is constructed by the norm of the difference between the original negative target image and the component region image predicted by the basic UNO model, and the difference between the norm of the difference between the original negative target image and the component region image predicted by the improved UNO model.

8. The component-level preference alignment method in personalized design image generation according to claim 7, characterized in that, The anchoring loss for each component is obtained by taking the norm of the difference between the original target image in frontal view and the image of each component region predicted by the improved UNO model.

9. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, The loss function also includes a consistency loss, which is obtained by summing the positive consistency sub-loss and the negative consistency sub-loss of the background mask for each component; The frontal consistency sub-loss of the background mask for each component is obtained by taking the norm of the difference between the background regions corresponding to each component of the predicted image obtained by predicting the frontal target image using the improved UNO model and the basic UNO model. The negative consistency sub-loss of the background mask for each component is obtained by taking the norm of the difference between the background regions corresponding to each component of the predicted image obtained by predicting the negative target image using the improved UNO model and the basic UNO model.

10. The component-level preference alignment method in personalized design image generation according to claim 1, characterized in that, The loss function ,in, For preference alignment loss, the For consistency loss, For normalized time step parameters, This is a hyperparameter.

Images