Feature-enhanced advertising service scene image generation method

Abstract

The invention provides a feature-enhanced advertisement service scene image generation method. The method comprises the following steps: S1, training data set preparation: collecting and preprocessing advertisement and e-commerce picture data sets; s2, selecting and initializing a basic model: selecting an appropriate stable diffusion large model as the basic model, and carrying out initial testing on the basic model; s3, Lora model training: training a Lora model on the basis of the stable diffusion large model; s4, model integration: integrating the trained Lora model parameters into a stable diffusion large model to realize characteristic enhanced image generation; and S5, generating and optimizing an image. According to the method, a customized feature enhancement model is efficiently trained at low cost, and then the model is integrated to a diffusion model to realize an image generation method of an advertising service scene, so that the problems of deficiencies in feature extraction and personalized generation of a traditional method and the problems of low efficiency and large resource consumption in fine adjustment of a large model are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method for generating a feature-enhanced advertising business scene image. Background Art

[0002] In today's era of high-efficiency life and convenient development, the e-commerce industry is booming, and the online advertising industry is also expanding rapidly. In the production of advertisements and product promotion posters, the demand for high-quality and personalized customized images is growing. In recent years, image generation technology has been continuously iterating and has made significant progress in the field. In particular, image generation methods based on diffusion models, such as the Stable Diffusion model, can achieve good results in high-resolution and detail-rich image generation. Image generation applications based on large models are also constantly updated and developed. The underlying model based on the diffusion model has also evolved from Stable Diffusion 1.5 to Stable Diffusion XL, and then to Stable Diffusion 3. However, the application of these models in advertising business scenarios still faces some challenges, such as feature enhancement and customized generation requirements.

[0003] The main existing technologies in the field of image generation include image generation based on generative adversarial networks (GANs) and image generation based on variational autoencoders (VAEs), as well as diffusion model image generation methods represented by Stable Diffusion.

[0004] The problem with the existing technology is that the Stable Diffusion large model can already generate diverse and high-quality images, but it still faces the following major challenges when dealing with specific business scenarios such as advertising. When faced with the need for feature enhancement, advertising and e-commerce images need to be able to accurately reflect product characteristics and brand style. Due to the training data, the underlying basic model is difficult to effectively process and enhance these features. On the other hand, when dealing with personalized needs, different advertising and e-commerce platforms require customized image generation capabilities, and need to continuously transform stylized image generation to meet the needs of different brands and advertising content. In addition, in terms of generation efficiency and cost, the generation of high-quality images usually requires a lot of computing resources and time. The cost of training the underlying model alone and retraining the model weights is too high and inefficient. Reducing generation costs and improving efficiency are issues of concern to the industry.

[0005] The difficulty of solving the above problems is: if you want to add characteristic data to the basic diffusion model for training, you need to train and recalculate the overall parameters of the large model. For the current large model with a huge number of parameters, fine-tuning to adapt to its downstream tasks will result in huge overhead. If you deal with advertising business scenarios alone, you will also face problems of low efficiency and high resource consumption.

[0006] The significance of solving the above problems is to reduce the cost of fine-tuning the underlying model, improve efficiency, and ultimately achieve customized image generation with enhanced scene features. Summary of the invention

[0007] The present invention provides a method for generating feature-enhanced advertising business scene images, which utilizes the characteristics of the frozen pre-trained model weights of the low-rank adaptation technology to reduce the weight calculation through the low-rank matrix, so as to achieve low-cost and efficient training of a customized feature-enhanced model, and then integrate it into the diffusion model to achieve image generation for an advertising business scene, thereby solving the shortcomings of traditional methods in feature extraction and personalized generation and the problems of low efficiency and high resource consumption in fine-tuning large models.

[0008] The technical solution of the present invention is as follows: The feature-enhanced advertising business scene image generation method of the present invention comprises the following steps: S1. Preparation of training data set: Collecting and preprocessing advertising and e-commerce image data sets to ensure data diversity and quality; S2. Basic model selection and initialization: Selecting a suitable stable diffusion large model as the basic model and performing an initial test on it; S3. Low-rank adaptive model training: Based on the stable diffusion large model, training a low-rank adaptive model to extract and enhance the features of a specific advertising business scene; S4. Model integration: Integrating the trained low-rank adaptive model parameters into the stable diffusion large model to achieve feature-enhanced image generation; and S5. Image generation and optimization.

[0009] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S1, image data related to the advertising business scene is collected to ensure that the data set contains various advertising types and scenes.

[0010] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S2, a suitable basic model is selected on Civitai.

[0011] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S3, the low-rank adaptive model realizes feature extraction and enhancement by learning the product characteristics and brand style in the advertising image, so that the incremental parameter matrix of the full parameter fine-tuning Represented as two low-rank matrices approximate.

[0012] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S3, the weight of the basic model , the incremental parameter matrix of full parameter training updated by the old method of training all parameters Also belongs to The incremental parameter matrix of the full parameter training is trained using low-rank adaptation technology Represented as two smaller matrices with different parameters, the and , the low-rank matrix is ​​approximated as: ,in for , for The parameter weight matrix of low-rank adaptation, rank Much smaller than ; Given input , after adding low-rank adaptation, the output , .

[0013] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S4, the integration process improves the generation ability and effect of the model through multi-task learning and feature fusion, and injects the parameter weights trained by the low-rank adaptive model into the cross-attention layer of the U-Net network of the stable diffusion denoising process.

[0014] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S4, the basic principle of the stable diffusion large model is to use a step-by-step denoising process, and the denoising model body is a U-Net structure, which is used to generate random noise. In the example, the conditional information is concatenated and the noise is gradually reduced as , and finally decoded by the latent space decoder to generate the image Image. In the denoising process, the trained low-rank adaptation model is integrated into the cross-attention layer of the denoising U-Net network, and the trained low-rank weight matrix Merge into base model parameters In the process, the customized features of advertising and e-commerce business scenarios are diffused into the denoising model, so that the denoising result has the enhanced features of the advertising business scenario, and finally the image generation function with customized feature enhancement is completed.

[0015] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, in step S5, the model input conditions are set according to the advertising business needs, and after the preliminary image is generated, post-processing optimization is performed to finally generate a high-quality advertising image that meets the requirements.

[0016] Optionally, in the above-mentioned feature-enhanced advertising business scene image generation method, the model input conditions include text description and product attributes; and the post-processing optimization includes color correction and detail enhancement.

[0017] According to the technical solution of the present invention, the beneficial effects produced are: The feature-enhanced advertising business scene image generation algorithm of the present invention utilizes the customized characteristics of low-rank adaptive technology to realize the feature-enhanced advertising business scene image generation method, and finally realizes that the model can generate pictures that meet the needs of advertising production and e-commerce products, so as to solve the problem of high cost and low efficiency of fine-tuning of the underlying model. Specifically, the present invention realizes a customized advertising image generation method based on a diffusion model integrated with a low-rank adaptive model. In view of the problem that the pictures generated by the basic diffusion model do not have the ability to accurately reflect the characteristics and brand style of the advertising promotion products, in order to cope with the needs of advertising and e-commerce business for product feature enhancement, and the problem that the old method of retraining the overall model weight is costly and inefficient, the present invention effectively realizes a low-cost and efficient method to meet the needs of advertising business scenarios for customized features and personalized pictures. The method of the present invention, according to the low-rank adaptive technology, has low data requirements when fine-tuning a large model, and only updates and recalculates the weight parameters of the new injection layer to achieve lightweight and efficient training, which can realize low-cost extraction of customized features for diffusion model generation of pictures, and finally realizes the image generation function based on the customized feature enhancement of the diffusion model, so as to flexibly cope with the huge advertising business scenario for personalized pictures.

[0018] In order to better understand and illustrate the concept, working principle and effect of the present invention, the present invention is described in detail below through specific embodiments in conjunction with the accompanying drawings: BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0020] Figure 1 is a flow chart of the feature-enhanced advertising business scene image generation method of the present invention; Figure 2 It is a schematic diagram of low-rank adaptation (Lora) model training involved in the method of the present invention; Figure 3 It is a schematic diagram of image generation of a diffusion model integrated with a low-rank adaptation model involved in the method of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical method and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. These examples are only illustrative and not limiting of the present invention.

[0022] The feature-enhanced advertising business scene image generation method of the present invention is an efficient customized image generation method that meets the needs of advertising business scenes and e-commerce pictures. By combining the trained Lora model with the stable diffusion (Stable Diffusion) large model, the large model uses a lightweight method to use a low-rank decomposition matrix to inject between cross-attention layers, enhance customized features, and generate images with details and brand style according to advertising needs, so as to achieve high-quality generation of advertising images. In addition, by increasing the training of the Lora model with a data set that meets the needs, the extraction and enhancement capabilities of specific product features and brand styles are improved, so that the large model can generate pictures that meet specific needs and meet advertising business scenes on the basis of generating high-quality pictures, enrich product features and background information, and enable stable diffusion (Stable Diffusion) to integrate the content perception capabilities of advertising business scenes while retaining the excellent generation capabilities of the underlying large model, and finally realize that the model can generate pictures that meet the needs of advertising production and e-commerce products, so as to solve the problems of high cost and low efficiency of fine-tuning of the underlying model.

[0023] The principle of the present invention is: to utilize the frozen pre-trained model weights of the low-rank adaptation technology to reduce the characteristics of weight calculation through the low-rank matrix, use data with advertising business needs or brand feature style to train the Lora model, and then integrate it into the underlying Stable Diffusion model, so as to improve the model training efficiency, save computing resources, energy and time, and further realize the customized feature image generation function for advertising business scenarios.

[0024] The method of the present invention provides a method for generating customized feature-enhanced images. A stylized dataset is established by collecting and preprocessing advertisements and e-commerce images. The high-efficiency and low-cost fine-tuning large model capabilities of low-rank adaptation technology are utilized to train a model with characterized and customized image generation for advertisement application scenarios. In this way, high-quality customized advertisements or e-commerce images can be generated to meet the needs of characterized and stylized image generation for a large number of advertisement business scenarios.

[0025] like Figure 1 As shown, the feature-enhanced advertising business scene image generation method of the present invention comprises the following steps from start to finish: S1. Training dataset preparation, dataset collection and data preprocessing: Collect and preprocess advertising and e-commerce image datasets to ensure data diversity and quality. The data preprocessing steps include image normalization and data enhancement to improve the model training effect, that is, to improve the effect and robustness of model training. After data preprocessing, the image feature map is obtained. .

[0026] Specifically, step S1 includes: collecting image data related to advertising business scenarios, ensuring that the data set contains various advertising types and scenarios. The data set is used to train the low-rank adaptation (Lora) model. The number of single feature data sets is usually 10-50, which has the characteristics of sufficient inclusion and high quality. After preprocessing, it is used as the feature input for model training .

[0027] S2. Basic model selection and initialization: Select a suitable stable diffusion large model as the basic model and conduct initial testing on it to enable it to have the basic ability to handle advertising and e-commerce image generation and ensure that it can meet the needs of advertising and e-commerce image generation.

[0028] Specifically, step S2 includes: after collecting complete training data, it is necessary to select a model suitable for the advertising business scenario from many basic underlying models based on the stable diffusion model. A suitable basic model can be selected on Civitai, and the Lora model can be trained on this basis.

[0029] S3. Lora model training: Based on the Stable Diffusion large model, the Lora model is trained to extract and enhance the features of specific advertising business scenarios.

[0030] The Lora model extracts and enhances features by learning the product characteristics and brand style in the advertising images, making the incremental parameter matrix of full parameter fine-tuning Represented as two low-rank matrices Approximation. Specifically, the low-rank adaptation (Lora) model is used to train the characteristics of specific advertising business scenarios to extract features such as product characteristics and brand style. During the training process, adversarial training and feature alignment techniques are used to improve feature extraction and enhancement effects.

[0031] like Figure 2 As shown in the figure, it is a diagram of the training principle of the Lora model. In the previous step, the Stable Diffusion model has been selected as the basic model. , the incremental parameter matrix of full parameter training updated by the old method of training all parameters Also belongs to The incremental parameter matrix of all parameter training is trained using low-rank adaptation technology Represented as two smaller matrices with different parameters, the and , the low-rank matrix is ​​approximated as: ,in for , for The parameter weight matrix of low-rank adaptation, rank Much smaller than Given input , output after adding Lora , During training, the pre-trained parameters Being frozen means Although it participates in forward propagation and backward propagation, its corresponding gradient ∂𝐿 / ∂ , and its parameters will not be updated. Thus, a low-cost model solution that can extract and enhance the characteristics of advertising business scenarios and specific styles can be obtained flexibly and efficiently.

[0032] S4. Model integration: Integrate the trained Lora model parameters into the Stable Diffusion large model to achieve feature-enhanced image generation. The integration process improves the model's generation capability and effect through multi-task learning and feature fusion. Inject the parameter weights trained by the Lora model into the cross-attention layer of the U-Net network in the Stable Diffusion denoising process.

[0033] The image generation function is based on the Stable Diffusion model. After training, the Lora model should be integrated into the Stable Diffusion model for customized feature-enhanced image generation. Figure 3 As shown in the figure, the basic principle of the stable diffusion model is to gradually denoise the random noise. The main body of the denoising model is a U-Net structure. In the process, the conditional information such as the prompt words are connected and gradually reduced to , and finally passes through the latent space decoder (i.e. Figure 3 In the denoising process, the trained Lora model is integrated into the cross attention layer of the denoising U-Net network, and the trained low-rank weight matrix Merge into base model parameters In this way, the customized features of advertising and e-commerce business scenarios are diffused into the denoising model, so that the denoising result has the enhanced features of the advertising business scenario, and finally the image generation function with customized feature enhancement is completed.

[0034] S5. Image generation and optimization: According to the advertising business needs, set the model input conditions, such as text description, product attributes, etc. After generating the preliminary image, perform post-processing optimization, such as color correction and detail enhancement, and finally generate high-quality advertising images that meet the requirements.

[0035] After the diffusion process after model integration in step S4, after gradual denoising, it is possible to generate pictures with characteristics. According to the specific advertising business needs, it is also necessary to adjust the model input conditions, such as prompt words and product attributes, including color correction and detail enhancement, to finally achieve high-quality feature-enhanced image generation for advertising business scenarios.

[0036] The method of the present invention is designed for customized feature enhancement. The method first builds a stylized dataset by collecting and preprocessing advertising and e-commerce images. Then, low-rank adaptation techniques are used to fine-tune large models in an efficient and cost-effective manner. Through this training, the model can generate images with specific features and customized application scenarios to meet the needs of high-quality, customized image generation in advertising and e-commerce businesses. This method not only improves the quality and personalization of images, but also optimizes the use of resources, bringing flexible image generation solutions to the advertising and e-commerce industries.

[0037] Combined with Table 1, the method of the present invention is a feature-enhanced advertising business scene image generation method based on the Stable Diffusion large model and the Lora model training. By integrating the Lora model (the parameters of the model are shown in Table 1), it can effectively extract and enhance specific advertising business features and generate high-quality, personalized advertising and e-commerce pictures. By injecting a low-rank trainable matrix into a model as deep as the cross-attention layer, efficient and influential changes and adjustments to the original model are achieved. Because the weights of the pre-trained model are frozen, only training is done on the new injected layer, and only the weights of the new layer are saved, a considerable amount of training resources and space costs are saved, and efficient, high-quality, low-cost training and large model fine-tuning are achieved. At the same time, the needs of high-quality customized feature enhancement are met, and the training cost is reduced to facilitate flexible response to various scene stylization update requirements, and finally customized scene feature enhanced image generation is achieved.

[0038] Table 1 Comparison of parameters between the integrated customized LoRA model of the present invention and the large model with full parameter fine-tuning Integrate customized LoRA model Fine-tune large models with all parameters Training time On a specific image dataset, training a LoRA model may take up to 3 hours The same data set takes 6-9 hours Number of model parameters About 500,000 new parameters added to the specific LoRA model The number of parameters of a complete large model is in the billions or even more Training resource requirements (GPU memory, etc.) 8GB - 16GB GPU memory 32GB or more GPU memory Model performance (measured by task-specific metrics) In the task of generating images in a specific style, the accuracy can reach over 80% The accuracy rate in the same task is about 75% Model Customizability high Low Model update difficulty Relatively easy, only a few parameters need to be updated Update is more difficult The above description is the best embodiment based on the concept and working principle of the invention. The above embodiment should not be understood as limiting the protection scope of the present claims, and other implementations and combinations of implementations according to the concept of the present invention belong to the protection scope of the present invention.

Claims

1. A method for generating a feature-enhanced advertising business scene image, characterized in that: The following steps are involved: S1. Training dataset preparation: Collect and preprocess advertising and e-commerce image datasets to ensure data diversity and quality; S2. Base model selection and initialization: Select an appropriate stable diffusion model as the base model and perform initial testing on it; S3. Low-rank adaptive model training: Based on the stable diffusion large model, a low-rank adaptive model is trained to extract and enhance the features of specific advertising business scenarios; S4. Model integration: integrating the trained low-rank adaptive model parameters into the stable diffusion large model to achieve feature-enhanced image generation; and S5. Image generation and optimization.

2. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S1, image data related to advertising business scenarios are collected to ensure that the data set contains various advertising types and scenarios.

3. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S2, a suitable base model is selected on Civitai.

4. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S3, the low-rank adaptive model extracts and enhances features by learning the product characteristics and brand style in the advertising image, so that the incremental parameter matrix of the full parameter fine-tuning Represented as two low-rank matrices approximate.

5. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S3, the weights of the base model , the incremental parameter matrix of full parameter training updated by the old method of training all parameters Also belongs to The incremental parameter matrix of the full parameter training is trained using low-rank adaptation technology Represented as two smaller matrices with different parameters, the and , the low-rank matrix is ​​approximated as: ,in for , for The parameter weight matrix of low-rank adaptation, rank Much smaller than ; Given input , after adding low-rank adaptation, the output , .

6. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S4, the integration process improves the generation ability and effect of the model through multi-task learning and feature fusion, and injects the parameter weights trained by the low-rank adaptive model into the cross-attention layer of the U-Net network of the stable diffusion denoising process.

7. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S4, the basic principle of the stable diffusion model is to use a step-by-step denoising process. The denoising model is mainly a U-Net structure, which is used to generate random noise. In the example, the conditional information is concatenated and the noise is gradually reduced as , and finally decoded by the latent space decoder to generate the image Image. In the denoising process, the trained low-rank adaptation model is integrated into the cross-attention layer of the denoising U-Net network, and the trained low-rank weight matrix Merge into base model parameters In the process, the customized features of advertising and e-commerce business scenarios are diffused into the denoising model, so that the denoising result has the enhanced features of the advertising business scenario, and finally the image generation function with customized feature enhancement is completed.

8. The method for generating a feature-enhanced advertising business scene image according to claim 1, characterized in that: In step S5, according to the advertising business requirements, the model input conditions are set, and after the preliminary image is generated, post-processing optimization is performed to finally generate a high-quality advertising image that meets the requirements.

9. The method for generating a feature-enhanced advertising business scene image according to claim 8, characterized in that: The model input conditions include text description and product attributes; the post-processing optimization includes color correction and detail enhancement.

Images