A brand creative advertisement picture generation method and system based on a multi-modal large model

Abstract

The application discloses a brand creative advertisement picture generation method and system based on a multi-modal large model, which comprises the following steps: S1, a multi-modal large language model is used to analyze a brand asset package, extract a feature vector, construct a brand feature matrix, and output a structured layout prompt text; S2, a text input space layout generation model is used to generate a space layout priori graph; S3, a diffusion model is constructed, the priori graph is injected into a constraint macroscopic composition through a space condition control network, the brand feature matrix is injected into a reinforced constraint as a key-value pair at a cross-attention layer, and an initial creative advertisement picture is generated after denoising; and S4, a multi-modal large language model is inputted for evaluation, a local repair mask is generated if a blank area is damaged, peripheral pixels are frozen for local redrawing, and a final standard advertisement picture is outputted. The application belongs to the technical field of image processing, and can solve the problems of low brand asset fidelity and lack of advertisement composition semantics in the prior art through structural improvement of the multi-modal large model and the diffusion model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image generation method based on deep learning. Background Technology

[0002] With the development of artificial intelligence technology, large-scale image models are widely used in various image creation applications. However, when applied to the generation of brand creative advertising images, existing technologies have the following shortcomings: Brand equity fidelity is low, and existing technologies typically rely on text prompts to control image generation, failing to accurately reproduce brand logos, standard colors, and specific product packaging. Using post-generation compositing or image-to-image conversion methods results in unnatural lighting and stylistic dissonance between brand equity and the background environment.

[0003] Lacking semantics for ad composition, existing generation models lack ad layout logic, and the generated images often have issues such as the core product being obscured by background elements or insufficient space for ad copy, resulting in images that lack commercial value for direct advertising and requiring extensive manual re-layout.

[0004] Therefore, there is an urgent need to develop a method and system for generating brand creative advertising images based on a multimodal large model to solve the problems in the existing technology. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for generating brand creative advertising images based on a multimodal large model. Through structural improvements to the multimodal large model and diffusion model, it can solve the problems of low brand asset fidelity and lack of advertising composition semantics in the prior art. Moreover, the structure is simple and easy to use, thus solving the problems of low brand asset fidelity and lack of advertising composition semantics mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for generating brand creative advertising images based on a multimodal large model includes the following steps: S1: Multimodal Brand Asset Analysis and Feature Extraction. The system receives a brand asset package and advertising / marketing intent text, uses a multimodal large language model to analyze the brand asset package, extracts brand color feature vectors, logo shape feature vectors, and product appearance feature vectors, and concatenates them to construct a brand feature matrix. Simultaneously, it outputs structured layout prompt text based on the advertising / marketing intent text. The brand asset package includes brand logo images, visual recognition system color values, and product images; the advertising / marketing intent text includes product promotional descriptions. S2: Advertising and Marketing Intent Understanding and Spatial Layout Map Generation. The system inputs structured layout prompt text into the spatial layout generation model to generate a spatial layout prior map containing different grayscale or color masks. S3: Brand Conditional Diffusion Generation Model Construction and Image Generation. The system constructs a diffusion model including a U-shaped network and a cross-attention layer. The spatial layout prior map is input into the spatial condition control network to extract spatial control parameters and inject them into the U-shaped network. At the same time, the brand feature matrix and advertising marketing intent text features are injected as key-value pairs in the cross-attention layer. The system is characterized by extracting ambient lighting parameters from the spatial layout prior map to construct ambient lighting field features. In the feature decoding layer of the U-shaped network, the ambient lighting field features are injected as lighting control conditions to guide the brand asset area to adaptively match ambient lighting during the denoising process. The initial creative advertising image is generated through iterative denoising. S4: Multi-level evaluation and dynamic feedback. The system uses a multimodal large language model to evaluate the initial creative advertising image at multiple levels. Evaluation indicators include visual quality, semantic consistency, business compliance, and lighting and shadow rationality. Among them, visual quality indicators include brand logo clarity, color consistency with brand standard colors, and the naturalness of lighting and shadow in the generated image. Semantic consistency indicators include the matching degree between advertising marketing intent and generated content, and the relevance between advertising copy and image elements. Business compliance indicators include the standardization of brand logo use and the legal compliance of advertising content. Lighting and shadow rationality indicators include the matching degree between the lighting direction, intensity, and reflectivity of the brand asset area and the background environment. Based on the evaluation results, the system dynamically adjusts the spatial layout prior map or feature injection weights. S5: Local Repair and Iterative Optimization. If the multi-level evaluation detects that the white space is damaged or the lighting and shadow are not reasonable enough, the system outputs the position coordinates of the area and generates a local repair mask. It freezes all pixels in the initial creative advertisement image except for the local repair mask, performs local diffusion redrawing on the mask area, and inputs the repaired image back into the multi-level evaluation module for iterative optimization until all evaluation indicators meet the preset thresholds, and outputs the final standard advertisement image.

[0007] By adopting the above technical solution, and by introducing ambient lighting field features into the diffusion model and injecting them into the feature decoding layer, it is possible to achieve physical-level adaptive matching between brand assets and the generated environment in terms of lighting direction, intensity, and reflection. At the same time, by combining multi-level evaluation and local redrawing that includes the rationality of light and shadow, the problem of fragmented light and shadow and unrealistic composite feeling in the generated advertising image is fundamentally solved, and advertising image generation with high visual realism and commercial usability is achieved.

[0008] As a further aspect of the present invention: In step S1, the multimodal large language model consists of a visual encoder and a language decoder; the visual encoder converts the image into a visual feature sequence, and the language decoder understands the text and performs logical reasoning; the multimodal large language model is fine-tuned and trained before use, and the training data includes manually labeled samples. Each sample includes a historical brand asset image, the corresponding feature description text, and a standard feature vector. The model parameters are optimized using an autoregressive loss function.

[0009] By adopting the above technical solution and fine-tuning the multimodal large language model, the model's ability to accurately identify and extract features of brand visual elements is improved, ensuring that the subsequently generated brand asset information is accurate and error-free.

[0010] As a further aspect of the present invention: In step S2, the spatial layout generation model is an image generation model based on a conditional generative adversarial network; the spatial layout generation model undergoes supervised training before use, and the training data includes excellent advertising design drafts and their corresponding layout mask images and layout description text; during the training process, the layout description text is used as a conditional input, and the model weights are updated with the goal of minimizing the cross-entropy loss between the generated mask image and the real mask image.

[0011] By adopting the above technical solution and generating adversarial network models through supervised training conditions, the system possesses advertising composition logic and can accurately generate spatial layout prior diagrams that conform to commercial typography standards.

[0012] As a further aspect of the present invention: In step S3, the diffusion model includes a Unet (U-shaped network) for progressively denoising and generating images, and the U-shaped network contains several cross-attention layers; the spatial condition control network is an auxiliary network parallel to the U-shaped network, used to extract features of the spatial layout prior map and inject them as spatial control parameters into the corresponding layers of the U-shaped network.

[0013] By adopting the above technical solution and constructing a U-shaped network and a parallel auxiliary network, precise spatial constraints on the macroscopic composition of the generated image are achieved, ensuring that the generated advertising image conforms to the preset spatial region division.

[0014] As a further aspect of the present invention: In step S3, in the cross-attention layer, the latent space features of the image generate a query matrix; based on the text features extracted from the advertising and marketing intent text, a text key matrix and a text value matrix are generated; the brand feature matrix is ​​linearly transformed to generate a brand key matrix and a brand value matrix; the text key matrix is ​​concatenated with the brand key matrix, and the text value matrix is ​​concatenated with the brand value matrix; then attention calculation is performed to subject the corresponding region to enhanced constraints at the brand color and shape feature levels.

[0015] By adopting the above technical solution, and by performing specific splicing and calculation on the query, key, and value matrices at the cross-attention layer, deep integration of brand features and image features is achieved, ensuring pixel-level fidelity of brand colors and shapes during the generation process.

[0016] As a further aspect of the present invention: In step S3, the system uses a pre-trained illumination estimation network to analyze the spatial layout prior map, extracts the global illumination direction vector and ambient light intensity scalar, and combines the material reflectivity attribute in the brand feature matrix to construct the ambient illumination field features.

[0017] By adopting the above technical solution, and by extracting the global illumination direction and intensity and combining it with the material reflectivity to construct the ambient illumination field features, the abstract ambient light and shadow information can be transformed into calculable physical parameters, providing an accurate data foundation for the illumination injection of the subsequent feature decoding layer.

[0018] As a further aspect of the present invention: in the feature decoding layer of the U-shaped network, the ambient light field features are fused with the feature map of the decoding layer through a learnable affine transformation module. Specifically, the feature map is scaled and offset at the channel level according to the ambient light field features to inject the lighting control conditions.

[0019] By adopting the above technical solution, the feature map is scaled and offset at the channel level through a learnable affine transformation module. This allows for the lightweight integration of lighting control conditions into the feature decoding process, ensuring that the generated image accurately reflects the ambient lighting effect at the pixel level.

[0020] As a further aspect of the present invention: In step S5, if the multimodal large language model detects that the white space is invaded by redundant elements, it outputs the position coordinates of the area and generates a local repair mask; the system freezes all pixels in the initial creative advertisement image except for the local repair mask, and only performs local diffusion redrawing on the mask area to restore the white space.

[0021] By adopting the above technical solution, the layout defects in the advertisement image are effectively repaired, the blank areas are restored, and the commercial conversion rate of the image is improved by accurately locating the damaged area and freezing the non-target pixels for local redrawing.

[0022] This invention also discloses a brand creative advertising image generation system based on a multimodal large model, comprising: The brand asset analysis module is used to receive brand asset packages and marketing intent texts, and use a multimodal large language model to extract brand feature matrices and structured layout prompt texts; A spatial layout planning unit is used to generate a spatial layout prior diagram based on the structured layout prompt text. A dual-condition guided diffusion generation module is connected to the brand asset analysis module and the spatial layout planning module. It is used to receive the brand feature matrix and the spatial layout prior map, extract the ambient lighting parameters in the spatial layout prior map to construct ambient lighting field features, and generate an initial creative advertising image through the feature injection mechanism of the spatial condition control network and the cross attention layer, as well as by injecting the ambient lighting field features as lighting control conditions into the feature decoding layer of the U-shaped network. The multimodal assessment and local repair module, connected to the dual-condition guided diffusion generation module, is used to receive the initial creative advertising image, perform compliance assessment using a multimodal large language model, locally redraw non-compliant areas, and output the final standard advertising image. By adopting the above technical solutions and through the coordinated operation of the four core modules, a complete advertising image generation system integrating input and output, generation and repair has been constructed, realizing an automated and high-quality creative advertising image generation process.

[0023] As a further aspect of the present invention: the dual-condition guided diffusion generation module includes a diffusion model unit, a spatial control network unit, and a feature injection unit; the diffusion model unit includes a U-shaped network and several cross-attention layers; the spatial control network unit is an auxiliary network parallel to the U-shaped network, used to extract features of the spatial layout prior map and inject them into the Unet network (U-shaped network) as spatial control parameters; the feature injection unit is used to inject the brand feature matrix as key-value pairs in the cross-attention layers to strengthen constraints.

[0024] By adopting the above technical solution and refining the internal structure of the dual-condition guided diffusion generation module, the specific execution units for spatial control and feature injection are clarified, thereby improving the controllability and fusion accuracy of the system in the image generation process.

[0025] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves pixel-level fidelity of brand identity and standard color by injecting a brand feature matrix into the cross-attention layer of the diffusion model, thus solving the problems of low fidelity of brand assets and style fragmentation.

[0026] 2. This invention, by generating a priori spatial layout diagram to constrain the macroscopic composition of the diffusion model, ensures that the generated advertising image contains a clear main visual area and a text white space area, eliminating the need for manual secondary layout and solving the problem of not being able to directly place ads due to a lack of semantic meaning in the advertising composition.

[0027] 3. This invention uses a multimodal large language model to perform compliance assessment and locate the repair area, which can automatically correct layout errors during the generation process and ensure the stability and usability of the output advertising image quality.

[0028] 4. This invention extracts spatial control parameters through a parallel auxiliary network, which can accurately guide the region division of the image and improve the consistency between the generated image and the preset spatial layout.

[0029] Other features and advantages of the present invention will be disclosed in detail in the following detailed description and accompanying drawings. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the steps of a method and system for generating brand creative advertising images based on a multimodal large model, as described in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] In this embodiment of the invention, a method and system for generating brand creative advertising images based on a multimodal large model are described below. Figure 1 As shown, it includes the following: S1: Extract the brand asset feature matrix using a multimodal large language model and generate advertising layout prompt text; S2: Use conditional generative adversarial networks to generate spatial layout prior maps to constrain the macroscopic composition of images; S3: In the process of generating images by denoising using the diffusion model, the brand asset feature matrix is ​​injected into the latent space of the image through the spatial condition control network and the cross attention layer, so as to achieve pixel-level fidelity and natural integration of brand assets. S4: Compliance assessment and local repair are performed using a multimodal large language model to ensure that the generated ad images comply with commercial typography standards.

[0033] Specifically, it includes the following: Example 1 This embodiment provides a method for generating brand creative advertising images based on a multimodal large model, specifically including the following steps: S1: Multimodal Brand Asset Analysis and Feature Extraction The system receives a brand asset package and advertising and marketing intent text input by the user. The brand asset package includes the brand logo image, color values ​​of the visual recognition system, and product images. For example, the user can input the logo image of a sunscreen brand, the standard color RGB value of RGB(255,120,50), a product image of the sunscreen, and the advertising and marketing intent text "Summer sunscreen promotion, beach background".

[0034] The system uses a multimodal large language model to parse brand asset packages. The multimodal large language model consists of a visual encoder and a language decoder. The visual encoder uses a Vision Transformer structure to convert images into visual feature sequences, and the language decoder uses a Transformer structure to understand text and perform logical reasoning.

[0035] Before use, the multimodal large language model undergoes fine-tuning training. The training data includes manually labeled samples, with each sample consisting of historical brand asset images, corresponding feature description text, and standard feature vectors.

[0036] During training, an autoregressive loss function is used to optimize the model parameters. The formula for calculating the autoregressive loss function is as follows:

[0037] Where T represents the length of the output sequence, This represents the t-th output flag. This represents all tags preceding the t-th tag. denoted as the sequence of visual features of the input image, and P represents the probability predicted by the model. This formula is used to measure the model's ability to correctly predict the current text tag given image features and preceding text.

[0038] The system extracts the brand color feature vector. Identifier shape feature vector and product appearance feature vectors The above vectors are concatenated to construct a brand feature matrix. At the same time, it outputs structured layout prompts based on the advertising and marketing intent, such as "the upper quarter is blank sky, the middle half is the main body of the sunscreen product, and the lower quarter is the beach background".

[0039] S2: Understanding Advertising and Marketing Intent and Generating Spatial Layout Maps The system inputs the structured layout prompt text into the spatial layout generation model, which is an image generation model based on conditional generative adversarial networks, and includes a generator and a discriminator.

[0040] The spatial layout generation model undergoes supervised training before use. The training data includes excellent advertising design drafts and their corresponding layout mask images and layout description text. During training, the layout description text is used as a conditional input, and the model weights are updated with the goal of minimizing the cross-entropy loss between the generated mask image and the real mask image.

[0041] The formula for calculating the cross-entropy loss function is as follows:

[0042] Where N represents the total number of pixels in the image, and C represents the number of layout regions, such as white space, subject area, and background area, which are three categories in total. This indicates that the i-th pixel belongs to the true label of category c. This represents the probability that the i-th pixel predicted by the model belongs to category c. This formula is used to measure the pixel-level difference between the layout mask image generated by the model and the real mask image.

[0043] The spatial layout generation model generates a spatial layout prior image based on the input structured layout prompt text. The spatial layout prior image is an image containing different grayscale or color masks to represent the region division of the image. For example, the pixel value of the white space area is set to 0, the pixel value of the main body area is set to 128, and the pixel value of the background area is set to 255.

[0044] S3: Brand Conditioned Diffusion Generation Model Construction and Image Generation The system constructs a diffusion model that combines spatial constraints and feature injection. The diffusion model includes a Unet network for progressively denoising and generating images, and the Unet network contains several cross-attention layers.

[0045] The system inputs the spatial layout prior map into the spatial condition control network, which is an auxiliary network parallel to the Unet network. The spatial condition control network is used to extract the features of the spatial layout prior map and inject them as spatial control parameters into the corresponding layers of the Unet network, thereby constraining the macroscopic mapping.

[0046] Simultaneously, in the cross-attention layer of the diffusion model, text features extracted based on the advertising and marketing intent text, along with the brand feature matrix, are injected as key-value pairs. In the cross-attention layer, image latent space features are linearly mapped to generate a query matrix Q; text features extracted based on the advertising and marketing intent text are linearly mapped to generate a text key matrix. and text value matrix The brand feature matrix is ​​transformed linearly to generate the brand key matrix. Brand Value Matrix .

[0047] Text key matrix Brand Key Matrix spliced ​​as Text value matrix Brand Value Matrix The concatenation is done to form V', and then attention is calculated using the following formula:

[0048] in, The dimension of the key vector is used to scale the dot product result to prevent gradient vanishing. This formula means that when calculating attention weights, image features are simultaneously matched with text features and brand features for similarity, so that the corresponding regions are subject to reinforcement constraints at the level of brand color and shape features.

[0049] Based on the above two conditions, the system generates the initial creative advertisement image through several iterative denoising processes. .

[0050] S4: Multimodal visual compliance verification and local repair The system will generate the initial creative ad image. The image is input into a multimodal large language model for compliance assessment. The multimodal large language model judges the image based on preset assessment indicators, including whether the white space is destroyed, whether the markings are clear, and whether the subject is prominent.

[0051] If the multimodal large language model detects that the white space is invaded by redundant elements, it outputs the location coordinates of the area and generates a local repair mask, and the system freezes the initial creative ad image. For all pixels except those in the local repair mask, during the local diffusion redrawing process, the system only performs noise addition and denoising operations within the mask area. The calculation formula for the redrawn pixels is as follows:

[0052] in, This represents the final standard advertisement image output. This represents the initial creative advertising image. This represents the pixels after local redrawing, and Mask represents the local repair mask matrix. This formula is used to ensure that the repair process only affects the damaged white space area, leaving the pixels in other areas unchanged, in order to restore the white space area and output the final standard advertising image.

[0053] Example 2 Based on the method of Embodiment 1, this embodiment provides a brand creative advertising image generation system based on a multimodal large model, including the following modules: The brand asset analysis module receives brand asset packages and marketing intent texts, and uses a multimodal large language model to extract brand feature matrices and structured layout prompt texts. This module contains a visual encoder and a language decoder, and uses an autoregressive loss function for fine-tuning training of instructions to achieve accurate extraction of brand color features, logo shape features and product appearance features.

[0054] The spatial layout planning module, connected to the brand asset analysis module, receives structured layout prompt text and generates a spatial layout prior map using a spatial layout generation model. This module has a built-in generator and discriminator based on conditional generative adversarial networks. Through training with the cross-entropy loss function, it transforms the text into a spatial region partitioning map containing different grayscale or color masks.

[0055] The dual-condition guided diffusion generation module, connected to the brand asset analysis module and the spatial layout planning module, is used to receive the brand feature matrix and the spatial layout prior map. Through the feature injection mechanism of the spatial condition control network and the cross attention layer, it generates the initial creative advertising image. This module specifically includes a diffusion model unit, a spatial control network unit, and a feature injection unit.

[0056] The diffusion model unit includes a Unet network and several cross-attention layers; The spatial control network unit is an auxiliary network running parallel to the Unet network. It is used to extract features from the spatial layout prior map and inject them into the Unet network as spatial control parameters. The feature injection unit is used to concatenate the brand feature matrix with the text key-value matrix after linear transformation in the cross-attention layer, and to perform reinforcement constraint calculation.

[0057] The multimodal assessment and local repair module, connected to the dual-condition guided diffusion generation module, receives the initial creative advertising image, performs compliance assessment using a multimodal large language model, and locally redraws non-compliant areas to output the final standard advertising image. This module locates defective areas and generates local repair masks based on preset assessment indicators. Combining the mask matrix with the redraw pixel calculation formula, local repair is completed while freezing non-target pixels.

[0058] Example 3 The difference between this embodiment and Embodiment 1 is that a multi-level evaluation and dynamic feedback mechanism is added in step S4, and iterative optimization is implemented in step S5.

[0059] The specific execution process is as follows: (1) Multi-level evaluation: After generating the initial creative advertising image, the system calls a multimodal large language model to evaluate the image in three aspects: 1. Visual Quality: Image features are extracted through a visual encoder to calculate the clarity of the brand logo, such as edge sharpness. The gradient magnitude is calculated using the Laplacian operator, and a threshold of ≥0.8 is considered acceptable. The color difference between the generated color and the brand's standard color, such as RGB(255,120,50), is calculated using the CIEDE2000 color difference formula (International Commission on Illumination 2000 color difference formula), and a threshold of ≤5% is considered acceptable. 2. Semantic Consistency: The advertising and marketing intent text "Summer sunscreen promotion, beach background" is input into the language decoder to extract the key semantics "sunscreen", "beach" and "promotion". The semantic tags of the product subject and beach background in the generated image are then extracted and matched with the semantic tags using the CLIP model (contrastive language-image pre-trained model). A threshold of ≥80% is considered satisfactory. 3. Business Compliance: Check whether the use of the brand logo complies with the VI specifications, such as the logo size accounting for ≥10% of the image area, the position being located in the center of the main area, and verification through object detection models such as YOLO (You Only Look Once).

[0060] (2) Dynamic feedback: If the evaluation results do not meet the standards, the system adjusts the generated parameters according to the indicator type: If the visual quality is substandard, such as excessive color difference, the weight of color features in the brand feature matrix will be increased, for example, from 0.5 to 0.7. If semantic consistency is not met, such as low matching degree, the feature weight of the advertising and marketing intent text in the cross-attention layer is increased, such as from 0.6 to 0.8; If business compliance is not met, such as if the signage is misaligned, the position parameters of the main area in the spatial layout prior drawing will be adjusted, for example, the center coordinates of the main area will be adjusted from (0.5, 0.5) to (0.45, 0.5).

[0061] (3) Iterative optimization: After adjusting the parameters, the system regenerates the initial creative ad image and performs another multi-level evaluation.

[0062] If the white space is still damaged, such as the beach background intruding into the white space, then proceed to step S5: generate a local repair mask, locate the damaged area through the semantic segmentation model, freeze the pixels in the non-mask area, perform local diffusion redrawing on the mask area, and only perform the noise addition-denoising process on the mask area. After redrawing, the system will input the repaired image back into the multi-level evaluation module until all indicators meet the preset thresholds, such as visual quality score ≥90, semantic consistency ≥85, and commercial compliance 100%, and then output the final standard advertising image.

[0063] Example 4 The difference between this embodiment and Embodiment 1 is that this embodiment adds an adaptive fusion mechanism based on ambient lighting guidance in step S3, and adds evaluation and repair of the rationality of light and shadow in steps S4 and S5.

[0064] The specific execution process is as follows: (1) Construction of ambient lighting field features: While inputting the spatial layout prior map into the spatial condition control network, the system also inputs it into the pre-trained lighting estimation network. This network predicts the spherical harmonic lighting coefficient based on the convolutional neural network, thereby extracting the global lighting direction vector and the ambient light intensity scalar. Meanwhile, material reflectivity attributes, including surface roughness and metallicity, are extracted from the product images in the brand asset package using a material recognition model; the illumination direction vector, ambient light intensity scalar, and material reflectivity attributes are cascaded and spliced ​​together to construct an ambient light field feature vector.

[0065] (2) Illumination control condition injection: In the feature decoding layer of the U-shaped network, a learnable affine transformation module is set up; the ambient illumination field feature vector is input into this module, and the scaling parameters and offset parameters are generated by mapping through the fully connected layer; For the feature map in the feature decoding layer, a channel-by-channel affine transformation is performed using the following formula:

[0066] Where F is the original feature map, For scaling parameters, For offset parameters, This indicates element-wise multiplication; through this method, ambient lighting information is deeply integrated into the feature decoding process to guide the noise reduction generation of brand asset areas, so that the generated product or logo surface has highlights and shadows that conform to physical laws.

[0067] (3) Light and shadow rationality assessment: In the multi-level assessment stage, light and shadow rationality index is added; the system extracts the lighting features of the brand asset area and the background area in the generated image through the lighting consistency discriminator, that is, a pre-trained residual network, calculates the cosine similarity between the two, and sets a threshold of ≥90% to meet the standard; at the same time, it calculates the consistency between the highlight reflection direction of the brand asset area and the global lighting direction vector extracted in step (1), and sets a threshold deviation of ≤15 degrees to meet the standard.

[0068] (4) Dynamic feedback and repair: If the light and shadow rationality assessment does not meet the standard, such as excessive deviation in the reflection direction, the system increases the weight of the ambient light field features in the feature decoding layer, that is, increases the scaling parameter. The gradient update step size; if the light and shadow of the blank area or asset area are severely broken, then enter S5 local repair. After generating the local repair mask, only the mask area is subjected to the noise addition-denoising process. In the noise reduction process, the injection weight of the ambient lighting field features is strengthened until the light and shadow rationality index reaches the preset threshold, and the final standard advertising image is output.

[0069] This invention provides a method and system for generating brand creative advertising images based on a multimodal large model. Through structural improvements to the multimodal large model and diffusion model, it can solve the problems of low brand asset fidelity and lack of semantic meaning in advertising composition in existing technologies.

[0070] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0071] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for generating brand creative advertising images based on a multimodal large model, characterized in that, Includes the following steps: S1: Multimodal Brand Asset Analysis and Feature Extraction. The system receives a brand asset package and advertising / marketing intent text, uses a multimodal large language model to analyze the brand asset package, extracts brand color feature vectors, logo shape feature vectors, and product appearance feature vectors, and concatenates them to construct a brand feature matrix. Simultaneously, it outputs structured layout prompt text based on the advertising / marketing intent text. The brand asset package includes brand logo images, visual recognition system color values, and product images; the advertising / marketing intent text includes product promotional descriptions. S2: Advertising and Marketing Intent Understanding and Spatial Layout Map Generation. The system inputs structured layout prompt text into the spatial layout generation model to generate a spatial layout prior map containing different grayscale or color masks. S3: Brand Conditional Diffusion Generation Model Construction and Image Generation. The system constructs a diffusion model including a U-shaped network and a cross-attention layer. The spatial layout prior map is input into the spatial condition control network to extract spatial control parameters and inject them into the U-shaped network. At the same time, the brand feature matrix and advertising marketing intent text features are injected as key-value pairs in the cross-attention layer. The system is characterized by extracting ambient lighting parameters from the spatial layout prior map to construct ambient lighting field features. In the feature decoding layer of the U-shaped network, the ambient lighting field features are injected as lighting control conditions to guide the brand asset area to adaptively match ambient lighting during the denoising process. The initial creative advertising image is generated through iterative denoising. S4: Multi-level evaluation and dynamic feedback. The system uses a multimodal large language model to evaluate the initial creative advertising image at multiple levels. Evaluation indicators include visual quality, semantic consistency, business compliance, and lighting and shadow rationality. Among them, visual quality indicators include brand logo clarity, color consistency with brand standard colors, and the naturalness of lighting and shadow in the generated image. Semantic consistency indicators include the matching degree between advertising marketing intent and generated content, and the relevance between advertising copy and image elements. Business compliance indicators include the standardization of brand logo use and the legal compliance of advertising content. Lighting and shadow rationality indicators include the matching degree between the lighting direction, intensity, and reflectivity of the brand asset area and the background environment. Based on the evaluation results, the system dynamically adjusts the spatial layout prior map or feature injection weights. S5: Local Repair and Iterative Optimization. If the multi-level evaluation detects that the white space is damaged or the lighting and shadow are not reasonable enough, the system outputs the position coordinates of the area and generates a local repair mask. It freezes all pixels in the initial creative advertisement image except for the local repair mask, performs local diffusion redrawing on the mask area, and inputs the repaired image back into the multi-level evaluation module for iterative optimization until all evaluation indicators meet the preset thresholds, and outputs the final standard advertisement image.

2. The method for generating brand creative advertising images based on a multimodal large model according to claim 1, characterized in that, In step S1, the multimodal large language model consists of a visual encoder and a language decoder; the visual encoder converts the image into a visual feature sequence, and the language decoder understands the text and performs logical reasoning; the multimodal large language model is fine-tuned and trained before use, and the training data includes manually labeled samples. Each sample includes a historical brand asset image, the corresponding feature description text, and a standard feature vector. The model parameters are optimized using an autoregressive loss function.

3. The method for generating brand creative advertising images based on a multimodal large model according to claim 1, characterized in that, In step S2, the spatial layout generation model is an image generation model based on conditional generative adversarial networks. The spatial layout generation model is trained under supervision before use. The training data includes excellent advertising design drafts and their corresponding layout mask images and layout description text. During the training process, the layout description text is used as a conditional input, and the model weights are updated with the goal of minimizing the cross-entropy loss between the generated mask image and the real mask image.

4. The method for generating brand creative advertising images based on a multimodal large model according to claim 1, characterized in that, In step S3, the diffusion model includes a U-shaped network for progressively denoising and generating images, and the U-shaped network contains several cross-attention layers; the spatial condition control network is an auxiliary network parallel to the U-shaped network, used to extract features of the spatial layout prior map and inject them as spatial control parameters into the corresponding layers of the U-shaped network.

5. The method for generating brand creative advertising images based on a multimodal large model according to claim 4, characterized in that, In step S3, in the cross-attention layer, the latent space features of the image generate a query matrix; based on the text features extracted from the advertising and marketing intent text, a text key matrix and a text value matrix are generated; the brand feature matrix is ​​linearly transformed to generate a brand key matrix and a brand value matrix; the text key matrix is ​​concatenated with the brand key matrix, and the text value matrix is ​​concatenated with the brand value matrix; then attention calculation is performed to subject the corresponding region to enhanced constraints at the brand color and shape feature levels.

6. The method for generating brand creative advertising images based on a multimodal large model according to claim 1, characterized in that, In step S3, the system uses a pre-trained illumination estimation network to analyze the spatial layout prior map, extracts the global illumination direction vector and ambient light intensity scalar, and combines the material reflectivity attribute in the brand feature matrix to construct the ambient illumination field features.

7. The method for generating brand creative advertising images based on a multimodal large model according to claim 6, characterized in that, In the feature decoding layer of the U-shaped network, the ambient light field features are fused with the feature map of the decoding layer through a learnable affine transformation module. Specifically, the feature map is scaled and offset at the channel level according to the ambient light field features to inject the lighting control conditions.

8. The method for generating brand creative advertising images based on a multimodal large model according to claim 1, characterized in that, In step S5, if the multimodal large language model detects that the white space is invaded by redundant elements, it outputs the position coordinates of the area and generates a local repair mask; the system freezes all pixels in the initial creative advertisement image except for the local repair mask, and only performs local diffusion redrawing on the mask area to restore the white space.

9. A brand creative advertising image generation system based on a multimodal large model, characterized in that, include: The brand asset analysis module is used to receive brand asset packages and marketing intent texts, and use a multimodal large language model to extract brand feature matrices and structured layout prompt texts; A spatial layout planning unit is used to generate a spatial layout prior diagram based on the structured layout prompt text. A dual-condition guided diffusion generation module is connected to the brand asset analysis module and the spatial layout planning module. It is used to receive the brand feature matrix and the spatial layout prior map, extract the ambient lighting parameters in the spatial layout prior map to construct ambient lighting field features, and generate an initial creative advertising image through the feature injection mechanism of the spatial condition control network and the cross attention layer, as well as by injecting the ambient lighting field features as lighting control conditions into the feature decoding layer of the U-shaped network. The multimodal evaluation and local repair module, connected to the dual-condition guided diffusion generation module, is used to receive the initial creative advertising image, perform compliance evaluation using a multimodal large language model, locally redraw non-compliant areas, and output the final standard advertising image. The dual-condition guided diffusion generation module includes a diffusion model unit, a spatial control network unit, and a feature injection unit. The diffusion model unit includes a U-shaped network and several cross-attention layers.

10. A brand creative advertising image generation system based on a multimodal large model according to claim 9, characterized in that, The spatial control network unit is an auxiliary network running parallel to the U-shaped network. It is used to extract features from the spatial layout prior map and inject them into the U-shaped network as spatial control parameters. The feature injection unit is used to inject the brand feature matrix as key-value pairs into the cross-attention layer for enhanced constraints.

Images