Ad multi-modal content creation and placement decision system based on generative agent

Abstract

This invention relates to a multimodal advertising content creation and delivery decision-making system based on generative intelligent agents, belonging to the field of advertising design technology. It includes a content generation module, a content evaluation module, and a multi-objective delivery decision-making module. This invention innovatively introduces a brand baseline feature vector space. By calculating the Euclidean distance and semantic offset of multimodal features, it transforms abstract brand tone into a precisely quantifiable brand deviation penalty value, effectively avoiding the drift of brand visual and semantic features. Furthermore, it dynamically balances predicted conversion revenue with the brand deviation penalty. When the material triggers the deviation penalty, it utilizes a gradient backpropagation path to accurately trace the source of abnormal features, targeting and extracting the offset semantics and automatically converting them into negative prompts. It also fine-tunes and forces alignment of the generation conditions, significantly reducing the computational cost of inference and rendering in large multimodal models and greatly improving the agility and controllability of advertising content iteration.

Description

Technical Field

[0001] This invention relates to the field of advertising design technology, and in particular to an advertising multimodal content creation and delivery decision-making system based on generative intelligent agents. Background Technology

[0002] In existing computational advertising recommendation chains, the generation and distribution decisions of multimodal creatives typically rely on reinforcement learning or supervised learning models with click-through rate or conversion rate as the single objective function. In actual engineering operations, due to the low signal-to-noise ratio and ease of collection of short-term behavioral data, the recommendation system quickly guides the generative model to converge to the feature region that can evoke strong sensory stimulation when updating weight gradients. As the frequency of feedback iterations increases, the distribution distance between the multimodal feature vector set output by the generated model and the baseline feature space initially set by the brand continues to expand. In the process of pursuing short-term click revenue, the model inevitably outputs materials that highly cater to the curiosity, resulting in serious drift of the brand's visual and semantic features, which in turn causes substantial decay and damage to the unified brand image and asset value in the long term. To address the aforementioned technical deficiencies, a solution is proposed. Summary of the Invention

[0003] The purpose of this invention is to innovatively introduce a brand baseline feature vector space, and by calculating the Euclidean distance and semantic offset of multimodal features, transform the abstract brand tone into a brand deviation penalty value that can be accurately quantified. This effectively avoids the drift of brand visual and semantic features, and then dynamically balances the predicted conversion benefits with the brand deviation penalty. When the material triggers the deviation penalty, the gradient backpropagation path is used to accurately trace the source of abnormal features, target and extract the offset semantics and automatically convert them into negative prompt words. The generation conditions are fine-tuned and forced aligned, which not only significantly reduces the computational power consumption of inference and rendering of multimodal large models, but also greatly improves the agility and controllability of advertising content iteration.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: an advertising multimodal content creation and delivery decision-making system based on generative intelligent agents, comprising a content generation module, a content evaluation module, and a multi-target delivery decision-making module, wherein: The multimodal content generation module has a built-in generative agent based on a large language model and a diffusion model. It is used to receive initial prompt words and basic image and text features input by the user, and output several multimodal advertisement drafts containing images and narration audio tracks before sending them to the content evaluation module. The content evaluation module is used to receive the multimodal ad draft, map the multimodal ad draft into a high-dimensional feature vector containing visual, text and audio features through a multimodal feature extraction network, calculate the Euclidean distance and semantic offset between the high-dimensional feature vector and the preset brand benchmark feature vector space, and output the brand deviation penalty value to the content optimization module. The multi-objective delivery decision module includes a delivery evaluation unit and a content optimization unit. The delivery evaluation unit has a built-in reinforcement learning network. The reward function of the reinforcement learning network includes a predicted conversion revenue term and a brand asset constraint term that is negatively correlated with the brand deviation penalty value. When the brand deviation penalty value exceeds a preset dynamic safety threshold, a content optimization signal is generated and sent to the content optimization unit. When the brand deviation penalty value does not exceed the dynamic safety threshold, a content delivery signal is generated; The content optimization unit is used to acquire content optimization signals. Based on the multimodal ad draft, it locates the offset semantics that cause high brand deviation penalty values ​​through gradient backpropagation path, transforms the offset semantics into negative prompt words to correct the input conditions of the generative agent, forces the text output to align with the brand baseline text style, and generates a new multimodal ad draft to be sent to the content evaluation module again.

[0005] Furthermore, the system receives initial prompt words and basic image / text features from the user terminal, and performs dimensionality reduction and alignment operations on the initial prompt words and basic image / text features using a built-in cross-modal encoder. Specifically: The initial prompt words are converted into word embedding vectors using the built-in word segmenter; The basic image and text features are used to extract local and global feature maps through the built-in visual transformer; The word embedding vectors are jointly encoded with local and global feature maps in a unified latent feature space to generate an initial context condition vector with temporal relevance.

[0006] Furthermore, the specific process for outputting the multimodal advertising draft is as follows: S11. The large language model has a pre-built advertising logic architecture tree. Taking the context condition vector as input, it performs autoregressive inference and decouples the output to produce structured multimodal generation instructions. The multimodal generation instructions specifically include: Text modality, namely advertising voice-over scripts and on-screen subtitles; Visual constraint instructions, which are storyboard description prompts corresponding to each time stamp, include scene lighting and shadow settings, product subject shot size, and camera movement logic; S12. Based on the visual features of the diffusion model and cross-attention, a built-in diffusion model is generated, and the visual constraint instructions are obtained as conditional inputs. At the same time, in the reverse denoising generation stage, the diffusion model adopts a control mechanism that introduces a spatial control network. S13. The diffusion model integrates the semantic features of the scene description prompts with the basic image and text features through a cross-attention mechanism. Through multi-step denoising iteration, it generates a continuous image frame sequence that conforms to the semantics of the prompts and retains the physical features of the product. S14. Obtain the advertising voice-over script, generate the narration audio track through the text-to-speech model, and extract the Mel-frequency cepstral coefficients and beat features from the generated narration audio track. Based on the peaks, troughs and emotional pauses of the speech, dynamically stretch and crop the time axis of the continuous image frame sequence to achieve the alignment of the physical timestamps of the visual scene switching and the audio accent nodes. S15. The time-aligned image frame sequence, narration audio track, and text modality are merged and rendered to form a preliminary multimodal advertising draft.

[0007] Furthermore, before outputting to the content evaluation module, the multimodal advertising draft is tagged with feature indexes, that is, the key control vectors of the image frame sequence, the narration audio track and the text modality and their timestamp metadata are encapsulated into the additional data header of the draft.

[0008] Furthermore, after obtaining the multimodal advertising draft, a pre-defined multimodal feature extraction network is used to decouple and analyze the draft, resulting in a high-dimensional feature vector, specifically including: Visual feature extraction specifically involves extracting keyframes from the draft video stream, using a visual converter to extract color distribution, subject composition, and lighting features, and outputting a high-dimensional visual feature vector. ; Text feature extraction specifically involves extracting on-screen subtitles through optical character recognition and combining this with speech recognition transcription of the narration audio track. The resulting text is then input into a pre-trained large-scale language understanding model to extract deep semantic embedding vectors. ; Audio feature extraction specifically involves using an audio convolutional neural network to extract the Mel spectrogram of the narration track, and extracting speech rate, pitch, and emotional fluctuations as acoustic feature vectors. .

[0009] Furthermore, the content evaluation module is used to acquire the brand's visual identity guidelines, brand tone description text, and core taboo word library. Through offline representation learning using a pre-built contrastive language-image pre-trained large model and audio encoder, a set of high-dimensional anchor vectors is generated as the reference space vector. The reference space vector is used as a safe generation boundary for brand assets, where As a visual anchor point for the brand, This is the brand's baseline acoustic attribute vector. This is the brand's baseline text vector.

[0010] Furthermore, the specific process for outputting the brand deviation penalty is as follows: S21. Obtain the high-dimensional feature vector, and calculate the difference between the high-dimensional feature vector and the baseline feature vector using different distance metric algorithms: The Euclidean distance algorithm is used to measure the absolute distance, i.e., the visual offset value, between the draft's visual features and the brand's visual anchor points in multidimensional space. To measure the difference between physical vision and visual features, the specific formula is as follows: ; Where i = 1, 2, 3, ..., n, and n is the number of terms in the high-dimensional visual feature vector; The above formula is also applicable to calculating the physical offset of the acoustic eigenvector, obtaining the acoustic offset value. ; The cosine similarity is used to calculate the angle between the draft text vector and the brand's baseline text vector to measure the difference between the text and the inherent logic of emotion, i.e., the text offset value. The specific calculation formula is as follows: ; S22. A dynamic weight and hard constraint step function are introduced to calculate the brand deviation penalty value. The specific mathematical model for calculation is expressed as follows: ; in, , and These are the preset deviation weighting coefficients for visual, text, and audio modalities, with values ​​dynamically assigned based on the sensitivity of the current ad placement. This is a nonlinear step function that triggers the safety red line.

[0011] Furthermore, the specific process of the new draft multimodal advertising is as follows: S31. Obtain the content optimization signal, which embeds the brand deviation penalty value exceeding the safety threshold and its corresponding high-dimensional feature vector. Using the preset gradient backpropagation path, calculate the gradient of the penalty loss function with respect to the intermediate latent variables of the generation model. By identifying the attention layer with the strongest gradient response, accurately locate the latent variable features that cause high brand deviation. S32. Through a multimodal inverse decoder, the latent variable features are mapped inversely to offset semantics in the natural language space, triggering a condition correction mechanism to transform the extracted offset semantics into negative prompt words for the diffusion model and the large language model. S33. The preset brand baseline text vector is hard-injected as a system-level prefix instruction, the generation temperature parameter of the large language model is dynamically lowered, and the corrected copy and voice-over script that conform to the brand baseline style are forcibly regenerated. S33. For visual anomalies: Automatically generate a mask based on the attention region corresponding to the latent variable features, and input the corrected prompt words into the diffusion model to perform a local redrawing operation. During the redrawing process, the pixel features of the non-mask area are preserved as is, and only the visual elements that cause the deviation within the mask are replaced to obtain the corrected video frame sequence. For audio-text anomalies: extract the regenerated corrected text and dubbing script, call the text-to-speech model to generate a new audio track, and use the dynamic time warping algorithm to remap the new audio track onto the timeline of the original video frame to achieve seamless replacement; S34. The revised video frame sequence, the new audio track, the revised text and dubbing script are repackaged and merged to generate a new multimodal advertising draft.

[0012] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: This generative agent-based multimodal advertising content creation and delivery decision-making system innovatively introduces a brand baseline feature vector space. By calculating the Euclidean distance and semantic offset of multimodal features, it transforms the abstract brand tone into a precisely quantifiable brand deviation penalty value, effectively avoiding the drift of brand visual and semantic features. It then dynamically balances the predicted conversion revenue with the brand deviation penalty. When the material triggers the deviation penalty, it uses the gradient backpropagation path to achieve accurate source tracing of abnormal features, targets and extracts the offset semantics and automatically converts them into negative prompt words, and fine-tunes and forces alignment of the generation conditions. This not only significantly reduces the computational power consumption of inference and rendering of the multimodal large model, but also greatly improves the agility and controllability of advertising content iteration. Attached Figure Description

[0013] Figure 1 A schematic diagram of the overall structure of the present invention is shown; Figure 2 A schematic diagram of the overall method flow of the present invention is shown. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Example: Figure 1-2 As shown, the advertising multimodal content creation and delivery decision-making system based on generative intelligent agents includes a content generation module, a content evaluation module, and a multi-objective delivery decision-making module, wherein: The multimodal content generation module has a built-in generative agent based on a large language model and a diffusion model. It is used to receive initial prompt words and basic image and text features input by the user, and output several multimodal advertisement drafts containing images and narration audio tracks before sending them to the content evaluation module. The system receives initial prompts (text feature sequences) and basic image and text features (such as product main images, brand logos, and basic 3D model assets) from the user client. It then uses a built-in cross-modal encoder to perform dimensionality reduction and alignment operations on the initial prompts and basic image and text features. Specifically: The initial prompt words are converted into word embedding vectors using the built-in word segmenter; The basic image and text features are used to extract local and global feature maps through the built-in visual transformer; The word embedding vectors are jointly encoded with local and global feature maps in a unified latent feature space to generate an initial context condition vector with temporal relevance.

[0016] The specific process for generating a draft multimodal advertisement is as follows: S11. The large language model has a pre-built advertising logic architecture tree. Taking the context condition vector as input, it performs autoregressive inference and decouples the output to produce structured multimodal generation instructions. The multimodal generation instructions specifically include: Text modality, namely advertising voice-over scripts and on-screen subtitles; Visual constraint instructions, which are storyboard description prompts corresponding to each time stamp, include scene lighting and shadow settings, product subject shot size, and camera movement logic; S12. Based on the visual features of the diffusion model and cross-attention, a built-in diffusion model is generated, and the visual constraint instructions are obtained as conditional inputs. At the same time, in the reverse denoising generation stage, the diffusion model adopts a control mechanism that introduces a spatial control network. S13. The diffusion model integrates the semantic features of the scene description prompts with the basic image and text features through a cross-attention mechanism. Through multi-step denoising iteration, it generates a continuous image frame sequence that conforms to the semantics of the prompts and retains the physical features of the product. S14. Obtain the advertising voice-over script, generate the narration audio track through the text-to-speech model, and extract the Mel-frequency cepstral coefficients and beat features from the generated narration audio track. Based on the peaks, troughs and emotional pauses of the speech, dynamically stretch and crop the time axis of the continuous image frame sequence to achieve the alignment of the physical timestamps of the visual scene switching and the audio accent nodes. S15. The time-aligned image frame sequence, narration audio track, and text modality are merged and rendered to form a preliminary multimodal advertising draft.

[0017] Before being output to the content evaluation module, the multimodal advertising draft is characterized by feature indexing and tagging. This involves encapsulating the key control vectors and timestamp metadata of the image frame sequence, narration audio track, and text modality into the draft's additional data header. This allows the content optimization unit to directly use this metadata to accurately locate the specific modality layer and specific timestamp node that caused the deviation through the gradient backpropagation path after the subsequent content evaluation module calculates the brand deviation penalty value. This eliminates the need to blindly reconstruct the entire video stream frame by frame, thereby significantly reducing the computational overhead of the system.

[0018] The content evaluation module is used to receive the multimodal ad draft, map the multimodal ad draft into a high-dimensional feature vector containing visual, text and audio features through a multimodal feature extraction network, calculate the Euclidean distance and semantic offset between the high-dimensional feature vector and the preset brand benchmark feature vector space, and output the brand deviation penalty value to the content optimization module. The content evaluation module is used to obtain the brand's visual identity specifications (such as the brand's standard color library and white space guidelines), brand tone description text, and core taboo word library. It then uses a pre-built contrastive language-image pre-trained large model and an audio encoder for offline representation learning to generate a set of high-dimensional anchor vectors as the baseline space vector. The reference space vector is used as a safe generation boundary for brand assets, where As a visual anchor point for the brand, This is the brand's baseline acoustic attribute vector. This is the brand's baseline text vector.

[0019] After obtaining the multimodal advertisement draft, a pre-defined multimodal feature extraction network is used to decouple and analyze the draft, resulting in a high-dimensional feature vector, specifically including: Visual feature extraction specifically involves extracting keyframes from the draft video stream, using a visual converter to extract color distribution, subject composition, and lighting features, and outputting a high-dimensional visual feature vector. ; Text feature extraction specifically involves extracting on-screen subtitles through optical character recognition and combining this with speech recognition transcription of the narration audio track. The resulting text is then input into a pre-trained large-scale language understanding model to extract deep semantic embedding vectors. ; Audio feature extraction specifically involves using an audio convolutional neural network to extract the Mel spectrogram of the narration track, and extracting speech rate, pitch, and emotional fluctuations as acoustic feature vectors. .

[0020] The specific process for outputting the brand deviation penalty is as follows: S21. Obtain the high-dimensional feature vector, and calculate the difference between the high-dimensional feature vector and the baseline feature vector using different distance metric algorithms: The Euclidean distance algorithm is used to measure the absolute distance, i.e., the visual offset value, between the draft's visual features and the brand's visual anchor points in multidimensional space. To measure the difference between physical vision and visual features, the specific formula is as follows: ; Where i = 1, 2, 3, ..., n, and n is the number of terms in the high-dimensional visual feature vector; The above formula is also applicable to calculating the physical offset of the acoustic eigenvector, obtaining the acoustic offset value. ; The cosine similarity is used to calculate the angle between the draft text vector and the brand's baseline text vector to measure the difference between the text and the inherent logic of emotion, i.e., the text offset value. The specific calculation formula is as follows: ; S22. A dynamic weight and hard constraint step function are introduced to calculate the brand deviation penalty value. The specific mathematical model for calculation is expressed as follows: ; in, , and These are the preset deviation weighting coefficients for visual, text, and audio modalities, with values ​​dynamically assigned based on the sensitivity of the current ad placement. A nonlinear step function to trigger the safety red line; when the feature network identifies that the draft contains hard-to-prohibit brand words or seriously violating visual elements (such as competitor logos), Output a very large constant (such as 10^6), causing the calculated brand deviation penalty value to surge instantly.

[0021] The multi-objective delivery decision module includes a delivery evaluation unit and a content optimization unit. The delivery evaluation unit has a built-in reinforcement learning network. The reward function of the reinforcement learning network includes a predicted conversion revenue term and a brand asset constraint term that is negatively correlated with the brand deviation penalty value. When the brand deviation penalty value exceeds a preset dynamic safety threshold, a content optimization signal is generated and sent to the content optimization unit. When the brand deviation penalty value does not exceed the dynamic safety threshold, a content delivery signal is generated; The content optimization unit is used to acquire content optimization signals. Based on the multimodal ad draft, it locates the offset semantics that cause high brand deviation penalty values ​​through gradient backpropagation path, transforms the offset semantics into negative prompt words to correct the input conditions of the generative agent, forces the text output to align with the brand baseline text style, and generates a new multimodal ad draft to be sent to the content evaluation module again.

[0022] The specific process for generating a new multimodal ad draft is as follows: S31. Obtain the content optimization signal, which embeds the brand deviation penalty value exceeding the safety threshold and its corresponding high-dimensional feature vector. Using the preset gradient backpropagation path, calculate the gradient of the penalty loss function with respect to the intermediate latent variables of the generation model. By identifying the attention layer with the strongest gradient response, accurately locate the latent variable features that cause high brand deviation. For example, is it a specific pixel area at the visual level (such as an inappropriate element appearing in a picture), or a specific token in a text / audio sequence (such as an exaggerated leading verb).

[0023] S32. Through a multimodal inverse decoder, the latent variable features are mapped inversely to offset semantics in the natural language space, triggering a condition correction mechanism to transform the extracted offset semantics into negative prompt words for the diffusion model and the large language model. For example, if the original draft is penalized due to excessive visual color saturation and vulgar copywriting, the system will write labels such as "high saturation, fluorescent color, sensationalism, and suggestive online language" into the negative prompt word set. Mathematically, this is equivalent to forcibly pushing the generation trajectory away from the manifold space where these negative features are located during the classifier-free process of the diffusion model.

[0024] S33. The preset brand baseline text vector is hard-injected as a system-level prefix instruction, the generation temperature parameter of the large language model is dynamically lowered, and the revised copy and dubbing script that conform to the brand baseline style are forcibly regenerated to ensure the semantic rigor and compliance of the new text. S33. For visual anomalies: Automatically generate a mask based on the attention region corresponding to the latent variable features, and input the corrected prompt words into the diffusion model to perform a local redrawing operation. During the redrawing process, the pixel features of the non-mask area are preserved as is, and only the visual elements that cause the deviation within the mask are replaced to obtain the corrected video frame sequence. For audio-text anomalies: extract the regenerated corrected text and dubbing script, call the text-to-speech model to generate a new audio track, and use the dynamic time warping algorithm to remap the new audio track onto the timeline of the original video frame to achieve seamless replacement; S34. The revised video frame sequence, the new audio track, the revised text and dubbing script are repackaged and merged to generate a new multimodal advertising draft.

[0025] This invention innovatively introduces a brand baseline feature vector space. By calculating the Euclidean distance and semantic offset of multimodal features, it transforms the abstract brand tone into a precisely quantifiable brand deviation penalty value, effectively avoiding the drift of brand visual and semantic features. This allows for a dynamic balance between predicted conversion revenue and brand deviation penalty. When the material triggers the deviation penalty, the gradient backpropagation path is used to accurately trace the source of abnormal features, target and extract the offset semantics, and automatically convert them into negative prompt words. The generation conditions are fine-tuned and forced aligned, which not only significantly reduces the computational power consumption of inference and rendering of multimodal large models, but also greatly improves the agility and controllability of advertising content iteration.

[0026] The size of the interval and threshold is set to facilitate comparison. The size of the threshold depends on the amount of sample data and the number of bases set by those skilled in the art for each set of sample data; as long as it does not affect the ratio between the parameter and the quantized value.

[0027] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation. In the two embodiments provided in this application, it should be understood that the disclosed apparatus and system can be implemented in other ways; for example, the apparatus embodiments described above are merely illustrative, for example, the division of modules is merely a logical functional division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed; another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the apparatus or module can be electrical, mechanical or other forms. The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multimodal advertising content creation and delivery decision-making system based on generative intelligent agents, characterized in that, It includes a content generation module, a content evaluation module, and a multi-target delivery decision module, among which: The multimodal content generation module has a built-in generative agent based on a large language model and a diffusion model. It is used to receive initial prompt words and basic image and text features input by the user, and output several multimodal advertisement drafts containing images and narration audio tracks before sending them to the content evaluation module. The content evaluation module is used to receive the multimodal ad draft, map the multimodal ad draft into a high-dimensional feature vector containing visual, text and audio features through a multimodal feature extraction network, calculate the Euclidean distance and semantic offset between the high-dimensional feature vector and the preset brand benchmark feature vector space, and output the brand deviation penalty value to the content optimization module. The multi-objective delivery decision module includes a delivery evaluation unit and a content optimization unit. The delivery evaluation unit has a built-in reinforcement learning network. The reward function of the reinforcement learning network includes a predicted conversion revenue term and a brand asset constraint term that is negatively correlated with the brand deviation penalty value. When the brand deviation penalty value exceeds a preset dynamic safety threshold, a content optimization signal is generated and sent to the content optimization unit. When the brand deviation penalty value does not exceed the dynamic safety threshold, a content delivery signal is generated; The content optimization unit is used to acquire content optimization signals. Based on the multimodal ad draft, it locates the offset semantics that cause high brand deviation penalty values ​​through gradient backpropagation path, transforms the offset semantics into negative prompt words to correct the input conditions of the generative agent, forces the text output to align with the brand baseline text style, and generates a new multimodal ad draft to be sent to the content evaluation module again.

2. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, The system receives initial prompt words and basic image / text features from the user client. It then uses a built-in cross-modal encoder to perform dimensionality reduction and alignment operations on the initial prompt words and basic image / text features. Specifically: The initial prompt words are converted into word embedding vectors using the built-in word segmenter; The basic image and text features are used to extract local and global feature maps through the built-in visual transformer; The word embedding vectors are jointly encoded with local and global feature maps in a unified latent feature space to generate an initial context condition vector with temporal relevance.

3. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, The specific process for generating a draft multimodal advertisement is as follows: S11. The large language model has a pre-built advertising logic architecture tree. Taking the context condition vector as input, it performs autoregressive inference and decouples the output to produce structured multimodal generation instructions. The multimodal generation instructions specifically include: Text modality, namely advertising voice-over scripts and on-screen subtitles; Visual constraint instructions, which are storyboard description prompts corresponding to each time stamp, include scene lighting and shadow settings, product subject shot size, and camera movement logic; S12. Based on the visual features of the diffusion model and cross-attention, a built-in diffusion model is generated, and the visual constraint instructions are obtained as conditional inputs. At the same time, in the reverse denoising generation stage, the diffusion model adopts a control mechanism that introduces a spatial control network. S13. The diffusion model integrates the semantic features of the scene description prompts with the basic image and text features through a cross-attention mechanism. Through multi-step denoising iteration, it generates a continuous image frame sequence that conforms to the semantics of the prompts and retains the physical features of the product. S14. Obtain the advertising voice-over script, generate the narration audio track through the text-to-speech model, and extract the Mel-frequency cepstral coefficients and beat features from the generated narration audio track. Based on the peaks, troughs and emotional pauses of the speech, dynamically stretch and crop the time axis of the continuous image frame sequence to achieve the alignment of the physical timestamps of the visual scene switching and the audio accent nodes. S15. The time-aligned image frame sequence, narration audio track, and text modality are merged and rendered to form a preliminary multimodal advertising draft.

4. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, Before being output to the content evaluation module, the multimodal advertising draft is tagged with feature indexes, which means that the key control vectors of the image frame sequence, the narration audio track and the text modality and their timestamp metadata are encapsulated into the additional data header of the draft.

5. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, After obtaining the multimodal advertisement draft, a pre-defined multimodal feature extraction network is used to decouple and analyze the draft, resulting in a high-dimensional feature vector, specifically including: Visual feature extraction specifically involves extracting keyframes from the draft video stream, using a visual converter to extract color distribution, subject composition, and lighting features, and outputting a high-dimensional visual feature vector. ; Text feature extraction specifically involves extracting on-screen subtitles through optical character recognition and combining this with speech recognition transcription of the narration audio track. The resulting text is then input into a pre-trained large-scale language understanding model to extract deep semantic embedding vectors. ; Audio feature extraction specifically involves using an audio convolutional neural network to extract the Mel spectrogram of the narration track, and extracting speech rate, pitch, and emotional fluctuations as acoustic feature vectors. .

6. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, The content evaluation module is used to obtain the brand's visual identity guidelines, brand tone description text, and core taboo word list. It then uses a pre-built contrastive language-image pre-trained large model and an audio encoder for offline representation learning to generate a set of high-dimensional anchor vectors as the baseline space vector. The reference space vector is used as a safe generation boundary for brand assets, where As a visual anchor point for the brand, This is the brand's baseline acoustic attribute vector. This is the brand's baseline text vector.

7. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, The specific process for outputting the brand deviation penalty is as follows: S21. Obtain the high-dimensional feature vector, and calculate the difference between the high-dimensional feature vector and the baseline feature vector using different distance metric algorithms: The Euclidean distance algorithm is used to measure the absolute distance, i.e., the visual offset value, between the draft's visual features and the brand's visual anchor points in multidimensional space. To measure the difference between physical vision and visual features, the specific formula is as follows: ； Where i = 1, 2, 3, ..., n, and n is the number of terms in the high-dimensional visual feature vector; The above formula is also applicable to calculating the physical offset of the acoustic eigenvector, obtaining the acoustic offset value. ; The cosine similarity is used to calculate the angle between the draft text vector and the brand's baseline text vector to measure the difference between the text and the inherent logic of emotion, i.e., the text offset value. The specific calculation formula is as follows: ； S22. A dynamic weight and hard constraint step function are introduced to calculate the brand deviation penalty value. The specific mathematical model for calculation is expressed as follows: ； in, , and These are the preset deviation weighting coefficients for visual, text, and audio modalities, with values ​​dynamically assigned based on the sensitivity of the current ad placement. This is a nonlinear step function that triggers the safety red line.

8. The advertising multimodal content creation and delivery decision-making system based on generative intelligent agents according to claim 1, characterized in that, The specific process for generating a new multimodal ad draft is as follows: S31. Obtain the content optimization signal, which embeds the brand deviation penalty value exceeding the safety threshold and its corresponding high-dimensional feature vector. Using the preset gradient backpropagation path, calculate the gradient of the penalty loss function with respect to the intermediate latent variables of the generation model. By identifying the attention layer with the strongest gradient response, accurately locate the latent variable features that cause high brand deviation. S32. Through a multimodal inverse decoder, the latent variable features are mapped inversely to offset semantics in the natural language space, triggering a condition correction mechanism to transform the extracted offset semantics into negative prompt words for the diffusion model and the large language model. S33. The preset brand baseline text vector is hard-injected as a system-level prefix instruction, the generation temperature parameter of the large language model is dynamically lowered, and the corrected copy and voice-over script that conform to the brand baseline style are forcibly regenerated. S33. For visual anomalies: Automatically generate a mask based on the attention region corresponding to the latent variable features, and input the corrected prompt words into the diffusion model to perform a local redrawing operation. During the redrawing process, the pixel features of the non-mask area are preserved as is, and only the visual elements that cause the deviation within the mask are replaced to obtain the corrected video frame sequence. For audio-text anomalies: extract the regenerated corrected text and dubbing script, call the text-to-speech model to generate a new audio track, and use the dynamic time warping algorithm to remap the new audio track onto the timeline of the original video frame to achieve seamless replacement; S34. The revised video frame sequence, the new audio track, the revised text and dubbing script are repackaged and merged to generate a new multimodal advertising draft.

Images