AI-based ASA advertising keyword and material generation system

Abstract

This paper proposes an AI-based ASA advertising keyword and creative generation system. This system uses a knowledge graph combined with XLNet to generate keywords and a multimodal Transformer encoder to process advertising creatives. This system employs multi-task learning for effect estimation and layered multi-armed bandit-based delivery optimization, achieving end-to-end automated processing from understanding advertising demand to delivery optimization.

Description

Technical Field

[0001] The present invention relates to the field of advertising technology, and in particular to an AI-based ASA advertising keyword and material generation system. Background Art

[0002] Apple Search Ads (ASA) is a key channel for mobile app promotion. Compared to other mobile advertising formats, ASA ads are placed directly on the Apple App Store's search results page, offering high contextual relevance and targeting, making them a powerful tool for reaching high-intent users and improving promotion effectiveness. However, ASA ads have a wide keyword coverage and are difficult to monetize. Key technical challenges in optimizing ASA ads include how advertisers can quickly and accurately generate keywords and ad creatives that are highly aligned with their products based on their vast library of creative resources and product information, thereby improving advertising effectiveness.

[0003] Traditional ASA ad optimization relies primarily on manual keyword configuration and creative design. Ad optimizers require a deep understanding of the product and industry, as well as the ability to develop targeted keyword combinations and creative strategies to generate high-quality advertising content. This process is not only time-consuming and labor-intensive, but also susceptible to experience limitations and cognitive blind spots. The resulting ads often lack generalizability and are unable to flexibly respond to changing market conditions and user needs. Furthermore, keyword screening and creative design lack quantitative metrics and real-time feedback, making it difficult for advertisers to accurately predict the effectiveness of newly added keywords and creatives. This results in low budget utilization and a difficult ROI.

[0004] In recent years, the development of artificial intelligence (AI) technology has brought new breakthroughs to ASA ad optimization. Large-scale language models (LLMs) such as BERT and GPT in the natural language processing (NLP) field have made significant progress in understanding semantics and generating text. Multimodal machine learning models have also matured, capable of processing diverse information formats such as text, images, and video, making it possible for computers to "understand" advertising content. Furthermore, AI technologies such as knowledge graphs and reinforcement learning are increasingly being applied in real-world scenarios. These cutting-edge technological advances provide new insights and approaches for ASA ad generation and optimization.

[0005] The present invention conducts research on three core issues in the generation of ASA advertising keywords and materials: 1) How to automatically generate ASA delivery keywords that are highly matched with the product based on product information and historical advertising delivery data to improve advertising effectiveness; 2) How to combine the historical performance data of keywords to accurately predict the delivery effect of new keywords (such as click-through rate, conversion rate, etc.); 3) How to automatically generate advertising materials (including copy, pictures, videos) that are consistent with the brand image, and optimize the materials in real time according to the data during the delivery process. The present invention innovatively proposes an AI-based ASA advertising keyword and material generation system. The system uses powerful natural language understanding capabilities and XLNet's advantages in text generation and multimodal information processing to efficiently analyze advertisers' advertising needs, accurately grasp product selling points and audience characteristics, and intelligently construct advertising keywords and materials that are in line with the brand tone. On this basis, the present invention designs a set of dynamic optimization algorithms to count advertising effect data in real time and continuously optimize keyword combination strategies and creative presentation methods. Experimental results show that this method can significantly improve key performance indicators such as the click-through rate and conversion rate of ASA advertisements. While improving the delivery effect, it greatly improves the automation and efficiency of advertisement optimization, providing a new breakthrough for ASA advertising practice. Summary of the Invention

[0006] In order to solve the above problems, the present invention provides an ASA advertising keyword and material generation system based on artificial intelligence, the structure of which is shown in the following figure: Figure 1 As shown, the system includes the following modules:

[0007] ASA advertising demand understanding module, which completes the understanding of advertising demand according to the following steps:

[0008] Step 1.1: Use the deep learning model of BiLSTM-CRF architecture to analyze advertiser needs;

[0009] Step 1.2 introduces the multi-head attention mechanism MHA to enhance advertising intent understanding:

[0010] Q, K, V = H Q，K，V W Q，K，V ;

[0011]

[0012] MHA(Q,K,V)=concat(head1,...,head k )W O ;

[0013] Among them, H Q，K，V The input feature matrix representing advertising demand, W Q，K，Vrepresents the learnable weight matrix, d represents the dimension of the attention head, head k represents the output of the kth attention head, concat represents the vector concatenation operation, W O represents the output mapping matrix, Q, K, and V represent query, key, and value matrices, respectively.

[0014] ASA advertising keyword generation module, which generates advertising keywords according to the following steps:

[0015] Step 2.1: Precise advertising keyword mining based on knowledge graph:

[0016]

[0017] in, represents the knowledge subgraph of the advertising product p, (h, r, t) represents the head entity, relationship and tail entity triples in the subgraph, v h , v r , v t Represent their vector representations respectively, ||·||2 represents the L2 norm, and argmin represents finding the parameter value that minimizes the objective function.

[0018] Step 2.2: XLNet-based advertising keyword expansion:

[0019] P(w i |S \i )=XLNet(S|w i =[MASK]);

[0020]

[0021] Among them, S represents the advertising seed keyword sequence, w i represents the i-th word, h i Represents the last layer w of XLNet i The hidden state of the position, Q(w i ) represents the quality assessment score of the advertising keyword, b i and b j represents the bias term, w i and w j Represents word vector, S \i represents the sequence except the i-th word, and [MASK] represents the mask tag.

[0022] ASA ad creative generation module, which generates ad creatives according to the following steps:

[0023] Step 3.1 Design a multimodal Transformer encoder to process advertising creatives:

[0024] h (m)=MMTE(x (m) , p (m) ), m∈{t,i,v};

[0025] Among them, x (m) Represents the content feature vector of the advertising material in mode m, p (m) represents the position feature vector, m represents the modality type, t, i, v represent text, image, and video modalities respectively, and h (m) represents the modal features of the output, and MMTE stands for Multimodal Transformer Encoder.

[0026] Step 3.2 introduces cross-modal matching loss to optimize ad creatives:

[0027]

[0028] in, Represents a set of advertising samples in a batch, Represents the feature representation of the advertising sample x under modality m, x + represents the positive sample that matches the sample x, x′ represents other samples in the batch, and · represents the dot product operation.

[0029] Advertisement effect estimation module, which estimates the advertisement effect according to the following steps:

[0030] Step 4.1 Design the feature interaction layer to model advertising creatives and keywords:

[0031] z ad =MLP(e ad )⊙v ad ;

[0032] z kw =MLP(e kw )⊙v kw ;

[0033] z=[z ad , z kw , z ad ⊙z kw ];

[0034] Among them, e ad and e kw are the Embedding vectors of advertising creativity and keywords respectively, v ad and v kw are the additional feature vectors of advertising creativity and keywords respectively, ⊙ is the Hadamard product (element-wise product), [·, ·] represents vector concatenation, and MLP stands for multi-layer perceptron.

[0035] Advertisement delivery optimization module, which optimizes ad delivery according to the following steps:

[0036] Step 5.1 Use the Hierarchical Multi-Armed Bandit Strategy (HMAB) for ad delivery optimization:

[0037]

[0038] Among them, a represents the advertising strategy, EU PK (a) represents the expected utility calculated based on prior knowledge, n a represents the number of times strategy a is selected, n i represents the number of selections of the i-th strategy, α represents the exploration factor (used to balance exploration and utilization), Represents the total number of selections for all strategies.

[0039] Step 5.2: Use evolutionary algorithms to optimize advertising bids:

[0040] Fitness(b)=ω1×CTR b +ω2×Rank b -ω3×CPC b +ω4×Quality b ;

[0041] Among them, b represents the advertising bid vector, CTR b Represents the estimated click rate, Rank b Represents the ad ranking score, CPC b stands for cost per click, Quality b Represents the advertising quality score, ω1, ω2, ω3, and ω4 represent the weight coefficients of click-through rate, ranking, cost, and quality, respectively.

[0042] The present invention also provides a storage medium for generating computer-implemented instructions for AI-based ASA advertising keywords and materials. This storage medium can be read by a computer device, and when executed by a processor, the stored computer program instructions enable full-process intelligent processing of ASA advertising. The storage medium includes a module for deeply understanding advertiser needs. This module analyzes needs through a deep learning model, enhances understanding of advertising intent using a multi-head attention mechanism (MHA), constructs an input feature matrix for advertising needs, and performs feature processing using a learnable weight matrix, achieving precise understanding and structured expression of advertising needs.

[0043] The core functional modules of the storage medium include an advertising keyword generation system based on knowledge graphs and XLNet, a multimodal Transformer-based material processing engine, a multi-task learning-based effect estimation framework, and a layered multi-armed bandit-based delivery optimization strategy. The keyword generation system constructs a product knowledge subgraph for precise keyword mining and leverages the XLNet model for dynamic keyword expansion. The material processing engine uniformly processes multimodal content such as text, images, and videos, optimizing the quality of ad creative generation through cross-modal matching loss. The effect estimation framework uses a feature interaction layer to model the correlation between ad creative and keywords, estimating multiple performance metrics through multi-task learning. The delivery optimization strategy integrates the HMAB algorithm and evolutionary algorithms to achieve multi-level intelligent optimization and bid adjustments from campaigns to ad groups.

[0044] To ensure the efficient operation of the system, the storage medium also includes a complete data management and system monitoring module. The data management module is responsible for establishing the index structure of the advertising material library, storing and maintaining the entity relationship data, model parameters and training data of the knowledge graph, and supporting incremental data updates and garbage collection mechanisms. The system monitoring module records the system operation status and performance indicators in real time, monitors the model prediction effect, adaptively adjusts the system parameter configuration, and realizes anomaly detection and fault recovery. The storage medium can be a computer-readable storage medium in various forms such as ROM, RAM, disk, CD, etc., which has a special area for storing intermediate calculation results, evaluation indicators and other data, as well as a cache area for temporary data storage during program execution to ensure the stability and efficiency of the system operation.

[0045] This paper significantly improves the accuracy and coverage of keyword generation through the innovative combination of knowledge graphs and XLNet. Furthermore, it employs a multimodal Transformer architecture to achieve unified processing and generation of text, image, and video materials, ensuring semantic consistency across different forms of material through cross-modal matching losses. A multi-task learning-based effect prediction framework addresses the problem of modeling correlations between prediction tasks through a feature interaction layer and uncertainty weighting, maintaining high accuracy even in data-sparse scenarios.

[0046] At the system level, the Hierarchical Multi-Armed Bandit (HMAB) optimization strategy implements multi-level intelligent optimization from campaigns to ad groups. Combined with bid optimization using evolutionary algorithms, this significantly improves the automation of ad delivery and budget efficiency. The entire system utilizes end-to-end automation to significantly reduce manual intervention costs while maintaining strong market adaptability and the ability to quickly adjust strategies based on delivery data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1This is the system structure diagram of this application. DETAILED DESCRIPTION

[0048] XLNet, a Transformer-like language model proposed by Google, has outperformed BERT on multiple NLP tasks. Unlike traditional language models, XLNet utilizes the Permutation Language Modeling training paradigm, which allows it to model bidirectional dependencies within text sequences and enhances semantic understanding and contextual awareness. Applying XLNet to automatically generate advertising copy can fully exploit the semantic information in the source material library, learning the brand's language style and industry terminology, and ultimately generating fluent, natural, and engaging advertising language.

[0049] Ad optimization requires comprehensive consideration of material information from different modalities, such as text, images, and videos. XLNet uses a Two-Stream Self-Attention mechanism that can flexibly fuse feature representations from different modalities. This mechanism maps inputs from different modalities to the same semantic space by introducing an identifier vector (Modality Indicator) and a position vector (Position Embedding), thereby modeling the interactions and connections between modalities. On this basis, the present invention designs a Multi-modal Transformer Encoder (MMTE) that implements end-to-end representation learning of text, image, and video features:

[0050] h (m) =MMTE(x (m) , p (m) ), m∈{t, i, υ} (1);

[0051] Where x represents the input content feature vector, p represents the input position feature vector, and t, i, and v represent the text, image, and video modalities, respectively. MMTE adaptively fuses information from different modalities through a self-attention mechanism, ultimately obtaining a unified multimodal feature representation h. Based on this representation, the present invention further designs a cross-modal matching (CM) loss function:

[0052]

[0053] in is the set of advertising samples in a batch. Formula (2) explicitly models the cross-modal consistency of advertising materials by maximizing the matching degree of different modal features in the same advertisement.

[0054] The intelligent ASA advertising optimization system proposed in this paper primarily consists of four modules: a demand understanding module, a keyword generation module, a material generation module, and an effectiveness estimation module. The demand understanding and keyword generation modules provide a structured representation of advertising demand and automatically generate keywords. The material generation module leverages XLNet's text generation and multimodal feature fusion capabilities to automatically generate high-quality graphics, text, and video materials. The effectiveness estimation module uses machine learning algorithms based on historical data to predict key performance indicators for advertising creatives and keyword combinations. These four modules work together to ultimately output an optimized advertising placement configuration.

[0055] The demand understanding module is designed to convert advertising demands provided by advertisers in natural language into structured information representations. Traditional demand understanding relies primarily on manually designed keyword extraction rules and template matching, which has limited generalization capabilities. This paper combines advanced natural language processing technology to design an end-to-end advertising demand understanding model based on deep learning.

[0056] The backbone network of the model uses a BiLSTM-CRF architecture. The BiLSTM layer models the contextual information of the input text sequence, and the CRF layer further considers the constraints between slot labels to decode the optimal slot filling path. In addition, the present invention introduces a multi-head attention (MHA) layer before the BiLSTM layer:

[0057]

[0058] The MHA layer uses multiple attention heads to learn different semantic perspectives of the input sequence in parallel, improving the accuracy of intent recognition and slot filling.

[0059] This paper pre-trains the model on a large-scale advertising corpus and designs a specific intent slot annotation system for ASA advertising based on business scenarios. Given an advertising demand description, the model accurately parses key information such as product type, audience characteristics, and promotion objectives, and outputs a standardized JSON representation. This structured demand representation effectively supports subsequent keyword development, material generation, and other steps.

[0060] High-quality keywords are the foundation of ASA advertising. This paper proposes a keyword generation method that integrates knowledge graphs and deep learning. This method consists of two main steps: targeted keyword mining and generalized keyword expansion.

[0061] First, based on the product information provided by advertisers, this paper uses a commercial large-scale model to build a product knowledge graph. The knowledge graph uses products as the central node and links related concepts through various semantic relationships (such as attributes, functions, usage scenarios, etc.). This paper uses a graph embedding method based on TransE to learn low-dimensional vector representations of product nodes:

[0062]

[0063] in is the knowledge subgraph of product p, (h, r, t) is the triple in the subgraph, representing the head entity, relation and tail entity respectively. TransE assumes v h +v r ≈v t , by minimizing the translation error to learn the embedding representation of entities and relations. On this basis, the present invention designs a heuristic scoring mechanism that comprehensively considers factors such as the semantic relevance between keywords and products, industry popularity, and historical click-through rate, and mines a set of highly targeted seed keywords from the knowledge graph.

[0064] Seed keywords reflect the precise grasp of product selling points, but their number is limited and it is difficult to meet the coverage requirements of advertising. Therefore, in the second step, the present invention generates more related words by expanding generalized keywords based on seed keywords. Inspired by the MaskLanguageModel and MLM paradigms, this method uses XLNet as a generator, takes seed keywords as input, and generates generalized keywords through iterative mask prediction. Specifically, let S = {w1, w2, ..., w K} is a seed keyword sequence. The present invention randomly masks a proportion of words in it and inputs it into XLNet to obtain the probability distribution of the masked words:

[0065] P(w i |S \i )=XLNet(S|w i =[MASK]);

[0066] Based on this distribution, a new word w′ is sampled. i The original masked words are replaced to obtain a new sequence S′. The above process is repeated until a sufficient number of generalized keywords are generated. To ensure the quality of the generated words, the present invention sets constraints such as top-k sampling and introduces a quality evaluator in the last layer of XLNet to dynamically adjust the weights of the generated words:

[0067]

[0068] h i is the last layer w of XLNet i The hidden state of the position, Q(w i) is the output of the quality evaluator w i Quality score. The resulting generalized keywords are highly semantically related to the seed keywords and are more diverse in form, helping to attract a wide range of potential users.

[0069] By combining targeted and generalized keywords, this method improves keyword coverage while ensuring relevance. Numerous real-world advertising applications demonstrate that the knowledge graph-based keyword generation method, combined with XLNet, can significantly increase the impression volume and click-through rate of ASA ads.

[0070] High-quality advertising creativity is key to improving advertising effectiveness. Traditional creative design relies primarily on experienced creative professionals, which is costly and inefficient. This paper proposes a multimodal, multi-task, end-to-end creative generation method that automatically generates highly personalized graphic, text, and video materials from an advertising corpus.

[0071] This method uses XLNet as the basis of the generative model and incorporates specific designs for the ASA material generation task: 1) Introducing advertising industry corpus in the pre-training stage to enhance the model's understanding and generation capabilities of advertising copy; 2) Using the multimodal Transformer encoder (Formula (1)) as the underlying feature extractor to achieve unified modeling of image, text, and video features; 3) Designing multiple decoders to complete different material generation subtasks (such as copy, keyframes, video editing, etc.), and using cross-modal matching loss (Formula (2)) to achieve collaborative training between decoders; 4) Adding an external knowledge injection mechanism to the decoder to use prior information such as knowledge graphs to guide material generation.

[0072] During model training, the present invention constructs a large-scale advertising material corpus (including copy, images, and videos) into a series of "keyword, advertising creative" pairs. The keyword information is passed to each decoder through the attention mechanism to guide the generation of the corresponding material. For example, for the copy generation task, the model's goal is to maximize the following conditional probability:

[0073]

[0074] Where y is the generated copy sequence, x is the input keyword sequence, is the knowledge graph corresponding to the keyword. Similarly, the key frame generation task is based on x and The video editing task further optimizes the shot composition based on the key frames through reinforcement learning, achieving the best video quality in terms of information content, aesthetics, and rhythm.

[0075] This paper trains a multimodal creative generation model on a massive corpus of advertising creatives and designs a human-graded evaluation system to assess the quality of the generated creatives. Offline experiments and online A / B testing demonstrate that the model can automatically design highly engaging and personalized advertising creatives based on input keywords, significantly improving both average click-through and conversion rates. Furthermore, the model exhibits excellent generalization capabilities, enabling it to generate high-quality creatives for new categories and scenarios, significantly saving advertisers significant creative design costs.

[0076] Predicting advertising effectiveness is crucial for optimizing advertising delivery strategies. This paper proposes a deep effectiveness prediction model based on Multi-task Learning (MTL), which can simultaneously predict key indicators such as click-through rate (CTR), conversion rate (CVR), and retention rate (RR) for both ad creatives and keyword combinations.

[0077] The model takes ad creative (ad) and keyword (kw) as input, and first maps them into dense vectors through their respective Embedding layers. Due to the variety of creatives and keywords, this paper further introduces the FeatureInteraction layer to model the high-level interaction features between the two:

[0078] zad=MLP(ead)⊙vad zkw=MLP(ekw)⊙vkw z=[zad, zkw, zad⊙zkw];

[0079] e ad and e kw are the embedding vectors of creativity and keywords respectively, and ⊙ is the Hadamard product. By introducing MLP transformation and second-order interaction term z ad ⊙z kw , this layer can flexibly learn the nonlinear relationship between features.

[0080] The output z of the FeatureInteraction layer is passed into multiple Task-specific MLP classifiers to predict different target indicators:

[0081]

[0082] These classifiers share the underlying feature z, so they can explicitly model the relationship between different indicators while maintaining a certain degree of task differentiation through independent classification layers. The model training process adopts a joint learning paradigm, combining the losses of multiple tasks:

[0083] L=λ1L CTR +λ2L CVR +λ3L RR ;

[0084] λ i is the weight coefficient of each task. In addition, the present invention also introduces a Gate mechanism after each classifier to adaptively adjust the task weight:

[0085]

[0086] in is a learnable parameter vector. The gate mechanism enables the model to dynamically assign task weights based on input ideas and keywords, improving estimation flexibility.

[0087] This MTL-based effect prediction model leverages the correlation between different metrics to accurately predict ad effectiveness even in sparse scenarios. Offline experiments show that it improves the AUC by an average of 3%-5% compared to traditional independent prediction models.

[0088] The goal of advertising delivery optimization is to find keyword combinations and creative strategies to maximize overall advertising revenue while meeting budget constraints. This paper proposes a hierarchical delivery optimization framework that optimizes keyword and creative combination strategies, bids, and budget allocation from top to bottom.

[0089] The top layer uses a multi-armed bandit (MAB) model, treating each keyword or creative as a bandit arm. As the delivery process progresses, MAB explores the expected returns of different arms through ∈-greedy and uses the UCB algorithm to balance exploration and utilization, gradually converging to the globally optimal delivery combination.

[0090] Based on Multi-armedBandit (MAB), the present invention takes into account the specific constraints and goals of the ASA scenario and designs a series of improvements: 1) Introducing a hierarchical bandit structure (HierarchicalMAB, HMAB), first optimizing budget allocation at the Campaign level, and then optimizing keywords and creative combinations at the Ad Group level to improve optimization efficiency; 2) Designing a reward function based on expected utility (EU), while considering multiple factors such as click-through rate, conversion rate, and cost, to reasonably balance advertising revenue and cost; 3) Adding prior knowledge (PK) guidance to the bandit algorithm, making full use of information such as click-through rate provided by the effect prediction model, and accelerating strategy convergence. The modified HMAB reward can be expressed as:

[0091]

[0092] Where a is a combination strategy, EU PK (a) Expected utility calculated based on prior knowledge, n ais the number of times strategy a has been selected, n i is the number of selections for the i-th arm, and α is the exploration factor.

[0093] In Multi-armed Bandit, after MAB obtains a combination strategy, the lower layer uses an evolutionary algorithm (EA) to further optimize keyword bids. Unlike traditional manual price adjustment rules, EA starts from multiple randomly generated bidding plans and iteratively searches for the optimal bid through operations such as selection, crossover, and mutation. Among them, the Fitness function comprehensively considers factors such as ad click-through rate, ranking, and click price:

[0094]

[0095] b is the bid vector, CTR b Rank b CTR and ranking estimated by the effect estimation model, CPC b is the click price, Quality b is the creative quality score, ω i is the weight coefficient. After each round of iteration, the bid solutions with higher scores are retained and cross-mutated, while low-quality solutions are eliminated. This evolution continues for multiple rounds until the bid converges.

[0096] To validate the effectiveness of our proposed method, we conducted large-scale comparative experiments and analyses in real-world ASA advertising scenarios. The experimental dataset encompassed advertising data from multiple advertisers across multiple industries, spanning three months. We focused on evaluating the effectiveness of three modules: keyword optimization, creative generation, and impact estimation.

[0097] This paper uses an A / B testing framework to conduct experiments. Each advertiser's traffic is randomly divided into a control group (CG) and a test group (TG), with a 1:1 traffic ratio. The control group uses the advertiser's original advertising strategy, while the test group uses the optimization method proposed in this paper. The improvement effect of the test group compared to the control group is observed.

[0098] The evaluation indicators used in the experiment mainly include:

[0099]

[0100] The baseline models for experimental comparison include:

[0101] 1) Classic TF-IDF keyword extraction and manual configuration (TF-IDF+Manual)

[0102] 2) Template Creation

[0103] 3) DeepFFM CTR Prediction Model for Independent Tasks (DeepFFM)

[0104] This paper compares the performance of the knowledge graph + XLNet (KG+XLNet) keyword generation method proposed in this paper with the TF-IDF+Manual baseline. The results are shown in Table 1:

[0105] Table 1:

[0106]

[0107] It can be seen that the KG+XLNet method has achieved significant improvement in all industries, with a significant increase in average CTR and CTR respectively. Among them, the e-commerce and gaming industries performed particularly well. The present invention believes that the main reasons are: on the one hand, the products in these two industries are updated and iterated quickly, homogeneous competition is fierce, and advertisers have a strong demand for precise reach; on the other hand, the knowledge graph can better explore the segmented attributes of the product, and combined with the powerful generation ability of XLNet, it can obtain both accurate and diverse keyword combinations. The improvement in the education industry is relatively small. The reason is that the user search habits in this industry are relatively stable, and advertisers have strong subjective preferences for keywords, which to a certain extent limits the space for algorithm optimization. Overall, the KG+XLNet method can fully integrate industry knowledge and language priors to automatically generate customized keywords, bringing substantial improvement in results for advertisers.

[0108] This paper compares the performance of our proposed Multimodal XLNet (MM-XLNet) material generation model with the TemplateCreation baseline. Because TemplateCreation relies on manual configuration and is labor-intensive, we selected three representative industries for comparison. The results are shown in Table 2:

[0109] Table 2:

[0110]

[0111] Experimental results demonstrate that the MM-XLNet model outperforms traditional template-based methods in generating both image and video assets. Further analysis reveals that the assets generated by MM-XLNet are superior in terms of visual appeal and information completeness. Thanks to the model's deep understanding of images and text, the generated assets are more tailored to user needs. Furthermore, through the fusion modeling of multimodal information, MM-XLNet can better grasp the inherent connections between different asset formats, ensuring that the generated images and videos are highly aligned with the advertiser's brand tone and product selling points.

[0112] The present invention also built a multi-task learning (MTL) effect prediction model proposed in the present invention, and compared it with the traditional independent task DeepFFM model. The results are shown in Table 3. Two learning paradigms, single-task loss (STL Loss) and multi-task joint loss (MTL Loss), were used in the experiment. It can be seen that the model using the MTL learning paradigm achieved better results for both CTR and CVR prediction tasks, indicating that joint learning helps to share and transfer knowledge between different tasks. In addition, the dynamic MTL Loss with the addition of the Gate mechanism is further improved compared to the simple static weighted MTL Loss, which confirms the value of the Gate mechanism in modeling the importance differences of different tasks and improving the generalization of the model.

[0113] Table 3:

[0114]

[0115] To further verify the superiority of the MTL model, the present invention statistically analyzed the overall effect of advertising delivery after adopting different effect prediction models, as shown in Table 4. It can be seen that whether HMAB optimization or manually configured delivery strategy is adopted, the effect prediction based on the MTL model (Ours) makes the overall ROI and CPA better than the DeepFFM model, which once again confirms the effect improvement brought about by multi-task learning by modeling the connection between different indicators and alleviating data sparsity. It is worth mentioning that the HMAB optimization framework also has obvious advantages over manual rules in various indicators. This is due to the fact that HMAB can continuously learn and dynamically adjust strategies, and make full use of the effect prediction results to guide the decision-making process of advertising delivery.

[0116] Table 4:

[0117]

[0118] In summary, large-scale online experimental results confirm the effectiveness of the series of optimization methods proposed in this paper. The knowledge graph combined with XLNet enhances the richness and accuracy of keyword expansion; the multimodal XLNet material generation model provides personalized, diverse, and high-quality advertising creatives; the MTL effect prediction model significantly improves the prediction of key metrics such as click-through rate and conversion rate; and the hierarchical MAB optimization framework enables adaptive adjustment of strategies to maximize advertising returns. The combined application of these methods enables intelligent and automated implementation of all aspects of advertising, significantly improving advertisers' overall return on investment (ROI).

[0119] This paper addresses the three core challenges of ASA ad optimization: keyword development, creative generation, and performance estimation. It proposes an end-to-end solution based on knowledge graphs and multimodal learning. This solution innovatively combines natural language understanding, the structured representation of knowledge graphs, and the advantages of XLNet in text generation and multimodal feature modeling, achieving significant performance improvements in massive ad delivery scenarios.

[0120] Specifically, the main contributions of the present invention are as follows:

[0121] 1) It is proposed to introduce knowledge graphs into ASA advertising optimization, explicitly model prior knowledge such as product attributes and user characteristics based on knowledge graphs, and learn the vectorized representation of products through graph neural networks to realize automatic mining of targeted keywords.

[0122] 2) Improvements to the XLNet model enable end-to-end generation of multimodal creatives, including advertising copy, images, and videos, through iterative mask prediction, multimodal feature fusion, and cross-modal consistency constraints. The resulting personalized creative effectively matches product selling points with user preferences, enhancing ad appeal and conversion rates.

[0123] 3) We propose an effect prediction framework under the multi-task learning paradigm. We model the high-level interaction between advertising creatives and keywords through the Feature Interaction layer, and introduce a Gate mechanism to dynamically adjust the weights of different task objectives. This significantly improves the prediction of key indicators such as click-through rate and conversion rate in sparse data environments.

[0124] 4) We designed a hierarchical HMAB delivery optimization strategy, enabling intelligent budget allocation and delivery portfolio optimization at different granularities, both within campaigns and ad groups. This highly adaptive framework allows for real-time strategy adjustments based on changing circumstances, leveraging performance estimates to guide delivery decisions and maximize advertisers' ROI.

Claims

1. AI-based ASA advertising keyword and material generation system, characterized by: ASA stands for Apple Search AD, and the system includes the following modules: ASA advertising demand understanding module; ASA advertising keyword generation module, the ASA advertising keyword generation module generates advertising keywords according to the following steps: Step 2.1: Accurate advertising keyword mining based on knowledge graph, using The graph embedding method learns low-dimensional vector representations of product nodes: ； in, The knowledge subgraph representing the advertising product p, Represents the head entity, relationship and tail entity triples in the subgraph, Represent their vector representations respectively, represents the L2 norm, It represents the parameter value that minimizes the objective function. It uses a heuristic scoring mechanism to comprehensively consider the semantic relevance between keywords and products, industry popularity, and historical click-through rate factors to mine a set of highly targeted seed keywords from the knowledge graph. Step 2.2 is based on Ad keyword expansion, using As a generator, it takes seed keywords as input and generates generalized keywords through iterative mask prediction. The probability distribution of mask words is: ； set up Sampling constraints, and The last layer introduces a quality evaluator to dynamically adjust the weight of the generated words: ； in, Represents the advertising seed keyword sequence, represents the i-th word, Represents the last layer of XLNet The hidden state of the position, Represents the ad keyword quality evaluation score output by the quality evaluator. and represents the bias term, and represents the word vector, represents the sequence except the i-th word, Represents the mask mark; ASA ad creative generation module, which generates ad creatives according to the following steps: Step 3.1 Design a multimodal Transformer encoder to process advertising creatives: ； in, Represents the content feature vector of the advertising material in mode m, represents the position feature vector, Represents the modal type, Represent text, image, and video modes respectively. represents the modal characteristics of the output, represents the multimodal Transformer encoder; Step 3.2 introduces cross-modal matching loss to optimize ad creative. The cross-modal matching loss function is: ； in, Represents a set of advertising samples in a batch, Representative advertising samples In modal The following feature representation, Representatives and samples Matched positive samples, Represents other samples in the batch, Represents the dot product operation; Advertising effect estimation module; Advertisement delivery optimization module; The system also includes a storage medium that can be read by a computer device, and the computer program instructions stored therein are executed by a processor to realize the full-process intelligent processing of ASA advertisements.

2. The AI-based ASA advertising keyword and material generation system according to claim 1 is characterized in that: The ASA advertising demand understanding module understands advertising demand according to the following steps: Step 1.1 Use The deep learning model of the architecture analyzes advertiser needs; Step 1.2 introduces the multi-head attention mechanism MHA to enhance advertising intent understanding: ； ； ； in, The input feature matrix representing advertising demand, represents the learnable weight matrix, represents the dimension of the attention head, represents the output of the kth attention head, Represents vector concatenation operation, represents the output mapping matrix, Represent the query, key and value matrices respectively.

3. The AI-based ASA advertising keyword and material generation system according to claim 1 is characterized in that: The advertising effect estimation module estimates the advertising effect according to the following steps: Step 4.1 Design the feature interaction layer to model advertising creatives and keywords: ； ； ;in, and are the embedding vectors of advertising creativity and keywords respectively, and are the additional feature vectors of advertising creativity and keywords respectively, is the Hadamard product, represents vector concatenation, stands for Multilayer Perceptron.

4. The AI-based ASA advertising keyword and material generation system according to claim 1, characterized in that: The advertising delivery optimization module optimizes advertising delivery according to the following steps: Step 5.1 Use the hierarchical multi-armed bandit strategy HMAB for ad delivery optimization: ； in, Represents advertising strategy, represents the expected utility calculated based on prior knowledge, Representative Strategy The number of selections, Representative The number of strategy selections, represents the exploration factor used to balance exploration and exploitation, represents the total number of selections for all strategies; Step 5.2: Use evolutionary algorithms to optimize advertising bids: ； in, represents the advertising bid vector, Represents the estimated click-through rate, Represents the ad ranking score, stands for cost per click, Represents the ad quality score. Represent the weight coefficients of click-through rate, ranking, cost and quality respectively.

Images