Advertisement putting method and device based on machine learning

Abstract

The invention discloses an advertisement putting method and device based on machine learning, and relates to the technical field of digital advertisements. The method comprises the following steps: obtaining a delivery dimension, a pre-delivery creativity and historical interaction data, and constructing a decision model comprising a response prediction network and a reinforcement learning network; iteratively optimizing model parameters by adopting a multi-strategy co-evolution algorithm which sequentially executes information weighting, information rotation, chain combination and global information learning; inputting the user and media features and the remaining budget in the real-time bidding request into a trained model to generate a target bidding strategy; performing feature extraction and dimension reduction coding on the pre-put creativity based on the putting dimension, and matching similar creativity materials; and generating and executing an instruction in combination with a bidding strategy and a similar creative historical effect, and updating interaction data in real time. According to the invention, through multi-strategy optimization and similar creativity matching, the problems of low advertisement putting precision, high manual dependence and difficult cold start in the prior art are solved, and the putting efficiency and ROI are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital advertising, and particularly relates to an advertisement putting method and device based on machine learning. BACKGROUND

[0002] With the rapid development of mobile Internet and big data technology, digital marketing has become the core channel for enterprises to promote products. In particular, the popularization of real-time bidding (RTB) advertising mechanism makes the advertisement putting change from the traditional "purchase media position" to a more accurate "purchase target audience". In the RTB ecosystem, when a user visits a media page, an advertisement transaction platform (Ad Exchange) will send a bidding request to a demand-side platform (DSP), and the DSP needs to judge the value of the traffic according to user information, context environment and other data within a very short time (usually tens of milliseconds), and decide whether to bid and the specific bid amount, in order to obtain the maximum advertising effect within the budget.

[0003] The existing advertisement putting technology mainly relies on artificial experience or rule-based automatic system. In the traditional putting mode, the advertisement optimizer needs to manually set the crowd targeting condition, bidding strategy and budget allocation according to historical experience, and needs to continuously and closely monitor the advertisement putting data for manual adjustment. This way not only has low operation efficiency, consumes a lot of labor cost, and is extremely dependent on the personal experience of the optimizer, the subjectivity of the decision is strong, and it is difficult to cope with massive, high-dimensional real-time bidding data. With the increasing dispersion of media channels and the diversification of consumer behavior patterns, simply relying on artificial or simple rules cannot meet the needs of large-scale and fine-grained putting.

[0004] Although some advertising systems have begun to introduce machine learning techniques (such as logistic regression, simple neural networks) in recent years to predict click-through rate (CTR) or conversion rate (CVR) to assist bidding, the existing technical solutions still have significant defects. First, in the management and matching of advertising creatives, the existing technologies mostly rely on manual labeling of materials or only based on simple keyword matching, failing to fully utilize computer vision and other technologies for deep feature extraction and analysis of creative materials, resulting in low matching degree of advertising content with user interest and media environment, and making it difficult to achieve truly personalized creative recommendation. Second, in terms of bidding strategy, the existing methods often treat each bidding request as an independent event or use a static linear bidding formula, lacking global consideration and dynamic adaptation ability to changes in the remaining budget state of the advertising account, all-day traffic distribution and market competition environment, easily leading to uneven budget allocation, such as budget depletion before the traffic peak period or missing high-quality traffic. In addition, the training and parameter optimization of the existing model usually uses traditional gradient descent algorithm, which is easy to fall into local optimal solution when facing complex non-convex optimization problem, resulting in limited generalization ability and prediction accuracy of the model, and unable to continuously maintain optimal delivery effect in the highly dynamic real-time bidding environment.

[0005] Therefore, there is an urgent need for an advertising delivery method that can deeply mine the value of historical interaction data, intelligently match creative materials and adaptively optimize bidding strategies. SUMMARY

[0006] The present application provides a machine learning-based advertising delivery method and device, which solves the problems of low advertising delivery precision, high manual intervention and poor strategy adaptability in the prior art.

[0007] In a first aspect, the present application provides a machine learning-based advertising delivery method, which comprises:

[0008] Obtaining delivery dimension information, pre-delivery creative materials and historical advertising delivery interaction data of the advertising project to be delivered; the historical advertising delivery interaction data includes user physical behavior feature data, context environment data and corresponding historical delivery feedback labels;

[0009] Based on the historical advertising delivery interaction data, an advertising delivery decision model is constructed; the advertising delivery decision model includes a user response prediction network for predicting user click-through rate and a reinforcement learning network for generating bidding strategies;

[0010] The multi-strategy co-evolution algorithm is used for initializing and iteratively optimizing training of the hyperparameters of the advertisement launching decision model, so as to obtain a trained advertisement launching decision model; the multi-strategy co-evolution algorithm sequentially performs information weighted learning, rotation information learning, chain combination learning and global information learning to update the model parameter vector until a convergence condition is met;

[0011] Real-time receiving of an advertisement bidding request, and parsing of real-time user features and media environment features in the bidding request;

[0012] Inputting the real-time user features, the media environment features and the remaining budget state of the current advertisement account into the trained advertisement launching decision model to generate a target bidding strategy for the bidding request;

[0013] Based on the launching dimension information, feature extraction and dimension reduction coding are performed on the pre-launching creative material, and similar creative materials matching the feature coding of the pre-launching creative material are queried in the preset creative material library;

[0014] According to the target bidding strategy and the historical launching effect data of the similar creative material, a final advertisement launching instruction is generated and launching operation is performed, and feedback data after launching is collected in real time to update the historical advertisement launching interaction data.

[0015] Preferably, the step of initializing and iteratively optimizing training of the hyperparameters of the advertisement launching decision model by using the multi-strategy co-evolution algorithm comprises:

[0016] A network topology structure of the advertisement launching decision model is constructed, and network weight parameters are randomly initialized between a preset upper limit and a lower limit of the parameters to generate an initial population comprising a plurality of parameter vectors;

[0017] For each parameter vector in the population, an adaptability value thereof on the historical advertisement launching interaction data is calculated; the adaptability value is calculated based on a loss function between a model prediction result and a historical feedback label;

[0018] According to a preset number of iterations, a parameter vector is updated by sequentially performing an information weighted learning strategy for global exploration, a rotation information learning strategy for local development, a chain combination learning strategy for feature fusion, and a global information learning strategy for mutation disturbance;

[0019] The target optimal vector after iteration is taken as the final network weight parameter of the advertisement launching decision model.

[0020] Preferably, a specific update formula of the information weighted learning strategy is:

[0021]

[0022] wherein the fitness value is used to calculate the weighted weight The calculation of the weighted weight is based on three different parameter vectors randomly selected The fitness of the parameter vector

[0023] wherein, represents the parameter vector in the first training process, represents the parameter vector after information weighted learning; represents the parameter vector after information weighted learning; is a random number between 0 and 1.

[0024] Preferably, the specific update formula of the rotation information learning strategy is:

[0025]

[0026] wherein, represents the parameter vector in the first training process, represents the parameter vector after information weighted learning; represents the parameter vector after information weighted learning; represents the optimal parameter vector in the current iteration; is an inertia weight, which decreases linearly with the increase of the iteration number; is a spiral path factor, which is used to control the shape of the search path; is a random number between 0 and 1, which is used to control the step size of the spiral search.

[0027] Preferably, the reinforcement learning network adopts a deep network architecture based on multi-level budget constraints, and the construction of the advertisement delivery decision model includes:

[0028] Modeling the advertisement delivery process as a set of multi-level Markov decision processes according to different levels of budget constraints, defining the state space , action space and reward function ;

[0029] The state space includes the current remaining budget , the current remaining bidding step number and the feature vector of the current bidding request ;

[0030] For each level of Markov decision process, a corresponding deep network sub-model is constructed, and the input state variable is processed;

[0031] ​​Generating a target bidding strategy for the bidding request, specifically including calculating a weighted sum of all sub-model outputs, with the formula being:

[0032]

[0033] wherein, is the total number of layers, is the weight coefficient of the i-th layer, is the output value of the i-th layer sub-model, is the i-th layer's reduced observation value of the state.

[0034] Preferably, the reduction processing of the input state variable includes reduction of the remaining budget and the remaining step length, with the reduction formula being:

[0035]

[0036]

[0037] wherein, is the reduced remaining step length of the i-th layer, is the reduced remaining budget of the i-th layer, is the total number of bidding steps, is a preset reduction ratio factor; the weight coefficient satisfies wherein . .

[0038] Preferably, the pre-delivery creative material includes a video material, and based on the delivery dimension information, feature extraction and dimension reduction coding are performed on the pre-delivery creative material, including:

[0039] extracting frame rate feature vectors and resolution feature vectors of each frame image in the video material;

[0040] respectively performing average pooling dimension reduction processing on the frame rate feature vectors and the resolution feature vectors to obtain reduced frame rate feature vectors and reduced resolution feature vectors;

[0041] splicing and feature converting the reduced frame rate feature vectors and the reduced resolution feature vectors to generate a global feature vector of the video material;

[0042] hash coding the global feature vector to generate a video feature code of the video material;

[0043] ​​​​​​​Based on the video feature encoding and the preset check code, the stock creative materials with the matching feature encoding are retrieved from the creative material library as the similar creative materials.

[0044] Preferably, based on the target bidding strategy and the delivery effect data of the similar creative materials, the final advertisement delivery instruction is generated, including:

[0045] The historical click-through rate of the similar creative materials And the historical conversion rate ;

[0046] The estimated click-through rate of the current user output by the user response prediction network is combined To calculate the estimated value of the current bidding request ;

[0047] According to the target bidding action output by the reinforcement learning network , the final bid is calculated , the calculation formula is:

[0048]

[0049] Among them, The bid scaling factor adjusted by the action , the scaling factor is used to smooth the budget consumption in the whole delivery period;

[0050] If the final bid Is higher than the bidding base price and the current remaining budget is sufficient, the advertisement delivery instruction containing the final bid and the pre-delivery creative material is generated.

[0051] Preferably, the reinforcement learning network further includes a reward prediction network, which collects feedback data after delivery in real time to update the historical advertisement delivery interaction data, including:

[0052] Monitor the exposure, click and conversion events after the advertisement delivery, and generate experience data four-tuple containing state, action, reward and next state ;

[0053] Using the reward prediction network to predict the long-term return value Based on the current state And action , and the long-term return value is fused with the immediate reward To generate a corrected reward signal;

[0054] Using the corrected reward signal and the multi-strategy co-evolution algorithm to fine-tune the parameters of the reinforcement learning network online.

[0055] In the second aspect, the embodiments of the present application provide an advertisement delivery device based on machine learning, the device comprising:

[0056] The data acquisition module is used to acquire the ad delivery dimension information, pre-delivery creative materials, and historical ad delivery interaction data of the ad projects to be delivered; historical ad delivery interaction data includes user physical behavior characteristic data, contextual environment data, and corresponding historical delivery feedback tags.

[0057] The ad placement decision module is used to build an ad placement decision model based on historical ad placement interaction data. The ad placement decision model includes a user response prediction network for predicting user click-through rates and a reinforcement learning network for generating bidding strategies.

[0058] The decision model optimization module is used to initialize and iteratively optimize the hyperparameters of the advertising placement decision model using a multi-strategy co-evolutionary algorithm to obtain a trained advertising placement decision model. The multi-strategy co-evolutionary algorithm sequentially performs information weighted learning, rotation information learning, chain combination learning, and global information learning to update the model parameter vector until the convergence condition is met.

[0059] The bidding request receiving module is used to receive advertising bidding requests in real time and parse the real-time user characteristics and media environment characteristics in the bidding requests;

[0060] The target bidding strategy module is used to input real-time user characteristics, media environment characteristics, and the current remaining budget status of the advertising account into the trained advertising delivery decision model to generate a target bidding strategy for the bidding request.

[0061] The similar creative material acquisition module is used to extract features and perform dimensionality reduction encoding on the creative materials to be deployed based on the deployment dimension information, and to query similar creative materials that match the feature encoding of the pre-deployed creative materials in the pre-set creative material library;

[0062] The ad delivery module is used to generate the final ad delivery instructions and execute the delivery operation based on the target bidding strategy and historical performance data of similar creative materials, and to collect feedback data after delivery in real time to update historical ad delivery interaction data.

[0063] Beneficial effects:

[0064] By constructing an advertising delivery decision model incorporating a user response prediction network and a reinforcement learning network, and employing a multi-strategy co-evolutionary algorithm for iterative optimization—sequentially performing information weighting, information rotation, chain combination, and global information learning—this method effectively overcomes the tendency of traditional gradient descent algorithms to get trapped in local optima, significantly improving the model's generalization ability and prediction accuracy, thus maintaining robustness in highly dynamic real-time bidding environments. This method deeply integrates real-time user characteristics, media environment characteristics, and the current account's remaining budget status, using reinforcement learning to generate target bidding strategies. This solves the problem of uneven budget allocation caused by the lack of global consideration of budget status in existing technologies, achieving accurate capture of high-quality traffic under budget constraints. Furthermore, by extracting features and performing dimensionality reduction encoding on pre-delivery creative materials, and matching similar creative materials in a database, the historical performance of similar materials guides the delivery of new creatives, breaking through the limitations of traditional manual tagging or keyword matching and significantly improving the matching degree between ad content and user interests. This method achieves a fully automated closed loop from strategy optimization to creative matching, significantly reducing labor costs and improving ad click-through rates and overall ROI. Attached Figure Description

[0065] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0066] Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention;

[0067] Figure 2 This is a flowchart illustrating the steps of an advertising delivery method based on machine learning, as provided in an embodiment of the present invention.

[0068] Figure 3 This is a schematic diagram of the functional modules of an advertising delivery device based on machine learning provided in an embodiment of the present invention. Detailed Implementation

[0069] To more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is merely some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of aiding understanding the invention, but does not constitute a limitation on the invention. The solution of the present invention will be further described below in conjunction with the accompanying drawings.

[0070] Reference Figure 1 , Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0071] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0072] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0073] like Figure 1 As shown, the memory 1005, as a storage medium, may include an operating device, a data storage module, a network communication module, a user interface module, and an electronic program. In the illustrated electronic device, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the electronic device of the present invention can be disposed in the electronic device, and the electronic device calls the machine learning-based advertising delivery device stored in the memory 1005 through the processor 1001, and executes the machine learning-based advertising delivery method provided in the embodiments of the present invention.

[0074] Reference Figure 2This invention provides a machine learning-based advertising delivery method to address the technical challenges of traditional advertising delivery, such as high reliance on manual intervention, rigid bidding strategies, difficulties in cold start, and the tendency for model optimization to get trapped in local optima. By constructing a closed-loop automated delivery process, this embodiment can perceive market dynamics in real time, accurately predict user intent, and dynamically adjust bidding and creative strategies, thereby maximizing the advertiser's return on investment (ROI). The method process described in this embodiment may specifically include the following steps S101 to S107:

[0075] S101, obtain the placement dimension information, pre-placement creative materials, and historical ad placement interaction data of the ad project to be placed; the historical ad placement interaction data includes user physical behavior characteristic data, contextual environment data, and corresponding historical placement feedback tags.

[0076] This step aims to provide a comprehensive data foundation for subsequent model building and strategy formulation. The data acquisition module collects multi-source heterogeneous data through Application Programming Interface (API), Software Development Kit (SDK) tracking, and log servers. Specifically, step S101 includes the following sub-steps:

[0077] S1011, Obtain the placement dimension information for the advertising project to be placed. The placement dimension information defines the basic framework and constraints of the advertising campaign. The device receives user-configured parameters through the Advertiser Service Platform. This information specifically includes:

[0078] Basic attributes include the advertiser's industry category (e.g., e-commerce, gaming, finance), brand positioning, and target audience profile (age, gender, geographic distribution, spending power, etc.). Budget constraints include the total budget, daily budget cap, and budget smoothing strategy requirements (e.g., uniform or accelerated delivery). Bidding models support various models such as Cost Per Click (CPC), Cost Per Mille (CPM), and Optimized Cost Per Action (oCPA). Time and space constraints include the time period for ad placement (e.g., morning / evening peak hours), media channels (e.g., app list, website domain), and geo-fencing.

[0079] S1012, Acquire Pre-delivery Creative Materials. Creative materials are the core medium for advertising to reach users. The device receives multimedia materials uploaded by advertisers, including images (JPG, PNG), videos (MP4, MOV), text (titles, descriptions), and rich media (HTML5). For each pre-delivery creative material, the device assigns a unique material ID and extracts its metadata, such as resolution, duration, file size, and bitrate. Furthermore, the device performs preliminary analysis of the material content, such as extracting text information from images using Optical Character Recognition (OCR) and obtaining representative frames from videos using keyframe extraction technology, preparing for subsequent feature extraction.

[0080] S1013, Obtain historical ad interaction data. Historical data is the foundation for training machine learning models. This embodiment particularly emphasizes the mining of deep interaction data, rather than merely focusing on impressions and clicks. Historical ad interaction data includes: user physical behavior characteristic data, which refers to the microscopic physical signals generated by users when operating terminal devices, reflecting the user's subconscious interests and current state. Specifically, it includes:

[0081] Touch trajectory refers to the path, speed, and acceleration of a user's swipes on the screen. For example, a slow, meandering swipe might indicate that the user is reading carefully, while a fast, straight swipe might mean that the user is quickly browsing or ignoring content. Press pressure, if supported by the device hardware (e.g., 3D Touch), measures the pressure applied when the user taps the screen. High pressure values ​​are usually associated with a specific tap intention. Device attitude measures the device's tilt angle and movement using a gyroscope and accelerometer. For example, a user changing from lying down to sitting up, or a device moving from a stationary position, might indicate a shift in the user's attention.

[0082] Contextual data refers to the external environment and device status at the time of ad display. Specifically, it includes:

[0083] Network status—Wi-Fi, 4G, 5G, or weak network conditions—affects video ad loading strategies. Device status—battery level (users may be less willing to click on data-intensive ads in low-power mode), screen brightness, and volume settings (video ad effectiveness may be reduced in silent mode). Time and context—specific timestamps, whether it's a holiday, and the type of app the user is currently using (e.g., users may be more inclined to interact in social media apps, or to complete tasks quickly in utility apps).

[0084] Historical delivery feedback tags serve as the target variables for supervised learning. In addition to standard binary tags such as "Click" and "Conversion," these include continuous tags like "video playback duration," "landing page dwell time," and "scrolling depth." These tags form the foundation of multi-task learning.

[0085] S102, Based on the historical advertising interaction data, an advertising placement decision model is constructed; the advertising placement decision model includes a user response prediction network for predicting user click-through rates and a reinforcement learning network for generating bidding strategies.

[0086] This step constructs the core decision-making brain of the device, consisting of two cooperating deep neural networks: one responsible for "prediction" and the other for "decision-making." Specifically, step S102 includes the following sub-steps:

[0087] S1021, Construct the User Response Prediction Network. The User Response Prediction Network (URPN) aims to predict the probability (predicted click-through rate, pCTR) of a specific user responding to a specific advertisement based on the input feature vector. This network employs a deep neural network architecture, such as a Deep Factorization Machine (DeepFM) or a Deep Interest Network (DIN). The network input layer receives preprocessed features, including:

[0088] User features include the embedding vector of the user ID and user profile tags. Ad features include the embedding vector of the creative ID and material features (see S106 for details). Contextual features are the environmental data extracted in S1013. The network structure includes a feature interaction layer for automatically learning the combined relationships between features (e.g., the combined effect of "young male" + "sneaker ad" + "10 PM"). The output layer uses the Sigmoid activation function, outputting probability values ​​in the range (0,1).

[0089] S1022, Construct the Reinforcement Learning Network. A Reinforcement Learning Network (RLN) is used to generate optimal bidding strategies in a Real-Time Bidding (RTB) environment. Considering the criticality of budget constraints in advertising, this embodiment adopts a deep network architecture based on multi-level budget limitations. The construction process is as follows:

[0090] The model is based on a Markov Decision Process (MDP), which models the advertising delivery process as a multi-level set of MDPs. Each level corresponds to a different budget consumption stage or granularity.

[0091] Define the state space state space Including the current remaining budget Current remaining bidding steps (e.g., remaining campaign time or expected remaining impressions), and the feature vector of the current bidding request. (High-level features extracted from S1021). That is... .

[0092] Define action space ,action Defined as bid adjustment factor , used to adjust the base bid.

[0093] Define reward function ,award Set it as a KPI that advertisers care about, such as number of clicks or conversion amount.

[0094] Constructing deep network sub-models: For each level of the MDP, construct a deep network sub-model. Deep Q-Network (DQN) sub-models. These sub-models share some low-level parameters, but make independent decisions at higher levels for different budget granularities.

[0095] Strategy generation mechanism: The generation of the target bidding strategy for the bidding request specifically includes calculating the outputs of all sub-models. The weighted sum of the values. The formula is:

[0096]

[0097] In the formula: The total number of layers indicates the depth of the multi-level architecture. This is the index of the current layer number. . For the first The weight coefficients of the layers are used to balance the influence of different level sub-models. For the first Layer model output Value (action value function). For the first Calculated observations of layer-to-state conditions (the specific calculation method is detailed in S105).

[0098] S103, the hyperparameters of the advertising placement decision model are initialized and iteratively optimized using a multi-strategy co-evolutionary algorithm to obtain a trained advertising placement decision model; the multi-strategy co-evolutionary algorithm sequentially performs information weighted learning, rotational information learning, chain-based combination learning, and global information learning to update the model parameter vector until the convergence condition is met.

[0099] To overcome the shortcomings of traditional gradient descent algorithms, such as susceptibility to local optima and sensitivity to initial values ​​in non-convex optimization problems, this embodiment introduces a multi-strategy co-evolutionary algorithm. This algorithm treats the network weight parameters of the advertising decision-making model as evolving individuals, seeking the global optimum through collective intelligence. Specifically, step S103 includes the following sub-steps:

[0100] S1031, Initialize the population. Construct the network topology of the advertising delivery decision model, and within the preset parameter upper limit. With parameter lower limit The network weight parameters are randomly initialized to generate an initial population containing multiple parameter vectors. Let the population size be... , No. The parameter vector of each individual is denoted as... The initialization formula is:

[0101]

[0102] in, A random number between 0 and 1.

[0103] S1032, Calculate fitness. For each parameter vector in the population, calculate its fitness value on the historical ad delivery interaction data. The fitness value is calculated based on the loss function between the model prediction result and the historical feedback labels. For user response prediction networks, the loss function can be logarithmic loss (LogLoss); for reinforcement learning networks, fitness can be defined as the cumulative reward over the backtesting period. Let the... The fitness value of an individual is To subsequently calculate the weights, the loss function value is typically transformed into a maximization problem, for example... .

[0104] S1033 executes an information-weighted learning strategy. This strategy utilizes the fitness differences among individuals in the population to construct a weighted update vector. The specific update formula is as follows:

[0105]

[0106] in, , indicating the first The first training process A parameter vector (current individual); , indicating the first The parameter vector updated by each individual after information-weighted learning; , which are three different parameter vectors randomly selected from the current population (different from the three parameter vectors selected from the current population). ); , is a random number between (0,1), used to increase the random perturbation of the search; , where is the weighted weight calculated based on fitness. The weight is calculated by using the fitness value to calculate the weighted weight, and then calculating the fitness percentage based on three randomly selected parameter vectors. Let . The corresponding fitness values ​​are respectively The formula for calculating the weight is: This strategy uses fitness weighting, which allows better-performing individuals to have a greater influence on the direction of updates, thereby guiding the population to evolve towards high-fitness regions.

[0107] S1034 executes the Rotational Information Learning Strategy. Based on information-weighted learning, a rotation mechanism is introduced for local development updates, enhancing the algorithm's ability to detect local extrema. The specific update formula is as follows:

[0108]

[0109] In the formula: Indicates the first The first training session after information-weighted learning A parameter vector. This represents the parameter vector after learning rotational information. This represents the optimal parameter vector in the current iteration. This is an inertial weight that decreases linearly with the number of iterations. For example... ,in This represents the maximum number of iterations. This mechanism allows the algorithm to have a large exploration step size in the early stages and a small development step size in the later stages. This is the helical path factor, used to control the curvature shape of the logarithmic spiral; The value is a random number between (-1, 1), whose sign determines the direction of the spiral search (clockwise or counterclockwise), and whose absolute value determines the step size of the search. The algorithm can perform comprehensive and detailed exploration of local regions by mitigating random variations. This strategy simulates spiral search behavior, enabling individuals to conduct refined local searches around the current optimal solution, thus improving convergence accuracy.

[0110] S1035, Execute the Chain Combination Learning Strategy. To further promote information flow within the population and break local optima, this embodiment introduces a chain structure. The specific update formula is as follows:

[0111]

[0112] In the formula: Indicates the first The next iteration (corresponding to the above) ) A parameter vector after learning rotational information. This represents the parameter vector after chain-based combination learning. This is a perturbation factor used to control the update amplitude. The first combined learning factor controls the degree to which a position moves closer to the group's average dominant position. The second combined learning factor controls the degree of learning from neighboring individuals. The parameter vector of the next individual in the chain (if If the last individual is selected, then the first individual or a random individual can be chosen. This adjacent interaction mechanism forms a "chain reaction" that can effectively transmit high-quality information. This represents the average parameter vector, indicating the compromise direction between the current individual and the optimal individual.

[0113] S1036, Execute the Global Information Learning Strategy. Finally, to maintain population diversity and prevent premature convergence, a mutation-based global search is introduced. The specific update formula is:

[0114]

[0115] In the formula: Indicates the first During the training process, the first The parameter vector after learning by chain combination. Indicates the first The parameter vector after learning global information. This means that the mean is 0. The Gaussian variation coefficient is generated by the Gaussian variation of the variance. This is a Gaussian mutation control factor that can be dynamically adjusted (e.g., gradually decreased) during iterations. This strategy, by introducing random noise, gives the algorithm the ability to escape local optima, ensuring global convergence.

[0116] S1037, Iterative Control. S1033 to S1036 constitute a complete iterative cycle. After each iteration, the fitness of the newly generated population is calculated, and the global optimum is updated. The convergence criteria are determined as follows: The preset maximum number of iterations is reached. The improvement in the optimal fitness value is less than a preset threshold for several consecutive iterations. The fitness value reaches a preset target value. If the convergence criteria are met, training stops, and the target optimal vector after the iterations is used as the final network weight parameters of the advertising decision-making model; otherwise, the next iteration begins.

[0117] S104, receive advertising bidding requests in real time, and analyze the real-time user characteristics and media environment characteristics in the bidding requests.

[0118] After the model training is completed and deployed online, the device enters the real-time inference phase. Specifically, step S104 includes the following sub-steps:

[0119] S1041, Receive Bid Request. The ad delivery device receives bid requests in real time from the ad exchange via the OpenRTB protocol interface. This request is typically in JavaScript Object Notation (JSON) format and contains detailed information about the current exposure opportunity.

[0120] S1042, Parse Real-Time User Features. Based on the Device ID (such as Advertising Identifier IDFA, International Mobile Equipment Identity (IMEI), Open Anonymous Device Identifier (OAID)) or User ID in the request, retrieve the user's real-time features from the user profile database in the Data Management Platform (DMP) within milliseconds. Real-time user features include: static attributes (gender, age group, occupation); dynamic interests (recently viewed product categories, search keywords, list of installed applications); and real-time status (current activity level, last interaction time).

[0121] S1043, parse media environment characteristics. Parse the contextual information carried in the request: media attributes, App name, App category (e.g., news, video, social), page keywords. Ad placement attributes, ad size (e.g., 320x50), location (first screen, position in the news feed), display format (banner, interstitial, native). Spatiotemporal attributes, current timestamp, user's Global Positioning System (GPS) coordinates, city code, network type.

[0122] S105, input the real-time user characteristics, media environment characteristics and the current remaining budget status of the advertising account into the trained advertising delivery decision model to generate a target bidding strategy for the bidding request.

[0123] This step utilizes the trained model for real-time decision-making. Specifically, step S105 includes the following sub-steps:

[0124] S1051, construct state variables and perform transformation processing. First, the features obtained from S104 are vectorized (e.g., using embedding) to obtain feature vectors. Simultaneously, it allows real-time monitoring of advertising account devices to obtain the current remaining budget. and the current remaining bidding steps (For example, if the campaign period is 24 hours and the current time is 12:00, then there are 12 hours remaining, which can be converted into the remaining estimated number of bidding requests.) To adapt to the multi-level budget-constrained deep network architecture built in S1022, the state variables need to be tiered and discounted. The discount formula is:

[0125]

[0126]

[0127] In the formula: This is the index of the current layer number. . For the first The remaining step size after layer reduction. It represents the number of steps remaining within the "micro" time window of the current layer. For the first The remaining budget after tier discounting. It represents the budget quota allocated to the current tier's time window. This refers to the total number of bidding steps (the total duration or total number of requests during the entire campaign period). This represents the actual remaining total budget at the current moment. This represents the actual total number of steps remaining at the current moment. In this embodiment, the preset conversion ratio factor is used. The preferred setting is 10. This means that for each higher-level sub-model, the time granularity it focuses on expands tenfold, thereby achieving multi-scale budget control from minutes to hours to days. Through this adjustment, the model can perceive budget pressure at different time granularities. (Higher-level...) Focus on macro-level budget allocation, lower-level (small) Focus on micro-level bidding decisions.

[0128] S1052, Generate target bidding strategy.

[0129] The converted state variables The input is fed into the corresponding deep network sub-model, and the overall Q-value is calculated. The specific calculation formula is as follows:

[0130]

[0131] in, , where is the total number of levels, representing the depth of the multi-level budget constraint model; , is the index of the current layer. ; , for the first The weight coefficients of the layer satisfy the following conditions: (in This is used to balance the influence of sub-models at different levels; the lower the level (the more microscopic), the greater the weight. , for the first Layer model output Value estimation (action value function); , for the first The layer-wise reduced observations of the state, specifically including the reduced residual budget. and remaining steps (For the conversion formula, please refer to step S1051).

[0132] S106. Based on the delivery dimension information, perform feature extraction and dimensionality reduction encoding on the pre-delivery creative material, and query the preset creative material library for similar creative materials that match the feature encoding of the pre-delivery creative material.

[0133] To address the issue of a lack of historical data for new materials (cold start), this embodiment utilizes the historical performance of similar materials to aid decision-making. Specifically, step S106, taking video materials as an example, includes the following sub-steps:

[0134] S1061, Feature Extraction. Extract the frame rate feature vector and sharpness feature vector from each frame of the video footage. The frame rate feature vector is not a single FPS value, but rather a sequence of instantaneous frame rate fluctuations counted over a fixed time window (e.g., per second) during video playback, or a dynamically changing feature vector calculated based on inter-frame differences. It reflects the pace and smoothness stability of the video content. The frame rate feature vector reflects the smoothness and the intensity of dynamic changes in the video. The sharpness feature vector reflects the texture richness and blurriness of the video image. These underlying visual features directly affect the user's viewing experience.

[0135] S1062, Dimensionality Reduction. Average pooling is performed on both the frame rate feature vector and the resolution feature vector to reduce their dimensionality, resulting in a dimensionality-reduced frame rate feature vector and a dimensionality-reduced resolution feature vector. Pooling compresses the data size while preserving the main distribution information of the features, thus improving computational efficiency.

[0136] S1063, Feature Fusion and Encoding. The reduced frame rate feature vector and the reduced resolution feature vector are concatenated and transformed (e.g., through mapping via a fully connected layer) to generate a global feature vector for the video footage. The global feature vector is then hashed to generate the video feature code (a string of binary code) for the video footage. Commonly used hashing algorithms include Locality Sensitive Hashing (LSH) or Perceptual Hashing (pHash).

[0137] S1064, Similar Material Query. Based on the video feature encoding and a preset checksum, such as the Message-Digest Algorithm 5 (MD5) value of the material to prevent duplication, existing creative materials with matching feature encodings are retrieved from a preset creative material library as the similar creative materials. The matching criterion can be that the Hamming distance is less than a preset threshold. The creative material library stores a large number of historically deployed materials and their deployment effect data.

[0138] S107. Based on the target bidding strategy and the historical performance data of similar creative materials, generate the final ad delivery instruction and execute the delivery operation, and collect feedback data after delivery in real time to update the historical ad delivery interaction data.

[0139] This step is the final step in the closed loop, completing the decision execution and device evolution. Specifically, step S107 includes the following sub-steps:

[0140] S1071, Calculate the final bid. Obtain the historical click-through rate of the similar creative materials. and historical conversion rate This data provides prior knowledge of the potential performance of new content. Combined with the estimated click-through rate for the current user from the output of the User Response Prediction Network (URPN),... Calculate the estimated value of the current bid request. .For example:

[0141] ,

[0142] in, For target conversion cost, The similarity factor is used to determine the target bidding action (i.e., bid adjustment coefficient) output by the reinforcement learning network. ), calculate the final bid The calculation formula is:

[0143]

[0144] in, For the action Adjusted bid scaling factor. In this embodiment, It can be designed as an exponential function or a linear function, for example or Its core logic is: when the reinforcement learning model outputs an action... When the value is large (indicating a high willingness to bid), The value decreases, thus reducing the final bid. Increase the bid; conversely, decrease it. The scaling factor is used to smooth budget consumption throughout the entire campaign period. For example, if the action... The instruction to "actively bid" will then , making Increase; if the action If the instruction is "conservative bidding", then , making This reduces costs. This achieves a balance between budget control and value maximization.

[0145] S1072, Generate and execute the placement instruction. If the final bid... If the bid is higher than the floor price in the bidding request and there is sufficient remaining budget, an ad delivery instruction containing the final bid and the pre-delivery creative material, or its Content Delivery Network (CDN) link, is generated and sent to the ad exchange platform to participate in the bidding.

[0146] S1073, Data Acquisition and Model Update. Real-time monitoring of impressions, clicks, and conversions after ad delivery. Generation of empirical data quadruples containing state, action, reward, and next state. Among them, immediate rewards This could be the direct revenue generated from this demonstration (such as click value). To address the reward sparsity problem caused by ad conversion delays, the reinforcement learning network also includes a reward prediction network. This reward prediction network is used to predict rewards based on the current state. and actions Predicting long-term returns (For example, the predicted conversion probability within the next 24 hours). Combine the aforementioned long-term return value with the immediate reward. The signals are merged to generate a revised reward signal. The parameters of the reinforcement learning network are fine-tuned online using the corrected reward signal and the multi-strategy co-evolutionary algorithm (or combined with gradient descent). Simultaneously, newly collected interaction data is stored in a historical database for subsequent full-scale training, enabling continuous iteration and evolution of the device.

[0147] In summary, this invention, through constructing a multi-level budget-constrained reinforcement learning model and optimizing it using an innovative multi-strategy co-evolutionary algorithm, combined with content-based creative cold-start technology, achieves a highly efficient, intelligent, and adaptive advertising delivery method. This method not only accurately predicts user value but also dynamically adjusts bidding strategies under complex budget constraints, effectively improving the efficiency and effectiveness of advertising delivery.

[0148] This invention also provides an advertising delivery device based on machine learning, referring to... Figure 3 The diagram illustrates a functional block diagram of an advertising delivery device 300 based on machine learning according to the present invention. The device may include the following modules:

[0149] The data acquisition module 301 is used to acquire the advertising dimension information, pre-delivery creative materials, and historical advertising interaction data of the advertising project to be delivered; the historical advertising interaction data includes user physical behavior characteristic data, contextual environment data, and corresponding historical delivery feedback tags.

[0150] The advertising placement decision module 302 is used to build an advertising placement decision model based on historical advertising placement interaction data. The advertising placement decision model includes a user response prediction network for predicting user click-through rates and a reinforcement learning network for generating bidding strategies.

[0151] The decision model optimization module 303 is used to initialize and iteratively optimize the hyperparameters of the advertising placement decision model using a multi-strategy co-evolutionary algorithm to obtain a trained advertising placement decision model. The multi-strategy co-evolutionary algorithm sequentially performs information weighted learning, rotation information learning, chain combination learning, and global information learning to update the model parameter vector until the convergence condition is met.

[0152] The bidding request receiving module 304 is used to receive advertising bidding requests in real time and parse the real-time user characteristics and media environment characteristics in the bidding requests.

[0153] The target bidding strategy module 305 is used to input real-time user characteristics, media environment characteristics, and the current remaining budget status of the advertising account into the trained advertising delivery decision model to generate a target bidding strategy for the bidding request.

[0154] The similar creative material acquisition module 306 is used to extract features and perform dimensionality reduction encoding on the creative materials to be deployed based on the deployment dimension information, and to query similar creative materials that match the feature encoding of the pre-deployed creative materials in the pre-set creative material library;

[0155] The ad delivery module 307 is used to generate the final ad delivery instruction and execute the delivery operation based on the target bidding strategy and historical delivery performance data of similar creative materials, and to collect feedback data after delivery in real time to update historical ad delivery interaction data.

[0156] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0157] Memory, used to store computer programs;

[0158] When a processor executes a program stored in memory, it implements the machine learning-based advertising delivery method of the present invention.

[0159] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM), or non-volatile memory, such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0160] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0161] Furthermore, to achieve the above objectives, embodiments of the present invention also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the machine learning-based advertising delivery method of the embodiments of the present invention.

[0162] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable hardware devices (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0166] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0167] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A machine learning-based ad delivery method, characterized in that, The method includes: Obtain the ad placement dimension information, pre-placement creative materials, and historical ad placement interaction data for the ad projects to be placed; the historical ad placement interaction data includes user physical behavior characteristic data, contextual environment data, and corresponding historical placement feedback tags. Based on the historical ad delivery interaction data, an ad delivery decision model is constructed; the ad delivery decision model includes a user response prediction network for predicting user click-through rates and a reinforcement learning network for generating bidding strategies; The hyperparameters of the advertising placement decision model are initialized and iteratively optimized using a multi-strategy co-evolutionary algorithm to obtain a trained advertising placement decision model. The multi-strategy co-evolutionary algorithm sequentially performs information weighted learning, rotational information learning, chain combination learning, and global information learning to update the model parameter vector until the convergence condition is met. Receive advertising bidding requests in real time and analyze the real-time user characteristics and media environment characteristics in the bidding requests; The real-time user characteristics, media environment characteristics, and the current remaining budget status of the advertising account are input into the trained advertising delivery decision model to generate a target bidding strategy for the bidding request. Based on the aforementioned delivery dimension information, feature extraction and dimensionality reduction encoding are performed on the pre-delivery creative materials, and similar creative materials matching the feature encoding of the pre-delivery creative materials are queried in a pre-set creative material library; Based on the target bidding strategy and the historical performance data of similar creative materials, the final ad delivery instruction is generated and the delivery operation is executed. Feedback data after delivery is collected in real time to update the historical ad delivery interaction data.

2. The machine learning-based ad delivery method according to claim 1, characterized in that, The steps of initializing and iteratively optimizing the hyperparameters of the advertising delivery decision model using a multi-strategy co-evolutionary algorithm include: Construct the network topology of the advertising delivery decision model, and randomly initialize the network weight parameters between the preset upper and lower limits of parameters to generate an initial population containing multiple parameter vectors; For each parameter vector in the population, its fitness value on the historical ad delivery interaction data is calculated; the fitness value is calculated based on the loss function between the model prediction result and the historical feedback label; According to the preset number of iterations, the information weighted learning strategy is executed sequentially to perform global exploration and update of the parameter vector, the rotation information learning strategy is executed to perform local development and update, the chain combination learning strategy is executed to perform feature fusion and update, and the global information learning strategy is executed to perform mutation and perturbation update. The target optimal vector after the iteration is used as the final network weight parameter of the advertising delivery decision model.

3. The machine learning-based advertising delivery method according to claim 2, characterized in that, The specific update formula for the information-weighted learning strategy is as follows: The fitness value is used to calculate the weighted weights. The weighting is calculated based on three randomly selected different parameter vectors. The fitness percentage; In the formula, Indicates the first The first training process A parameter vector This represents the parameter vector after information weighting and learning; for A random number between [a certain number of points].

4. The machine learning-based advertising delivery method according to claim 2, characterized in that, The specific update formula for the rotation information learning strategy is as follows: In the formula, Indicates the first The first training session after information-weighted learning A parameter vector This represents the parameter vector after learning rotational information; This represents the optimal parameter vector in the current iteration; The inertial weight decreases linearly with the number of iterations. This is the spiral path factor, used to control the shape of the search path; for A random number between these values ​​is used to control the step size of the spiral search.

5. The machine learning-based advertising delivery method according to claim 1, characterized in that, The reinforcement learning network employs a deep learning architecture based on multi-level budget constraints. The network architecture, wherein the construction of the advertising delivery decision model includes: The advertising process is modeled as a set of multi-level Markov decision processes based on different levels of budget constraints, and the state space is defined. Action space and reward function ; The state space Including the current remaining budget Current remaining bidding steps and the feature vector of the current bidding request ; For each level of the Markov decision process, construct the corresponding depth The network sub-model is used to perform calculations on the input state variables; The generation of the target bidding strategy for the bidding request specifically includes calculating the outputs of all sub-models. The weighted sum of the values ​​is calculated using the following formula: In the formula, This represents the total number of floors. For the first Layer weight coefficients, For the first Layer model output value, For the first Layer-to-state The converted observation values.

6. The machine learning-based advertising delivery method according to claim 5, characterized in that, The process of reducing the input state variables includes reducing the remaining budget and remaining step size. The reduction formula is as follows: In the formula, For the first The remaining step size after layer reduction For the first The remaining budget after layer conversion Total number of bidding steps The preset conversion ratio factor; the weighting coefficient satisfy ,in .

7. The machine learning-based advertising delivery method according to claim 1, characterized in that, The pre-deployment creative materials include video materials. The step of extracting features and performing dimensionality reduction encoding on the pre-deployment creative materials based on the deployment dimension information includes: Extract the frame rate feature vector and sharpness feature vector of each frame in the video material; The frame rate feature vector and the resolution feature vector are respectively subjected to average pooling dimensionality reduction processing to obtain the dimensionality-reduced frame rate feature vector and the dimensionality-reduced resolution feature vector; The reduced frame rate feature vector and the reduced resolution feature vector are concatenated and transformed to generate the global feature vector of the video material. The global feature vector is hash-encoded to generate the video feature code of the video material; Based on the video feature encoding and the preset check code, existing creative materials with matching feature encodings are retrieved from the creative material library as the similar creative materials.

8. The machine learning-based advertising delivery method according to claim 1, characterized in that, The step of generating the final ad delivery instruction based on the target bidding strategy and the performance data of similar creative materials includes: Get the historical click-through rate of the similar creative materials. and historical conversion rate ; Combined with the estimated click-through rate for the current user output by the user response prediction network. Calculate the estimated value of the current bid request. ; Based on the target bidding action output by the reinforcement learning network Calculate the final bid The calculation formula is: in, For the action An adjusted bid scaling factor, which is used to smooth budget consumption throughout the entire campaign period; If the final bid If the bid is higher than the reserve price and there is sufficient remaining budget, an ad delivery instruction containing the final bid and the pre-delivery creative material will be generated.

9. The machine learning-based advertising delivery method according to claim 1, characterized in that, The reinforcement learning network further includes a reward prediction network, and the real-time collection of feedback data after ad delivery to update the historical ad delivery interaction data includes: Monitor ad impressions, clicks, and conversions after ad placement to generate a quadruple of experience data containing status, action, reward, and next status. ; The reward prediction network is used based on the current state. and actions Forecast long-term returns and combine the long-term return value with the immediate reward The signals are then merged to generate a revised reward signal. The parameters of the reinforcement learning network are fine-tuned online using the modified reward signal and the multi-policy co-evolutionary algorithm.

10. An ad delivery device based on machine learning, characterized in that, The device includes: The data acquisition module is used to acquire the advertising dimension information, pre-delivery creative materials, and historical advertising interaction data of the advertising project to be delivered; the historical advertising interaction data includes user physical behavior characteristic data, contextual environment data, and corresponding historical delivery feedback tags. An advertising placement decision module is used to construct an advertising placement decision model based on the historical advertising placement interaction data; the advertising placement decision model includes a user response prediction network for predicting user click-through rates and a reinforcement learning network for generating bidding strategies; The decision model optimization module is used to initialize and iteratively optimize the hyperparameters of the advertising placement decision model using a multi-strategy co-evolutionary algorithm to obtain a trained advertising placement decision model. The multi-strategy co-evolutionary algorithm sequentially performs information weighted learning, rotational information learning, chain combination learning, and global information learning to update the model parameter vector until the convergence condition is met. The bidding request receiving module is used to receive advertising bidding requests in real time and parse the real-time user characteristics and media environment characteristics in the bidding requests. The target bidding strategy module is used to input the real-time user characteristics, media environment characteristics, and the current remaining budget status of the advertising account into the trained advertising delivery decision model to generate a target bidding strategy for the bidding request. The similar creative material acquisition module is used to extract features and perform dimensionality reduction encoding on the pre-deployment creative materials based on the deployment dimension information, and query similar creative materials that match the feature encoding of the pre-deployment creative materials in the preset creative material library; The ad delivery module is used to generate the final ad delivery instruction and execute the delivery operation based on the target bidding strategy and the historical delivery performance data of similar creative materials, and to collect feedback data after delivery in real time to update the historical ad delivery interaction data.

Images