Advertisement recommendation method and system based on click rate prediction model

Abstract

The invention discloses an advertisement recommendation method and system based on a click-through rate prediction model, and relates to the technical field of advertisement recommendation, including collecting advertisement recommendation feature data, processing through feature engineering, and outputting a user feature vector; inputting a user interest vector into a click-through rate prediction model, training the click-through rate prediction model through a causal optimized cross entropy loss function, and outputting a click probability value for each advertisement; using the click probability value as an input of a reinforcement learning algorithm, combining user interaction behavior, and generating advertisement ranking results through dynamic optimization of a multi-objective reward function; updating a user interest bias value of a causal inference model in real time according to real-time feedback data in the user interaction behavior, and correcting the user interest vector offset; the invention can quickly respond to changes in user behavior by updating the user interest bias value in real time, and improve the timeliness of advertisement recommendation and user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of advertisement recommendation, and in particular to an advertisement recommendation method and system based on a click rate prediction model. Background Art

[0002] With the rapid development of Internet advertising technology, advertising recommendation systems driven by big data have gradually become an important tool for improving user experience and advertising delivery efficiency. Traditional advertising recommendation systems usually rely on collaborative filtering, content-based recommendation, and hybrid recommendation methods to predict the advertisements that users may be interested in by analyzing users' historical behavior data, advertising content characteristics, and the interactive relationship between users and advertisements. However, with the increasing dimensions of user behavior data and the increasing complexity of advertising content, existing recommendation methods face many bottlenecks in terms of accuracy and real-time performance. In recent years, the widespread application of machine learning, especially deep learning, has provided new technical means for advertising recommendation systems, among which recommendation methods based on click-through rate prediction models have become a research hotspot. This type of method uses a click-through rate prediction model to sort advertisements, and uses click-through rate (CTR) as the core indicator to measure user interest, which significantly improves the accuracy of advertising recommendations. However, how to further improve the performance of the click-through rate prediction model, fully explore user interest characteristics, and optimize advertising sorting strategies are still research difficulties in the current advertising recommendation field.

[0003] Although the advertising recommendation method based on the click-through rate prediction model has been widely used, the existing technology still has some shortcomings. First, most click-through rate prediction models usually rely on simple statistical features or manually designed features in the feature extraction stage, and fail to fully consider the temporal nature of user behavior characteristics and the impact of contextual environment on changes in user interests, resulting in the model's capture of user interests is not accurate enough. Secondly, most existing models are supervised and trained through traditional loss functions, ignoring the potential causal relationship in user behavior data, and are easily interfered by non-causal factors, resulting in deviations in click-through rate prediction results. In addition, in the advertising ranking stage, the existing technology is usually based on static optimization objectives, and fails to make full use of users' real-time feedback data to dynamically adjust the advertising ranking strategy, making it difficult to meet personalized recommendation needs. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an advertisement recommendation method based on a click rate prediction model to solve the problems in the prior art of not accurately capturing user interests and failing to meet personalized recommendation needs.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an advertisement recommendation method based on a click rate prediction model, which includes collecting advertisement recommendation feature data, processing it through feature engineering, and outputting a user feature vector;

[0008] Based on the user feature vector, the causal inference model is used to eliminate the interference of non-causal factors, extract the user interest bias, and generate the optimized user interest vector;

[0009] The specific steps of generating the optimized user interest vector are as follows:

[0010] Standardize the user feature vector after dimension reduction;

[0011] Arrange the standardized feature vectors in time sequence to form a time series matrix;

[0012] A causal inference model is built based on the time series matrix to extract the user interest bias. The expression is:

[0013] ,

[0014] in, Indicates the current time The user interest bias value, represents the time window, represents the attenuation reference coefficient, represents the time decay rate, represents the adjustment coefficient, represents the total number of eigenvalues, index variable representing the eigenvalue, Indicates the corresponding The weight of the feature value, represents the actual observed eigenvalue, represents the expected observed eigenvalue, represents the differential element of the integral, represents the integral variable, represents the time interval that determines the time decay rate;

[0015] According to the user's interest bias, the importance of each feature dimension in the time series matrix is ​​weighted;

[0016] Based on the weighted results, each feature dimension after importance weighting is screened and the optimized user interest vector is output;

[0017] The user interest vector is input into the click-through rate prediction model, and the click-through rate prediction model is trained through the causal optimization cross entropy loss function to output the click probability value of each advertisement.

[0018] The click probability value is used as the input of the reinforcement learning algorithm, combined with the user interaction behavior, and the advertising ranking results are generated by dynamically optimizing the multi-objective reward function;

[0019] According to the real-time feedback data in the user interaction behavior, the user interest bias of the causal inference model is updated in real time, and the user interest vector offset is corrected.

[0020] As a preferred solution of the advertisement recommendation method based on the click rate prediction model of the present invention, the steps of collecting advertisement recommendation feature data, processing through feature engineering, and outputting user feature vectors are as follows:

[0021] Collect user behavior data in real time and store it as a structured data table;

[0022] Use time windows to count user behavior characteristics in structured data tables;

[0023] Obtain advertising attributes and extract semantic vectors through bidirectional encoder representation to generate advertising features;

[0024] Record the context information when the ad is displayed, encode the time and geographic features, and associate the user behavior features with the ad features to form context features;

[0025] Combine context features with user behavior features to generate time-behavior cross features;

[0026] Use principal component analysis to reduce the dimension of time behavior cross features and output the user feature vector after dimension reduction.

[0027] As a preferred solution of the advertisement recommendation method based on the click rate prediction model of the present invention, wherein: according to the user interest bias, each feature dimension in the time series matrix is ​​weighted in importance, and the specific steps are as follows:

[0028] Use the arithmetic mean to calculate the mean of the feature dimensions over the entire time series one by one, and output the mean of the feature dimensions;

[0029] Use the arithmetic mean to calculate the mean of the user interest deviations over the entire time series;

[0030] Calculate the covariance based on the mean of the feature dimension and the mean of the user interest bias;

[0031] Combine all the calculated covariances into a correlation matrix between feature dimensions and user interest bias values ;

[0032] For the incidence matrix For each column in , construct the covariance vector and calculate the sum of the modulus of the covariance vectors of all feature dimensions;

[0033] The weight vector corresponding to each feature dimension is calculated according to the sum of the modulus lengths of the covariance vectors;

[0034] Multiply the weight vector by the corresponding element in the time series matrix and output the weighted time series matrix.

[0035] As a preferred solution of the advertisement recommendation method based on the click rate prediction model of the present invention, wherein: the user interest vector is input into the click rate prediction model, the click rate prediction model is trained by the causal optimized cross entropy loss function, and the click probability value of each advertisement is output. The specific steps are as follows:

[0036] Concatenate the user interest vector and the advertising features to construct the model input feature vector;

[0037] The Transformer network based on multi-head self-attention mechanism is used as the core framework of the click rate prediction model;

[0038] For the input layer in the Transformer network structure, the concatenated feature vector is input and linearly transformed to generate the query matrix, key matrix and value matrix;

[0039] Based on the query matrix, key matrix and value matrix, the interactive relationship between user interest features and advertising features is captured through the Transformer's multi-head self-attention mechanism;

[0040] Use multi-layer perceptron to further extract high-dimensional features from the fully connected layer in the Transformer network structure;

[0041] Use the Sigmoid activation function to map the high-dimensional features output by the fully connected layer to interval, generating the initial click probability;

[0042] In the cross entropy loss function Based on this, a causal penalty term is introduced to further optimize the cross entropy loss function. , the expression is:

[0043] ,

[0044] in, represents the cross entropy loss, represents the weight coefficient of the causal regularization term, represents the causal penalty term;

[0045] The causal penalty term expression is:

[0046] ,

[0047] in, represents the Frobenius norm;

[0048] Use the optimized cross entropy loss function to train the click rate prediction model;

[0049] The trained click rate prediction model is used to infer new samples to obtain the click probability value of each advertisement.

[0050] As a preferred solution of the advertisement recommendation method based on the click rate prediction model of the present invention, the specific steps of generating the advertisement ranking result are as follows:

[0051] The click probability value is concatenated with the user feature vector and the ad feature to form the state vector of the reinforcement learning algorithm.

[0052] Extract real-time feedback data from user interaction behaviors;

[0053] Combine real-time feedback data with the state vector to dynamically update the input features of the reinforcement learning algorithm to form a new state vector;

[0054] Represent all possible orderings of ad sequences as an action space;

[0055] After each ad ranking, calculate the immediate reward based on user feedback and optimize the multi-objective reward function;

[0056] According to the optimized multi-objective reward function, the action space is dynamically adjusted in combination with user feedback to generate advertising ranking results.

[0057] As a preferred solution of the advertisement recommendation method based on the click rate prediction model of the present invention, wherein: the user interest bias value of the causal inference model is updated in real time according to the real-time feedback data in the user interaction behavior, and the user interest vector offset is corrected. The specific steps are as follows:

[0058] Store the real-time feedback data from user interaction behaviors in a structured form in the database;

[0059] Perform data cleaning and real-time feature extraction on the data in the database, and organize the extracted features into user feature streams according to time series;

[0060] According to the user feature flow, by calculating the covariance of each feature value and the user interest bias value, the corresponding The weight of the feature value;

[0061] The user interest vector is corrected according to the real-time updated user interest bias.

[0062] In a second aspect, the present invention provides an advertisement recommendation system based on a click rate prediction model, comprising a feature processing module, a causal inference module, a prediction module, an advertisement ranking module and a feedback update module;

[0063] The feature processing module is used to collect advertisement recommendation feature data, process it through feature engineering, and output a user feature vector;

[0064] The causal inference module is used to extract the user interest bias value based on the user feature vector and generate an optimized user interest vector by using the causal inference model to eliminate the interference of non-causal factors;

[0065] The prediction module is used to input the user interest vector into the click rate prediction model, train the click rate prediction model through the causal optimization cross entropy loss function, and output the click probability value of each advertisement;

[0066] The advertisement ranking module is used to use the click probability value as the input of the reinforcement learning algorithm, combine the user interaction behavior, and generate the advertisement ranking result by dynamically optimizing the multi-objective reward function;

[0067] The feedback update module is used to update the user interest bias of the causal inference model in real time according to the real-time feedback data in the user interaction behavior, and correct the user interest vector offset.

[0068] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the advertising recommendation method based on the click-through rate prediction model as described in the first aspect of the present invention is implemented.

[0069] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the advertising recommendation method based on the click-through rate prediction model as described in the first aspect of the present invention.

[0070] The beneficial effects of the present invention are as follows: the present invention adopts a causal inference model to eliminate the interference of non-causal factors, thereby being able to more accurately extract user interest bias and optimize user interest vectors, thereby improving the prediction accuracy of the click-through rate prediction model; by introducing a reinforcement learning algorithm and combining user interaction behavior for dynamic optimization, real-time adjustment and multi-objective optimization of advertisement sorting are effectively achieved, thereby enhancing the flexibility and personalization of advertisement recommendations; in addition, the present invention can quickly respond to changes in user behavior by updating user interest bias in real time, thereby improving the timeliness of advertisement recommendations and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0072] Figure 1 This is a flow chart of the advertisement recommendation method based on the click rate prediction model in Example 1.

[0073] Figure 2 This is a module diagram of the advertisement recommendation system based on the click rate prediction model in Example 1. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0075] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0076] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0077] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides an advertisement recommendation method based on a click rate prediction model, comprising the following steps:

[0078] S1. Collect advertising recommendation feature data, process it through feature engineering, and output user feature vectors.

[0079] Furthermore, user behavior data (clicks, exposures, purchases, etc.) is collected in real time and stored as a structured data table (fields: user ID, behavior type, timestamp, dwell time, etc.);

[0080] Use time windows to count user behavior characteristics in structured data tables;

[0081] Obtain ad attributes (ad ID, category, budget, etc.), extract semantic vectors through bidirectional encoder representation (bidirectional Transformer), and generate ad features;

[0082] Record the context information when the ad is displayed (device operating system, browser, geographic location, timestamp, etc.), encode time and geographic features, and associate user behavior features with ad features to form context features;

[0083] Combine context features with user behavior features to generate time-behavior cross features;

[0084] Specifically, contextual features: describe the environment in which the ad is currently displayed, such as time (such as "8 o'clock in the morning"), geographic location (such as "Shanghai"), device type (such as "Android phone"), and ad category (such as "sports equipment"); user behavior features: describe the user's historical behavior data, such as number of clicks, browsing categories, purchase categories, and length of stay.

[0085] When combining two types of features: first directly concatenate context features and user behavior features to form a complete feature vector; then combine context features with user behavior features through cross calculation to generate new cross features. For example, you can count the user's click behavior in a certain time period based on time features, or count the user's browsing categories in a specific location based on geographic features; finally, integrate all cross features to generate time-behavior cross features, which reflect the user's behavior patterns under different context conditions.

[0086] Assume the following scenario:

[0087] The context features include: time feature: 8 o'clock in the morning; geographic location: Shanghai; device type: Android phone; advertising category: sports equipment.

[0088] User behavior characteristics include: number of clicks: clicked "sports equipment ads" 5 times in the past; browsing categories: browsed "running shoes" and "sports watches" in the past; purchase categories: purchased "running shoes" in the past; length of stay: the average stay time is 15 seconds.

[0089] Then through cross calculation, we can get: time × number of clicks: the user clicked on the "sports equipment ad" 3 times at "8 o'clock in the morning"; geographic location × browsing category: the user browsed "running shoes" in "Shanghai"; device type × purchase category: the user used "Android phone" to buy "running shoes"; advertisement category × dwell time: the total dwell time of the user on the "sports equipment" ad is 75 seconds (15 seconds × 5 clicks).

[0090] The final generated time behavior cross-feature is: [number of clicks at 8 am = 3 times, Shanghai: running shoes, Android phone: running shoes, sports equipment: 75 seconds]; this indicates that the user's behavioral characteristics under the current advertising context conditions can provide more accurate input for the advertising recommendation model.

[0091] Principal component analysis (PCA) is used to reduce the dimension of the temporal behavior cross-features and output the reduced-dimensional user feature vector.

[0092] S2. Based on the user feature vector, the causal inference model is used to eliminate the interference of non-causal factors, extract the user interest bias, and generate the optimized user interest vector.

[0093] Furthermore, the user feature vector after dimensionality reduction is standardized to ensure that the features of each dimension are within the same numerical range;

[0094] Arrange the standardized feature vectors in time series to form a time series matrix to capture the dynamic changes of user behavior and provide time series data support for the causal inference model;

[0095] A causal inference model is built based on the time series matrix to extract the user interest bias value, which is used to identify the relationship between real interest and external interference factors. The expression is: ,

[0096] in, Indicates the current time The user interest bias value, represents the time window, represents the attenuation reference coefficient, represents the time decay rate, represents the adjustment coefficient, represents the total number of eigenvalues, index variable representing the eigenvalue, Indicates the corresponding The weight of the feature value, represents the actual observed eigenvalue, represents the expected observed eigenvalue, represents the differential element of the integral, represents the integral variable, represents the time interval that determines the time decay rate;

[0097] Specifically, represents the time interval that determines the time decay rate, for example, when When the value of is large, it means that the current time If the time interval is far from the time point in the time window, the value calculated by the exponential decay function will be smaller, which means that the factors farther away from the current time have a greater impact on the current time. The influence of the user interest bias value decays at a set time rate The more it decays, the more The time interval represented is an important element in the entire mechanism that determines the time decay and reflects the degree of change in the impact of time distance on interest. It is also the key to the effect of the time decay rate.

[0098] It should also be noted that regarding the Indicates the attenuation reference coefficient, represents the time decay rate and Indicates the adjustment coefficient". The specific setting method is as follows:

[0099] Attenuation reference factor Setting: In application scenarios such as advertising recommendations, if you have a general understanding of the average initial interest level of users for various advertising topics from past business experience or market research, you can set the initial value accordingly. For example, after long-term observation, it is found that for popular category advertisements (such as electronic product advertisements), users generally have a medium to high interest level when they first come into contact with them. If the interest level is quantified (for example, measured in the range of 0-1), it can be initially set to around 0.6; while for relatively niche and highly professional advertisements (such as industrial machinery parts advertisements), users may have a lower initial interest, so it can be set to 0.2, etc.

[0100] Time decay rate Setting: Analyze the changing patterns of users' interest in advertisements over different time spans. For example, by observing the user's behavior data for seasonal advertisements (such as summer travel advertisements and winter warm-keeping products advertisements), it is found that users are very interested in such advertisements in the current season, but their interest drops rapidly after the corresponding season. Then, the time decay rate corresponding to such advertisements should be The value of should be set larger to reflect its rapid decay characteristics, and may be set to 0.5 or above; for some advertisements with strong versatility and relatively stable demand throughout the year (such as daily household goods advertisements), user interest changes relatively slowly over time, and the time decay rate It can be set to a smaller value, such as 0.1-0.2. In addition, it can also be determined by considering the behavioral differences of different types of user groups. For example, young user groups may have a faster interest in fresh and trendy ads, and their interest may decay quickly. In the advertising-related model for young groups, The value should be set relatively large; older user groups may be more interested in some classic and practical ads. The value should be smaller.

[0101] Adjustment coefficient Setting: If the difference between each feature and the expected feature value is found in the analysis (i.e. The impact of the difference in some features on the final user's interest bias value is expected to be relatively balanced. If you do not want the difference in one or several features to excessively affect the overall result, you can set it to a relatively small value, such as 0.1-0.3, so that the overall influence of the summation formula in the denominator is relatively mild, and each feature can play a relatively even role; on the contrary, if you want to highlight the regulatory effect of feature differences on interest bias, for example, if it is known that the deviation of certain key features (such as features related to users' recent purchase behavior) is crucial to judging interest, you can appropriately increase The value of can be set to 0.5-1 or even larger to strengthen the influence of these feature differences in the calculation.

[0102] According to the real interest intensity of users, each feature dimension in the time series matrix is ​​weighted in importance, the feature dimensions with significant causal relationships are retained, and the noise features are filtered out;

[0103] Specifically, the arithmetic mean is used to calculate the mean of the feature dimensions over the entire time series one by one, and the mean of the feature dimensions is output;

[0104] Use the arithmetic mean to calculate the mean of the user interest deviations over the entire time series, and output the mean of the user interest deviations;

[0105] Based on the mean of the feature dimension and the mean of the user interest bias, the covariance is calculated to measure the degree of coordinated change of the feature dimension and the interest bias relative to their respective average levels at different time points, that is, to measure the linear relationship between each feature dimension and the user interest bias. The larger the covariance, the more significant the impact of the feature dimension on the user interest bias.

[0106] Combine all the calculated covariances into a correlation matrix between feature dimensions and user interest bias values ;

[0107] For the incidence matrix For each column in , construct the covariance vector and calculate the sum of the modulus of the covariance vectors of all feature dimensions;

[0108] The weight vector corresponding to each feature dimension is calculated based on the sum of the modulus lengths of the covariance vectors to indicate its relative importance to the user's interest bias;

[0109] Multiply the corresponding elements in the weight vector time series matrix, that is, perform weighted processing on each feature dimension in the time series matrix, and finally obtain the weighted time series matrix, which indicates the contribution of the feature to the user's interest at that time point.

[0110] It should be noted that the weighted time series matrix can effectively highlight the influence of important features while suppressing the interference of noise features; the weighted matrix is ​​used to generate subsequent user interest vectors, so that the optimized user interest vectors can more accurately reflect the user's true interests.

[0111] Based on the weighted results, each feature dimension after importance weighting is screened and the optimized user interest vector is output;

[0112] The "user interest bias" plays a key role. It is a numerical indicator used to quantify the user's interest in related things (for example, in the case of advertising recommendation, it is the products, services or topics represented by various advertisements) at a specific time. It is usually set within a certain value range, for example, the common range can be between 0 and 1 (of course, other reasonable ranges may also be used according to the specific model settings and business scenario requirements). The higher the value, the stronger the user's interest in the related content at that moment; the lower the value, the weaker the interest;

[0113] Furthermore, it can be used to sort different advertisements and recommend those advertisements with higher interest bias values ​​to users first, so as to increase the probability of being clicked, paid attention to and even ultimately leading to conversions (such as purchasing products, using services, etc.) by users. At the same time, it can also continuously monitor the changes in user interest bias values ​​over time and adjust the recommendation strategy in real time. For example, when it is found that a user's interest bias value in a certain type of advertisement has dropped significantly, it will replace the recommendation with other advertisements with higher relevance that may reawaken interest, thereby realizing personalized and dynamic advertising recommendation services, better meeting user needs and improving advertising effectiveness.

[0114] S3. Input the user interest vector into the click-through rate prediction model, train the click-through rate prediction model through the causal optimized cross entropy loss function, and output the click probability value of each advertisement.

[0115] Furthermore, the user interest vector and the advertisement features are concatenated to construct the model input feature vector;

[0116] The Transformer network based on multi-head self-attention mechanism is used as the core framework of the click rate prediction model;

[0117] Among them, the Transformer network structure includes input layer, multi-head self-attention layer, fully connected layer and output layer, as follows:

[0118] Input layer: Input the concatenated feature vector.

[0119] Multi-head self-attention layer: captures the interactive relationship between user interest features and advertising features.

[0120] Fully connected layer: extract high-dimensional features through multi-layer perceptron (MLP).

[0121] Output layer: Use Sigmoid activation function to output the probability of ad click.

[0122] For the input layer in the Transformer network structure, the concatenated feature vector is input and linearly transformed to generate the query matrix, key matrix and value matrix;

[0123] Based on the query matrix, key matrix and value matrix, the interactive relationship between user interest features and advertising features is captured through the Transformer's multi-head self-attention mechanism;

[0124] Preferably, multi-head attention can calculate the feature interactions of multiple subspaces in parallel, improving the model's ability to model the complex relationship between interest features and advertising features.

[0125] Use multi-layer perceptron (MLP) to further extract high-dimensional features from the fully connected layer in the Transformer network structure;

[0126] Use the Sigmoid activation function to map the high-dimensional features output by the fully connected layer to interval, generating the initial click probability;

[0127] In the cross entropy loss function Based on this, a causal penalty term is introduced to further optimize the cross entropy loss function. , the expression is:

[0128] ,

[0129] in, represents the cross entropy loss, represents the weight coefficient of the causal regularization term, represents the causal penalty term;

[0130] The causal penalty term expression is:

[0131] ,

[0132] in, represents the Frobenius norm;

[0133] Preferably, the weights of non-causal features are directly constrained through the causal regularization term to reduce noise interference and improve the model's ability to focus on causal features. Unlike traditional regularization methods, the causal penalty term focuses on the correspondence between weights and causal features and is more targeted.

[0134] Use the optimized cross entropy loss function to train the click rate prediction model;

[0135] The trained click rate prediction model is used to infer new samples to obtain the click probability value of each advertisement.

[0136] S4. Use the click probability value as the input of the reinforcement learning algorithm, combine it with the user interaction behavior, and generate the advertising ranking result by dynamically optimizing the multi-objective reward function.

[0137] Furthermore, the click probability value is concatenated with the user feature vector and the ad feature to form the state vector of the reinforcement learning algorithm;

[0138] Extract real-time feedback data from user interaction behaviors;

[0139] Specifically, "user interaction behavior" refers to the behavioral data generated by users during their interaction with advertising platforms, recommendation platforms, or e-commerce websites; these behaviors reflect the user's interest and preference in the displayed content (such as advertisements or products), and usually include the following categories:

[0140] Click behavior: whether the user clicks on an ad or recommended content, which is a direct measure of user interest; dwell time: the length of time a user stays on an ad page, product detail page or recommended content, which is usually used to assess the user's in-depth interest or engagement; browsing behavior: including the number of pages the user browses, the distance and speed of sliding / scrolling, etc., reflecting the user's overall attention to the content; bounce rate: whether the user leaves the ad or page quickly after opening it (that is, without further interaction), which is used to evaluate the attractiveness of the content; conversion behavior: whether the user completes a target action, such as purchasing, registering, filling out a form, adding to favorites, adding to shopping carts, etc., which is an important indicator for measuring the effectiveness of advertising or recommendations; interaction frequency: the number of interactions a user has within a period of time, such as the number of times an ad is clicked or the frequency of browsing recommended content; negative feedback: whether the user has performed negative operations on the content, such as closing ads, marking as uninterested, complaining or blocking recommended content.

[0141] Combine real-time feedback data with the state vector to dynamically update the input features of the reinforcement learning algorithm to form a new state vector;

[0142] Represent all possible orderings of ad sequences as an action space;

[0143] Specifically, "all possibilities" refers to all possible permutations and combinations generated based on the number of a given set of advertisements. Specifically, for a sequence of m advertisements (m represents the index variable of the advertisements), the number of all possible orderings is m! (the factorial of m), that is, each order of advertisements is regarded as a possible action; these different orderings together constitute the action space of the reinforcement learning algorithm, and the algorithm will select an optimal ordering from these arrangements to maximize the reward function based on user feedback.

[0144] After each ad ranking, calculate the immediate reward based on user feedback and optimize the multi-objective reward function;

[0145] Based on the optimized multi-objective reward function, the action space is dynamically adjusted in combination with user feedback to generate ad ranking results;

[0146] For example, in the ad ranking of e-commerce platforms, the multi-objective reward function combines click-through rate (CTR), conversion rate (CVR) and advertiser return (ROI). When a user clicks on an ad but does not complete the purchase, the action space will be dynamically adjusted through user feedback (such as click behavior and dwell time), reducing the ranking probability of similar low-conversion ads, while giving priority to ads with high conversion rates and high returns in history, and finally generating a new ad ranking result to improve the overall advertising effect.

[0147] S5. Based on the real-time feedback data in the user interaction behavior, the user interest bias of the causal inference model is updated in real time, and the user interest vector offset is corrected.

[0148] Furthermore, the real-time feedback data of user interaction behaviors (such as click behaviors, dwelling time, conversion behaviors, etc.) are stored in the database in a structured form (such as user behavior table or time series data table);

[0149] It should be noted that real-time feedback data needs to include key information such as "user ID, advertising ID, click behavior, stay duration, timestamp", and be clearly stored in a structured form as a table or time series data.

[0150] Perform data cleaning (such as deduplication and removal of outliers) and real-time feature extraction (such as calculation of click-through rate, average dwell time, etc.) on the data in the database, and organize the extracted features into user feature streams according to time series;

[0151] According to the user feature flow, by calculating the covariance of each feature value and the user interest bias value, the corresponding The weight of the feature value;

[0152] Correct the user interest vector according to the real-time updated user interest bias value;

[0153] Specifically, the user interest vector is corrected through the following steps: first, according to the covariance of each feature value and the interest bias value in the user feature stream, the weight of each feature is dynamically calculated and the weight is normalized; then, these weights are multiplied with the corresponding feature values ​​in the user feature stream to form weighted features; then, combined with the time decay factor, the weighted features are time-weighted accumulated to reflect the dynamic change trend of the user's interest; finally, the weighted accumulation results are recombined into a corrected user interest vector to ensure that it can reflect the user's latest interest preferences in real time.

[0154] The present embodiment also provides an advertisement recommendation system based on a click-through rate prediction model, comprising: a feature processing module, a causal inference module, an estimation module, an advertisement ranking module and a feedback update module; the feature processing module is used to collect advertisement recommendation feature data, process it through feature engineering, and output a user feature vector; the causal inference module is used to eliminate non-causal factor interference based on the user feature vector by using the causal inference model, extract the user interest bias, and generate an optimized user interest vector; the estimation module is used to input the user interest vector into the click-through rate prediction model, train the click-through rate prediction model through the causal optimized cross entropy loss function, and output the click probability value of each advertisement; the advertisement ranking module is used to use the click probability value as the input of the reinforcement learning algorithm, combine the user interaction behavior, and generate the advertisement ranking result by dynamically optimizing the multi-objective reward function; the feedback update module is used to update the user interest bias of the causal inference model in real time according to the real-time feedback data in the user interaction behavior, and correct the user interest vector offset.

[0155] This embodiment also provides a computer device, which is suitable for the case of an advertisement recommendation method based on a click-through rate prediction model, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the advertisement recommendation method based on a click-through rate prediction model proposed in the above embodiment.

[0156] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0157] The present embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for implementing an advertisement recommendation method based on a click-through rate prediction model as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage device, a flash memory, a disk or an optical disk.

[0158] In summary, the present invention eliminates the interference of non-causal factors by adopting a causal inference model, can more accurately extract user interest bias, optimize user interest vector, and thus improve the prediction accuracy of the click-through rate prediction model; by introducing a reinforcement learning algorithm combined with user interaction behavior for dynamic optimization, real-time adjustment and multi-objective optimization of advertisement sorting are effectively realized, thereby enhancing the flexibility and personalization of advertisement recommendation; in addition, by updating user interest bias in real time, the present invention can quickly respond to changes in user behavior, thereby improving the timeliness of advertisement recommendation and user experience.

[0159] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An advertisement recommendation method based on a click rate prediction model, characterized in that: include, Collect ad recommendation feature data, process it through feature engineering, and output user feature vectors; Based on the user feature vector, the causal inference model is used to eliminate the interference of non-causal factors, extract the user interest bias, and generate the optimized user interest vector; The specific steps of generating the optimized user interest vector are as follows: Standardize the user feature vector after dimension reduction; Arrange the standardized feature vectors in time sequence to form a time series matrix; A causal inference model is built based on the time series matrix to extract the user interest bias. The expression is: Among them, I t represents the user interest bias at the current time t, T represents the time window, α represents the attenuation base coefficient, β represents the time attenuation rate, γ represents the adjustment coefficient, n represents the total number of eigenvalues, k represents the index variable of the eigenvalue, and w k represents the weight corresponding to the kth eigenvalue, f k represents the actual observed eigenvalue, h k represents the expected observed characteristic value, dτ represents the differential element of integration, τ represents the integration variable, and t-τ represents the time interval that determines the time decay rate; According to the user's interest bias, the importance of each feature dimension in the time series matrix is ​​weighted. The specific steps are as follows: Use the arithmetic mean to calculate the mean of the feature dimensions over the entire time series one by one, and output the mean of the feature dimensions; Use the arithmetic mean to calculate the mean of the user interest deviations over the entire time series; Calculate the covariance based on the mean of the feature dimension and the mean of the user interest bias; All the calculated covariances are combined into a correlation matrix Q between feature dimensions and user interest bias values; For each column in the correlation matrix Q, construct the covariance vector and calculate the sum of the modulus of the covariance vectors of all feature dimensions; The weight vector corresponding to each feature dimension is calculated according to the sum of the modulus lengths of the covariance vectors; Multiply the weight vector with the corresponding element in the time series matrix and output the weighted time series matrix; Based on the weighted results, each feature dimension after importance weighting is screened and the optimized user interest vector is output; The user interest vector is input into the click-through rate prediction model, and the click-through rate prediction model is trained through the causal optimization cross entropy loss function to output the click probability value of each advertisement. The click probability value is used as the input of the reinforcement learning algorithm, combined with the user interaction behavior, and the advertising ranking results are generated by dynamically optimizing the multi-objective reward function; According to the real-time feedback data in the user interaction behavior, the user interest bias of the causal inference model is updated in real time, and the user interest vector offset is corrected.

2. The advertisement recommendation method based on the click rate prediction model according to claim 1, characterized in that: The steps of collecting advertisement recommendation feature data, processing it through feature engineering, and outputting user feature vectors are as follows: Collect user behavior data in real time and store it as a structured data table; Use time windows to count user behavior characteristics in structured data tables; Obtain advertising attributes and extract semantic vectors through bidirectional encoder representation to generate advertising features; Record the context information when the ad is displayed, encode the time and geographic features, and associate the user behavior features with the ad features to form context features; Combine context features with user behavior features to generate time-behavior cross features; Use principal component analysis to reduce the dimension of time behavior cross features and output the user feature vector after dimension reduction.

3. The advertisement recommendation method based on the click rate prediction model according to claim 2, characterized in that: The user interest vector is input into the click rate prediction model, the click rate prediction model is trained through the causal optimization cross entropy loss function, and the click probability value of each advertisement is output. The specific steps are as follows: Concatenate the user interest vector and the ad features to construct the model input feature vector; The Transformer network based on multi-head self-attention mechanism is used as the core framework of the click rate prediction model; For the input layer in the Transformer network structure, the concatenated feature vector is input and linearly transformed to generate the query matrix, key matrix and value matrix; Based on the query matrix, key matrix and value matrix, the interactive relationship between user interest features and advertising features is captured through the Transformer's multi-head self-attention mechanism; Use multi-layer perceptron to further extract high-dimensional features from the fully connected layer in the Transformer network structure; Use the Sigmoid activation function to map the high-dimensional features output by the fully connected layer to the [0,1] interval to generate the initial click probability; Based on the cross entropy loss function L, a causal penalty term is introduced to further optimize the cross entropy loss function L. The expression is: L = B + λ·R; Among them, B represents the cross entropy loss, λ represents the weight coefficient of the causal regularization term, and R represents the causal penalty term; The causal penalty term expression is: in, represents the Frobenius norm; Use the optimized cross entropy loss function to train the click rate prediction model; The trained click rate prediction model is used to infer new samples to obtain the click probability value of each advertisement.

4. The advertisement recommendation method based on the click rate prediction model according to claim 3, characterized in that: The specific steps of generating the advertisement ranking result are as follows: The click probability value is concatenated with the user feature vector and the ad feature to form the state vector of the reinforcement learning algorithm. Extract real-time feedback data from user interaction behaviors; Combine real-time feedback data with the state vector to dynamically update the input features of the reinforcement learning algorithm to form a new state vector; Represent all possible orderings of ad sequences as an action space; After each ad ranking, calculate the immediate reward based on user feedback and optimize the multi-objective reward function; According to the optimized multi-objective reward function, the action space is dynamically adjusted in combination with user feedback to generate advertising ranking results.

5. The advertisement recommendation method based on the click rate prediction model according to claim 4, characterized in that: The method of updating the user interest bias value of the causal inference model in real time based on the real-time feedback data in the user interaction behavior and correcting the user interest vector offset is as follows: Store the real-time feedback data from user interaction behaviors in a structured form in the database; Perform data cleaning and real-time feature extraction on the data in the database, and organize the extracted features into user feature streams according to time series; According to the user feature stream, the weight of the corresponding k-th feature value is dynamically updated by calculating the covariance of each feature value and the user's interest bias value; The user interest vector is corrected according to the real-time updated user interest bias.

6. An advertisement recommendation system based on a click rate prediction model, based on the advertisement recommendation method based on a click rate prediction model according to any one of claims 1 to 5, characterized in that: Including feature processing module, causal inference module, estimation module, advertisement ranking module and feedback update module; The feature processing module is used to collect advertisement recommendation feature data, process it through feature engineering, and output a user feature vector; The causal inference module is used to extract the user interest bias value based on the user feature vector and generate an optimized user interest vector by using the causal inference model to eliminate the interference of non-causal factors; The prediction module is used to input the user interest vector into the click rate prediction model, train the click rate prediction model through the causal optimization cross entropy loss function, and output the click probability value of each advertisement; The advertisement ranking module is used to use the click probability value as the input of the reinforcement learning algorithm, combine the user interaction behavior, and generate the advertisement ranking result by dynamically optimizing the multi-objective reward function; The feedback update module is used to update the user interest bias of the causal inference model in real time according to the real-time feedback data in the user interaction behavior, and correct the user interest vector offset.

Images