System and method for real-time user response prediction of content presentation on client device

Through a real-time user response prediction system, Bayesian logistic regression model, etc., predict the possibility of users' interaction with digital content, solve the problems of waste of resources and high costs in the existing technology, and achieve efficient and accurate content presentation optimization.

CN119096259APending Publication Date: 2024-12-06SKILLZ PLATFORM INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280092477.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2021-12-30
Filing Date
2022-12-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art is difficult to effectively predict the user's response to digital content, resulting in waste of resources and increased costs, and it is difficult to determine which users will respond to digital content and interact.

Method used

Using a real-time user response prediction system, the user's response prediction system is used to receive feature information on the user's equipment, generate feature vectors, and use machine learning models such as Bayesian logistic regression to predict the user's interaction possibilities for content presentation, and determine the impression value and response value of content presentation. , thereby optimizing content presentation.

Benefits of technology

It improves computer processing efficiency, can handle a large number of user requests within strict time constraints, quickly absorb new data, accurately predict user responses, optimize content presentation, and reduce costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096259A_ABST
    Figure CN119096259A_ABST
Patent Text Reader

Abstract

In an aspect, a request to display information on a client device of a user may be received. A plurality of features related to the request may be extracted. The plurality of features may include at least one feature characterizing the user and at least one additional feature characterizing the client device. Feature vectors may be generated based on features related to the request. Predictive response values of the user to the presentation of the content may be generated using the predictive model and the feature vectors. The predicted response value may characterize a likelihood that the user interacts with the content presentation. A request response value may be determined based on the content presentation impression value and the feature vector. The request response value and the content presentation may be sent for display on the user's client device.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 266,186, filed on December 30, 2021, the entire contents of which are incorporated herein by reference. Background Art

[0003] Client devices can present a variety of digital content to users, including video, audio, images and combinations thereof. Content is usually developed and used to achieve a certain response of the user who contacts the content. Some content can encourage users to take certain actions, such as interacting or participating with client applications, purchasing products or services, or installing client applications. However, not all users will respond to the digital content presented to them, so presenting the same digital content to all users may cause a waste of resources and be ineffective. Determining whether a user will respond to digital content and which users will respond to digital content may be difficult, time-consuming and expensive. Summary of the invention

[0004] Systems and methods are provided for real-time user response prediction of content presentation on a client device. Related devices, methods, and articles are also described.

[0005] In one aspect, a request to display information on a client device of a user may be received. A plurality of features associated with the request may be extracted. The plurality of features may include at least one feature characterizing the user and at least one additional feature characterizing the client device. A feature vector may be generated based on the plurality of features associated with the request. A predicted response value of a user to a content presentation may be generated using a prediction model and the feature vector. The predicted response value may characterize the likelihood of the user interacting with the content presentation. When the predicted response value meets a predetermined threshold, a content presentation impression value may be generated using the predicted response value and a target value associated with the user. The target value may characterize the cost of the user's interaction with the content presentation. A request response value may be determined based on the content presentation impression value and the feature vector. The request response value and the content presentation may be sent for display on the client device of the user.

[0006] One or more features may be included in any feasible combination. For example, the prediction model may be a Bayesian Logistic Regression model. For example, the predicted response value may further characterize the possibility of downstream conversion in response to the user interacting with the content presentation. For example, the predicted response value and the target value associated with the user may be used to generate a content presentation impression value by multiplying the target value with the predicted response value. For example, the content presentation for display on the user's client device may be selected based on the feature vector. For example, in response to determining that the predicted response value of the user for content presentation does not meet a predetermined threshold, an alternative content presentation for display on the client device may be selected. For example, the prediction model and the feature vector may be used to generate an additional predicted response value for the user to present the alternative content. For example, when the additional predicted response value meets a predetermined threshold, the additional predicted response value and the target value associated with the user may be used to generate an additional content presentation value. In other examples, a filter model associated with the prediction model may be implemented. For example, implementing the filter model may include determining the additional possibility of the user converting in a specific category associated with one or more additional content presentations.

[0007] In another aspect, a system is provided, and the system may include at least one data processor and a memory storing instructions, the instructions configured to cause the at least one data processor to perform the operations described herein. The operations may include: receiving a request to display information on a user's client device; extracting multiple features associated with the request, the multiple features including at least one feature characterizing the user and at least one additional feature characterizing the client device; generating a feature vector based on the multiple features associated with the request; using a prediction model and the feature vector to generate a predicted response value of the user to a content presentation, the predicted response value characterizing the likelihood of the user interacting with the content presentation; when the predicted response value meets a predetermined threshold, using the predicted response value and a target value associated with the user to generate a content presentation impression value, the target value characterizing the cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; and sending the request response value and the content presentation for display on the user's client device.

[0008] One or more features may be included in any feasible combination. For example, the prediction model may be a Bayesian logistic regression model. For example, the predicted response value further characterizes the likelihood of downstream conversion of a user interacting with the content presentation. For example, at least one data processor may perform the following operations: generating a content presentation impression value using the predicted response value and the target value associated with the user by multiplying the target value with the predicted response value. For example, the operation may further include selecting a content presentation for display on the user's client device based on the feature vector. For example, the operation may further include: in response to determining that the user's predicted response value for content presentation does not meet a predetermined threshold, selecting an alternative content presentation for display on the client device. For example, the operation may further include: generating an additional predicted response value of the user for the alternative content presentation using the prediction model and the feature vector. For example, the operation may further include: implementing a filter model associated with the prediction model. For example, the operation of implementing the filter model may include: determining an additional likelihood of a user converting in a specific category associated with one or more additional content presentations.

[0009] In yet another aspect, a non-transitory computer program product (i.e., a physically implemented computer program product) is also described that stores instructions that, when executed by at least one data processor forming part of at least one computing system, cause the at least one data processor to perform operations described herein. The operations include: receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features including at least one feature characterizing the user and at least one additional feature characterizing the client device; generating a feature vector based on the plurality of features associated with the request; generating a predicted response value of the user to a content presentation using a prediction model and the feature vector, the predicted response value characterizing the likelihood of the user interacting with the content presentation; generating a content presentation impression value using the predicted response value and a target value associated with the user when the predicted response value meets a predetermined threshold, the target value characterizing the cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; and sending the request response value and the content presentation for display on the user's client device.

[0010] A computer system is also described, which may include one or more data processors and a memory connected to one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more operations described herein. In addition, these methods may be implemented by one or more data processors distributed in a single computing system or between two or more computing systems. Such computing systems may be connected, and data and / or commands or other instructions may be exchanged by one or more connections, by direct connections between one or more of the computing systems, etc., including connections by a network (e.g., the Internet, wireless wide area network, local area network, wide area network, wired network, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above embodiments will be more fully understood from the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are not intended to be drawn to scale. For clarity, not every component will be labeled in every drawing. In the drawings:

[0012] Figure 1 is a block diagram illustrating an exemplary system for real-time user response prediction;

[0013] Figure 2 is a block diagram illustrating an exemplary method for real-time user response prediction;

[0014] Figure 3 is a block diagram illustrating another exemplary method for predicting a user's response to a content presentation displayed on a client device; and

[0015] Figure 4 is a block diagram of an exemplary computing device that can perform one or more of the operations described herein, according to present embodiments. DETAILED DESCRIPTION

[0016] Certain exemplary embodiments will now be described to provide an overall understanding of the structure, function, manufacture and principle of use of the apparatus and method disclosed herein. One or more examples of these embodiments are shown in the accompanying drawings. It will be appreciated by those skilled in the art that the apparatus and method specifically described herein and shown in the accompanying drawings are non-restrictive exemplary embodiments, and the scope of the present invention is limited only by the claims. The features shown or described in conjunction with an exemplary embodiment may be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present invention. In addition, in the present invention, the eponymous components of the embodiments generally have similar features, and therefore in a specific embodiment, each feature of each eponymous component may not be fully described.

[0017] The present invention is directed to a system and method for predicting real-time user responses to content presentation on a client device. According to the present invention, an automated process can predict the likelihood that a user will participate in a content presentation or other digital content presented on its client device. Embodiments of the present invention can utilize appropriate machine learning models to predict such user responses in real time. The prediction can be used to calculate the impression value of content presentation, which can be used, for example, to generate, determine or optimize the response to a request to display or otherwise display content presentation on a user's client device. For purposes of discussion and not limitation only, the present invention will illustrate various aspects of the present invention with reference to content presentation on a mobile device. However, the present invention can be used in or with any appropriate type of system, for which the user's predicted response can be used to determine and select the type, format and content of information to be presented to the user in real time on the user's client device.

[0018] Embodiments of the present invention can improve computer processing efficiency by determining and selecting the type, format, and content of information to be presented to those users on client devices based on the predicted responses of the users rather than through time and processor intensive trial and error and other similar computer-based methods, especially when there are a large number of users and content presentations (e.g., millions, tens of millions, etc.) that must be processed within a fraction of a second. In addition, embodiments of the present invention can utilize linear models (e.g., logistic regression models, etc.) to make predictions within such strict time constraints and process millions of requests per second without using specialized computer hardware to make such predictions. Such linear models can also allow new data to be quickly absorbed to make the most accurate predictions. In addition, embodiments of the present invention can utilize Bayesian methods to quantify uncertainty in predictions and effectively balance content presentation inventory exploration and utilization that cannot be implemented through trial and error and other similar computer-based methods.

[0019] Figure 11 is a block diagram illustrating an exemplary system 100 for real-time user response prediction for content presentation on a client device. A server system 114 may provide functionality for developing a prediction model, utilizing the prediction model to predict in real time a user's response to content presentation, and generating, determining, or otherwise optimizing a response to a request to display a content presentation on a user's client device. In an embodiment, a content presentation may be any suitable type of digital content that can be served, displayed, or otherwise displayed to a user on a client device, such as, for example, a digital advertisement, a creative asset, or other presentation of digital content, and may include text, images, video, audio, or any combination thereof. The content presentation may be used for any suitable purpose to achieve a desired response from a user contacting the content presentation. The server system 114 may include software components and databases that may be deployed at, for example, one or more data centers 112 in one or more geographic locations. The software components of the server system 114 may include a model development module 116, a user response prediction module 118, and a response determination module 120. The software components may include subcomponents that may be executed on the same or different separate data processing devices. The databases of the server system 114 may include, for example, a user response data database 122 and a client device data database 124. The databases may exist in one or more physical storage systems or be cloud-based. The software components and data will be further described below.

[0020] like Figure 1As shown, the model development module 116 , the user response prediction module 118 , and the response determination module 120 may be in communication with a user response data database 122 and a client device data database 124 . The user response data database 122 may include data associated with users, user responses to content presentations, user information (e.g., age, gender, device identifier, etc.), content presentation response or bid history (e.g., publisher, category, bid, impression, sample auction, etc.), user response history (e.g., clicks, installs, in-app purchases, in-app revenue, number of purchases, content presentation interactions (e.g., view completions), etc.), content presentation performance metrics (e.g., click-through rate, cost per action, etc.), installation and post-install events (e.g., in-app purchases, content presentation revenue, etc.), publisher information (e.g., application, category, etc.), content presentation environment (e.g., Wi-Fi / cellular reflecting the context in which the content presentation is displayed), content presentation environment and / or information related to the content presentation, such as, for example, images, video, sound, text, etc., and may include descriptions of features or characteristics present in the content presentation (e.g., the format of the content presentation), content presentation location information, time / date (e.g., time of day, day of week, week of year, etc.), user geographic information (e.g., country, region, designated marketing area (DMA), etc.), etc. Other user response data is also possible. The client device data database 124 may include information or characteristics related to the user's client device, such as, for example, client device model, carrier, client device operating system, in-application events, client device characteristics (e.g., price, hardware, storage, etc.), summary statistics calculated based on such data and / or data from the user response data database 122, etc. Other client device information or characteristics are also possible.

[0021] The software application or its components can be executed on the user's client device, such as client device A 102, client device B 104, client device C 106, ..., client device N 108 (wherein N can be any appropriate natural number), and can be accessed and accessed through a network 110 (e.g., the Internet). Each client device can be any appropriate type of electronic device capable of executing the software application and communicating with the server system 114 through the network 110, such as a smart phone, a tablet computer, a laptop computer, a desktop computer or a personal computer, a game console, a set-top box, a smart TV, etc. Other client devices are also possible. In an alternative embodiment, the user response data database 122, the client device data database 124, or any part thereof can be stored on one or more client devices. Additionally or alternatively, the software components (e.g., model development module 116, user response prediction module 118 and / or response determination module 120) of the system 100 or any part thereof can be present on one or more client devices or used to perform operations on one or more client devices.

[0022] For example, software applications such as web-based or mobile applications (e.g., publisher applications) can be provided as end-user applications (e.g., provided by the publisher) to allow users to interact with the server system 114. In addition, appropriate digital content (e.g., content presentation) can be displayed to the user by or in association with the software application executed on the user's client device. The software application can be related to and / or provide a variety of functions and information, including, for example, entertainment (e.g., games, music, videos, etc.), business (e.g., word processing, accounting, spreadsheets, etc.), news, weather, finance, sports, etc. In a specific embodiment, the software application can provide a mobile game. The mobile game can be or include, for example, a sports game, an adventure game, a virtual poker game, a virtual board game, a puzzle game, a racing game, or any other appropriate type of mobile game. In an embodiment, the mobile game can be a game based on asynchronous competitive skills, in which players can compete with each other in the mobile game, but do not have to play the mobile game at the same time. In an alternative embodiment, the mobile game can be a game based on synchronous competitive skills, in which players can play the mobile game at the same time and can compete with each other in real time in the mobile game. In an embodiment, the content presentation may be presented on a client device outside of the software application and allow the user to interact or engage with the content presentation, for example by clicking or selecting on or within the content presentation. Additionally or alternatively, the software application may be configured to present the content presentation within the software application on a client device and allow the user to interact or engage with the content presentation.

[0023] According to an exemplary embodiment, the model development module 116 can be used to develop a user response prediction model to predict the user's response to content presentation in real time. In an embodiment, any appropriate predictive machine learning model can be appropriately trained and used to predict user responses in real time. In some embodiments of the present invention, the model development module 116 can adopt a Bayesian logistic regression model as a user response prediction model. Bayesian logistic regression is a probabilistic model that can automatically calculate the success probability of a new data point. In some embodiments of the present invention, the model development module 116 can build a model for a single entity (e.g., advertiser, etc.) or entity group (e.g., advertiser group, etc.) that expects to display one or more content presentations on the user's client device. The model development module 116 can arrange a predetermined number of classification features (e.g., 5, 10, 15 or any appropriate number) into a feature vector. The classification features in the feature vector can be classified or stored in the feature vector in any desired order. The feature vector can include a digital presentation of a combination of classification features, such as user attributes and / or other attributes. For example, the feature vector can include multiple numbers (e.g., integers), each of which represents a different classification feature. Depending on the expected number of classification features captured in the feature vector, the feature vector can have any appropriate length. In some embodiments of the present invention, one-hot encoding can be used for all or part of the feature vector to calculate, convert or otherwise map the numerical representation of each or any classification feature in the classification feature set, although other appropriate encoding techniques can be used. For the purpose of illustration and not limitation, a feature vector with a length or amplitude of M (where M is any appropriate natural number) may include the numerical representation of the classification features of a specific publisher application as the mth element of the feature vector. For the one-hot encoding of publisher application A, if the mth element of the feature vector is "1", this can indicate that the publisher application is A, otherwise the mth element is "0" (i.e., the publisher application is not A). Other digital representations of classification features are also possible. For example, the digital representation may depend on the types and characteristics of various user attributes and / or other attributes including the classification features in the feature vector, and how such classification features are converted or mapped to numbers (e.g., integers) by one-hot encoding or other appropriate encoding techniques. For purposes of illustration and not limitation, categorical features may include, for example, publisher application, country, time of day, day of week, Wi-Fi / cellular, region, DMA, carrier, device OS, ad format, in-app event, auction stream history (e.g., publisher and category), etc. Other categorical features are possible and will depend on the use case. All or any of these categorical features may be encoded (e.g., one-hot encoded) and converted to corresponding numerical representations (e.g., integers) in a feature vector.In some embodiments of the invention, users may be divided into multiple groups or cohorts to maximize information gain. For example, users having one or more similar characteristics may be grouped into a first group, while users having other similar characteristics may be grouped into a second, third, fourth, etc. group. Any suitable number of groups may be utilized, and each or any group may be detailed with desired constraints based on, for example, use case requirements.

[0024] In some embodiments of the present invention, the model development module 116 may use Bayesian logistic regression to determine the distribution of the weight coefficient w of the data for a given feature vector x, as shown in equation (1):

[0025] p(w|x,y)∝p(y|w,x)p(w)(1)

[0026] The prediction can be given by equation (2):

[0027] p(y=1|w,x)=σ(w·x) (2)

[0028] where w is extracted from the distribution in equation (1) (e.g., using Thompson sampling, etc.). More specifically, the posterior likelihood of the weight coefficients can be expressed as in equation (3):

[0029]

[0030] Among them, π i =σ(x i ·w) and “Bern” denotes the Bernoulli likelihood function. In equation (3), m 0 and q 0 The prior knowledge of the mean value and precision of each weight can be reflected respectively. According to equation (4), the model development module 116 can determine the most likely value of w for a given data, which can be called Maximum a Posteriori (MAP) estimation:

[0031]

[0032] The model development module 116 may then apply the Laplace approximation to estimate the posterior accuracy of each weight according to equation (5):

[0033]

[0034] In some embodiments of the invention, the prediction model built by model development module 116 may be trained on appropriate data (e.g., using an appropriate optimizer, such as, for example, mini-batch limited memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), etc.), and then periodically updated using incremental data as incremental data is received by server system 114. In an embodiment, application embeddings may be used to vectorize applications based on co-occurrences on client devices to extend predictions to similar applications (software applications) on similar client devices.

[0035] Other suitable prediction models can be utilized.For example, the prediction model can be or include, for example, one or more equations (for example, regression equations, etc.) and / or classifiers.In some embodiments, the prediction model can be or include a ridge (Ridge) regression model, a lasso (Lasso) regression model, an elastic net (Elastic Net) regression model, an adaptive lasso (Adaptive Lasso) regression model, a group lasso (Group Lasso) regression model, a random lasso (RandomizedLasso) regression model and / or other suitable regression models.Additionally or alternatively, the prediction model can be or include a classifier, such as, for example, one or more linear classifiers (for example, Fisher linear discriminator, logistic regression, naive Bayes classifier and / or perceptron), support vector machine (for example, least squares support vector machine), quadratic classifier, kernel estimation model (for example, k- nearest neighbor), enhancement (meta-algorithm) model, decision tree (for example, random forest, gradient enhancement tree), neural network and / or learning vector quantization model, etc.

[0036] In some embodiments of the present invention, the model development module 116 can utilize additional models to supplement or enhance the basic prediction model (e.g., Bayesian logistic regression model) as an integrated model. In an embodiment, the predictions from one or more additional models can be appropriately combined with each other and / or with the predictions generated by the basic prediction model (e.g., via multiplication, averaging, median, etc.) to improve or otherwise refine the predictions generated by the basic prediction model. Additionally or alternatively, the predictions from one or more additional models can be used as a filter to determine whether to submit a response (e.g., based on whether the predictions from one or more additional models are higher or lower than one or more predetermined or adjustable thresholds). The model development module 116 can utilize appropriate filter models to filter traffic in specific scenarios (e.g., for audience expansion, similar targets, etc.). In an embodiment, the filter model can be based on a point-to-point mutual information (PMI) score, wherein the PMI score can be a measure of the correlation between two events or the association between a feature and a category (e.g., a larger and / or different feature group). The PMI score can represent a quantitative measure of how much or how little the possibility of two events occurring simultaneously is. For example, a PMI score may represent a correlation between one or more of a plurality of features and conversions in, for example, a specific category of advertisers. A PMI score may compare the probability of two events occurring together to the probability if the two events occurred independently. For example, a PMI score of 0 for two events x and y may indicate that the specific events x and y are statistically independent. A positive PMI score may indicate that events x and y occur together more frequently than would be expected if they were independent events. A negative PMI score may indicate that events x and y occur together less frequently than expected. In some embodiments of the present invention, a PMI score for a pair of events x and y may be calculated according to equation (6):

[0037]

[0038] Equation (6) may be based on maximum likelihood estimation, where p(x, y) may be the joint probability of random variables x and y divided by the product of the individual probabilities p(x) and p(y). For example, according to some embodiments of the present invention, the following information may be determined: (i) x , which can be the number of observations of event x, (ii) y , which can be the number of observations of event y, and (iii) N, which can be the total size of the observation pool. Given such information, the probability of events x and y and the co-occurrence of x and y can be determined using equation (6) as follows: p(x) = o x / N and p(y)=o y / N, where p(x, y) can be the number of observed co-occurrences of x and y. For purposes of illustration and not limitation, the PMI score can be measured between the presence of a software application (e.g., a publisher application) on a user's client device and the fact that the user converted in a specific advertiser. Events for which correlation can be determined can include, for example, the extent to which the fact that a user has application A on their client device increases the likelihood that the user will make a purchase in application B. Co-occurrences of other events measured by the PMI score are also possible. Additionally or alternatively, the filter model can be based on an appropriate positive unlabeled classifier. For example, an appropriate model can be trained to distinguish between a list of high-quality users and an "average" user. Although the predictions themselves may be meaningless, such results can be used to rank users by quality.

[0039] In some embodiments of the present invention, if the filter probability meets (e.g., is greater than or less than) a predetermined or adjustable threshold, an installed base prediction model (e.g., a Bayesian logistic regression model that can predict the likelihood of a user installing a product identified in a content presentation) can be used to bid on an auction or auction-like process. In an embodiment, the threshold can be an adjustable hyperparameter, whose value can be used to control the trade-off between scale and performance of a given advertising campaign, etc. For example, the model development module 116 can use a filter model to predict category-level buyer likelihood. Category-level buyer likelihood can measure the likelihood that a user will convert into a buyer in a given category of advertisers, etc. If the category-level buyer likelihood prediction meets (e.g., is greater than) a predetermined or adjustable threshold (e.g., an adjustable hyperparameter, as discussed above), the model development module 116 can use the installed base prediction model to make appropriate user response predictions. Otherwise, the installed base prediction model will not be used to make user response predictions. Other filter models are also possible.

[0040] In an alternative embodiment, the model development module 116 can utilize an appropriate hierarchical model for the ensemble model as a base prediction model to, for example, aggregate data from relevant third parties (e.g., advertisers, etc.) while capturing third-party level effects (e.g., advertiser level effects, etc.). For example, the model development module 116 can train a multi-level Bayesian logistic regression (BLR) hierarchical model using top-level coefficients as lower-level priors for such models. In some embodiments of the present invention, the multi-level BLR hierarchical model can be a two-layer model with a category-level prior model and an application-level model. Other hierarchical models are also possible. Additionally or alternatively, the model development module 116 can use an appropriate multiplication model for the ensemble model with the base prediction model. For example, the multiplication model can decompose the data according to equation (7):

[0041] p(E | impression)=p(E | E1) p(E1 | impression) (7)

[0042] Among them, E1 can be a "pivot" event. The model development module 116 can train a p(E|E1) model on appropriate unattributed data and train a p(E1|impression) model on appropriate attribution data. For example, a p(install|impression) model can be trained on appropriate attribution data, while a p(purchaser|install) model can be trained on appropriate unattributed data. Other multiplicative models are also possible. For example, attributed data can be installation and post-install events that have been attributed to content presentation impressions of a particular entity. Unattributed or non-attributed data can be installation and post-install events that are not attributed to an entity (for example, either they are attributed to another user acquisition channel or they are spontaneous installations). For example, for unattributed or non-attributed data, there may be no information about the acquisition context, so a corresponding model can be built based on, for example, a user profile.

[0043] In some embodiments of the present invention, the model development module 116 can use a prediction model (e.g., a Bayesian logistic regression model) to predict the user's response to content presentation (e.g., click, install, in-app purchase, etc.) to generate an expected value p, which can be a probability value between 0 and 1. In other words, the predicted response can characterize the likelihood of the user interacting with the content presentation. In some embodiments of the present invention, the predicted response value can additionally or alternatively characterize the likelihood of the user's downstream conversion (i.e., the likelihood that the user takes some desired action in response to interacting with the content presentation). For example, the expected value p can represent the probability that the user converts after being exposed to the content presentation. Additionally or alternatively, the prediction model can also be used to predict continuous values ​​in some use cases, such as, for example, expected purchase frequency, expected in-app revenue, etc. In an embodiment, the user response prediction module 118 can use the expected value p from the model development module 116 to calculate the impression value of the content presentation. In some embodiments of the present invention, if the user response prediction p of the content presentation meets an appropriate predetermined or adjustable threshold (e.g., at 50%, 60%, 70%, etc. or above), the user response prediction module 118 may calculate an impression value for the content presentation. In an embodiment, the content presentation impression value may be a monetary (e.g., dollar) value for showing or otherwise displaying the content presentation impression to the user. In other words, the content presentation impression value may specify how much (in monetary value) one is willing to pay to display the content presentation to the user. If the user response prediction p does not meet (e.g., is below) the predetermined or adjustable threshold, the user response prediction module 118 may select another (alternative) content presentation for which the user response prediction p meets (e.g., is equal to or above) the predetermined or adjustable threshold. In some embodiments of the present invention, if the user response prediction p does not meet the predetermined or adjustable threshold, the alternative content presentation may not be selected (and no response provided).

[0044] The user response prediction module 118 may receive a request (e.g., a bid request as part of an auction process, etc., etc.) from a content presentation exchange 126 (e.g., an ad exchange, etc.) associated with a user's client device (e.g., via the network 110). The user response prediction module 118 may extract context (i.e., a plurality of classification features) from or otherwise related to the request to generate a feature vector x of a predetermined size (e.g., containing 5, 10, 15, or other suitable number of classification features). In addition to information that may be retrieved or otherwise derived from either or both of the user response data database 122 and / or the client device data database 124, information may also be obtained from the request itself. For purposes of illustration and not limitation, the feature vector x may include classification features such as, for example, publisher (e.g., application, category, etc.), location, user (e.g., age, gender, device identifier, etc.), time of day, day of the week, week of the year, geographic location (e.g., country, region, DMA, etc.), and / or other similar information. Other classification features are possible and will depend on the specific use case. In some embodiments of the present invention, the categorical features may be one-hot encoded to calculate, convert, or otherwise map the categorical features to a numerical representation of a feature vector x. The feature vector x may be passed to the model development module 116 to generate an expected value p based on the posterior weight distribution, as discussed above. The user response prediction module 118 may receive the expected value p from the model development module 116. The user response prediction module 116 may also utilize a target value y (e.g., click, install, engagement, purchase, number of purchases, revenue, etc.), which may be predetermined or calculated by the user response prediction module 118. In some embodiments of the present invention, the target value y may be a target cost per engagement (CPE), although other target values ​​or metrics are possible. The CPE price may be the price paid when a user engages with a content presentation (e.g., a digital advertisement) on their client device. Therefore, the content presentation impression value v may be calculated by the user response prediction module 118 according to equation (8):

[0045] v=p×y (8)

[0046] An impression is a view, display, presentation or other surface of a content presentation to a user on his client device. The content presentation impression value v may be the value of the content presentation impression. In some embodiments of the present invention, the content presentation impression value v may be used to determine an appropriate response to a request (e.g., an offer value in the form of a bid price in response to a request as part of, for example, an auction process).

[0047] In an embodiment, the content presentation impression value v is calculated by the user response prediction module 118 and the feature vector x can be passed to the response determination module 120. The response determination module 120 can be used to determine the amount or value of the response or other bid (e.g., the bid price b) to maximize the expected profit E[vb]. In an embodiment, the response determination module 120 can model the winning probability w as a function according to equation (9):

[0048]

[0049] Therefore, the response determination module 120 can calculate the response value or bid price b according to equation (10):

[0050] b = argmax b w (vb) (10)

[0051] Other methods of calculating the response value or bid price b are possible. The response determination module 120 may generate a response to the original request (e.g., a bid response to a request in an auction process, etc.), which may include a response value or bid price b, and optionally include an appropriate content presentation (e.g., retrieved from a user response data database 122 or otherwise generated), and the response is sent or otherwise sent to a content presentation trading platform 126 (e.g., via network 110). In an embodiment, a content presentation may be selected from a plurality or a group of content presentations. In some embodiments of the present invention, the response determination module 120 may utilize an appropriate model or technique (e.g., multi-arm beta / Bernoulli gambling, multivariate gambling, etc.) to select a content presentation (e.g., the best content presentation) based on a feature vector x. In an embodiment, contextual gambling may be used to determine a content presentation (e.g., the best content presentation). For example, a Bayesian logistic regression (LR) model or other similar model may be constructed (e.g., using a model development module 116). The Bayesian LR model may identify an appropriate posterior weight distribution w according to equation (11): i :

[0052] arg max i y i =σ(w·x); W i ~Ν(m,q) (11)

[0053] For example, the Bayesian LR model can employ Thompson sampling, which is an exploration / exploitation technique that operates on the posterior weight distribution during prediction. Other techniques for identifying appropriate posterior weight distributions are also possible. Therefore, the response determination module 120 can automatically adjust and generate, determine or optimize the response to the request (e.g., bids in response to requests in an auction process, etc.) in real time based on the user response prediction (and the content presentation impression value v calculated subsequently) and the feature vector x.

[0054] Figure 22 is a block diagram illustrating an exemplary method 200 for real-time user response prediction for content presentation on a client device. In some embodiments of the present invention, the method 200 may be performed as part of an auction or other similar process managed by the content presentation transaction platform 126. At block 205, the user response prediction module 118 may receive a request to display information on the user's client device (e.g., a bid request from the content presentation transaction platform 126 via the network 110 for an auction or auction-like process, etc.). At block 210, the user response prediction module 118 may extract a plurality of features associated with the request to generate a feature vector. For purposes of illustration and not limitation, the feature vector may include categorical features such as, for example, publisher (e.g., application, category, etc.), location, user (e.g., age, gender, device identifier, etc.), time of day, day of the week, week of the year, geographic location (e.g., country, region, DMA, etc.), and / or other similar information, although other features are possible. In some embodiments of the present invention, the categorical features may be one-hot encoded to calculate, convert, or otherwise map the categorical features to a numerical representation of the feature vector. At block 215, the user response prediction module 118 may select a content presentation to be displayed on the user's client device based on the feature vector. For example, the user response prediction module 118 may retrieve the content presentation from the user response data database 122. At block 220, the model development module 116 may utilize the prediction model and the feature vector to generate a predicted response of the user to the content presentation. In some embodiments of the present invention, the prediction model may be a Bayesian logistic regression model, etc. At block 225, the user response prediction module 118 may determine whether the predicted response meets (e.g., is greater than or equal to) an appropriate (predetermined or adjustable) threshold. If the predicted response does not meet (e.g., is lower than) the (predetermined or adjustable) threshold, the user response prediction module 118 may return to block 215 to select an alternative content presentation. In some embodiments of the present invention, if the predicted response does not meet (e.g., is lower than) the threshold, the method 200 may end at block 225 (i.e., not return to block 215 and not proceed to block 230). If the predicted response meets (e.g., is at or above) the threshold, then at block 230, the user response prediction module 118 may generate a content presentation impression value using the predicted response and a target value associated with the user. At block 235, the response determination module 120 may determine a response value to the request based on the content presentation impression value and the feature vector. At block 240, the response determination module 120 may send the response value and the selected content presentation in response to the request (e.g., to the content presentation transaction platform 126 via the network 110). When the response is accepted (e.g., accepted by the content presentation transaction platform 126 as part of an auction or auction-like process), the content presentation may be displayed on the user's client device.

[0055] Figure 3 300 is a block diagram illustrating another exemplary method 300 for predicting a user's response to content presentation on a client device. In an embodiment, at block 302, the user response prediction module 118 may receive a request to display information on a user's client device (e.g., a bid request from the content presentation trading platform 126 via the network 110, etc., as part of an auction or auction-like process). As described above, the information output or displayed on the user's client device may include digital advertisements, creative assets, etc., which may include text, images, videos, audio, etc. In addition, such information may be presented outside or inside a software application executed on the user's client device. In an embodiment, at block 304, the user response prediction module 118 may extract multiple features related to the request. These features may include at least one feature characterizing the user and at least one additional feature characterizing the client device. As described above, multiple features may include information such as, for example, data characterizing the publisher, the location of the content presentation on the screen of the client device, data associated with the user, such as, for example, age, gender, etc. In addition, multiple features may also include a geographic location specific to the user and / or the client device. The at least one characteristic characterizing the user may correspond to information such as age, gender, etc., and the at least one characteristic characterizing the client device may include a device identifier or other comparable information characterizing the client device.

[0056] In an embodiment, at block 306, the user response prediction module 118 can generate a feature vector. The feature vector can correspond to a list of a specific size, and can include, for example, 5, 10, or 15 features, although any appropriate number of features is possible. As discussed above, a feature vector can be a digital presentation of a combination of user attributes and / or other attributes. For example, a feature vector can include multiple digits and have any appropriate length according to the expected number of features captured in the feature vector. In some embodiments of the present invention, a unique hot encoding can be used for all or part of a feature vector to calculate the numerical presentation of each or any feature in a set of features. In an embodiment, one or more of the multiple features described above with respect to block 304 can be stored as a part of a feature vector generated at block 306.

[0057] In an embodiment, at block 308, the model development module 116 may generate a predicted response value for the user, in particular for a content presentation, i.e., a predicted response value for a content presentation that may be displayed on a user's client device. The predicted response value may be determined using the prediction model and feature vector generated at block 306. In an embodiment, the prediction model may be a Bayesian logistic regression model, etc. In an embodiment, the prediction model may be constructed by the model development module 116 and trained using, for example, a mini-batch finite memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), etc. Alternatively, in an embodiment, the prediction model may include a ridge, a lasso regression model, an elastic net regression model, an adaptive lasso regression model, a group lasso regression model, a random lasso regression model, and / or other suitable regression models. Additionally or alternatively, the prediction model can be or include a classifier, such as, for example, one or more linear classifiers (e.g., Fisher linear discriminator, logistic regression, naive Bayes classifier and / or perceptron), support vector machines (e.g., least squares support vector machine), quadratic classifier, kernel estimation model (e.g., k-nearest neighbors), boosting (meta-algorithm) model, decision tree (e.g., random forest, gradient boosting tree), neural network and / or learning vector quantization model, etc.

[0058] In an embodiment, at block 310, a content presentation impression value may be generated using a predicted response value and a target value associated with a user. It should be noted that the predicted response value may characterize the likelihood of a user interacting with a content presentation, and the target value may characterize the cost of a user interacting with the content presentation. In addition, it should be noted that a content presentation impression value may be generated when the predicted response value meets a predetermined or adjustable threshold. For example, the predicted response value may be expressed in the form of a probability value ranging from 0 to 1. It should be noted that other thresholds and value ranges may also be utilized. In addition, the target value may represent the cost of a user interacting with a content presentation displayed on a user's client device. As described above, the target value may correspond to a price associated with a specific instance of engagement between a user and a content presentation. It should be noted that other metrics may also be utilized.

[0059] In an embodiment, at block 312, response determination module 120 may determine a response value (i.e., request response value) based on the content presentation impression value and the feature vector. It should be noted that the response value (i.e., request response value) may correspond to an offer value or bid price for an opportunity to display the content presentation on the user's client device. As discussed above, such a request response value may be calculated using equation (10), which is repeated below:

[0060] b = argmax b w (vb) (10)

[0061] Finally, at block 314, response determination module 120 may be operable to send the request response value and the content presentation for display on the user's client device. For example, when the request response value is accepted (e.g., accepted by content presentation transaction platform 126 as part of an auction or auction-like process), the content presentation may be displayed on the user's client device. In an embodiment, the sending may be accomplished wirelessly or via a wired connection.

[0062] Figure 4 It is a block diagram of an exemplary computing device 400 that can perform one or more operations described in this article according to the present embodiment. Computing device 400 can be connected to other computing devices in a local area network, an intranet, an extranet and / or the Internet. Computing device 400 can be operated according to the capacity of a server machine in a client-server network environment, or can be operated according to the capacity of a client in a peer-to-peer network environment. Computing device 400 can be provided by a machine that can (sequentially or otherwise) perform a group of instructions (it specifies the action that this machine will take). In addition, although only a single computing device 400 is shown, the term "computing device" should also be considered to include the set of any computing devices that perform a group (or multiple groups) of instructions to perform method discussed herein alone or in conjunction.

[0063] The exemplary computing device 400 may include a computer processing device 402 (e.g., a general purpose processor, ASIC, etc.), a main memory 404, a static memory 406 (e.g., flash memory, etc.), and a data storage device 408 that can communicate with each other via a bus 430. The computer processing device 402 may be provided by one or more general purpose processing devices (e.g., a microprocessor, a central processing unit, etc.). In an illustrative example, the computer processing device 402 may include a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets or a processor that implements a combination of instruction sets. The computer processing device 402 may also include one or more special processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. According to one or more aspects of the present invention, the computer processing device 402 may be configured to perform the operations described herein to perform the operations and steps discussed herein.

[0064] The computing device 400 may further include a network interface device 412 that may communicate with a network 414. According to one or more aspects of the present invention, the data storage device 408 may include a machine-readable storage medium 428 on which one or more sets of instructions may be stored, for example, instructions for performing the operations described herein. The instructions 418 implementing the core logic instructions 426 may also reside in the main memory 404 and / or the computer processing device 402 during execution by the computing device 400, all or at least in part, which may also constitute computer-readable media. The instructions may further be sent or received via the network 414 via the network interface device 412.

[0065] Although the machine-readable storage medium 428 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) that store one or more sets of instructions. The term "computer-readable storage medium" should also be considered to include any medium that can store, encode, or carry a set of instructions for execution by a machine and causes the machine to perform the methods described herein. Therefore, the term "computer-readable storage medium" should be considered to include, but is not limited to, solid-state memories, optical media, magnetic media, and the like.

[0066] The subject matter and the embodiment of operation described in the present invention can be implemented in a digital electronic circuit including the structure disclosed in the present invention and its structural equivalent form, or computer software, firmware or hardware, or a combination of one or more thereof. The embodiment of the subject matter described in the present invention can be implemented as one or more computer programs, that is, a module of one or more computer program instructions encoded on a computer storage medium to be executed by a data processing device or to control the operation of a data processing device. Alternatively, or in addition, the program instructions can be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical, optical or electromagnetic signal), which is generated as coded information to be sent to a suitable receiver device to be executed by a data processing device. The computer storage medium can be or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof. In addition, although the computer storage medium is not a propagation signal, the computer storage medium can be the source or target of the computer program instructions encoded in the artificially generated propagation signal. The computer storage medium can also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks or other storage devices).

[0067] The operations described in the present invention may be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0068] The term "data processing device" includes various devices, means and machines for processing data, for example, including a programmable processor, a computer processing device, a computer, a system on a chip, or a plurality or combination of the foregoing. The computer processing device may include one or more processors, and the one or more processors may include a dedicated logic circuit, for example, a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), a central processing unit (CPU), a multi-core processor, etc. In addition to hardware, the device may also include code that creates an execution environment for the relevant computer program, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more thereof. The device and the execution environment can implement a variety of different computing model infrastructures, for example, network services, distributed computing, and grid computing infrastructures.

[0069] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative languages, procedural languages, or functional languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may (but does not necessarily) correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language resource), in a single file dedicated to a related program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on a single computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0070] The processes and logic flows described in the present invention may be performed by one or more programmable processors executing one or more computer programs to perform actions by running input data and generating output. The processes and logic flows may also be performed by a dedicated logic circuit (e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)), and the device may also be implemented as a dedicated logic circuit.

[0071] For example, a processor suitable for executing a computer program includes any one or more processors of both general and special microprocessors and any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic element of a computer is a processor for performing actions according to instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices (e.g., disks, magneto-optical disks, optical disks, solid-state drives, etc.) for storing data, or a computer may be operably connected to receive data from one or more large-capacity storage devices or send data to one or more large-capacity storage devices, or two situations. However, a computer may not necessarily have such a device. In addition, a computer may be embedded in another device (e.g., a smart phone, a mobile audio or media player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), etc.). Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CDROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0072] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD) monitor, a light emitting diode (LED) monitor, etc.) for displaying information to the user and a keyboard and pointing device (e.g., a mouse, a trackball, a touchpad, a stylus, etc.) that enables the user to provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user may be received in any form, including acoustic, voice, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices (e.g., single-point or multi-point resistive or capacitive tracking pads), voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices, and associated interpretation software, etc. In addition, the computer may interact with the user by sending resources to and receiving resources from devices used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0073] Embodiments of the subject matter described in this invention may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser that enables a user to interact with an implementation of the subject matter described in this invention), or any combination of one or more of the back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), peer-to-peer networks (e.g., ad hoc peer-to-peer networks), and the like.

[0074] The computing system may include a client and a server. The client and the server are usually far away from each other and generally interact through a communication network. The relationship between the client and the server is generated by means of a computer program that runs on a corresponding computer and has a client-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to a client device (e.g., for displaying data to a user interacting with the client device and receiving user input from the user). The data generated at the client device (e.g., the result of the user interaction) can be received from the client device at the server.

[0075] A system of one or more computers may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system, which, when executed, causes the system to perform actions. One or more computer programs may be configured to perform specific operations or actions by including instructions, which, when executed by a data processing device, causes the device to perform actions.

[0076] Reference throughout this disclosure to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this disclosure are not necessarily all referring to the same embodiment. Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or".

[0077] Although the present invention includes many specific implementation details, these details should not be interpreted as limitations on any invention scope or content that may be claimed, but should be interpreted as descriptions of features specific to specific embodiments of specific inventions. Certain features described in the context of separate embodiments in the present invention may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations, even initially claimed in this way, in some cases, one or more features in the claimed combination may be removed from the combination, and the claimed combination may point to a variant of a sub-combination or a sub-combination.

[0078] Similarly, although operations and / or logic flows are depicted in the accompanying drawings and / or described herein in a particular order, this should not be understood as requiring that these operations and / or logic flows be performed in the particular order or sequence shown, or requiring that all operations shown be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, but it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged as multiple software products.

[0079] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes described in the accompanying drawings do not necessarily require the specific order or sequence shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0080] The words "example" or "exemplary" are used herein to indicate use as an example, instance or illustration. Any aspect or design described herein as "example" or "exemplary" is not necessarily to be interpreted as preferred or superior to other aspects or designs. On the contrary, the use of the words "example" or "exemplary" is intended to present the concept in a specific way. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X includes A or B" is intended to mean any natural inclusive arrangement. That is, if X includes A; X includes B; or X includes both A and B, "X includes A or B" is satisfied in any of the above cases. In addition, the articles "one" and "a" used in this application and the appended claims should generally be interpreted as meaning "one or more", unless otherwise specified or clearly pointed to the singular form in the context. In addition, the terms "one embodiment" or "an embodiment" or "an implementation" or "an implementation" used throughout the text are not intended to mean the same embodiment or implementation, unless so described. In addition, the terms “first,” “second,” “third,” “fourth,” etc. as used herein are indicated as labels to distinguish different elements, and do not necessarily have ordinal meanings according to their numerical names.

[0081] In the above description and in the claims, phrases such as "at least one of" or "one or more of" may appear before a list of combinations of elements or features. The term "and / or" may also appear in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradictory to the context in which it is used, this phrase is intended to mean any individually listed element or feature, or any combination of a recorded element or feature with any other recorded element or feature. For example, the phrases "at least one of A and B"; "one or more of A and B"; and "A and / or B" are intended to mean "A alone, B alone, or A and B together", respectively. A similar interpretation is also intended for a list including three or more items. For example, the phrases "at least one of A, B, and C"; "one or more of A, B, and C"; and "A, B, and / or C" are intended to mean "A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together", respectively. Furthermore, the term “based on” as used above and in the claims is intended to mean “based, at least in part, on” such that non-recited features or elements are also permitted.

[0082] The above description of the implementation of the illustrated invention is not intended to be exhaustive or to limit the invention to the precise form disclosed. Although the specific implementation and examples of the invention are described herein for the purpose of illustration, various equivalent modifications are possible within the scope of the invention, as recognized by those skilled in the relevant art. Depending on the desired configuration, the subject matter described herein can be implemented in systems, devices, methods and / or articles. The implementations set forth in the above specification do not represent all implementations consistent with the subject matter described herein. On the contrary, they are only some examples consistent with aspects related to the described subject matter. Although some variations have been described in detail above, other modifications or additions may also be made. Specifically, in addition to those features and / or variations set forth herein, further features and / or variations may also be provided. For example, the above implementations may be directed to various combinations and sub-combinations of the disclosed features, and / or combinations and sub-combinations of several further features disclosed above. Other implementations may be within the scope of the appended claims.

Claims

1. A method comprising: receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features comprising at least one feature characterizing a user and at least one additional feature characterizing a client device; generating a feature vector based on a plurality of features associated with the request; Generate a predicted response value of a user to content presentation using a prediction model and the feature vector, wherein the predicted response value represents a likelihood of the user interacting with the content presentation; When the predicted response value meets a predetermined threshold, generating a content presentation impression value using the predicted response value and a target value associated with the user, the target value representing a cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; as well as The request response value and the content presentation are sent for display on the user's client device.

2. The method according to claim 1, wherein: The prediction model is a Bayesian logistic regression model.

3. The method according to claim 1, wherein: The predicted response value further characterizes the likelihood of a downstream conversion in response to a user interacting with the content presentation.

4. The method according to claim 1, wherein: Generating a content presentation impression value using the predicted response value and a target value associated with the user includes: multiplying the target value by the predicted response value.

5. The method according to claim 1, further comprising: A content presentation is selected for display on a user's client device based on the feature vector.

6. The method according to claim 1, further comprising: In response to determining that the predicted user response value to the content presentation does not satisfy a predetermined threshold, an alternative content presentation is selected for display on the client device.

7. The method according to claim 6, further comprising: The prediction model and the feature vector are used to generate an additional predicted response value of the user to the alternative content presentation.

8. The method according to claim 7, further comprising: When the additional predicted response value meets a predetermined threshold, an additional content presentation impression value is generated using the additional predicted response value and a target value associated with the user.

9. The method according to claim 1, further comprising: Implement the filter model associated with the prediction model.

10. The method according to claim 9, wherein: Implementing the filter model includes determining an additional likelihood that the user will transition within a specific category associated with one or more additional content presentations.

11. A system comprising: at least one data processor; as well as A memory storing instructions that, when executed by at least one data processor, cause the at least one data processor to perform operations including: receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features comprising at least one feature characterizing a user and at least one additional feature characterizing a client device; generating a feature vector based on a plurality of features associated with the request; Generate a predicted response value of a user to a content presentation using the prediction model and the feature vector, wherein the predicted response value represents a likelihood of the user interacting with the content presentation; When the predicted response value meets a predetermined threshold, generating a content presentation impression value using the predicted response value and a target value associated with the user, the target value representing a cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; as well as The request response value and the content presentation are sent for display on the user's client device.

12. The system according to claim 11, wherein: The prediction model is a Bayesian logistic regression model.

13. The system according to claim 11, wherein: The predicted response value further characterizes the likelihood of a downstream conversion in response to a user interacting with the content presentation.

14. The system according to claim 11, wherein: The at least one data processor generates a content presentation impression value using the predicted response value and a target value associated with the user by multiplying the target value and the predicted response value.

15. The system according to claim 11, wherein: The operations further include: A content presentation is selected for display on a client device of the user based on the feature vector.

16. The system of claim 11, wherein: The operations further include: In response to determining that the predicted user response value to the content presentation does not satisfy a predetermined threshold, an alternative content presentation is selected for display on the client device.

17. The system of claim 16, wherein: The operations further include: The prediction model and the feature vector are utilized to generate additional predicted response values ​​of the user to the alternative content presentation.

18. The system of claim 11, wherein: The operations further include: Implement the filter model associated with the prediction model.

19. The system of claim 18, wherein: Operations implementing the filter model include determining additional likelihoods for a user to transition within a specific category associated with one or more additional content presentations.

20. A non-volatile computer program product storing executable instructions which, when executed by at least one data processor forming part of at least one computing system, perform operations comprising: receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features comprising at least one feature characterizing a user and at least one additional feature characterizing a client device; generating a feature vector based on a plurality of features associated with the request; Generate a predicted response value of a user to a content presentation using the prediction model and the feature vector, wherein the predicted response value represents a likelihood of the user interacting with the content presentation; When the predicted response value meets a predetermined threshold, generating a content presentation impression value using the predicted response value and a target value associated with the user, the target value representing a cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; as well as The request response value and the content presentation are sent for display on the user's client device.