Systems and methods for real-time user response prediction regarding content presentation on a client device

Abstract

In an aspect, a request for displaying information on a user's client device can be received. A plurality of features associated with the request can be extracted. The plurality of features can include at least one feature that characterizes the user and at least an additional feature that characterizes the client device. A feature vector based on the features associated with the request can be generated. A predicted response value of the user to the content presentation can be generated using the predictive model and the feature vector. The predicted response value can characterize the likelihood of the user interacting with the content presentation. A request response value can be determined based on the content presentation impression value and the feature vector. The request response value and the content presentation can be transmitted for display on the user's client device.

Description

[Background technology]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 63 / 266,186, filed December 30, 2021, the entire contents of which are incorporated herein by reference.

[0002] Client devices can present a wide variety of digital content to users, including video, audio, images, and combinations thereof. Content is often developed and used to achieve a certain response from users exposed to the content. Some content can prompt users to take a certain action, such as interacting with or engaging with a client application, purchasing a product or service, or installing a client application. However, not all users will respond to the digital content presented to them, and therefore presenting the same digital content to all users can result in wasted resources and be unproductive. Determining whether users will respond to digital content and which users will respond can be difficult, time-consuming, and expensive. Summary of the Invention [Means for solving the problem]

[0003] A system and method are provided for real-time user response prediction for content presentation on a client device. Related apparatus, techniques, and articles are also described.

[0004] In an aspect, a request for displaying information on a user's client device can be received. A plurality of features associated with the request can be extracted. The plurality of features can include at least one feature that characterizes the user and at least an additional feature that characterizes the client device. A feature vector based on the plurality of features associated with the request can be generated. A predicted response value of the user to the content presentation can be generated using the prediction model and the feature vector. The predicted response value can characterize the likelihood of the user interacting with the content presentation. A content presentation impression value can be generated using the predicted response value and a target value associated with the user when the predicted response value satisfies a pre-determined threshold. The target value can characterize a cost of the user's interaction with the content presentation. A request response value can be determined based on the content presentation impression value and the feature vector. The request response value and the content presentation can be transmitted for display on the user's client device.

[0005] The one or more features can be included in any feasible combination. For example, the predictive model can be a Bayesian logistic regression model. For example, the predicted response value can further characterize the likelihood of a downstream conversion of the user in response to interacting with the content presentation. For example, a content presentation impression value can be generated using the predicted response value and a target value associated with the user by multiplying the target value and the predicted response value. For example, a content presentation for display on the user's client device can be selected based on the feature vector. For example, an alternative content presentation for display on the client device can be selected in response to determining that the user's predicted response value to the content presentation does not satisfy a pre-determined threshold. For example, an additional predicted response value of the user to the alternative content presentation can be generated using the predictive model and the feature vector. For example, the additional content presentation value can be generated using the additional predicted response value and a target value associated with the user when the additional predicted response value satisfies a pre-determined threshold. In another embodiment, a filter model associated with the predictive model can be implemented. For example, implementation of a filter model can include determining an additional likelihood that a user will interact with a particular category that is associated with one or more additional content presentations.

[0006] In another aspect, a system is provided and may include at least one data processor and a memory storing instructions configured to cause the at least one data processor to perform operations described herein. The operations can include receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features including at least one feature characterizing the user and at least an additional feature characterizing the client device; generating a feature vector based on the plurality of features associated with the request; generating a predicted response value for the user to the content presentation using the predictive model and the feature vector, the predicted response value characterizing a likelihood of the user interacting with the content presentation; generating a content presentation impression value using the predicted response value and a target value associated with the user when the predicted response value satisfies a predetermined threshold, the target value characterizing a cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; and transmitting the request response value and the content presentation for display on the user's client device.

[0007] The one or more features can be included in any feasible combination. For example, the predictive model can be a Bayesian logistic regression model. For example, the predicted response value further characterizes the likelihood of a downstream conversion of the user in response to interacting with the content presentation. For example, the at least one data processor can perform an operation of generating a content presentation impression value using the predicted response value and a target value associated with the user by multiplying the target value and the predicted response value. For example, the operation can further include selecting a content presentation for display on the client device of the user based on the feature vector. For example, the operation can further include selecting an alternative content presentation for display on the client device in response to determining that the user's predicted response value to the content presentation does not satisfy a predetermined threshold. For example, the operation can further include generating an additional predicted response value of the user to the alternative content presentation using the predictive model and the feature vector. For example, the operation can further include implementing a filter model associated with the predictive model. For example, the acts of implementing a filter model can include determining an additional likelihood that a user will convert within a particular category that is associated with one or more additional content offers.

[0008] In yet another aspect, a non-transitory computer program product (i.e., a physically embodied computer program product) is also described which stores instructions that, when executed by at least one data processor forming part of at least one computing system, cause the at least one data processor to perform the operations described herein. The operations include receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features including at least one feature characterizing the user and at least an additional feature characterizing the client device; generating a feature vector based on the plurality of features associated with the request; generating a predicted response value for the user to the content presentation using the predictive model and the feature vector, the predicted response value characterizing a likelihood of the user interacting with the content presentation; when the predicted response value satisfies a predetermined threshold, generating a content presentation impression value using the predicted response value and a target value associated with the user, the target value characterizing a cost of the user's interaction with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; and transmitting the request response value and the content presentation for display on the user's client device.

[0009] A computer system is also described, which may include one or more data processors and a memory coupled to the one or more data processors. The memory may store, temporarily or permanently, instructions that cause at least one processor to perform one or more of the operations described herein. In addition, the method may be implemented by one or more data processors, either in a single computing system or distributed among two or more computing systems. Such computing systems may be connected and may exchange data and / or commands or other instructions or the like through one or more connections, including connections such as via a network (e.g., the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, or the like), via a direct connection between one or more of the computing systems, or the like. [Brief description of the drawings]

[0010] The above-described embodiments will be more fully understood from the following detailed description taken in conjunction with the accompanying drawings, in which: The drawings are not intended to be drawn to scale, and for purposes of clarity, not every component may be labeled in every drawing.

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example system for real-time user response prediction.

[0012] [Diagram 2] FIG. 2 is a block diagram illustrating an example method for real-time user response prediction.

[0013] [Diagram 3]FIG. 3 is a block diagram illustrating another example method for prediction of a user's response to a content presentation displayed on a client device.

[0014] [Figure 4] FIG. 4 is a block diagram of an example computing device that may perform one or more of the operations described herein in accordance with the present embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Detailed Description Certain exemplary embodiments will now be described to provide a general understanding of the principles of the structure, function, manufacture, and use of the devices and methods disclosed herein. One or more examples of these embodiments are illustrated in the accompanying drawings. Those skilled in the art will appreciate that the devices and methods specifically described herein and illustrated in the accompanying drawings are non-limiting exemplary embodiments, and the scope of the present invention is defined only by the claims. Features illustrated or described in connection with one exemplary embodiment may be combined with features of other embodiments. Such modifications and variations are intended to be included within the scope of the present invention. Furthermore, in this disclosure, similarly named components of an embodiment generally have similar characteristics, and therefore, within a particular embodiment, each feature of each similarly named component is not necessarily described in full detail therewith.

[0016] The present invention is directed to a system and method for real-time user response prediction regarding content presentation on a client device. In accordance with the present invention, an automated process can predict the likelihood that a user will engage with a content presentation or other digital content presented on their client device. An embodiment of the present invention can predict such user responses in real-time using a suitable machine learning model. The prediction can be used to calculate an impression value of the content presentation, which can be used, for example, to generate, determine, or optimize a response to a request to display or otherwise present the content presentation on the user's client device. Solely for purposes of discussion, and not limitation, the present disclosure will refer to content presentation on a mobile device to illustrate various aspects of the present invention. However, the present invention can be used in and with any suitable type of system for which a user's predicted response can be used to determine and select the type, format, and content of information to present to the user on their client device in real-time.

[0017] An embodiment of the present invention can improve computer processing efficiency, especially when there are a large number of users and a large number of content submissions (e.g., millions, tens of millions, etc.) that must be processed within a fraction of a second, by determining and selecting the type, format, and content of information to present to users on client devices based on their predicted responses, rather than through time- and processor-intensive trial-and-error and other such computer-based methodologies. In addition, an embodiment of the present invention can use a linear model (e.g., a logistic regression model or equivalent) to make predictions within such strict time constraints and process millions of requests per second without using specialized computer hardware to make such predictions. Such a linear model can also allow for the rapid incorporation of new data to make the most accurate predictions. Furthermore, an embodiment of the present invention can use a Bayesian approach to quantify the uncertainty in the predictions and efficiently balance content submission inventory exploration and exploitation that is unattainable through other such trial-and-error and computer-based methodologies.

[0018] FIG. 1 is a block diagram illustrating an exemplary system 100 for real-time user response prediction for content presentation on a client device. The server system 114 can provide functionality for developing predictive models, using the predictive models to predict user responses to content presentations in real-time, and generating, determining, or otherwise optimizing responses to requests for the content presentation to be presented on the user's client device. In an embodiment, the content presentation can be any suitable type of digital content that can be served, presented, or otherwise displayed to a user on a client device, such as, for example, a digital advertisement, a creative asset, or other presentation of digital content or the like, and can include text, images, video, audio, or any combination thereof. The content presentation can be used for any suitable purpose to achieve a desired response from a user exposed to the content presentation. The server system 114 can include software components and databases that can be deployed, for example, in one or more data centers 112 in one or more geographic locations. The software components of the server system 114 may include a model development module 116, a user response prediction module 118, and a response determination module 120. The software components may include subcomponents that may run on the same or different individual data processing devices. The databases of the server system 114 may include, for example, a user response data database 122 and a client device data database 124. The databases may reside in one or more physical storage systems or be cloud-based. The software components and data will be further described below.

[0019] 1, the model development module 116, the user response prediction module 118, and the response determination module 120 can communicate with a user response data database 122 and a client device data database 124. The user response data database 122 can store data associated with users, user responses to responsive content presentations, user information (e.g., age, gender, device identifier, etc.), content presentation response or bid history (e.g., publisher, category, bid, impression, sampled auction, etc.), user response history (e.g., clicks, installs, in-app purchases, in-app revenue, number of purchases, content presentation interactions (e.g., completed views), etc.), content presentation performance metrics (e.g., click-through rate, cost per action, etc.), install and post-install events (e.g., in-app purchases, content presentation revenue, etc.), The content presentation may include publisher information (e.g., application, category, etc.), content presentation context (e.g., Wi-Fi / cellular to reflect the context in which the content presentation was displayed), the content presentation and / or information related to the content presentation, such as images, video, sound, text, and the like, and may include a description of the characteristics or features present in the content presentation (e.g., the format of the content presentation), content presentation placement information, time / date (e.g., time of day, day of week, week number of year, etc.), user geographic information (e.g., country, region, designated marketing area (DMA), etc.), and the like. Other user response data is also possible. The client device data database 124 may include information or characteristics related to the user's client device, such as, for example, client device model, carrier, client device operating system, in-app events, client device characteristics (e.g., price, hardware, storage, etc.), summarized statistics calculated from such data and / or data from the user response data database 122, and the like. Other client device information or characteristics are also possible.

[0020] The software application or components thereof may be executed on users' client devices, such as client device A 102, client device B 104, client device C 106, ..., client device N 108 (where N may be any suitable natural number), and may be accessed and accessed through a network 110 (e.g., the Internet). Each of the client devices may be any suitable type of electronic device capable of executing the software application and communicating with the server system 114 through the network 110, such as, for example, a smart phone, tablet computer, laptop computer, desktop or personal computer, gaming console, set-top box, smart television, or the like. Other client devices are also possible. In alternative embodiments, the user response data database 122, the client device data database 124, or any portion thereof, may be stored on one or more client devices. Additionally or alternatively, software components for system 100 (e.g., model development module 116, user response prediction module 118, and / or response determination module 120), or any portions thereof, may reside on or be used to perform operations on one or more client devices.

[0021] For example, software applications such as web-based or mobile applications (e.g., publisher apps) can be provided (e.g., by a publisher) as end-user applications to enable users to interact with the server system 114. Additionally, suitable digital content (e.g., content presentations) can be displayed to users through or in association with software applications executing on the user's client device. The software applications can relate to and / or provide a wide variety of functions and information including, for example, entertainment (e.g., games, music, videos, etc.), business (e.g., word processing, accounting, spreadsheets, etc.), news, weather, financing, sports, etc. In an embodiment, the software application can provide a mobile game. The mobile game can be or include, for example, a sports game, an adventure game, a virtual playing card game, a virtual board game, a puzzle game, a racing game, or any other suitable type of mobile game. In an embodiment, the mobile game can be an asynchronous competitive skill-based game in which players can compete against each other within the mobile game but need not play the mobile game at the same time. In an alternative embodiment, the mobile game can be a synchronous competitive skill-based game in which players can simultaneously play the mobile game and compete against each other within the mobile game in real time. In some embodiments, the content presentation can be presented on the client device outside of the software application and allow a user to interact with or engage with the content presentation, for example, by clicking or selecting on or within the content presentation. Additionally or alternatively, the software application can be configured to present the content presentation within the software application on the client device and allow a user to interact with or engage with the content presentation.

[0022] According to an exemplary embodiment, the model development module 116 can develop a user response prediction model and be used to predict user responses to content presentations in real time. In an embodiment, any suitable predictive machine learning model can be suitably trained and used to predict user responses in real time. In some implementations of the present invention, a Bayesian logistic regression model can be used by the model development module 116 as the user response prediction model. Bayesian logistic regression is a probabilistic model that can automatically calculate the probability of success for a new data point. In some implementations of the present invention, the model development module 116 can build a model for a single entity (e.g., an advertiser or equivalent) or a group of entities (e.g., a group of advertisers or equivalent) that desire to display one or more content presentations on a user's client device. The model development module 116 can arrange a predetermined number of categorical features (e.g., 5, 10, 15, or any suitable number) in a feature vector. The categorical features in the feature vector can be categorized or stored in the feature vector in any desired order. A feature vector may comprise a numerical representation of a combination of categorical features, such as user attributes and / or other attributes. For example, a feature vector may include a plurality of numbers (e.g., integers), each number representing a different categorical feature. A feature vector may be of any suitable length depending on the desired number of categorical features to be captured in the feature vector. In some implementations of the invention, one-hot encoding may be used for all or a portion of a feature vector to compute, convert, or otherwise map a numerical representation for each or any categorical feature in a set of categorical features, although other suitable encoding techniques may also be used. For illustrative purposes, but not by way of limitation, a feature vector having a length or size M, where M is any suitable natural number, may include a numerical representation for a categorical feature of a particular publisher app as the mth element of the feature vector.To one-hot encode publisher app A, if the m-th element of the feature vector is "1", this can represent that the publisher app is A, otherwise the m-th element is "0" (i.e., the publisher app is not A). Other numeric representations of categorical features are also possible. For example, the numeric representation may depend on the types and characteristics of the various user attributes and / or other attributes that make up the categorical features in the feature vector, and how such categorical features are converted or mapped to numbers (e.g., integers) by one-hot encoding or other suitable encoding techniques. For illustrative purposes, but not by way of limitation, categorical features may include, for example, publisher app, country, time of day, day of week, Wi-Fi / cellular, region, DMA, carrier, device OS, ad format, in-app events, auction stream history (e.g., publisher and category), and the like. Other categorical features are also possible and would depend on the use case. All or any of these categorical features can be encoded (e.g., one-hot encoded) and converted to a corresponding numerical representation (e.g., integers) in the feature vector. In some implementations of the invention, users can be split into multiple groups or cohorts to maximize information gain. For example, users with one or more similar characteristics can be grouped into a first cohort, while users with other similar characteristics can be grouped into a second, third, fourth, etc. cohort. Any suitable number of cohorts can be used, and each or any of the cohorts can be defined with desired constraints, for example, based on use case requirements.

[0023] In some implementations of the present invention, the model development module 116 may use Bayesian logistic regression to determine the distribution of weighting coefficients w given the data in the feature vector x, as shown in equation (1). [ka] The prediction can be given by equation (2). [ka] where w is drawn from the distribution in equation (1) (e.g., using Thompson sampling or equivalent). More specifically, the posterior probabilities for the weighting coefficients can be expressed as in equation (3). [ka] During the ceremony, [ka] where "Bern" represents the Bernoulli likelihood function. In equation (3), m 0 and q 0 may reflect prior knowledge about the average and accuracy of each weight, respectively. The model development module 116 may determine the most likely value of w given the data, which may be referred to as the maximum a posteriori (MAP) estimate, according to equation (4). [ka] The model development module 116 may then apply a Laplace approximation to estimate the posterior probability precision of each weight according to equation (5). [ka] In some implementations of the invention, the predictive model built by the model development module 116 can be trained on suitable data (e.g., using a suitable optimizer such as mini-batch limited memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) or equivalent) and then periodically updated with incremental data as it is received by the server system 114. In one embodiment, app embedding can be used to vectorize apps based on co-occurrence on a client device to extend predictions to similar apps (software applications) on similar client devices.

[0024] Other suitable predictive models can also be used. For example, the predictive model can be or include, for example, one or more equations (e.g., regression equations or equivalents) and / or classifiers. In some embodiments, the predictive model can be or include, for example, ridge, lasso, elastic net, adaptive lasso, group lasso, randomized lasso, and / or other suitable regression models. Additionally or alternatively, the predictive model can be or include, for example, one or more linear classifiers (e.g., Fisher's Linear Discriminant, logistic regression, naive Bayes classifier, and / or perceptron), support vector machines (e.g., least squares support vector machines), quadratic classifiers, kernel estimation models (e.g., k-nearest neighbors), boosting (meta-algorithm) models, decision trees (e.g., random forests, gradient boosting trees), neural networks, and / or learning vector quantization models, and other classifiers.

[0025] In some implementations of the present invention, the model development module 116 can use additive models to complement or extend a base predictive model (e.g., a Bayesian logistic regression model) as an ensemble model. In an embodiment, predictions from one or more additive models can be suitably combined with each other and / or with predictions generated by the base predictive model (e.g., via multiplication, averaging, median, etc.) to improve or otherwise refine the predictions generated by the base predictive model. Additionally or alternatively, predictions from one or more additive models can be used as filters to determine whether a response should be submitted (e.g., based on whether a prediction from one or more additive models is above or below one or more pre-determined or adjustable thresholds). The model development module 116 can use suitable filter models to filter traffic in certain scenarios (e.g., for audience expansion, lookalike targeting, etc.). In an embodiment, the filter model can be based on a self-mutual information (PMI) score, which can be a measure of correlation between two events or association between a feature and a class (e.g., a larger and / or different group of features). A PMI score can represent a quantified measure of how likely or unlikely two events are to occur together. For example, a PMI score can represent a correlation between one or more of a number of features and conversions, for example, within a particular category of advertisers. A PMI score can compare the probability of two events occurring together with this probability that would occur if the two events were independent. For example, a PMI score of 0 for two events x and y can represent that the particular events x and y are statistically independent. A positive PMI score can represent that events x and y occur together more frequently than would be expected if they were independent events. A negative PMI score can represent that events x and y occur together less frequently than would be expected.In some implementations of the present invention, the PMI score for a pair of events x and y may be calculated according to equation (6). [ka] Equation (6) can be based on a maximum likelihood estimate, where p(x,y) can be the joint probability of random variables x and y divided by the product of their individual probabilities p(x) and p(y). For example, the following information can be determined according to some implementations of the present invention: x , which can be the number of observations of event x, and (ii) o y , which may be the number of observations of event y, and (iii) N, which may be the total size of the pool of observations. Given such information, the probability of the co-occurrence of events x and y and x and y is given by: p(x)=o x / N and p(y)=o y / N, where p(x,y) can be determined using equation (6), where p(x,y) is the number of observed co-occurrences of x and y. For illustrative purposes, but not by way of limitation, a PMI score can be measured between the presence of a software application (e.g., a publisher app) on a user's client device and the fact that the user converted within an advertiser. Events between which correlations can be determined can include, for example, the degree to which the fact that a user has app A on their client device increases the likelihood that the user will make a purchase within app B. Co-occurrences of other events as measured by PMI scores are also possible. Additionally or alternatively, the filter model can be based on a suitable positive unlabeled classifier. For example, a suitable model can be trained to distinguish between a list of good users and an "average" user. While predictions may not be meaningful by themselves, such results can be used to rank users by quality.

[0026] In some implementations of the present invention, bids for an auction or auction-like process can be made using an install-based predictive model (e.g., a Bayesian logistic regression model that can predict the likelihood of a user to install a product identified in a content presentation) if the filter probability meets (e.g., is above or below) a pre-determined or adjustable threshold. In an embodiment, the threshold can be an adjustable hyperparameter, the value of which can be used to control the trade-off between scale and performance for a given advertising campaign or equivalent. For example, the model development module 116 can use the filter model to predict a category-level buyer likelihood. The category-level buyer likelihood can measure the likelihood that a user will convert to a buyer within a given category of advertiser or equivalent. If the category-level buyer likelihood prediction meets (e.g., is above) a pre-determined or adjustable threshold (e.g., an adjustable hyperparameter as discussed above), the model development module 116 can make a suitable user response prediction using the install-based predictive model. Otherwise, user response prediction would not be performed using the installed base prediction model. Other filter models are also possible.

[0027] In an alternative embodiment, the model development module 116 can use a suitable hierarchical model for the ensemble model as a base predictive model, for example, to pool data from related third parties (e.g., advertisers or the like) while capturing third-party level effects (e.g., advertiser level effects or the like). For example, the model development module 116 can train a multilevel Bayesian logistic regression (BLR) hierarchical model using upper level coefficients as initial values ​​for lower levels of such a model. In some implementations of the present invention, the multilevel BLR hierarchical model can be a two-tier model with category level initial values ​​and an app level model. Other hierarchical models are also possible. Additionally or alternatively, the model development module 116 can use a suitable multiplicative model for the ensemble model along with the base predictive model. For example, the multiplicative model can decompose the data according to equation (7). [ka] where E1 can be a "pivot" event. The model development module 116 can train a p(E|E1) model on suitable unattributed data and a p(E1|impression) model on suitable attributable data. For example, a p(install|impression) model can be trained on suitable attributable data, while a p(purchaser|install) model can be trained on suitable unattributed data. Other multiplicative models are also possible. For example, attributable data can be install and post-install events that are attributed to a content presentation impression of a particular entity. Unattributed or non-attributed data can be install and post-install events that are not attributed to an entity (e.g., either of them is attributed to another user acquisition channel or they are organic installs). For example, for unattributed or non-attributed data, there may be no information about the acquisition context, and thus a corresponding model can be built, for example, based on a user profile.

[0028] In some implementations of the present invention, the model development module 116 can predict a user's response to a content presentation (e.g., click, install, in-app purchase, etc.) using a predictive model (e.g., a Bayesian logistic regression model) to generate an expected value p, which can be a probabilistic value between 0 and 1. In other words, the predicted response can characterize the user's likelihood of interacting with the content presentation. In some implementations of the present invention, the predicted response value can additionally or alternatively characterize the user's likelihood of downstream conversion (i.e., the user's likelihood of taking a desired action in response to interacting with the content presentation). For example, the expected value p can represent the probability that a user will convert after being exposed to the content presentation. Additionally or alternatively, the predictive model can also be used in some use cases to predict persistent values, such as, for example, expected purchase frequency, expected in-app revenue, and the like. In an embodiment, the user response prediction module 118 can use the expected value p from the model development module 116 to calculate an impression value for the content presentation. In some implementations of the present invention, the user response prediction module 118 can calculate an impression value for a content presentation if the user response prediction p for the content presentation satisfies (e.g., is at or exceeds) a suitable pre-determined or adjustable threshold. In an embodiment, the content presentation impression value can be a value in currency (e.g., dollars) of showing or otherwise displaying the content presentation impression to a user. In other words, the content presentation impression value can define an amount (in monetary value) that is willing to be paid to show the content presentation to a user. If the user response prediction p does not satisfy (e.g., is below) the pre-determined or adjustable threshold, the user response prediction module 118 can select another (alternative) content presentation for which the user response prediction p satisfies (e.g., is at or above) the pre-determined or adjustable threshold.In some implementations of the present invention, an alternative content presentation may not be selected (and a response provided) if the user response prediction p does not satisfy a predetermined or tunable threshold.

[0029] The user response prediction module 118 may receive (e.g., over the network 110) a request (e.g., a bid request or equivalent as part of an auction process or equivalent) from a content presentation exchange 126 (e.g., an ad exchange or equivalent) associated with the user's client device. The user response prediction module 118 may extract context (i.e., multiple category features) from or otherwise associated with the request to generate a feature vector of a predetermined size x (e.g., containing 5, 10, 15, or other suitable number of category features). Information may be obtained from the request itself in addition to, or as an alternative to, information that may be retrieved or otherwise derived from one or both of the user response data database 122 and / or the client device data database 124. For illustrative purposes, but not by way of limitation, the feature vector x may include categorical features such as publisher (e.g., app, category, etc.), placement, user (e.g., age, gender, device identifier, etc.), time of day, day of week, week of year, geographic location (e.g., country, region, DMA, etc.), and / or other similar information. Other categorical features are also possible and would depend on the particular use case. In some implementations of the present invention, the categorical features may be one-hot encoded, and the categorical features may be calculated, converted, or otherwise mapped to a numerical representation for the feature vector x. The feature vector x may be passed to the model development module 116, which may generate an expected value p based on a posterior probability weighted distribution as described above. The user response prediction module 118 may receive the expected value p from the model development module 116. The user response prediction module 116 may also use a target value y (e.g., clicks, installs, engagement, purchases, number of purchases, revenue, or the like), which may be pre-determined or calculated by the user response prediction module 118.In some implementations of the present invention, the target value y can be a target cost per engagement (CPE), although other target values ​​or metrics are possible. The CPE price can be a price paid when a content presentation (e.g., a digital advertisement) is engaged by a user on their client device. Thus, the content presentation impression value v can be calculated by the user response prediction module 118 according to equation (8). [ka] An impression is a viewing, viewing, presentation, or other surfacing of a content presentation to a user on their client device. A content presentation impression value v can be a value of a content presentation impression. In some implementations of the invention, the content presentation impression value v can be used to determine an appropriate response to a request (e.g., a bid value in the form of a bid price, for example, in response to a request as part of an auction process or equivalent).

[0030] In one embodiment, the content presentation impression value v and the feature vector x calculated by the user response prediction module 118 can be passed to a response determination module 120. The response determination module 120 can be used to determine an amount or value (e.g., a bid price b) to respond or otherwise bid to maximize the expected profit E[vb]. In one embodiment, the response determination module 120 can model the win probability w as a function according to equation (9). [ka] As a result, the response determination module 120 can calculate the response value or bid price b according to equation (10). [ka] Other methods of calculating the response value or bid price b are also possible. The response determination module 120 generates a response to the original request (e.g., a bid response to a request in an auction process or equivalent), which may include the response value or bid price b, and optionally a suitable content offer (e.g., retrieved from the user response data database 122 or otherwise generated), and can transmit or otherwise send the response to the content offer exchange 126 (e.g., via the network 110). In an embodiment, the content offer can be selected from a plurality or set of content offers. In some implementations of the present invention, the response determination module 120 can select a content offer (e.g., an optimal content offer) based on the feature vector x using a suitable model or technique (e.g., multi-armed bandit beta / Bernoulli bandit, multivariate bandit, or equivalent). In an embodiment, a contextual bandit can be used to determine the content offer (e.g., an optimal content offer). For example, a Bayesian logistic regression (LR) model or other similar model can be constructed (e.g., using the model development module 116). The Bayesian LR model calculates the preferred posterior probability weighted distribution w i can be identified. [ka] For example, the Bayesian LR model can use Thompson sampling, which is an exploration / exploitation technique that operates on the posterior probability weight distribution during prediction. Other techniques for identifying a suitable posterior probability weight distribution are also possible. Thus, the response determination module 120 automatically adjusts, generates, determines, or optimizes a response to a request (e.g., a bid in response to the request in an auction process or equivalent) based on the user response prediction (and the subsequently calculated content presentation impression value v) and the feature vector x in real time.

[0031] FIG. 2 is a block diagram illustrating an exemplary method 200 for real-time user response prediction for content presentation on a client device. In some implementations of the invention, the method 200 can be implemented as part of an auction or other similar process managed by the content presentation exchange 126. In block 205, the user response prediction module 118 can receive a request (e.g., a bid request or equivalent for an auction or auction-like process from the content presentation exchange 126 over the network 110) and display information on the user's client device. In block 210, the user response prediction module 118 can extract a number of features associated with the request to generate a feature vector. For illustrative purposes, but not limited to, the feature vector can include categorical features such as publisher (e.g., app, category, etc.), placement, user (e.g., age, gender, device identifier, etc.), time of day, day of week, week number of year, geographic location (e.g., country, region, DMA, etc.), and / or other similar information, although other features are also possible. In some implementations of the present invention, the categorical features may be one-hot encoded and the categorical features may be computed, converted, or otherwise mapped to a numerical representation for the feature vector. In block 215, the user response prediction module 118 may select a content presentation for display on the user's client device based on the feature vector. For example, the user response prediction module 118 may retrieve the content presentation from the user response data database 122. In block 220, the model development module 116 may use the predictive model and the feature vector to generate a predicted response of the user to the content presentation. In some implementations of the present invention, the predictive model may be a Bayesian logistic regression model or equivalent. In block 225, the user response prediction module 118 may determine whether the predicted response satisfies (e.g., exceeds or equals) a suitable (predetermined or adjustable) threshold.If the predicted response does not meet (e.g., is below) the (predetermined or adjustable) threshold, the user response prediction module 118 may return to block 215 and select an alternative content presentation. In some implementations of the present invention, the method 200 may end at block 225 (i.e., not return to block 215 or proceed to block 230) if the predicted response does not meet (e.g., is below) the threshold. If the predicted response meets (e.g., is at or above) the threshold, in block 230, the user response prediction module 118 may generate a content presentation impression value using the predicted response and a target value associated with the user. In block 235, the response determination module 120 may determine a response value for the request based on the content presentation impression value and the feature vector. At block 240, the response determination module 120 may transmit the response value and the selected content offer in response to the request (e.g., over the network 110 to the content offer exchange 126). The content offer may be displayed on the user's client device once the response is accepted (e.g., accepted by the content offer exchange 126 as part of an auction or auction-like process).

[0032] FIG. 3 is a block diagram illustrating another exemplary method 300 for predicting a user response to content presentation on a client device. In an embodiment, at block 302, the user response prediction module 118 may receive a request to display information on the user's client device (e.g., a bid request or equivalent from the content presentation exchange 126 over the network 110 as part of an auction or auction-like process). As mentioned above, the information output or displayed on the user's client device may include digital advertisements, creative assets, and the like, which may include text, images, video, audio, and / or the like. Furthermore, such information may be presented outside of or within a software application executing on the user's client device. In an embodiment, at block 304, the user response prediction module 118 may extract a number of features associated with the request. These features may include at least one feature that characterizes the user and at least one additional feature that characterizes the client device. As noted above, the features may include information such as data characterizing a publisher, location of placement of a content presentation on a screen of a client device, data associated with a user, such as age, gender, etc. Additionally, the features may also include a geographic location specific to the user and / or client device. At least one feature characterizing a user may correspond to information such as age, gender, etc., and at least one feature characterizing a client device may include a device identifier or other comparable information characterizing a client device.

[0033] In an embodiment, in block 306, the user response prediction module 118 may generate a feature vector. The feature vector may correspond to a list of a particular size, for example, including 5, 10, or 15 features, although any suitable number of features is possible. As discussed above, the feature vector may be a numerical representation of a combination of user attributes and / or other attributes. For example, the feature vector may include multiple numbers and be of any suitable length depending on the desired number of features to be captured in the feature vector. In some implementations of the invention, one-hot encoding may be used for all or a portion of the feature vector to compute a numerical representation of each or any feature in the set of features. In an embodiment, one or more of the multiple features described above with respect to block 304 may be stored as part of the feature vector generated in block 306.

[0034] In an embodiment, at block 308, the model development module 116 may generate a predicted response value of the user to the content presentation, particularly, i.e., a content presentation that may be displayed on the user's client device. The predicted response value may be determined using the predictive model and the feature vector generated at block 306. In an embodiment, the predictive model may be a Bayesian logistic regression model or equivalent. In an embodiment, the predictive model may be constructed and trained by the model development module 116 using, for example, a mini-batch limited memory Broyden-Fletcher-Goldfarb-Shanno regression (L-BFGS) or equivalent. Alternatively, in an embodiment, the predictive model may include a ridge, lasso, elastic net, adaptive lasso, group lasso, randomized lasso, and / or other suitable regression model. Additionally or alternatively, the predictive model may be or include a classifier such as, for example, one or more linear classifiers (e.g., Fisher's Linear Discriminant, logistic regression, a naive Bayes classifier, and / or a perceptron), a support vector machine (e.g., a least squares support vector machine), a quadratic classifier, a kernel estimation model (e.g., k-nearest neighbors), a boosting (meta-algorithm) model, a decision tree (e.g., random forest, gradient boosting trees), a neural network, and / or a learning vector quantization model.

[0035] In an embodiment, at block 310, a content presentation impression value may be generated using the predicted response value and a target value associated with the user. It is noted that the predicted response value may characterize the likelihood of the user interacting with the content presentation, and the target value may characterize the cost of the user's interaction with the content presentation. It is further noted that the content presentation impression value may be generated when the predicted response value satisfies a pre-determined or adjustable threshold. For example, the predicted response value may be represented in the form of a probability value ranging from 0 to 1. It is noted that other thresholds and value ranges may also be utilized. Furthermore, the target value may represent a cost of the user's interaction with the content presentation, which is displayed on the user's client device. As noted above, the target value may correspond to a price associated with a particular instance of engagement between the user and the content presentation. It is noted that other metrics may also be utilized.

[0036] In an embodiment, at block 312, the response determination module 120 may determine a response value (i.e., a requested response value) based on the content presentation impression value and the feature vector. Note that the response value (i.e., the requested response value) may correspond to a bid amount or bid price for an opportunity to display the content presentation on the user's client device. As discussed above, such a requested response value may be calculated using equation (10), which is repeated below: [ka]

[0037] Finally, in block 314, the response determination module 120 may operate to transmit the request response value and the content presentation for display on the user's client device. For example, the content presentation may be displayed on the user's client device once the request response value is approved (e.g., approved by the content presentation exchange 126 as part of an auction or auction-like process). In an embodiment, the transmission may be implemented wirelessly or via a wired connection.

[0038] FIG. 4 is a block diagram of an exemplary computing device 400 that may perform one or more of the operations described herein according to the present embodiment. The computing device 400 may be connected to other computing devices in a LAN, an intranet, an extranet, and / or the Internet. The computing device 400 may operate in the capacity of a server machine in a client-server network environment or in the capacity of a client in a peer-to-peer network environment. The computing device 400 may be provided by a personal computer (PC), a set-top box (STB), a server, a network router, a switch or bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that define actions to be taken by the machine. Furthermore, although only a single computing device 400 is illustrated, the term "computing device" shall also be taken to include any collection of computing devices that individually or together execute a set (or sets) of instructions to perform the methods discussed herein.

[0039] The exemplary computing device 400 may include a computer processing device 402 (e.g., a general purpose processor, an ASIC, etc.), a main memory 404, a static memory 406 (e.g., a flash memory or equivalent), and a data storage device 408, which may communicate with each other via a bus 430. The computer processing device 402 may be provided by one or more general purpose processing devices, such as a microprocessor, a central processing unit, or equivalent. In an illustrative embodiment, the computer processing device 402 may comprise a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or a processor implementing a combination of instruction sets. The computer processing device 402 may also comprise one or more special purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or equivalent. The computing device 402 may be configured to execute the operations described herein in accordance with one or more aspects of the present disclosure to perform the operations and steps discussed herein.

[0040] The computing device 400 may further include a network interface device 412, which may be in communication with a network 414. The data storage device 408 may include a machine-readable storage medium 428 on which one or more sets of instructions may be stored, such as instructions for performing operations described herein according to one or more aspects of the disclosure. The instructions 418 implementing the core logic instructions 426 may also reside, completely or at least partially, within the main memory 404 and / or the computer processing device 402 during execution thereof, with the computing device 400, main memory 404, and computer processing device 402 also constituting computer-readable media. Instructions may further be transmitted or received via the network interface device 412 over the network 414.

[0041] Although the machine-readable storage medium 428 is shown in the illustrative embodiment as being a single medium, the term "computer-readable storage medium" should be taken to include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more sets of instructions. The term "computer-readable storage medium" should also be taken to include any medium capable of storing, encoding, or carrying a set of instructions for execution by a machine, causing the machine to perform the methods described herein. The term "computer-readable storage medium" should thus be taken to include, but is not limited to, solid-state memory, optical media, magnetic media, and the like.

[0042] The subject matter and embodiments of the operations described in this disclosure can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this disclosure and their structural equivalents, or in a combination of one or more of them. The subject matter described in this disclosure can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by or to control the operation of a data processing device. Alternatively, or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Also, a computer storage medium is not a propagating signal, although a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium can also be, or be contained in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0043] The operations described in this disclosure may be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0044] The term "data processing apparatus" encompasses any kind of apparatus, device, and machine for processing data, including, as examples, a programmable processor, a computer processing device, a computer, a system on a chip, or a plurality or combination of the above. A computer processing device can include one or more processors, which may include special purpose logic circuitry, e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), a central processing unit (CPU), a multi-core processor, etc. An apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of these. The apparatus and execution environment can implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0045] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative, procedural, or functional languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored within part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language resource), within a single file dedicated to the program, or within multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program can be deployed to run on one computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0046] The processes and logic flows described in this disclosure may be implemented by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows may also be implemented by, and apparatus may also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0047] Processors suitable for executing computer programs include, by way of example, both general-purpose and special-purpose microprocessors, as well as any one or more processors of any kind of digital computer. In general, the processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing actions according to instructions, and one or more memory devices for storing instructions and data. In general, the computer will also include, or be operatively coupled to receive data from, or transfer data to, one or more mass storage devices, such as magnetic disks, magneto-optical disks, optical disks, solid-state devices, or the like, for storing data. However, a computer need not have such devices. A computer can also be embedded in another device, such as a smartphone, a mobile audio or media player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0048] To provide for interaction with a user, embodiments of the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, light emitting diode (LED) monitor, or equivalent, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse, trackball, touchpad, stylus, or equivalent, by which the user may provide input to the computer. Other types of devices can be used to provide interaction with the user as well, e.g., feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive trackpads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and equivalents. In addition, the computer can interact with a user by sending resources to and receiving resources from devices used by the user, for example, by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0049] An embodiment of the subject matter described in this disclosure can be implemented in a computing system that includes a back-end component, e.g., as a data server, or includes a middleware component, e.g., an application server, or includes a front-end component, e.g., a client computer having a graphical user interface or web browser through which a user may interact with an implementation of the subject matter disclosed herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), peer-to-peer networks (e.g., ad-hoc peer-to-peer networks), and the like.

[0050] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., HTML pages) to a client device (e.g., for purposes of displaying data to a user interacting with the client device and receiving user input from the user). Data generated at the client device (e.g., results of user interaction) can be received from the client device at the server.

[0051] One or more computer systems may be configured to perform particular operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that, when in operation, causes the system to perform the actions. One or more computer programs may be configured to perform particular operations or actions by containing instructions that, when executed by a data processing device, cause the device to perform the actions.

[0052] Throughout this disclosure, references to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic is described in connection with an embodiment that is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this disclosure are not necessarily all referring to the same embodiment. In addition, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or."

[0053] Although the present disclosure contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features specific to certain embodiments of a particular invention. Certain features described in the present disclosure in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Also, although features may be described above as acting in a combination and may be initially claimed as such, one or more features from the claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0054] Similarly, although operations and / or logic flows are depicted in the figures and / or described herein in a particular order, this should not be understood as requiring such operations and / or logic flows to be performed in the particular order shown, or in a sequential order, or that all of the illustrated operations be performed, to achieve desirable results. In some situations, multitasking and parallel processing may be advantageous. Also, it should be understood that the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and that the program components and systems described may generally be integrated together in a single software product or packaged into multiple software products.

[0055] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

[0056] The word "embodiment" or "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "embodiment" or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word "embodiment" or "exemplary" is intended to present concepts in a specific manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X includes A or B" is intended to mean any of the natural inclusive permutations. That is, if X includes A, X includes B, or X includes both A and B, then "X includes A or B" is satisfied under any of the foregoing cases. In addition, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more," unless otherwise specified or clear from the context to be directed to the singular. Furthermore, use of the terms "in an embodiment" or "one embodiment" or "an implementation" or "one implementation" throughout is not intended to refer to the same embodiment or implementation unless so described. Furthermore, the terms "first," "second," "third," "fourth," etc. as used herein are meant as labels to distinguish between different elements and do not necessarily have an ordinal meaning according to their numerical designations.

[0057] In the above description and claims, phrases such as "at least one of" or "one or more of" may occur followed by a conjunctive list of elements or features. The term "and / or" may also occur within a list of two or more elements or features. Unless otherwise contradicted by the context in which it is used, implicitly or explicitly, such a phrase is intended to mean any of the listed elements or features individually or in combination with any of the other listed elements or features. For example, the phrases "at least one of A and B," "one or more of A and B," and "A and / or B" are intended to mean "A only, B only, or A and B together," respectively. A similar interpretation is also intended with respect to lists containing three or more items. For example, the phrases "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, and / or C" are intended to mean "A only, B only, C only, A and B together, A and C together, B and C together, or A and B and C together." Additionally, use of the term "based on" above and in the claims is intended to mean "based at least in part on," such that unrecited features or elements are also permissible.

[0058] The above description of the illustrated implementations of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed. Although specific implementations of the invention and examples thereof are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as one of ordinary skill in the art would recognize. The subject matter described herein can be embodied in systems, devices, methods, and / or articles, depending on the desired configuration. The implementations described in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. Although some variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those described herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of some additional features disclosed above. Other implementations may also be within the scope of the following claims.

Claims

1. 1. A method comprising: receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features including at least one feature characterizing the user and at least an additional feature characterizing the client device; generating a feature vector based on the plurality of features associated with the request; and generating a predicted response value of the user to a content presentation using a predictive model and the feature vector, the predicted response value characterizing a likelihood of the user interacting with the content presentation; generating a content presentation impression value using the predicted response value when the predicted response value meets a predetermined threshold and a target value associated with the user, the target value characterizing a cost of interaction of the user with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; transmitting the request response value and the content representation for display on the user's client device; and A method comprising:

2. The method of claim 1 , wherein the predictive model is a Bayesian logistic regression model.

3. The method of claim 1 , wherein the predicted response value further characterizes a likelihood of a downstream conversion of the user in response to interacting with the content presentation.

4. The method of claim 1 , wherein generating the content presentation impression value using the predicted response value and a target value associated with the user comprises multiplying the target value and the predicted response value.

5. The method of claim 1 , further comprising selecting the content representation for display on the user's client device based on the feature vector.

6. 2. The method of claim 1, further comprising: selecting an alternative content presentation for display on the client device in response to determining that the user's predicted response value to the content presentation does not satisfy the predetermined threshold.

7. The method of claim 6 , further comprising using the predictive model and the feature vector to generate an additional predicted response value of the user to the alternative content presentation.

8. 8. The method of claim 7, further comprising generating an additional content presentation impression value using the additional predicted response value and a target value associated with the user when the additional predicted response value satisfies the predetermined threshold.

9. The method of claim 1 , further comprising implementing a filter model associated with the predictive model.

10. 10. The method of claim 9, wherein implementing the filter model includes determining an additional likelihood that the user will convert within a particular category associated with one or more additional content offers.

11. 1. A system comprising: at least one data processor; A memory storing instructions that, when executed by the at least one data processor, cause the at least one data processor to: receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features including at least one feature characterizing the user and at least an additional feature characterizing the client device; generating a feature vector based on the plurality of features associated with the request; and generating a predicted response value of the user to a content presentation using a predictive model and the feature vector, the predicted response value characterizing a likelihood of the user interacting with the content presentation; generating a content presentation impression value using the predicted response value when the predicted response value meets a predetermined threshold and a target value associated with the user, the target value characterizing a cost of interaction of the user with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; transmitting the request response value and the content representation for display on the user's client device; and A memory and A system comprising:

12. The system of claim 11 , wherein the predictive model is a Bayesian logistic regression model.

13. The system of claim 11 , wherein the predicted response value further characterizes a likelihood of a downstream conversion of the user in response to interacting with the content presentation.

14. 12. The system of claim 11, wherein the at least one data processor performs an operation of generating the content presentation impression value using the predicted response value and a target value associated with the user by multiplying the target value and the predicted response value.

15. The operation further comprises: The system of claim 11 , further comprising selecting the content representation for display on the user's client device based on the feature vector.

16. The operation further comprises:

12. The system of claim 11, further comprising: selecting an alternative content presentation for display on the client device in response to determining that the user's predicted response value to the content presentation does not satisfy the predetermined threshold.

17. The operation further comprises:

20. The system of claim 16, further comprising using the predictive model and the feature vector to generate an additional predicted response value of the user to the alternative content presentation.

18. The operation further comprises: The system of claim 11 , comprising implementing a filter model associated with the predictive model.

19. 20. The system of claim 18, wherein the act of implementing the filter model includes determining an additional likelihood that the user will convert within a particular category associated with one or more additional content offers.

20. 1. A non-transitory computer program product, the non-transitory computer program product storing executable instructions which, when executed by at least one data processor forming part of at least one computing system, receiving a request to display information on a user's client device; extracting a plurality of features associated with the request, the plurality of features including at least one feature characterizing the user and at least an additional feature characterizing the client device; generating a feature vector based on the plurality of features associated with the request; and generating a predicted response value of the user to a content presentation using a predictive model and the feature vector, the predicted response value characterizing a likelihood of the user interacting with the content presentation; generating a content presentation impression value using the predicted response value when the predicted response value meets a predetermined threshold and a target value associated with the user, the target value characterizing a cost of interaction of the user with the content presentation; determining a request response value based on the content presentation impression value and the feature vector; transmitting the request response value and the content representation for display on the user's client device; and 2. A non-transitory computer program product performing operations including:

Images