COMPUTER SYSTEM FOR PROVIDING BLACK BOX COMPUTER SYSTEM INPUTS

Abstract

A computer system for determining an input to a black-box information processing system in an environment is provided, comprising a first memory, a second memory, a model creation module, and an input determination module. The black-box information processing system produces one or more non-deterministic observables based on the input, which remains the same over a period of time. The first memory stores historical data, and the second memory stores an environmental model created by the model creation module and modeling one or more stochastic processes of the one or more observables based on the historical data.The input determination module applies the environmental model to derive probability functions from an optimization criterion and one or more optimization constraints and calculates a conditional probability density to determine an optimized input according to the conditional probability.

Description

Title of the invention: COMPUTER SYSTEM FOR PROVIDING INPUTS TO THE BLACK BOX COMPUTER SYSTEM DOMAIN

[0001] The present disclosure relates to a specific architecture and implementation of a computer system for determining an input to a black box information processing system. CONTEXT

[0002] The technical world is nowadays widely distributed. Companies offer computing services to requesters, such as other companies or individuals. The computing services require inputs in specific forms and provide outputs to the requesters. However, the requesters do not know how the underlying information processing systems, which are used to provide the computing services, determine the outputs based on the inputs, i.e., the systems providing the computing services are the so-called black-box information processing systems for the requesters.

[0003] Applicants may need to optimize their inputs provided to the black-box information processing system so that the output meets certain requirements. This task becomes more difficult if the black-box information processing system does not provide deterministic outputs but faces some sort of randomness. Therefore, applicants need improved computer systems that are capable of determining the input to interact with these non-deterministic black-box information processing systems. SUMMARY

[0004] In this context, a computer system for determining an input to a black-box information processing system in an environment is provided. The black-box information processing system produces one or more non-deterministic observables for each execution based on the input, wherein the input to the black-box information processing system remains the same over a period of time, wherein said one or more observables relate to aggregated values over a given period of time for a given input.

[0005] The computer system comprises a first memory storing historical data related to entry into historical time periods and said one or more observables and a second memory storing an environmental model, in which the environmental model estimates said one or more observables based on an input.

[0006] The computer system further comprises a model creation module connected to the first and second memories, wherein the model creation module is configured to receive, from the first memory, the historical data, create the environmental model by modeling one or more stochastic processes of said one or more observables based on the historical data, and store the environmental model in the second memory.

[0007] The computer system further comprises an input determination module connected to a second memory, wherein the input determination module is configured to receive, from a second memory, the environmental model, receive, via a communication interface, an optimization criterion linked to one or more of said one or more observables, receive, via the communication interface, one or more optimization constraints, apply the environmental model to derive probability functions of the optimization criterion and said one or more input-dependent optimization constraints, calculate a conditional probability density according to which a given input reaches the highest or lowest value for the optimization criterion, given that said one or more optimization constraints is / are based on the probability functions,and determine an optimized input according to the conditional probability. ,

[0008] In some embodiments, the system further comprises a communication module configured to transmit the determined optimized input to the black-box information processing system for application during the next time period. In additional embodiments, the communication module is further configured to obtain the one or more observables from the black-box information processing system for the next time period.

[0009] In some embodiments, modeling one or more stochastic processes includes training one or more probability density regression neural networks based on one or more features specific to the entities in the environment and historical data. In additional embodiments, the one or more probability density regression neural networks are trained on the same inputs, wherein the last two layers vary according to the respective observable. In still further embodiments, the one or more probability density regression neural networks are continuously updated with one or more observables received from the black-box information processing system for the time periods ul In further embodiments, updating of said one or more probability density regression neural networks is performed periodically at time periods defined and / or triggered by the volume of data related to said one or more observables received from the black-box information processing system for subsequent time periods after the last update of said one or more probability density regression neural networks.

[0010] In some embodiments, the time period corresponds to a day, a week, a month, or a year. In other embodiments, the black-box information processing system is located remotely from at least the input determination model. In other embodiments, the optimization criterion and / or one or more optimization constraints is / are entered by a user at a user device and transmitted via the communication interface to the input determination module. In other embodiments, the optimization constraints relate to one or more observables.

[0011] In some embodiments, the one or more stochastic processes of the one or more observables are based on stochastic distributions of one or combinations of a Poisson, Gamma, Binomial, Beta, log normal distribution. In other embodiments, the optimized input is determined according to an expected value of the conditional probability density. In other embodiments, the first memory, the second memory, the model building module, and the input determining module are implemented in a distributed manner.

[0012] In other embodiments, the black-box information processing system is an online advertising auction system, in a market environment, in which the input is a bid, wherein the aggregated observables over a period of time include one or more successful bids, a number of bids, a number of wins, a number of clicks, a number of conversions, a bid placement hit, and a revenue achieved by the bids. Brief Description of the Drawings

[0013] The foregoing and following objects, features and advantages of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings, in which like numerals are used to represent like elements, wherein:

[0014] [Fig.l] is a schematic overview of the computer system for determining an input in a black box information processing system.

[0015] [Fig.2] is another schematic overview of the computer system according to modes of realization.

[0016] [Fig.3] shows the interaction of the computer system with the black box information processing system and the client devices according to embodiments.

[0017] [Fig.4] is an example of a representation of a neural network implemented in certain embodiments.

[0018] [Fig.5] shows the possible density distributions of the observables according to embodiments.

[0019] [Fig.6] shows graphs related to a dependency of an optimization criterion and an optimization constraint with respect to the offer according to embodiments.

[0020] [Fig.7] shows an example of probability calculation and conditional probability tional according to embodiments.

[0021] [Fig. 8] is a schematic representation of a computer system, which may implement some or all of the functionalities described herein. DETAILED DESCRIPTION

[0022] The present invention relates to a computer system for determining an input to a black-box information processing system. The computer system described herein may be implemented by a single computer, server, or the like that includes all of the components required to perform the methods. Alternatively, the computer system may be distributed among multiple processing systems, such as computers, servers, and the like, located in a geographic area. Additionally or alternatively, some components of the computer system described herein may be located remotely from one another, e.g., in the cloud. Therefore, different configurations of the computer system are conceivable as long as all of the components described herein are present and connected to one another as required.

[0023] Black-box information processing systems are systems that can be viewed in terms of input and output (or transfer characteristics), without knowing their internal workings. The implementation is "opaque" (black). A black-box information processing system often refers to equipment provided by a vendor or service provider for the purpose of using the vendor's product or the service provider's service. Often, the vendor or service provider keeps and maintains the equipment for processing (i.e., the black-box information processing system), and the requesting company or requesting department using the black box typically has no knowledge of the internal workings at this time.

[0024] A ubiquitous example of the use of a black-box information processing system concerns online advertising auction marketplaces, such as, e.g., provided by metasearch platforms. Companies acting as demanders within the meaning of the present disclosure transmit a bid in order to be displayed as the first search results on these auction-identified marketplaces. The bid represents an input to the black-box information processing system, i.e., the provided auction system. The companies do not know how this auction system works. Furthermore, winning with a bid is not deterministic and can be seen as a stochastic process.

[0025] Therefore, auctions are usually conducted by third parties, if the companies submitting bids do not have control at the event level. Some bidding strategies and targeting options to segment bids are proposed, but the companies do not know the inner workings. How precisely the auctions are conducted and the factors that affect the probability of winning a given auction are not known. Such platforms that provide online auction marketplaces are therefore a non-limiting example of a black-box information processing system, which only allows interaction by changing bids and periodically observing the aggregated number of wins, clicks and bookings (in batches).

[0026] The challenge with metasearch platforms is to choose the right offers per marketed good or, in general, the item such as hotels, flights, restaurants, events, tourism services, and the like. The objective may be, e.g., to maximize revenue, while being limited by business metrics, such as keeping Return on Advertising Spend (ROAS) above a certain desired threshold. This challenge arises - of course - for every black-box information processing system in an environment that takes one or more inputs and provides non-deterministic observables. The objective may vary, e.g., because it may be required to maximize the electricity supplied when dealing with a renewable energy grid information processing system when keeping electricity production in wind or solar farms at a safe level.

[0027] To overcome this challenge and optimize the input provided in the black-box information processing system, a specific architecture and implementation of a computer system based on a machine learning solution is presented herein, which determines the optimal input, e.g., the optimal bid, subject to the optimization criteria, e.g., business constraints for batch-based third-party auctions. The presented solution is directly applicable to metasearch platforms, but is not limited to them.

[0028] [Fig.l] is a schematic overview of the computer system 10 for determining an input to a black-box information processing system. The black-box information processing system produces one or more non-deterministic observables for each execution of the black-box information processing system based on the input, which generally remains the same over a period of time. In some examples, a period of time relates to one or more hours, one or more days, one or more weeks, one or more months, or one or more years, depending on the underlying black-box information processing system. The update frequency is not fixed but is related to the domain and the data. The one or more observables relate to aggregated values over a given period of time for a given input.

[0029] For example, the input may be the offer on a metasearch platform in a marketplace environment, in which entities such as hotels (or flights, events, or the like) may be presented to consumers. The offer may remain the same for a day, e.g., for all searches performed during the day, the offer to be presented to customers for a specific hotel will remain the same. The one or more observables in this example may then relate to one or more successful bids, a number of auctions, a number of wins, a number of clicks, a number of conversions, a bid ranking hit, and a revenue achieved by the bids.

[0030] The computer system 10 described herein comprises a first memory 11, a second memory 12, a model creation module 13 and an input determination module 14. The first memory 11 stores historical data related to the different inputs during historical time periods and one or more observables in these historical time periods.

[0031] In the example of a metasearch auction, the historical input data and observables may take the format as shown in Table 1. [Tables 1] Week Bid Auction Earnings Clicks Conversions Costs Revenue 1 0.8% 1000 900 60 2 50 800 2 1.1% 900 700 70 5 80 2 000

[0032] In Table 1, the first column defines the calendar week in a year, the second column indicates the offer as a percentage of the cost of the desired stay. The third column shows the number of auctions carried out with the bid during the respective calendar week. The number of wins from the total auctions is shown in the fourth column, and the number of consumer clicks is shown in the fifth column. Clicks mean that the consumer clicked through to the respective hotel website displayed on the metasearch website or app user interface. The sixth column then shows the number of conversions, i.e., the number of consumers who actually booked the stay at the respective hotel. The last two columns then represent the bid costs and the revenue achieved through conversions.

[0033] In the metasearch example, if a bid is among the highest bids, an auction is won and the hotel appears at the very top of the search results. The precise position and order are determined by the marketplace's own optimization, but unknown to the bidder. The online user may click on the advertised hotel from the list. If this happens, the absolute bid is paid to the marketplace, which incurs costs. If no click is made, then no costs will be associated with the auction. This action is known as Cost-Per-Click (CPC) bidding. For example, a customer may search for a three-night stay at a hotel in Berlin. The hotel on which a bid should be placed may cost €300 for three nights, meaning that a 1% bid would be worth €3.If there is a click, the online user is taken to the hotel's own website where they may or may not make a reservation. If the reservation occurs, a conversion event is recorded and the revenue corresponds to the value of the reservation. Typically, the online user may choose to book a longer stay or an upgraded room, in which case the revenue is higher than the searched value used to determine the cost due for a click. For each time period until the bid changes, a number of these auctions take place and at the end of the period the aggregated result is observed, as shown in Table 1. Costs and revenue are summed over all conversion click events, respectively, while all other fields are counts.

[0034] The model creation module 13 is connected to the first memory 11 and the second memory 12. The model creation module 13 may be connected to the first memory 11 and the second memory 12 by means of a bus or other wired connection, if the model creation module 13 and the memories 11, 12 are implemented in a single computer or server. Since at least the first memory 11 may be needed to store more data, it may be advantageous in some embodiments to have at least the first memory 11 separate from the model creation module 13 and / or the second memory 12. For example, the first memory 11 may be cloud storage. In such examples, in which the model creation module 13, the first memory 11, and / or the second memory 12 are located remotely and implemented by different servers (or similar processing systems), components 11, 12 and / or 13 may be connected via a wide area network, a global network, the Internet, or a similar network, which may be a public or private network, and may include multiple interconnected networks, as is known in the art.

[0035] The model creation module 13 is configured to create an environmental model by modeling one or more stochastic processes of said one or more observables based on the historical data stored in the first memory 11. Accordingly, the model creation module 13 obtains historical data from the first memory 11, as indicated by the arrow 101. The model creation module 13 creates the environmental model and stores the model in a second memory 12, as indicated by the arrow 102.

[0036] The environmental model may be any type of model suitable for modeling said one or more stochastic processes of one or more observables. The purpose of the environmental model is to evaluate as accurately as possible what would be observed if a constant input were applied over a period of time. Therefore, the environmental model may model all stochastic processes that define the outcome, i.e., the observables. In one embodiment, the environmental model is a set of one or more neural networks, e.g., one or more probability density regression neural networks.

[0037] Typically, stochastic processes are probability distributions that describe the probability of the occurrence of distinct events. Probability distributions can be of different types and can be one or a combination of Poisson, binomial, geometric, Beta, Boltzmann, Gamma, normal, logarithmic, Zeta, or any other suitable distribution. The distribution type of the observables can be determined in a preprocessing step, e.g., by an administrator estimating the distribution type, the respective distribution parameters will then be determined as described below. Alternatively, a machine learning model can also estimate the distribution type and the same or another machine learning model can estimate the distribution parameters.

[0038] The distributions of observables can be modeled as random variables described by latent random variables to be evaluated. The latent variables to be evaluated are therefore the parameters that govern the shape of these distributions. In general, the entities for which inputs are provided, e.g., hotels or events, or wind or solar farms, in the environment share certain characteristics, which are denoted by z herein. By denoting the input by x and assuming that there are n independent random variables (observables) to be evaluated, the environmental model ronmental determines the functions: [Math 1] y Yn~D {Z- ( X, Z )} j If yn is one of the independent observables n which is distributed according to the distribution D defined by certain parameters dependent on the observable j which are the latent random variables of interest.

[0039] Learning functions on high-dimensional feature spaces can be achieved with machine learning, such as neural networks. The use of features allows the model to learn signals that are invisible in the captured data. The stochastic nature of the processes to be learned may suggest Bayesian modeling via variational architectures. However, when the volume of input data is large, learning the latent variables directly can be challenging. The use of probability density regression neural networks may therefore be advantageous in some embodiments.

[0040] In the metasearch example, the metasearch marketplace can be modeled with CPC bids via 6 independent random variables represented by six or more latent variables to be learned, as presented in Table 2. For brevity, the generic feature dependency is omitted but only the latent variables that depend on the bid are highlighted. All latent variables are denoted by small Greek letters while observables are denoted by capital letters of the Roman alphabet. [Tables 2] Latent Variable Bid Dept. Observable Description Observable Distribution Auction Rate T no Estimated Auction per Time Period T~Poisson(r) with r-Gamma Win Rate v(.x) yes Estimated Wins per Time Period for Bid x W~Binom(T, v(x)) with v{x)~Beta{a[x), 0(x}), constrained so that v(0) =0, limv(x) = 1, Click Probability f । no Estimated clicks per time period C^Binom^W, y^ with y ~Beta{a}, Conversion probability P2 no Estimated conversions per time period C2~Binom[ W, y,,) with y2~Beta{a2> 0^) relative to the average value sought V no Estimated average costs per click per time period V-LogNormal ( &v ) relative to arbitrage A no Estimated average relative difference between the value sought and the reserved value A~LogNormal ( cta )

[0041] For brevity, only the explicit supply dependency is shown in Table 2. All quantities also implicitly depend on the features z, which for hotels may consist of categorical features, such as location, star rating, number of rooms, and the like. To build the environmental model using neural networks, six neural networks all taking the same inputs (e.g., x and z) would be needed, and for which, e.g., the last two layers vary depending on the latent variable to be evaluated. The penultimate layer can then produce the latent variable and the last layer is the distribution layer corresponding to the distribution of the observable cor- respondent.

[0042] The input determination module 14 is connected to the second memory 12, e.g., by means of a bus or other cable connection if the input determination module 14 and the second memory 12 are implemented in a single computer or server. It may be advantageous in some embodiments to have a second memory 12 separate from the input determination module 14. For example, the second memory 12 may be cloud storage. In such examples, in which the input determination module 14 and the second memory 12 are located remotely and implemented by different servers (or similar processing systems), the input determination module 14 and the second memory 12 may be connected via a wide area network, a global network, the Internet, or a similar network which may be a public network, a private network, and may include multiple interconnected networks, as is known in the art.

[0043] The input determination module 14 is configured to receive, from the second memory 12, the environmental model, which was created by the model creation module 13 as shown by the arrow 103. The input determination module 14 is further configured to receive, via a communication interface 15, an optimization criterion related to one or more of said one or more observables, as shown by the arrow 104. The communication interface may be part of the input determination module 14 or may be a separate component, which may or may not be part of the computer system 10. The communication interface 15 may be implemented in hardware and / or in software and may allow the computer system 10 to communicate with external devices, e.g., client devices used by consumers or administrators.

[0044] The input determination module 14 is also configured to receive, via the communication interface 15, one or more optimization constraints as indicated by the arrow 105. The optimization criterion, which may also consist of a plurality of optimization criteria and may also be referred to as an objective, and said one or more optimization constraints may be received from the same or different computing devices. In some embodiments, the optimization criterion and / or the optimization constraints may be input by a user in a user device and transmitted via the communication interface 15 to the input determination module 14. In some embodiments, the optimization criterion and / or the optimization constraints may relate to one or more of said one or more observables.In the research example, the optimization criterion and / or optimization constraints can be business indicators linked to the observables.

[0045] The input determination module 14 is also configured to apply the environmental model to derive the probability functions of the optimization criterion and the one or more input-dependent optimization constraints. The probability functions of the optimization criterion and the one or more optimization constraints are, therefore, functions of the learned latent variables and, usually, input-dependent (such as, e.g., only winning bids that will lead to clicks, etc.). In addition, the input determination module 14 is configured to calculate a conditional probability density according to which a given input achieves the highest or lowest value for the given optimization criterion, given that the one or more optimization constraints is / are based on the probability functions.

[0046] This can be illustrated by the example of metasearch. The optimization criterion and optimization constraints are referred to as business indicators M^x) which are functions of the learned latent variables. Since they depend on supply, the business indicators are obtained as functions of supply. The problem to be solved is introduced by limiting some business indicators (i.e., considering them to be optimization constraints), while requiring the maximization of others (i.e., considering these to be optimization criteria) e.g. For example, the question to be solved to determine the optimal input is [Math 2] maxM0 ( x ), given {M^x) > Æj for a value k. Since the Mt ( x ) are derived from random variables, they are also random variables.

[0047] The inventor recognized that calculating the conditional probability density that a given bid achieves the greatest objective value, given that the constraints are satisfied, results in a solution for determining the entry, which is distributed as follows: The inventor recognized that a solution to determine entry could be reached by calculating the conditional probability density that a given bid achieves the largest objective value. [Math 3] X~P(Mq(x) >Mfix^x / >k)

[0048] Therefore the objective is not maximized globally but conditionally, which takes into account the trade-offs arising from the optimization constraints. The offer can then be determined with respect to this resulting random variable. To solve the problem on average over several periods, one possible choice is to take the expected value: [Math 4] x* =

[0049] Alternatively, a greedy choice may be to take the offer corresponding to the highest probability, or if exploration is desired, the X sample similar to Thompson sampling.

[0050] Finally, the input determination module 14 is configured to determine an optimized input according to the calculated conditional probability.

[0051] This can be further illustrated by the example of metasearch. A typical problem in metasearch offerings is to try to maximize margin while keeping the return on advertising investment (ROAS) above a predetermined threshold. The motivation is to ensure good value for advertising budgets while trying to maximize activity. The non-trivial nature of the problem stems from the ROAS which is a ratio. A ROAS of 10 might be a good result considered in isolation, but when the ROAS is limited to spending only 1 euro and obtaining 10 euros of revenue, this is not sufficient. A ROAS of 10 with a revenue of 1,000 euros would certainly be better. Furthermore, a minimum win rate of 60% is desirable to bring visibility to the entity (e.g., a hotel) for which the offer is placed.

[0052] To derive margin and ROAS in terms of the environmental model, revenue due to bookings and costs due to clicks can be defined as follows [Math 5] Rev(x) = VxAxC2, C2~Binom(T, [Math6] Cost(x) =XXVxCt, v(x)y ^respectively, with the properties of a conditional binomial distribution being used to contract the latent variable dependence of clicks Q and conversions C2. Margin (^o) and ROAS (Af j) are then defined as their difference and ratio, respectively, with the minimum win rate (Af2) added as an additional business indicator: [Math 7] M0(x) -Rev(x)-Cost(x), Af j(x) = , M2(x)=tRj

[0053] [Fig. 2] is another schematic overview of a computer system 20 according to embodiments. The computer system 20 further comprises the first memory 11, the second memory 12, the model creation module 13, the input determination module 14, and a communication interface, which perform the same methods as those described in relation to [Fig. 1]. In this example, the computer system 20 further comprises a communication module 26 configured to transmit the determined optimized input to the processing system. the black box information for application during the next time period.

[0054] Accordingly, the input determination module 14 provides the determined optimized input to the communication module 26 as shown by the arrow 206. The communication module 26 may, in some embodiments, reformat or convert the information of the determined optimized input received from the input determination module 14 so that the black box information processing system can understand the information. For example, the black box information processing system may require a specific format in which the input is provided and the negation module will convert the determined optimized input into a specific format.

[0055] The communication module 26 further provides the determined optimized input, e.g., in some embodiments after converting the information, received from the input determination module 14, into a specific format in the black box information processing system as shown by arrow 207. In some embodiments (although not shown in [Fig. 2]), the communication module 26 may also use the communication interface 15 to provide the input to the black box information processing system.

[0056] [Fig. 3] shows the interaction of the computing system with the black box information processing system and the client devices according to embodiments. The computing system 10, 20 is connected to at least one client device 31 and to a black box information processing system 32. The computing system 10, 20 and at least one client device 31 and / or the black box information processing system may be connected by cable or wirelessly and / or may be connected via a wide area network, a global network, the Internet or a similar network, which may be a public or private network, and may include multiple interconnected networks, as is known in the art. Via the connection 301, the computing system 10, 20 may receive the optimization criterion and the one or more optimization constraints.Via connection 302, the computing system 10, 20 may provide the optimized input to the black box information processing system and may also, in some embodiments, obtain the one or more observables from the black box information processing system for the next period of time.

[0057] In some embodiments, the black box information processing system 32 is located remotely from the computer system 10, 20, in particular, from the input determination module 14 and / or from the communication module 26. In some embodiments, the black box information processing system 32 is an online advertising auction system in a market environment as used to illustrate the architecture and implementation above. Think for example, the input is a bid and the aggregated observables over a period of time include one or more successful bids, a number of auctions, a number of wins, a number of clicks, a number of conversions, a bid ranking cost, and a revenue achieved by the bids.

[0058] As mentioned above, the environmental model may be based on machine learning. The stochastic processes of the observables and latent variables may be modeled by training one or more probability density regression neural networks based on one or more features specific to the entities in the environment and historical data. [Fig. 4] is an example of a high-level representation of such a probability density neural network 40 implemented in some embodiments. The observables are random variables, while the latent variables defining the distribution shapes are scalars. Essentially, the neural network 40 better stimulates the stochastic process that generates the observable.

[0059] The neural network 40 is applied to a feature vector 41 which comprises the input x of the black-box information processing system 32 and the features z of a respective entity. The input layer 41 processes the input vector 4L. The neural network 40 may have several hidden layers 402, 403, 404, 405. It is noted that the number of layers is not limited to what is shown in [Fig. 4] but can be significantly larger. The same is true for the number of nodes in each layer. In the example of [Fig. 4], the neural network 40 has four hidden layers but can also have more or fewer. Some of these hidden layers may be the built-in layers for preprocessing the features z. The neural network 40 may be a network with typical regularizations whose depth depends on the amount of data to be processed.

[0060] The penultimate layer, in Fig. 4 designated by the number 406, may comprise i nodes for each of the latent variables of the distribution Dn to be determined. The last layer (not shown explicitly in Fig. 4) may be a distribution layer of type Dn, providing a distribution for the input feature vector {x, z}. When using probability density regression neural networks, the distributions are not determined impartially but are based on a modeling choice. Therefore, the distribution assumed for each observable is a model definition. The probability density regression neural networks determine the respective parameters, i.e. the latent variables, of the distribution of the input feature vectors.In alternative embodiments, true Bayesian neural networks can also be applied, in which the distribution types are not fixed but inferred. While such models in principle provide a more ap understanding. depth of the underlying modeled processes, they are harder to train and require more data. Therefore, probability density regression neural networks can be seen as a suitable choice in some embodiments.

[0061] In the non-limiting neural network example 40 of Fig. 4, Dn is a normal distribution and the latent variables include the mean (^) 42 and the standard deviation (a) 43. A negative log-likelihood of the observations with respect to Dn may be, e.g., used as the loss function to train the neural network 40. To construct the environmental model in the meta-search example, six neural networks that all take the same inputs, but for which the last two layers vary depending on the latent variable to be evaluated, may be applied as described above. The penultimate layer outputs the latent variable and the last layer is the distribution layer corresponding to the distribution of the corresponding observable.

[0062] In some embodiments, the neural network 40 (or any other suitable machine learning algorithm for predicting stochastic distributions of the observables) is trained before application but may also be continuously updated with new data received for the one or more observables from the black-box information processing system for a subsequent (i.e., next) period of time. By continuously updating or retraining the neural network, it can be ensured that the neural network 40 correctly models the stochastic processes underlying the environment. The updating of the neural network 40 may occur at predefined periods of time, such as once a day, a week, a month, or the like. Additionally or alternatively, the updating of the neural network 40 may be triggered by the amount of data collected.In the metasearch example, the update can be triggered based on the number of datasets (e.g., the number of auctions) received from the metasearch platform for different bids for different hotels. If the number exceeds a threshold, the update of the neural network 40 can be initiated. Alternatively, if the number of datasets is very small, it may not be worth updating the neural network since the observations are noisier and may lead to model degradation.

[0063] In the metasearch example, latent variables of the distributions for a property (hotel) in question are determined to predict the outcome(s) of auctions for that hotel over a range of bids. [Fig. 5] illustrates such learned distributions and presents possible density distributions of observables according to embodiments, e.g., according to the metasearch embodiment. In the top graph of [Fig. 5], the win rate is displayed with respect to the CPC in percentage. This is a clear bid-dependent distribution, as also highlighted in Table 2 above. The higher the percentage CPC, the higher the probability of winning. The lower graphs in [Fig.5] show different distributions with different expected values / variances. The distributions on the lower graph side will generally belong to observables that are not directly bid-dependent, e.g., bids, clicks, conversions, searched value, and arbitrage as presented in Table 2 above.

[0064] [Fig.6] shows graphs related to a dependency of an optimization criterion and an optimization constraint on the bid according to embodiments. As already mentioned above, a typical problem in metasearch bidding is to try to maximize the margin while keeping the ROAS above a certain predetermined threshold. In addition, a minimum win rate above a threshold is also desirable to bring visibility to the entity (e.g., a hotel) for which the bid is placed. [Fig.6] illustrates the dependency of the ROAS on the bid and the margin (the dependency of the win rate on the bid is illustrated in the left part of [Fig.5]).

[0065] Assuming that the margin is to be maximized while having a ROAS greater than 10. The upper graph of [Fig.6] shows the supply-dependent constraint (ROAS) but because ROAS is a random variable, it is not obvious what value is the optimal input to the black-box information processing system near where the expected value of ROAS crosses the threshold of 10. The lower graph of [Fig.6] illustrates the margin, which is (at a lower CPC almost always) higher, the higher the CPC.

[0066] Therefore, the conditional probability density of X is evaluated to find the probability distribution to maximize the margin while satisfying both trading conditions. [Fig.7] shows an example of probability and conditional probability calculation for such embodiments. The upper graph of [Fig.7] indicates that the likelihood of the ROAS constraint being satisfied gradually decreases (after its maximum) with increasing bid, as shown in line 71. The win rate requires a sufficiently large bid to be satisfied, as shown in line 72. The lower graph of [Fig.7] further shows that an unconstrained margin maximum, as shown in line 73, is outside the region where the ROAS constraint is satisfied, which can be seen by comparing lines 71 and 73.

[0067] The lower graph of [Fig.7] also shows that when the probabilities of ROAS and margin are combined by evaluating the conditional probability, as explained above, the optimal bid probability density can be easily found, which corresponds to line 74. The input determination module 14 can then choose as its next bid the expected value of this condition probability, which here is just above 0.002% of the CPC.

[0068] [Fig. 8] shows a schematic representation of internal components of a computer system 80 implementing the functionality of one or more components 11, 12, 13, and 14, as described herein. The computer system 80 includes at least a processor 81, a user interface 82, a network interface 83, and a main memory 86 that communicate with each other via a bus 85. Optionally, the computer system 80 may further include a static memory 87 and a hard disk drive (not shown) that also communicate with each other via the bus 85. A video display, an alphanumeric input device, and a cursor control device may be provided as examples of the user interface 82. Furthermore, the computer system 80 may also include one or more graphics processing units (GPUs) 84.

[0069] The GPUs 84 may also include a plurality of GPU cores or streaming multiprocessors, which include different novel components, such as at least one register, at least one cash and / or shared memory, and a plurality of ALUs, FPUs, a tensor processing unit (TPU) or tensor cores, and / or other optional processing units. The GPUs may perform multiple concurrent computations, thereby enabling distribution of training processes and acceleration of machine learning operations.

[0070] The main memory 86 may be random access memory (RAM) and / or any other volatile memory. The main memory 86 may store the program code 88a and may also store additional program data 88 required to provide the functionality described herein. In addition, the main memory 86 may also include a cache 89.

[0071] In one aspect, a computer program comprising instructions is provided. These instructions, when the program is executed by a computer, cause the computer to perform the methods described herein. The program code embodied in each of the systems described herein is capable of being individually or collectively distributed as a program product in various forms. In particular, the program code may be distributed using a computer-readable storage medium containing computer-readable program instructions to cause a processor to perform aspects of the embodiments described herein.

[0072] Computer-readable storage media that are inherently non-transitory may include removable and non-removable volatile or non-volatile tangible media implemented in any method or technology for storing information. formations, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media may further include random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disc read-only memory (CD-ROM) or other optical storage device, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and that can be read by a computer.

[0073] A computer-readable storage medium should not be construed as transient signals per se (e.g., radio waves or other electromagnetic waves propagating, electromagnetic waves propagating through a transmission medium such as a waveguide, or electrical signals transmitted through a cable). Computer-readable program instructions may be downloaded to a computer, other type of programmable data processing apparatus, or other device from a computer-readable storage medium or to an external computer or external storage device via a network.

[0074] It should be noted that although particular embodiments and variations have been described herein, any modifications and alternatives will be obvious to those skilled in the art. In particular, the examples are provided by way of illustration of the principles and to provide a number of specific methods and arrangements for implementing these principles.

[0075] In some embodiments, the functions and / or acts specified in the diagrams, sequence diagrams, and / or block diagrams may be reordered, processed serially, and / or processed simultaneously without departing from the scope of the present disclosure. In addition, the diagrams, sequence diagrams, and / or block diagrams may include more or fewer blocks than those illustrated in accordance with the embodiments of the disclosure.

[0076] The terminology used herein is for the sole purpose of describing particular embodiments and is not intended to limit the embodiments of the disclosure. It is further understood that the terms "includes" and / or "comprising," when used in this specification, specify the presence of stated features, integers, methods, operations, elements and / or components but do not preclude the presence or addition of one or more other features, integers, methods, operations, elements, components and / or groups thereof. Furthermore, to the extent that the terms "includes," "having," "has," "with," "composed of”, or variations thereof are used either in the detailed description or in the claims, such terms are intended to be inclusive in the same manner as the term “comprising”.

[0077] While a description of the various embodiments has illustrated the method and these embodiments have been described in great detail, applicants do not intend to restrict or limit in any way the scope of the appended claims to such details. Other advantages and modifications will be readily apparent to those skilled in the art. The disclosure in its broader aspects is therefore not limited to the specific details, representative apparatus and method, and illustrative examples shown and described. Accordingly, the disclosed embodiments should be understood as being provided by way of example, for the purpose of teaching general features and principles but should not be understood as limiting the scope which is as defined in the appended claims.

Claims

Claims

1. The computer system for determining an input to a black-box information processing system within an environment, wherein the black-box information processing system produces one or more non-deterministic observables for each execution based on the input, wherein the input to the black-box information processing system remains the same over a period of time, wherein said one or more observables relate to aggregated values over a given period of time for a given input, the system comprising: a. a first memory storing historical data related to the input over historical time periods and said one or more observables, b. a second memory storing an environmental model, wherein the environmental model evaluates said one or more observables based on an input; c. a model creation module connected to the first and second memories, wherein the model creation module is configured to: i. receive, from the first memory, the historical data; ii. create the environmental model by modeling one or more stochastic processes of said one or more observables based on the historical data; and iii. storing the environmental model in the second memory; and d. an input determination module connected to the second memory, wherein the input determination module is configured to: i. receive, from the second memory, the environmental model; ii. receive, via a communication interface, an optimization criterion linked to one or more of said one or more observables; iii. receive, via the communication interface, one or more optimization constraints; iv. applying the environmental model to derive probability functions of the optimization criterion and the one or more input-dependent optimization constraints; v. calculating a conditional probability density according to which a given input achieves the highest or lowest value for the optimization criterion, given that the one or more optimization constraints is / are based on the probability functions; and vi. determining an optimized input according to the conditional probability.

2. The system of claim 1 further comprises a communication module configured to transmit the determined optimized input to the black box information processing system for application purposes during the following time period.

3. The system of claim 2, wherein the communication module is further configured to obtain the one or more observables from the black box information processing system for the following time period.

4. The system of any preceding claim, wherein modeling one or more stochastic processes comprises training one or more probability density regression neural networks based on one or more features specific to entities in the environment and historical data.

5. The system of claim 4, wherein said one or more probability density regression neural networks are trained on the same inputs, wherein the last two layers vary according to the respective observable.

6. The system of claim 5, wherein said one or more probability density regression neural networks are continuously updated with one or more observables received from the black box information processing system for subsequent time periods.

7. The system of claim 6, wherein updating said one or more probability density regression neural networks is executed periodically at periods of time defined and / or triggered by the amount of data related to said one or more observables received from the black-box information processing system for subsequent periods of time after the last update of said one or more probability density regression neural networks

8. The system of any preceding claim, wherein the time period is a day, a week, a month or a year.

9. The system of any preceding claim, wherein the black box information processing system is located remotely from the input determination model at least.

10. The system according to any one of the preceding claims, wherein the optimization criterion and / or one or more optimization constraints is / are entered by a user in a user device and transmitted via the communication interface to the input determination module.

11. The system of any preceding claim, wherein the optimization constraints are related to one or more observables.

12. The system of any preceding claim, wherein said one or more stochastic processes of said one or more observables are based on stochastic distributions of one or combinations of a Poisson, Gamma, binomial, Beta, log normal distribution.

13. The system of any preceding claim, wherein the optimized input is determined according to an expected value of the conditional probability density.

14. The system of any preceding claim, wherein the first memory, the second memory, the model creation module, and the input determination module are implemented in a distributed manner.

15. The system of any preceding claim, wherein the black-box information processing system is an online advertising auction system, in a market environment, wherein the input is a bid, wherein the observables aggregated over a period of time comprise one or more bids retained, a number of bids, a number of wins, a number of clicks, a number of conversions, a bid placement hit, and a revenue achieved by the bids.

Images