Automatically determining parameter values

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for automatically determining parameter values that control or influence the provision of content by a content platform are provided. In one aspect, evaluation points are identified for a parameter. Each evaluation point includes an evaluation parameter value for the parameter and a metric value corresponding to a metric of a digital component provided by the content platform. A first model is generated using the set of evaluation points. A second model is generated based on the first model and a harvesting function that is based on an average value and a confidence interval of the first model and a configurable exploration weight that controls a priority of exploration for evaluating the parameter. A next parameter value to evaluate is determined from the second model, and the content platform is configured to provide the digital component using the next parameter value.

Description

Technical Field

[0001] This manual generally covers data processing and automatically determining parameter values ​​that control or influence content provided by the content platform. Background Technology

[0002] The video streamed to a user may include one or more digital components, which typically overlay the original video stream. The overlay content may be provided to the user within a rectangular area that covers a portion of the original video screen. Digital components may also include in-stream content played before, during, or after the original video stream. The provision of video and digital components may be controlled by a content platform (e.g., a video system) based on parameter values ​​of various parameters, such as the time interval between the presentation of multiple digital components presented alongside the video, the frequency or likelihood of selecting different types of digital components, or other types of parameters.

[0003] As used throughout this document, the phrase "digital component" refers to a discrete unit of digital content or digital information (e.g., a video clip, audio clip, multimedia clip, image, text, or other unit of content). A digital component may be stored electronically on a physical storage device as a single file or a collection of files, and may take the form of a video file, audio file, multimedia file, image file, or text file. For example, a digital component may be content designed to complement the content of a video or other resource. More specifically, a digital component may include digital content related to the resource content (e.g., a digital component may relate to the same or related topics as / content on the video). Therefore, the provision of digital components can complement and often enhance the content of a webpage or application. Summary of the Invention

[0004] Generally, an innovative aspect of the subject matter described in this specification can be embodied in a method comprising: performing multiple iterations to identify parameter values, based on which a content platform controls the provision of digital components having video content, wherein each of the multiple iterations comprises: identifying a set of evaluation points for the parameter, wherein each evaluation point includes an evaluation parameter value for the parameter and a metric value corresponding to a measure of the digital components provided by the content platform, wherein the metric value of the evaluation point is determined based on data generated by the content platform using the evaluation parameter values ​​of the evaluation points to provide digital components; generating a first model using the set of evaluation points; generating an average value and confidence interval for the first model; and based on the first model and the sample... The acquisition function generates a second model, wherein the acquisition function is based on the average value of the first model, the confidence interval of the first model, and configurable exploration weights that control the priority of exploration used to evaluate parameters; the next parameter value to be evaluated is determined from the second model; the content platform is configured to use the next parameter value to provide digital components with video content; the next metric is determined based on data generated by the content platform using the next parameter value to provide digital components; and a specific parameter value that results in the highest metric value or satisfies a specific threshold is determined from the parameter values ​​and corresponding metric values ​​of the parameters determined during the plurality of iterations; the specific parameter value is used to configure the content platform to control or select the digital components providing video content during production. Other embodiments of this aspect include corresponding methods, apparatus, and computer programs encoded on a computer storage device and configured to perform the actions of the method. These and other embodiments may each optionally include one or more of the following features.

[0005] Specific embodiments of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. The techniques described in this specification provide, for example, a parameter tuning system that can automatically adjust parameters of another system, such as a content platform. A content platform may have, for example, hundreds or even thousands of parameters (e.g., parameters controlling the time interval between the presentation of digital components, parameters representing the probability of selecting a certain type of digital component). Manually tuning each parameter may be infeasible. Therefore, automatically tuning parameters through a parameter tuning system makes it possible to tune a greater number of parameters than using manual tuning methods. Content platforms have access to limited resources, such as data storage, processor time, administrator time, real-time experimental data, and network bandwidth, to name just a few. Limited resources mean that a limited number of experiments can be conducted on the content platform to determine the effects (e.g., metrics) that a content platform might produce while being configured with specific parameter values. Therefore, a brute-force approach to evaluating every possible parameter value is infeasible. With hundreds of parameters, each with multiple parameter value options, a combinatorial explosion can occur, where a brute-force approach would require evaluation of an infeasible number of parameter value options. That is, the limited resources of the content platform for experimentation would be exhausted before all possible parameter values ​​have been evaluated. Other methods, such as random evaluation of parameter values, can be inefficient because parameter values ​​that do not lead to the desired metric can continue to be evaluated, even though they are not good parameter value candidates. For example, by randomly selecting parameter values, parameter values ​​that are close together can be chosen, which wastes resources because evaluating similar parameters often yields very small values ​​to determine a better parameter value. The better parameter value used in this specification is a parameter value that results in a better metric derived from data generated by the content platform with that parameter value, compared to a worse metric derived from data generated by the content platform with different parameter values ​​configured. A better metric is a metric that the content platform provider and / or the entity using the content platform prefers or desires (compared to other metrics). A better metric may correspond to the content platform's objectives or the entities using the content platform. Entity objectives are described in more detail below. A better metric can be a value obtained when the content platform is configured with a specific parameter value, compared to a worse metric obtained when the content platform is configured with different parameter values.

[0006] Conversely, the parameter tuning system described in this specification selects parameter values ​​to be evaluated automatically and iteratively based on an acquisition function, which, for example, identifies parameter values ​​that have the highest predictive metric (relative to the metric corresponding to other parameter values) and / or the parameter values ​​with the highest potential (relative to other parameter values) in an unexplored parameter value space. Compared to results obtainable from brute-force or randomized methods, the selection process used by the parameter tuning system can lead to better parameter values ​​being selected for evaluation (and implementation) more quickly. By selecting better parameter values ​​more quickly, fewer experiments are needed to determine them, and therefore, the resources consumed by the content platform from running experiments to evaluate the selected parameters can be reduced. In this way, the parameter tuning system's selection of parameter values ​​can lead to faster configuration of the content platform with better parameter values ​​and improved resource efficiency (relative to other methods, such as brute-force or randomized methods).

[0007] Besides saving resources due to fewer experiments, configuring the content platform with better parameter values ​​can also lead to reduced or saved resources once the content platform is configured using parameter values ​​determined, for example, by a parameter tuning system using the techniques described above (as further described throughout the specification). For example, example metrics may include video abandonment rate (VAR), which indicates the percentage of users who abandon videos, and view pass rate (VLR), which indicates the percentage of users who watch digital components presented alongside videos. The parameter tuning system can automatically determine new parameter values ​​for one or more parameters (e.g., the time interval parameter for digital components) using the selection process described above (which is further described throughout the specification), which, once configured in the content platform, result in better video abandonment rate and VLR metrics. For some metrics, better metric values ​​are associated with more efficient utilization. For example, for video abandonment rate and VLR metrics, resource consumption of the content platform can be more efficient after configuring the content platform with new parameter values ​​that have been selected / determined (e.g., by a parameter tuning system) to produce a better video abandonment rate or a better VLR. In this example, efficient resource consumption is achieved because the provided video content and digital components are more likely to be consumed compared to the time period before configuring the new parameter values. In other words, the processing cycles, network bandwidth, and other resources consumed in providing video content and digital components are more resource-efficient because the likelihood of client devices receiving video content and digital components without the user consuming them is reduced.

[0008] As another example, the parameter tuning system can select better parameter values ​​that, once configured in the content platform, result in better resource-specific metrics, such as latency-specific metrics, such as the latency for delivering video content or digital components to a user device. For example, the parameter tuning system can determine parameter values ​​that, if implemented, will result in lower latency-specific metrics compared to the time period before the parameter values ​​were implemented. As another example, the parameter tuning system can select better parameter values ​​that, once configured, result in certain metrics, such as the total digital component interaction count over a specific time period, which can be achieved by providing fewer digital components compared to the same total digital component interaction count previously achieved in a previous time period when the content platform was configured with previous parameter values. For example, the parameter tuning system can select parameter values ​​that, once configured in the content platform, cause the content platform to select digital components that interact at a higher rate compared to the digital components selected by the content platform in a previous time period when other parameter values ​​were configured. As another example, the parameter tuning system can select better parameter values ​​that, once configured in the content platform, result in better metrics for various metrics (e.g., total digital component interaction counts over a time period) that consume the same amount of resources as were previously consumed in a previous time period when the content platform was configured with the previous parameter values.

[0009] The parameter tuning platform can automatically determine other parameter values ​​that improve resource utilization compared to those not automatically determined by the platform. For example, the platform can determine the values ​​of various parameters that limit the number of digital components allocated to a given user by the content platform. For instance, limiting the allocation of digital components by the content platform for various reasons can lead to less use of processor and network resources. Examples of parameters that the platform can automatically determine, and that the content platform can use to limit the number of allocated digital components, include: 1) a frequency cap parameter, which limits the total number of digital components that the content platform can allocate to each user within a specific time period; and 2) a maximum repetition count parameter, which limits the number of times the content platform can allocate the same digital component to the same user within a specific time period.

[0010] As another example, the parameter adjustment system can automatically determine the values ​​of various parameters that the content platform can use to control the number of digital components participating in the digital component selection auction conducted by the content platform. Limiting the number of digital components participating in the auction conducted by the content platform saves processing resources compared to not using parameters to limit the number of auction participants. Example parameters that the parameter adjustment platform can automatically determine and that the content platform can use to limit the number of digital components participating in the content auction include: 1) different parameters each controlling the maximum number of digital components in a specific format (e.g., skippable format, non-skipable format) that can participate in the content selection auction conducted by the content platform; 2) different parameters each controlling the maximum number of video digital components with a specific video length range (e.g., 0-1 minute, 1-5 minutes, 5-10 minutes, greater than 10 minutes) that can participate in the digital component selection auction conducted by the content platform (e.g., the content platform can use these parameters to allow fewer longer videos and more shorter videos to participate in the auction); 3) each Different parameters controlling the maximum number of digital components that can participate in auctions for users in a particular country (e.g., the content platform can use these parameters to allow a smaller number of auction participants for selecting digital components for users in a first country, compared to a larger number of auction participants for selecting digital components for users in a second country); and 4) each controlling different parameters controlling the maximum number of video digital components with a specific creative quality (e.g., high quality, medium quality, low quality as determined by the content platform's quality analyzer) that can participate in digital component selection auctions conducted by the content platform (e.g., the content platform can use these parameters to allow fewer lower quality videos and more higher quality videos to participate in the auction).

[0011] Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the specification, drawings, and claims. Attached Figure Description

[0012] Figure 1 This is a block diagram of an example environment used to automatically determine parameter values.

[0013] Figure 2A This is an example curve plotted above showing the evaluation points with parameters.

[0014] Figure 2B This is an example graph showing the actual function of the parameters.

[0015] Figure 3A It is a graph showing the mean and confidence interval.

[0016] Figure 3B This is a graph showing the acquisition function.

[0017] Figure 4 Example pseudocode for automatically determining parameter values ​​is shown.

[0018] Figure 5 This is a flowchart of an example process for automatically determining parameter values.

[0019] Figure 6 This is a block diagram of an example computer system that can be used to perform the above operations. Detailed Implementation

[0020] As summarized below and described throughout this specification, the techniques described herein enable the automatic determination of parameter values ​​that control or influence content provided by a content platform. In some embodiments, the techniques described herein provide a parameter adjustment system that determines parameter values ​​to be provided to a content platform (e.g., for content provided by the content platform) or to another type of system. For example, content platform parameters can control or influence content provided by the content platform. When different parameter values ​​are configured for one or more content platform parameters, the parameter adjustment system can evaluate experimental results from experiments performed on the content platform. The parameter adjustment system can iteratively select new parameter values ​​to evaluate based on the experimental results. As described above, the iterative selection of parameters by the parameter adjustment system can lead to the selection of better parameter values ​​faster than achievable using other methods. A better parameter value is a parameter value that produces a better metric from the data generated by the content platform when the content platform is configured with better parameter values, compared to a worse metric derived from data generated by the content platform when the content platform is configured with previous parameter values.

[0021] More specifically, the parameter tuning system can identify a set of evaluation points corresponding to parameters from previous experiments during the current iteration of evaluating the content platform's parameters. Each evaluation point includes an evaluation parameter value and a metric. The metric is determined based on data generated when the content platform used the evaluation parameter values ​​to provide digital components during previous experiments.

[0022] For example, an example parameter could be the amount of time interval between the presentation of digital components during a video viewing session. Example parameter values ​​could be ten seconds, thirty seconds, etc. The example metric is the video abandonment rate, indicating how frequently users abandon a video viewing session. A parameter adjustment system (or experimental system) can run a first experiment on the time interval parameter using a thirty-second parameter value. The content platform can track user interaction data including the start and stop times of the video viewing session during the first experiment. The parameter adjustment system (or content platform) can generate a video abandonment rate metric based on the user interaction data generated during the first experiment. For example, an example video abandonment rate metric value generated based on user interaction data from the first experiment could be 10%. That is, the first experiment could indicate that when the time interval parameter is thirty seconds, ten percent of users abandon the video prematurely (i.e., before the video ends). Therefore, the evaluation point for the time interval parameter in the first experiment includes the thirty-second parameter value and a 10% metric value. The parameter adjustment system can conduct a second experiment, the result of which is, for example, another evaluation point for the time interval parameter including a ten-second parameter value and a 15% metric value. That is, the second experiment can indicate that when the time interval parameter is ten seconds, 15 percent of users prematurely abandon the video.

[0023] After the parameter tuning system has identified a set of evaluation points for the parameters used in previous experiments, it can use this set to generate a first model. For example, the parameter tuning system can generate a first model that fits the evaluation points, such as a Gaussian model. The parameter tuning system can generate the mean and confidence interval of the first model. The parameter tuning system can then use an acquisition function based on the first model to generate a second model, which is based on the mean of the first model, the confidence interval of the first model, and configurable exploration weights that control the priority of the exploration used to evaluate the parameters. The exploration is described in more detail below.

[0024] The parameter tuning system can determine at least one next parameter value to be evaluated from the second model. For example, the parameter tuning system can determine the next parameter value that results in the highest acquisition function value generated from the second model (or, in this case, the parameter tuning system can select a predetermined number of parameter values ​​with the highest acquisition function value). Once the parameter tuning system has selected one or more parameter values ​​to be evaluated, it can configure the content platform to use the next parameter value when delivering digital components during the next experiment. The parameter tuning system (or experimental system) can then conduct the next experiment. The parameter tuning system can determine the next metric based on the data generated by the content platform delivering digital components using the next parameter value during the next experiment.

[0025] The parameter adjustment system can evaluate the next metric to determine whether it is an improved metric compared to a previous metric performed when configuring the content platform using previously evaluated parameter values. In some cases, the parameter adjustment system can determine the evaluation parameter value to be evaluated in the next experiment for use in non-experimental (e.g., production) applications within the content platform. For example, the parameter adjustment system can determine that the next metric is greater than a predetermined threshold (e.g., the predetermined threshold could be a desired or target metric). As another example, the parameter adjustment system can be configured to perform a predetermined number of experiments, and the most recent experiment could be the last experiment in that predetermined number. After the parameter adjustment system has determined the evaluation parameter value to be selected for non-experimental use, the content platform can be configured to use the evaluation parameter value to control or select digital components during production to deliver video content provided by the content platform.

[0026] As another example, the parameter tuning system may determine to execute yet another experiment. For instance, the system may determine that the next metric will not exceed a predetermined threshold, or it may determine to execute at least one more experiment before a predetermined number of experiments have already been performed. In this case, the system can create an updated set of evaluation points by adding additional evaluation points to the existing set. Each added evaluation point includes the parameter value evaluated during the last experiment and the corresponding metric derived from the data generated by the content platform during that last experiment. As described above, the updated set of evaluation points can be used in new experiments (e.g., to create a first model).

[0027] During each experimental iteration, the parameter tuning system can configure exploration weights to control the priority of exploration during the current experiment. Prioritizing exploration can lead to selecting the next parameter value to be evaluated with a higher confidence interval, which may correspond to a range of parameter values ​​that have been explored to be smaller than the range of other parameter values. The exploration method can be compared to the mining method. The mining method may correspond to continuing to explore the range of parameter values ​​with a higher predictive metric compared to the range of other parameter values. For example, prioritizing mining can be achieved by reducing the exploration weights that could lead to selecting the next parameter value to be evaluated with a higher average value. In some cases, the parameter tuning system prioritizes exploration in early experiments and mining in later experiments. For example, because the number of evaluation points increases with the number of iterations, the first model can represent a more accurate fit to the current set of evaluation points in later iterations, and therefore, the first model can be used in later iterations to produce more accurate predictions of the metric compared to previous iterations. Because of the more accurate predictions of the metric in later iterations, the parameter tuning system can prioritize mining in later iterations. Furthermore, because a larger range of unexplored parameter values ​​exists in early iterations compared to later iterations, the parameter adjustment system can prioritize exploration in early iterations. Other methods can be used to balance the exploration / mining tradeoff in the parameter adjustment system. While exploration weights have been described, in some implementations, the exploration / mining tradeoff is achieved by also or additionally configuring mining weights. See below for reference. Figures 1 to 6 These features, along with additional features and benefits, will be described in further detail.

[0028] Further regarding the descriptions throughout this document, users may be provided with controls that allow them to choose whether and when the system, program, or feature described herein can enable the collection of user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current location) and whether the user sends content or communications from the server. Additionally, some data may be processed in one or more ways before it is stored or used to remove personally identifiable information. For example, a user's identity may be processed to make it impossible to determine personally identifiable information about the user, or, where location information is available, the user's geographic location (such as city, zip code, or state) may be generalized to make it impossible to determine the user's specific location. Therefore, users can control what information about themselves is collected, how that information is used, and what information is provided to them.

[0029] Figure 1This is a block diagram of an example environment 100 for automatically determining parameter values. Example environment 100 includes a network 104. Network 104 may include a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof. Network 104 may also include any type of wired and / or wireless network, satellite network, cable network, Wi-Fi network, mobile communication network (e.g., 3G, 4G, etc.), or any combination thereof. Network 104 may utilize communication protocols, including packet-based and / or datagram-based protocols, such as Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or other types of protocols. Network 104 may also include multiple devices that facilitate network communication and / or the formation of a network, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, or combinations thereof.

[0030] Network 104 connects client device 102, content platform 106, content provider 108, and parameter adjustment system 110. Example environment 100 may include many different content platforms 106, client devices 102, and content providers 108.

[0031] Content platform 106 is a computing platform for content distribution (such as, for example, reference). Figure 1 (Described as a web server or other data processing device). Example content platform 106 includes search engines, social media platforms, video sharing platforms, new platforms, data aggregator platforms, or other content sharing platforms. Each content platform 106 can be operated by a content platform service provider. Each component of content platform 106 is a software component that includes instructions executed by a processing entity such as a processor.

[0032] Content platform 106 can publish and make available its own content. For example, content platform 106 could be a news platform that publishes its own news articles. Content platform 106 can also display content (e.g., digital components) provided by one or more content providers 108 that are not part of content platform 106. In the example above, the news platform could also display third-party content provided by one or more content providers 108. As another example, content platform 106 could be a data aggregator platform that does not publish its own content but instead aggregates and displays third-party content provided by different content providers 108.

[0033] In some implementations, content platform 106 may store certain information about client devices (e.g., device preference information, content consumption information, etc.). This user information can be used by the content platform, for example, to customize content provided to client device 102, or to make it easy to access specific content frequently accessed by client device 102. In some implementations, content platform 106 may not store such device information on the platform; however, content platform 106 may still provide such information for storage on a specific server (separate from the content platform). Therefore, content platform 106 (also referred to herein as content platform / server 106 or simply server) refers to either the content platform storing such device information or the server storing such device information (separate from the content platform).

[0034] In some implementations, content platform 106 is a video service through which users can watch streamed video content. The video streamed to the user may include one or more digital components (e.g., provided by content provider 108) overlaid on the original video stream. For example, it may be desirable to provide overlay content on the underlying video stream, provide digital components to viewers of the video stream, and increase the amount of content delivered within a viewing area for a given video stream bandwidth. In addition to or as an alternative to video streaming scenarios, content platform 106 may include a video processor that processes video files to modify them to include overlay content, wherein the processed video file with overlay content is provided to client device 102 for display on client device 102. As another example, as a replacement for overlay content, content platform 106 may provide digital components presented to the user at scheduled breakpoints in the video stream (rather than overlaying the video stream).

[0035] Content platform 106 may maintain user activity log 112, which includes anonymous information corresponding to user activities involving content provided by content platform 106 and content provided by content provider 108 (e.g., digital components). For example, for video content, user activity log 112 may include information about the start and end times of video viewing sessions, the presentation of digital components with video content, user interactions with the presented digital components, and other user activity information.

[0036] The metric generator 114 of the content platform 106 can generate various types of metrics based on data in the user activity log 112. Metrics generated by the metric generator 114 can be stored in the metric database 116. Exemplary metrics may include metrics related to wait time or responsiveness, as described in more detail below. Other example metrics generated by the metric generator 114 may include a point-to-click rate indicating the percentage of users interacting with digital components, a view pass rate indicating the percentage of users who watch a particular video digital component to completion, an abandonment rate indicating the percentage of users who abandon a video content item, a revenue metric indicating revenue generated from presenting digital components, a return on investment metric, content and digital component presentation counts, and other types of metrics.

[0037] Some metrics generated by metric generator 114 may correspond to specific goals of users or entities using content platform 106 and / or user or entity satisfaction with content platform 106. For example, content platform 106 and content provider 108 may receive revenue from the presentation of digital components with content provided by content platform 106. Therefore, metrics such as revenue metrics, the count of presented digital components, etc., may be meaningful to the goals of content platform 106 or content provider 108 and may correspond to the goals of content platform 106 or content provider 108. Content provider 108, which provides digital components, may have other goals, such as return on investment, and therefore may be interested in other metrics, such as the interaction rate with the presented digital components.

[0038] As another example, some metrics generated by metric generator 114 may correspond to user satisfaction with content platform 106 among users of client devices 102 consuming content provided by content platform 106. For example, some metrics may correspond to user perception of the content provided by content platform 106 (and / or user perception of content platform 106 itself). For example, for video content, the video churn rate metric for videos presented along with digital components may be related to user perception of the presented digital components, the number and intervals of the presented digital components, etc. Thus, trade-offs can occur between different objectives of different entities. For example, a video creator may want to receive more revenue by scheduling more digital components to be presented with their video, but presenting more digital components may reduce user satisfaction with the video and may cause users to abandon the video prematurely (and thus preemptively replay some scheduled digital components). In addition, presenting more digital components may not necessarily increase the long-term return on investment for the provider of digital components, as presenting more digital components may reduce the interaction rate of the corresponding digital components, either because users are overwhelmed or annoyed by too many digital components, or because the relevance of the digital components to the presented resource content is generally reduced. Other metrics that may correspond to user satisfaction or user goals may include overall system wait time or responsiveness. For example, user satisfaction with content platform 106 typically decreases as wait times for playback content or digital components increase.

[0039] In some implementations, metric generator 114 can generate a combined metric that combines multiple different metrics. For example, the combined metric can be a combination of different metrics, each representing a goal of a different entity using content platform 106. For example, the combined metric can be based on a combination of sub-metrics including user acceptance metrics (e.g., video abandonment rate), revenue-related metrics (e.g., the count of digital components presented with the video), and digital component interaction rate metrics. In some cases, metric generator 114 can generate a combined metric by adjusting (e.g., multiplying) the corresponding sub-metric weight (e.g., a value ranging between 0 and 1) for each sub-metric value and aggregating (e.g., summing) the adjusted sub-metric values.

[0040] Content platform 106 provides content that can be controlled by various parameters 118. Parameters 118 can be configured by parameter setter 120, which can be an automated process and / or may include a user interface component for receiving parameter values ​​from an administrator. Parameters 118 may include, for example, one or more parameters that can be used by content platform 106 to control the time interval between the presentation of multiple digital components presented along with the video. Content platform 106 may use other parameters to control the frequency or likelihood of selecting different types of digital components (e.g., digital components with a certain type of content) for certain types of video. Typically, content platform 106 may include hundreds of different types of parameters.

[0041] The parameter setter 120 can set a given parameter to a specific value, which can affect the content provided by the content platform 106, as described above. The content provided by the content platform 106 according to different parameter values ​​can influence user activity, and thus, in turn, affect the aforementioned target-related metrics generated by the metric generator 114. For example, if the time interval parameter is reduced from thirty seconds to ten seconds by the parameter setter 120 (e.g., causing a new digital component to be displayed every ten seconds while the user is watching the video), the video creator's short-term revenue may increase due to the increased number of digital components presented. However, users may become annoyed, for example, by the repeated interruptions in the presentation of digital components before the video content is played, and therefore may spend less time watching content on the content platform 106, thus reducing the long-term revenue of both the content platform 106 and the content provider 108.

[0042] Another example parameter for content platform 106 is the user cost penalty parameter, which, for example, represents the magnitude of a potential reduction in long-term revenue costs if content platform 106 selects a digital component for playback at a certain point in time within a video. Content platform 106 can use the user cost penalty parameter value in an auction used to select digital components for playback with the video. Content platform 106 can use the user cost penalty parameter value to balance other factors, such as predicted interactions with digital components and predicted short-term revenue. The administrator of the content platform may initially be unaware of the value assigned to the user cost penalty parameter. As described in more detail below, parameter adjustment system 110 can be used to automatically determine the values ​​for the user cost penalty parameter (and other parameters).

[0043] The parameter setter 120, which modifies parameter values, may ultimately affect various metrics, but the administrator of the content platform 106 may not know the parameter values ​​(relative to other determined metric values ​​or relative to predetermined thresholds) specified for certain parameters to achieve the corresponding metric values. For example, there may be unknowns regarding parameters for a given metric. The situation of the parameter relative to the metric may be complex and may correspond to an unknown function with high degrees of freedom and nonlinearity, making it possible to directly calculate the predicted metric value for any given parameter value without knowing the function. Therefore, for the parameter setter 120, there may not be a direct way to calculate parameter values ​​that achieve other determined metric values ​​relative to the parameter or better metric values ​​relative to predetermined thresholds.

[0044] Content platform 106 can learn the impact of changes to metrics by using experimentation system 122 in response to parameter setter 120 changing parameter values. Experimentation system 122 can be configured with various experiments 124. Experiment 124 may include information specifying one or more values ​​to be evaluated during an experiment of one or more evaluation parameters. For example, the current value of a time interval parameter used to control the interval between digital components may be thirty seconds. Experiment 124 may specify different parameter values ​​to be evaluated during the experiment, such as ten seconds. Content platform 106 can use experimentation system 122 to segment live traffic between production (e.g., non-experimental) and experimental traffic 126. For example, content platform 106 can use experimentation system 122 such that content platform 106 processes a first portion (e.g., 99%) of a content request based on the current (e.g., production) parameter value, and content platform 106 processes a second portion (e.g., 1%) of the content request as experimental traffic 126 based on information specified in experiment 124. For example, parameter setter 120 may set parameter values ​​for the second portion of the content request based on information from experiment 124 during experiment 124.

[0045] Changing parameter values ​​by parameter setter 120 during the experiment can affect the content provided by content platform 106 (e.g., the video as a whole or digital components provided with / during the video). For example, content platform 106 may select certain types of digital components more or less frequently, or it may select different numbers of digital components. Users of client device 102 can react to the affected content provided by content platform 106. For example, users may interact with digital components more or less or in different ways (e.g., by clicking / selecting or spending time watching). As another example, users may interact differently, for example, with video content presented with digital components. For example, users receiving content in the experiment may tend to abandon video content at lower or higher frequencies, or they may abandon video content closer to or further from the end of the video. Content platform 106 can track and store these and other types of user interactions as experimental activity 128 during the experiment.

[0046] Metric generator 114 can generate experimental metrics 130 from experimental activity 128. Experimental metrics 130 may include information indicating which parameter values ​​are used in experiment 124. Experimental system 122 can compare the experimental metrics 130 generated by metric generator 114 from experimental activity 128 with corresponding non-experimental metrics in metric database 116 generated by metric generator 114 from non-experimental traffic from user activity to determine whether at least some experimental metrics 130 are superior to their corresponding non-experimental metrics. If an experimental metric 130 is superior to its corresponding non-experimental metric, parameter setter 120 can determine the parameter values ​​used during the experiment and set the corresponding parameters to those values ​​for subsequent (e.g., non-experimental) content delivery on content platform 106.

[0047] As mentioned above, conducting experiments on the experimental system 122 can be resource-intensive. The resources available for experimentation on the content platform 106 (e.g., processing time, data storage, administrator time, network bandwidth, real-time content requests for the experiment) may be limited. Therefore, a brute-force approach of trying all possible parameter values ​​for each parameter is not feasible. Other methods, such as randomly selecting parameter values, are generally ineffective because they may continue to select parameter values ​​that are poor choices for obtaining the desired metric.

[0048] As an alternative to brute-force, random, or other methods for selecting parameter values, parameter adjustment system 110 can automatically determine parameter values ​​by executing a parameter adjustment process. Each component of parameter adjustment system 110 is a software component comprising instructions executed by a processing entity such as a processor. Although parameter adjustment system 110 is described as adjusting parameters of content platform 106, parameter adjustment system 110 can adjust parameters of other types of systems. Although shown as separate from content platform 106, in some embodiments, some or all components of parameter adjustment system 110 may be included in content platform 106.

[0049] The driver 131 of the parameter adjustment system 110 can be used to control the parameter adjustment process. For example, an administrator can use driver 131 to start or stop the parameter adjustment process. As another example, driver 131 can be used to perform automatic parameter adjustment. For example, driver 131 can perform the parameter adjustment process automatically and / or periodically (e.g., weekly, monthly).

[0050] In response to a request or confirmation to begin the parameter tuning process, driver 131 may invoke parameter value selector 132. When tuning parameters, parameter value selector 132 may perform multiple parameter tuning iterations. During each parameter tuning iteration, parameter value selector 132 may automatically select one or more next parameter values ​​to be evaluated in the next experiment based on data from past experiments. As described in more detail below, parameter value selector 132 may be configured to perform a predetermined number of parameter tuning iterations during the parameter tuning process, or parameter value selector 132 may stop the parameter tuning process in response to determining that an experimental metric derived from experimental data evaluating parameter values ​​meets (e.g., reaches or exceeds) a threshold metric.

[0051] More specifically, the parameter value selector 132 can use evaluation points 134 corresponding to past experiments as input. As described in more detail below, the initial set of evaluation points 134 may correspond to experiments performed by the experimental system 122 using random parameter values. Other evaluation points 134 used by the parameter value selector 132 may correspond to experiments previously performed by the experimental system 122 using parameter values ​​previously selected by the parameter value selector 132 in an early iteration of the parameter tuning process. Each evaluation point 134 includes experimental metrics from previous experiments and parameter values ​​used by the content platform 106 to provide content during previous experiments. For example, the evaluator 135 may receive or access experimental data (e.g., experimental metric 130) generated by the metric generator 114 from the content platform 106 to generate evaluation points 134. The following is about Figure 2A Provide a more detailed explanation and description of the assessment points.

[0052] To generate initial evaluation points, parameter value selector 132 can randomly generate a predetermined number of random parameter values ​​for the parameter. For each random parameter value, driver 131 can use experiment file editor 136 to create an experiment file 138 including the random parameter value to be evaluated. Parameter tuning system 110 can provide experiment file 138 to experiment system 122. Experiment system 122 can use experiment file 138 to conduct random parameter value experiments, during which content platform 106 provides content based on the random parameter values ​​(e.g., as experiment traffic 126). As described above, during random parameter value experiments, parameter setter 120 can set the parameter to random parameter values, and content platform 106 can track experiment activities 128 that occur in response to experiment traffic 126. Metric generator 114 can generate experiment metrics 130 based on experiment activities 128, and experiment metrics 130 can be provided by content platform 106 to evaluator 135. Evaluator 135 can use the random parameter values ​​generated from the random parameter value experiments and the received metrics to populate the initial evaluation point 134 of the parameter.

[0053] During parameter tuning iterations, parameter value selector 132 obtains the current set of evaluation points 134 (e.g., initial evaluation points of the first iteration or evaluation points of subsequent iterations after the first iteration). Parameter value selector 132 uses the obtained evaluation points 134 to generate a first model 140. The first model 140 can be a probabilistic model that fits the evaluation points 134. For example, the first model 140 can be a Gaussian model that can produce predictions of metrics for unevaluated parameter values. Parameter value selector 132 can determine the mean 142 and confidence intervals 144 from the first model 140. For example, the confidence interval can correspond to the standard deviation value. The following is about... Figure 3A and Figure 4 The first model 140, the mean 142, and the confidence interval 144 are described in more detail.

[0054] After parameter value selector 132 generates a first model 140, it can use the mean 142 and confidence interval 144 derived from the first model 140 to generate a second model 146. For example, the second model 146 can be based on an acquisition function that is a combination of the mean 142, the confidence interval 144, and exploration weights that assign weights to the confidence interval 144. While the first model 140 represents a predicted metric of parameter values ​​fitted based on existing evaluation points, the second model 146 can represent the potential of unexplored parameter values.

[0055] Parameter value selector 132 can assign larger exploration weights in early parameter tuning iterations and smaller exploration weights in later parameter tuning iterations. Parameter value selector 132 can determine one or more next parameters to be evaluated based on the second model 146. For example, parameter value selector 132 can select parameter values ​​that produce the highest acquisition function value. The following section discusses... Figure 3B and Figure 4 The second model 146, the acquisition function, and exploration and mining are described in more detail.

[0056] For each parameter value selected by parameter value selector 132, driver 131 can use experiment file editor 136 to create an experiment file 138 including the selected parameter value. Parameter adjustment system 110 can provide experiment file 138 to experiment system 122, and experiment system 122 can conduct experiments during which content platform 106 provides content based on the selected parameter value (e.g., as experiment traffic 126). As described above, parameter setter 120 can set parameters to the selected parameter value, and content platform 106 can track experiment activities 128 that occur in response to experiment traffic 126. Metric generator 114 can generate experiment metrics 130 based on experiment activities 128, and experiment metrics 130 can be provided to evaluator 135. Experiment metrics 130 may include the aforementioned target-related metrics. As described above, target-related metrics may be combined metrics representing combinations of different targets of different entities using content platform 106.

[0057] In some implementations, evaluator 135 determines whether the received experimental metric for the selected parameter value satisfies a threshold metric. For example, the parameter adjustment process may be configured to stop when the metric generated by metric generator 114 from experiments using the selected parameter value satisfies (e.g., reaches or exceeds) a threshold metric (e.g., a predetermined, satisfactory, or desired metric). If the received experimental metric for the selected parameter value satisfies the threshold metric, parameter adjustment system 110 can determine that the parameter adjustment process for that parameter has been completed. On the other hand, if the selected parameter value does not satisfy (e.g., is less than) the threshold metric, the parameter adjustment process can continue to iterate through additional parameter adjustment iterations. As another example, if a predetermined number of parameter adjustment iterations have been performed on the parameter, parameter adjustment system 110 can also determine that the parameter adjustment process for the parameter has been completed.

[0058] In response to determining that the parameter tuning process has been completed, the parameter tuning system 110 can instruct the content platform 106 to use previously evaluated parameter values ​​assessed by the parameter value selector 132 for non-experimental traffic. For example, the parameter value selector 132 can select the previously evaluated parameter values ​​that result in the optimal evaluation metric. For example, the parameter setter 120 can set the parameter to the selected previously evaluated parameter value, and the content platform 106 can serve content based on the parameter value in response to subsequent requests for content. For example, the content platform 106 can use the selected previously evaluated parameter value to control or select digital components to serve video content during production.

[0059] If the parameter tuning system 110 determines that the parameter tuning process for a parameter has not yet reached a stopping point, the evaluator 135 can use the parameter values ​​selected by the parameter value selector 132 in previous iterations and the corresponding metric values ​​generated by the metric generator 114 from experiments using the parameter values ​​selected by the parameter value selector 132 in previous iterations to generate a new evaluation point 134 for the parameter, thus producing an updated evaluation point 134 (e.g., the new evaluation point can be added to the existing set of evaluation points 134). The parameter value selector 132 can use the updated evaluation point 134 for the parameter to perform the next parameter tuning iteration.

[0060] The parameter tuning system 110 can perform different parameter tuning processes for different parameters. Each parameter tuning process performed by the parameter tuning system 110 can include adjusting the parameter relative to a given metric. The parameter tuning system 110 can perform different parameter tuning processes for different metrics. The parameter tuning system 110 can execute different parameter tuning processes in parallel or sequentially. See below for reference. Figures 2A to 6 The additional structural and operational aspects of these components of the parameter adjustment system 110 and the parameter adjustment process are described.

[0061] Figure 2A This is an example graph 200 showing the evaluation points with parameters plotted above. The X-axis 202 corresponds to the value of the parameter. The parameter can be, for example, a real value between zero and one. The Y-axis 204 corresponds to the value of the metric generated by the metric generator 114, for example, when the content platform 106 provides content based on a specific parameter value during the experiment. Evaluation points 206, 208, 210, 212, and 214 are observation points, each including the evaluation parameter value and the corresponding metric value. For example, as shown by evaluation points 206, 208, 210, 212, and 214, when the parameter has values ​​of 0.8, 0.36, 0.4, 0.78, and 0.95, the corresponding metric values ​​generated by the metric generator 114 are 0.28, 0.55, 0.44, 0.61, and 0.44, respectively. For example, evaluation points 206, 208, 210, 212, and 214 can be included in the above description regarding... Figure 1The evaluation point 134 is described.

[0062] Evaluation point 212, with a parameter value of 0.78 and a metric value of 0.61, corresponds to the highest metric value among evaluation points 206, 208, 210, 212, and 214. While the parameter value of 0.78 at evaluation point 212 may be the optimal parameter value among the observation points, other unevaluated parameter values ​​could lead to better corresponding metric values ​​being generated by the metric generator 114 if configured by the parameter setter 120. However, as mentioned above, after observing only the current observation point, the parameter adjustment system 110 may not know the condition of the parameter values ​​or the "true function" of the parameter. The true function of the parameter can be a function that reflects the actual metric value of various possible parameter values. The true function (if known) can output a given metric value for a specific parameter value. Since the parameter adjustment system 110 does not know the true function, it can perform a parameter adjustment process for the parameter.

[0063] Figure 2B This is an example graph 250 showing the true function of the parameters. The X-axis 252, Y-axis 254, and evaluation points 256, 258, 260, 262, and 264 correspond to... Figure 2A The X-axis 202, Y-axis 204, and evaluation points 206, 208, 210, 212, and 214. The true function line 266 illustrates the true function or case for the metric given different parameter values. As mentioned above, the parameter tuning system 110 does not know the true function. Instead, the parameter tuning system 110 currently knows the evaluation points 256, 258, 260, 262, and 264 (which correspond to...). Figure 2A Evaluation points 206, 208, 210, 212, and 214). As mentioned above, evaluation point 262 (which corresponds to...) Figure 2A The evaluation point 212 has the highest observed metric value (e.g., 0.61), but as shown at point 268 on the true function line 266, the unevaluated parameter value of 0.23 (if configured by parameter setter 120) will result in a higher metric value (e.g., 0.80) being generated by metric generator 114. Parameter value selector 132 is configured to perform a parameter tuning process to find a better parameter value than the observed evaluation point, such as the parameter value of 0.23 at point 268.

[0064] Figure 3A This is a graph 300 showing the mean and confidence intervals. The X-axis 302, Y-axis 304, and evaluation points 306, 308, 310, 312, and 314 correspond to... Figure 2A The X-axis is 202, the Y-axis is 204, and the evaluation points are 206, 208, 210, 212, and 214. The true function line 316 corresponds to... Figure 2B The true function line is 266.

[0065] As mentioned above Figure 1 The parameter value selector 132 can generate a first model using evaluation points 306, 308, 310, 312, and 314. The first model (which may be first model 140) can be, for example, a Gaussian model fitted to evaluation points 306, 308, 310, 312, and 314. The first model is displayed as a Gaussian line 318 on graph 300. The Gaussian model may include observation points (e.g., evaluation points 306, 308, 310, 312, and 314) and predictions (e.g., average values ​​142) of points located between the observation points. For example, prediction point 315 includes a parameter value of 0.27 and a prediction metric of 0.64.

[0066] In some implementations, the first model is an aggregate model that combines multiple other models, each of which uses a different approach to fit evaluation points 306, 308, 310, 312, and 314. When the first model is an aggregate model, it may include the average of the average predicted metrics determined to be the average of the multiple other models.

[0067] The shaded areas on graph 300 (such as shaded area 320) represent confidence intervals (e.g., confidence interval 144) calculated by parameter value selector 132 from the first model. As shown in graph 300, since evaluation points 306, 308, 310, 312, and 314 are known or observed points, rather than corresponding to predicted metrics, the confidence intervals at and near evaluation points 306, 308, 310, 312, and 314 are low. For other parts of Gaussian line 318, larger confidence intervals exist, such as in parameter value ranges with fewer observed points, and therefore are explored less compared to other parameter value ranges. For example, vertical lines 322, 324, 326, and 328 show larger confidence intervals than in other areas of graph 300.

[0068] As described above, parameter value selector 132 can select the next parameter value to be evaluated based on a second model (e.g., second model 146) that uses a combination of the mean and confidence interval of the first model. Parameter value selector 132 can generate and use, for example, a sampling function to generate the second model, as described below. Figure 3B As shown below. (Regarding the following text...) Figure 4 In more detail, the acquisition function can have the general form shown in equation (1) below.

[0069]

[0070] The exploration weights, which can be set by the administrator and / or by the parameter value selector 132, can control the priority or bias used in the acquisition function to a confidence interval value relative to the mean. Higher exploration weights can result in higher priority being given by the parameter value selector 132 for exploration (e.g., exploration within a range of parameter values ​​that have not yet been explored). Lower exploration weights can reduce the priority of exploration by the parameter value selector 132, which can increase the priority of mining by the parameter value selector 132. Mining can refer to the parameter value selector 132 selecting the parameter value that the first model prediction will result in the highest metric. As described above, when the count of evaluated parameter values ​​is low, the parameter value selector 132 can initially prioritize exploration (e.g., by configuring a higher exploration weight) and subsequently prioritize mining as the count of evaluated parameter values ​​increases (e.g., by configuring a lower exploration weight). In summary, the parameter value selector 132 can first prioritize exploring unexplored ranges of parameter values ​​and then explore parameter values ​​predicted to have the highest metric, such as parameter values ​​close to the evaluated parameter values ​​that have so far resulted in the highest metric.

[0071] Assuming the acquisition function uses a combination of confidence interval values ​​and averages, the highest acquisition function value generated by the parameter value selector 132 using the activation function can be a parameter value with a relatively large confidence interval, a relatively large average, or a combination of a relatively large confidence interval and average relative to other parameter values. For example, prediction point 315 with a parameter value of 0.27 has the largest prediction metric value of 0.64 among the parameter values ​​and a confidence interval greater than other parameter values ​​(e.g., represented by the height of vertical line 322). Therefore, parameter value selector 132 can select the parameter value of 0.27 for prediction point 322 as the next parameter to be evaluated, as described below. Figure 3B More detailed description.

[0072] In some cases, parameter value selector 132 selects the next parameter to be evaluated for each subsequent experiment. In other cases, parameter value selector 132 can select more than one next parameter value to be evaluated for each subsequent experiment. Accordingly, parameter value selector 132 can select parameter values ​​other than the parameter value of 0.27 for prediction point 315. For example, prediction point 332 with a parameter value of 0.58 has a prediction metric of 0.35, which is lower than the prediction metric of 0.64 for prediction point 315. However, the confidence interval of the parameter value of 0.58 for prediction point 332 (e.g., represented by the height of vertical line 324) is greater than the confidence interval of prediction point 315. Based on the value of the exploration weight, the parameter value of 0.58 for prediction point can be selected as the next parameter to be evaluated. The relatively high confidence interval of the parameter value of 0.58 for prediction point 332 can indicate high potential (e.g., high potential for parameter value selector 132 to find the best or better parameter value compared to existing evaluation parameter values). The relatively high potential of the parameter value 0.58 may cause the parameter value selector 132 to select the parameter value 0.58 as the next parameter to be evaluated, even if the predicted metric of the parameter value 0.58 is lower than that of other predicted points. As another example, if the combination of the weighted confidence interval and the average of the predicted points 334 and 336 results in the highest (or within a predetermined number of highest values) acquisition function value, the parameter value selector 132 may select the parameter values ​​0.68 and 0.85 for the predicted points 334 and 336, respectively, as the subsequent parameter values ​​to be evaluated.

[0073] Figure 3B This is a graph 350 showing the acquisition function. The X-axis 352 corresponds to the parameter values. The Y-axis 354 corresponds to the value of the acquisition function given a specific parameter value. The acquisition function line 355 is plotted on the graph 350. The acquisition function is a function that can output a value used by the parameter value selector to select the next parameter value to evaluate (e.g., the acquisition function can produce values ​​that guide the parameter value selector 132 in exploring the parameter value space during the parameter tuning process). For example, the parameter value selector 132 can select the next parameter to evaluate by determining the parameter value with the highest acquisition function value. For example, the parameter value selector 132 can determine that point 356 on the acquisition function line 355 with a parameter value of 0.27 has the highest acquisition function value of 0.043. For example, point 356 is at the highest peak of the acquisition function line 355. Point 356 corresponds to the above regarding... Figure 3A The predicted point 315 is described. Although the parameter value 0.27 for point 356 can be selected by parameter value selector 132 based on a combination of confidence value (e.g., exploration) and average value (e.g., mining), parameter value selector 132 can primarily select the parameter value 0.27 based on the average value. Accordingly, the parameter value 0.27 can be considered primarily a "mining" parameter value.

[0074] The parameter value selector 132 can select other parameter values ​​to be evaluated based on the acquisition function. For example, the parameter value selector 132 can select parameter values ​​of 0.68 and 0.85 for points 358 and 360, respectively. Points 358 and 360 are located at other peaks of the acquisition function line 355. Points 358 and 360 correspond to... Figure 3A Prediction points 334 and 336. While parameter value selector 132 can select parameter values ​​0.68 and 0.85 based on a combination of confidence values ​​(e.g., exploration) and average values ​​(e.g., mining), parameter value selector 132 can primarily select parameter values ​​of 0.68 and 0.85 based on confidence values. Therefore, parameter values ​​of 0.68 and 0.85 can be considered primarily "exploratory" parameter values. The average values ​​of parameter values ​​0.68 and 0.85 can be influenced by the average values ​​of nearby known evaluation points 312 (e.g., where "nearby" means within a threshold distance of known evaluation points 312). The selection of parameter values ​​0.68 and 0.85 by parameter value selector 132 can be considered as further exploring the area around known evaluation points 312.

[0075] Figure 4 Example pseudocode 400 for automatically determining parameter values ​​is shown. In some implementations, the code corresponding to pseudocode 400 can be executed by any suitable data processing device, including, for example, those described above. Figure 1 The parameter value selector 132 and experimental system 122 are described.

[0076] In line 1, parameter value selector 132 sets the variable m to the number of trials per round. The number of trials per round can represent how many parameter values ​​are evaluated in each round.

[0077] In line 2, parameter value selector 132 sets the variable k to the number of experimental rounds.

[0078] In line 3, parameter value selector 132 sets the gamma variable to the exploration bias value. The gamma variable corresponds to the value mentioned above. Figure 3B The described exploration weights.

[0079] In line 4, parameter value selector 132 initializes the data_points array as an empty array.

[0080] In line 5, parameter value selector 132 sets the new_trials array to an empty array.

[0081] In line 6, parameter value selector 132 configures the first iteration construction to be repeated a total of m times (e.g., one iteration each time).

[0082] In line 7, during a given iteration constructed in the first iteration, parameter value selector 132 generates random parameter values ​​x[i].

[0083] In line 8, parameter value selector 132 adds random parameter values ​​to the new _test array.

[0084] In line 9, experimental system 122 performs experiments using random parameter values ​​from the new experimental array to generate metrics. .

[0085] In line 10, parameter value selector 132 creates evaluation points. And add the evaluation points to the data point array.

[0086] In line 11, parameter value selector 132 configures the second iteration construction to repeat a total of k-1 times. For example, as mentioned above... Figure 1 The parameter value selector 132 can be configured to perform a predetermined number of parameter adjustment iterations during the parameter adjustment process. The value k in line 11 corresponds to the predetermined number of parameter adjustment iterations.

[0087] In line 12, parameter value selector 132 creates a first model (e.g., first model 140) of the evaluation points in the fitted data point array.

[0088] In line 13, parameter value selector 132 resets the new test array to an empty array.

[0089] In line 14, parameter value selector 132 sets the new test array to an empty array, which clones the evaluation points in the data point array and stores the cloned evaluation points in a temporary array.

[0090] In line 15, parameter value selector 132 clones the first model and stores the cloned first model in a temporary object.

[0091] In line 16, parameter value selector 132 configures the third iteration construction to repeat a total of m times (e.g., one iteration each time).

[0092] In line 17, based on the first model (e.g., the fitted Gaussian model created in line 12), parameter value selector 132 uses the parameter value selector corresponding to... Figure 3B The upper confidence limit (UCB) function of the acquisition function shown determines the function value. The parameter value selector 132 uses the mean of the Gaussian model, the standard deviation of the Gaussian model (e.g., confidence level), and the gamma parameter to calculate the UCB function value. The gamma parameter is a parameter that can be configured by the parameter value selector 132 to control the priority of exploration (vs. mining), as described above regarding... Figure 3B The above and the following description regarding line 24.

[0093] In line 18, parameter value selector 132 determines the parameter value x[i] that has the maximum UCB function value (e.g., the maximum acquisition function value). For example, as Figure 3B As shown, point 356 corresponds to the maximum acquisition function value.

[0094] In line 19, parameter value selector 132 adds the parameter value x[i] with the maximum acquisition function value to the new _test array (e.g., which corresponds to including the maximum acquisition function value in the new test set for the next parameter adjustment iteration).

[0095] In line 20, parameter value selector 132 adds a new point to the temporary model, where the new point includes the X value x[i] with the highest acquisition function value and the Y value with the highest acquisition function value.

[0096] In line 21, parameter value selector 132 creates an adjusted first model by performing a fitting operation to fit points in a temporary model (e.g., where the temporary model includes new points with the highest acquisition function value). Including new points in the fitting operation causes the confidence value of x[i] to become zero in a later iteration of the third iteration construction (e.g., at line 17), which causes parameter value selector 132 to select other parameter values ​​with higher (e.g., non-zero) confidence values ​​for evaluation. That is, according to the ucb function in line 17, a higher confidence value leads to a higher acquisition function value. As an example, parameter value selector 132 may select points 358 and 360 in part based on the non-zero confidence values ​​of points 358 and 360.

[0097] In line 22, experimental system 122 performs experiments using parameter values ​​from the new experimental array to generate metrics. .

[0098] In line 23, parameter value selector 132 creates evaluation points. And add the evaluation points to the data_point array.

[0099] In line 24, parameter value selector 132 optionally adjusts the gamma variable to correspond to different exploration weights. (See above regarding...) Figure 3BThe exploration weight can be higher in early iterations and lower in later iterations (so that parameter value selector 132 prioritizes exploration in early iterations and mining in later iterations). Parameter value selector 132 can decrease the value of the gamma parameter during each iteration of the second iteration construction starting from line 11. Parameter value selector 132 can reduce the gamma parameter to zero during the final iteration of the second iteration construction, so that in the final iteration, parameter value selector 132 uses only the mean of the Gaussian model and not the standard deviation of the Gaussian model to determine the UCB value. After adjusting the gamma value during a given iteration, parameter value selector 132 can execute the next iteration unless the final iteration has already been executed.

[0100] Figure 5 This is a flowchart of an example process 500 used for automatically determining parameter values. The operation of process 500 is described below by... Figures 1 to 4 The components of the system described and depicted herein perform the operation. The operation of process 500 is described below for illustrative purposes only. The operation of process 500 can be performed by any suitable device or system (e.g., any suitable data processing apparatus). The operation of process 500 can also be implemented as instructions stored on a computer-readable medium that may be non-transitory. Execution of the instructions causes one or more data processing apparatuses to perform the operation of process 500.

[0101] The parameter adjustment system 110 performs multiple iterations to identify parameter values, which the content platform uses to control the delivery of digital components containing video content (at 502). In each of the multiple iterations, the parameter adjustment system performs operations 504 to 516, each of which is described below.

[0102] The parameter adjustment system 110 identifies the set of evaluation points for the parameters (at point 504). For example, see the reference above. Figures 1 to 4 The parameter value selector 132 identifies evaluation points 134. Each evaluation point includes an evaluation parameter value for that parameter and a metric value corresponding to a metric provided by the content platform 106 for the digital component. The metric generator 114 determines the metric value for the evaluation point based on data generated by the content platform 106 during previous experiments using the evaluation parameter values ​​of the evaluation points to provide the digital component.

[0103] The parameter tuning system 110 uses the evaluation point set to generate the first model (at position 506). For example, as referenced above. Figures 1 to 4 As described above, parameter value selector 132 can generate a first model 140 that fits the current evaluation point set. The first model can be a Gaussian model that fits the current evaluation point set. As another example, and as mentioned above... Figure 3AThe first model can be an aggregation model of multiple other models, each of which uses a different method to fit the current evaluation point set. For each parameter value, the first model may include an average prediction metric, which is determined based on the average of the prediction metrics generated by the various other models.

[0104] The parameter adjustment system 110 generates the mean and confidence interval (at 508) of the first model. For example, see the reference above. Figures 1 to 4 The parameter value selector 132 can generate an average value 142 and a confidence interval 144. The average value of points other than the current evaluation point set corresponds to the predicted metric generated by the first model. The average value of the current evaluation point set is a known metric, not a predicted metric. Therefore, the confidence value of the current evaluation point set is zero (e.g., since the metric is known for the current evaluation point set, the parameter tuning system can have full confidence in the metric of the current evaluation point set). The confidence value of points other than the current evaluation point set is non-zero, and the greater the distance of a given point from the known current evaluation point, the higher the confidence value. That is, the confidence value of a given point represents the confidence of the parameter tuning system in the predicted metric at that point, and a higher confidence value represents a lower confidence of the parameter tuning system in the predicted metric.

[0105] The parameter tuning system 110 uses an acquisition function to generate a second model (at 510) based on a first model. The acquisition function is based on the average value of the first model, the confidence interval of the first model, and configurable exploration weights that control the priority of exploration used to evaluate the parameters. For example, as referenced above... Figures 1 to 4 The parameter value selector 132 can generate a second model 146. When determining the next parameter value to be evaluated from the second model, the parameter adjustment system 110 can use exploration weights to control the priority of exploring new parameter values. The exploration weights can correspond to the weights of confidence intervals in the acquisition function. The exploration weights can be higher in early parameter evaluation iterations and lower in later parameter evaluation iterations.

[0106] The parameter adjustment system 110 determines the next parameter value to be evaluated (at position 512) from the second model. For example, as referenced above. Figures 1 to 4 The parameter value selector 132 can determine the next parameter value to be evaluated based on the highest acquisition function value generated from the second model 146. The parameter value selector 132 can determine more than one next parameter value to be evaluated. For example, the parameter value selector 132 can determine the parameter value corresponding to a maximum predetermined number of highest acquisition function values.

[0107] The parameter adjustment system 110 configures the content platform 106 to use the next parameter value to provide digital components with video content (at 514). For example, as referenced above.Figures 1 to 4 The parameter adjustment system 110 can provide the experimental file 138 to the experimental system 122 to configure the experimental system 122 to perform the experiment based on the next parameter value.

[0108] The parameter adjustment system 110 uses data generated by the digital component, provided by the content platform 106, to determine the next metric (at 516). For example, as referenced above. Figures 1 to 4 The evaluator 135 can receive experimental metrics 130 from the content platform 106.

[0109] 000108 The parameter adjustment system 110 determines a specific parameter value (at 518) from the parameter values ​​and corresponding metric values ​​of the parameters determined during the plurality of iterations that result in the highest metric value or satisfy a specific threshold. In some embodiments, and as referenced above... Figures 1 to 4 The parameter adjustment system 110 selects from various parameter values ​​the parameter value that results in the highest metric value (relative to other metric values ​​determined for other parameter values). Alternatively, and also as referenced above... Figures 1 to 4 The parameter adjustment system 110 selects a parameter value from a variety of parameter values ​​that satisfies (e.g., reaches or exceeds) a threshold value for the parameter.

[0110] In some implementations, the plurality of iterations are determined based on a metric from the current iteration that satisfies (e.g., reaches or exceeds) a threshold or the maximum number of iterations reached.

[0111] The parameter adjustment system 110 configures the content platform to use specific parameter values ​​(as determined at 518) to control or select digital components that provide video content during production (at 520).

[0112] Figure 6 This is a block diagram of an example computer system 600 that can be used to perform the operations described above. System 600 includes a processor 610, memory 620, storage device 630, and input / output device 640. Each of components 610, 620, 630, and 640 may be interconnected, for example, using a system bus 650. Processor 610 is capable of processing instructions for execution within system 600. In some embodiments, processor 610 is a single-threaded processor. In another embodiment, processor 610 is a multi-threaded processor. Processor 610 is capable of processing instructions stored in memory 620 or stored on storage device 630.

[0113] Memory 620 stores information within system 600. In one embodiment, memory 620 is a computer-readable medium. In some embodiments, memory 620 is a volatile memory cell. In another embodiment, memory 620 is a non-volatile memory cell.

[0114] Storage device 630 provides high-capacity storage for system 600. In some embodiments, storage device 630 is a computer-readable medium. In various embodiments, storage device 630 may include, for example, a hard disk drive, an optical disk drive, a storage device shared over a network by multiple computing devices (e.g., cloud storage devices), or some other high-capacity storage device.

[0115] Input / output device 640 provides input / output operations for system 600. In some embodiments, input / output device 640 may include one or more of a network interface device (e.g., an Ethernet card), a serial communication device (e.g., an RS-232 port), and / or a wireless interface device (e.g., an 802.11 card). In another embodiment, input / output device may include a driver device configured to receive input data and send output data to peripheral device 660 (e.g., a keyboard, printer, and display device). However, other embodiments, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc., may also be used.

[0116] Although Figure 6 An example processing system is described herein, but implementations of the subjects and functional operations described herein may be carried out in other types of digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed herein and their equivalents), or in a combination of one or more of them.

[0117] Embodiments of the subject matter and operation described in this specification may be implemented in digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their equivalents), or in combinations thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium (or media) for execution by or control of the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on artificially generated propagating signals (e.g., machine-generated electrical, optical, or electromagnetic signals) that are generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof, or may be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof. Furthermore, although the computer storage medium is not a propagating signal, it may be a source or destination of computer program instructions encoded in artificially generated propagating signals. Computer storage media may also be one or more separate physical components or media (e.g., multiple CDs, discs or other storage devices), or may be included in one or more separate physical components or media (e.g., multiple CDs, discs or other storage devices).

[0118] The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0119] The term "data processing apparatus" includes all kinds of devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, systems-on-a-chip, or a combination of the foregoing. The apparatus may include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement a variety of different computing model infrastructures, such as network services, distributed computing, and grid computing infrastructures.

[0120] Computer programs (also referred to as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, objects, or other units suitable for a computing environment. A computer program may, but does not necessarily, correspond to a file in a file system. A program may be stored as a portion of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to the program in question, or multiple co-located files (e.g., a file storing one or more modules, subroutines, or code sections). Computer programs can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communication network.

[0121] The processes and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform actions by manipulating input data and producing output. The processes and logic flows can also be executed by special-purpose logic circuitry, and the apparatus can be implemented as special-purpose logic circuitry, such as FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit).

[0122] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for performing actions according to instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data (e.g., magnetic disks, magneto-optical disks, or optical disks), or be operatively coupled to receive data from or transfer data to one or more mass storage devices for storing data, or both. However, a computer does not necessarily have to have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example: semiconductor storage devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into dedicated logic circuitry.

[0123] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and pointing device (e.g., a mouse or trackball) that the user can use to provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0124] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer with a graphical user interface or web browser through which a user can interact with embodiments of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected via any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0125] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship arises from computer programs running on separate computers that have a client-server relationship with each other. In some embodiments, the server sends data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with the client device and to receive user input from the user interacting with the client device). Data generated at the client device (e.g., the result of user interaction) may be received at the server from the client device.

[0126] While this specification contains numerous specific implementation details, these details should not be construed as limiting any invention or potentially claimed scope, but rather as descriptions of features characteristic of particular embodiments of a particular invention. Certain features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, one or more features from a claimed combination may, in some cases, be removed from said combination, and the claimed combination may be for sub-combinations or variations thereof.

[0127] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential manner, or to perform all the operations shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0128] Therefore, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequence shown to achieve the desired result. In some embodiments, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method, comprising: Multiple iterations are performed to identify parameter values, and the content platform controls the provision of digital components with video content based on these parameter values, wherein each of the multiple iterations includes: Identify a set of evaluation points for the parameter, wherein each evaluation point includes an evaluation parameter value for the parameter and a metric value corresponding to a metric provided by the content platform for a digital component, wherein the metric value for the evaluation point is determined based on data generated by the content platform using the evaluation parameter value of the evaluation point to provide a digital component; The first model is generated using the evaluation point set; Generate the mean and confidence interval of the first model; A second model is generated based on the first model and the acquisition function, wherein the acquisition function is based on the average value of the first model, the confidence interval of the first model, and configurable exploration weights that control the priority of exploration for evaluating the parameters; Determine the next parameter value to be evaluated from the second model; Configure the content platform to use the next parameter value to provide digital components with the video content; and The next metric is determined based on data generated by the digital components provided by the content platform using the next parameter value; and From the parameter values ​​and corresponding metric values ​​of the parameters determined during the plurality of iterations, determine the specific parameter value that leads to the highest metric value or satisfies a specific threshold; The specific parameter values ​​are used to configure the content platform to control or select digital components that provide the video content during production.

2. The computer-implemented method of claim 1, further comprising determining from the second model at least one other parameter value to be evaluated, in addition to the next parameter value.

3. The computer-implemented method according to claim 1 or 2, wherein generating the first model includes generating the first model as a model for fitting the current evaluation point set.

4. The computer-implemented method according to any one of claims 1 to 3, wherein when the next parameter value to be evaluated is determined from the second model, the exploration weight controls the priority of exploring new parameter values.

5. The computer-implemented method according to claim 4, wherein the exploration weights correspond to the weights of the confidence intervals in the acquisition function.

6. The computer-implemented method according to any one of claims 1 to 5, wherein the exploration weight is higher in early parameter evaluation iterations and lower in late parameter evaluation iterations.

7. The computer-implemented method according to any one of claims 1 to 6, wherein each evaluation point in the initial evaluation point set includes randomly generated parameter values.

8. The computer-implemented method according to any one of claims 1 to 7, wherein determining the next parameter value to be evaluated from the second model includes determining a parameter value having a corresponding highest acquisition function value.

9. A computer-implemented system, comprising: One or more storage devices for storing instructions; as well as One or more data processing devices are configured to interact with the one or more storage devices and, when executing the instructions, perform the operation of the method according to any one of claims 1 to 8.

10. A computer-readable medium storing instructions that, when executed by one or more data processing devices, cause the one or more data processing devices to perform operations according to any one of claims 1 to 8.

Images