Choosing a Content Delivery Network with Agents

Abstract

To provide a method for assigning a content delivery network that may most likely result in a higher quality of service to a playback session for a video.SOLUTION: A method includes determining an input value for a request for content and retrieving a plurality of states for a plurality of content delivery networks, the state including a component that is based on variance. A plurality of scores for a plurality of content delivery networks is generated based on the input value and the plurality of states. A score for a content delivery network is generated using a parameter value for the component and the input value. The method also includes selecting a content delivery network from the plurality of content delivery networks based on the score for the content delivery network and outputs information on the selected content delivery network to cause the selected content delivery network to service the request to deliver the content.SELECTED DRAWING: Figure 2

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001]

[0001] Pursuant to 35 U.S.C. § 119(a), this application is entitled to and claims the benefit of the filing date of Chinese Patent Application No. 202211632854.2, filed December 19, 2022, entitled "Selection of Content Delivery Networks Using Agents," the contents of which are incorporated herein by reference in their entirety for all purposes. [Background technology]

[0002]

[0002] A video distribution system may use multiple content delivery networks (CDNs) to deliver videos to clients. When a client requests a video, the video distribution system may select one of the content delivery networks to handle the request during a playback session. Upon selecting a content delivery network, the client communicates with the content delivery network to stream the video.

[0003]

[0003] Different content delivery networks may provide different quality of service (QoS) to different clients. Therefore, the content delivery network assigned to a client may affect the playback experience during a playback session. For example, a first content delivery network may provide a first quality of service, and a second content delivery network may provide a second quality of service to a client for playing a video. When the first quality of service playback experience and the second quality of service playback experience differ, it may be desirable for the video delivery system to assign to the playback session for the video a content delivery network that may be most likely to provide a higher quality of service. However, using random selection, the video delivery system may optionally not assign a content delivery network to a request for a video.

[0004]

[0004] The included drawings are for illustrative purposes and serve only to provide examples of possible structures and operations for the disclosed inventive systems, apparatus, methods, and computer program products. These drawings in no way limit any changes in form and detail that may be made by those skilled in the art without departing from the spirit and scope of the disclosed implementations. [Brief explanation of the drawings]

[0005] [Figure 1]

[0005] FIG. 1 illustrates a simplified system for selecting a content delivery network, according to some embodiments. [Figure 2]

[0006] 1 illustrates a more detailed example of a CDN selection system, according to some embodiments. [Figure 3]

[0007] 1 illustrates a simplified flowchart of a method for performing QoS prediction, according to some embodiments. [Figure 4]

[0008] 1 illustrates a simplified flowchart of a method for selecting a content delivery network using an allocation method, according to some embodiments. [Figure 5]

[0009] 1 illustrates a graph illustrating upper confidence limits, according to some embodiments. [Figure 6]

[0010] 1 illustrates a simplified flowchart of a method for performing content delivery network selection using a confidence upper bound, according to some embodiments. [Figure 7]

[0011] 1 illustrates an example of a state computed using an agent, according to some embodiments. [Figure 8]

[0012] 1 illustrates a video streaming system in communication with multiple client devices over one or more communication networks, according to one embodiment. [Figure 9]

[0013] 1 illustrates a diagram of a device for viewing video content and advertisements. DETAILED DESCRIPTION OF THE INVENTION

[0006]

[0014] Techniques for a video distribution system are described herein. In the following description, for purposes of explanation, numerous examples and specific details are set forth to provide a thorough understanding of some embodiments. Some embodiments, as defined by the claims, may include some or all of the features in these examples, alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

[0007]

[0015] As discussed above, a video delivery system may use multiple content delivery networks (CDNs) to deliver content to clients. When a client requests an instance of media content, the video delivery system may select one of the content delivery networks to process the request during a playback session. The content delivery network (CDN) selection system may include a prediction engine and an allocation engine. The prediction engine may predict a performance metric. The performance metric may be based on the delivery of the media content to the client, but may also be a metric for other applications, such as the performance of a profile in an adaptive bitrate algorithm. The metric may be quality of service (QoS) performance, although other metrics may be used. In some embodiments, the CDN selection system may predict QoS performance scores for multiple content delivery networks that can deliver the media content to the client. After predicting the QoS performance scores, the allocation engine may select a content delivery network from the available content delivery networks based on the predicted QoS performance scores for each content delivery network, such as by selecting the content delivery network with the top-ranked (e.g., highest) QoS performance score.

[0008]

[0016] Traditionally, predicting QoS performance can be modeled as a machine learning problem. For example, features to be input into a machine learning model can be selected, historical data for the features can be collected, and a large amount of historical data can be used to train the model to predict QoS performance. One drawback of using a machine learning model is that the machine learning model predicts QoS performance expectations to leverage historical observations. The expectations can be an attempt to maximize performance based on historical data. This means that a content delivery network that experiences poor QoS performance can have a poor QoS performance score predicted by the machine learning model. If there are few QoS historical observations for each content delivery network, the average QoS performance can be at high risk of being insufficient, and the content delivery network may not be selected frequently by the allocation engine (e.g., if the allocation engine is biased to select content delivery networks with higher scores). However, the content delivery network may actually be capable of providing good QoS performance over time if selected. However, the content delivery network may not be selected due to a poor QoS performance score generated based on a small number of historical observations of poor QoS performance.

[0009]

[0017] Some embodiments provide advantages over conventional methods by using an agent to perform QoS performance prediction. The agent may include parameters for measuring exploitation and exploration. The exploitation may be expected performance (e.g., mean), and the exploration may be potential, which may be based on the variance of a random variable. The QoS performance score using exploitation and exploration may be an upper confidence bound (UCB). The confidence bound may be an upper bound based on the expected performance and exploration of potential, which is described in more detail below. Thus, the agent may generate a prediction based on exploitation of historical data and also the potential for the content delivery network to deliver the predicted performance. This provides a more accurate prediction of QoS performance because the QoS prediction may compensate for a small number of historical observations, which may allow a content delivery network to still be selected to explore its potential performance even when a small number of observations may be received.

[0010]

[0018] Additionally, machine learning model methods may suffer from generalizability shortcomings. That is, QoS performance may change over time due to various factors, leading to degradation of the machine learning model's output when conditions change and the model does not adapt. To update the machine learning model's output, the machine learning model needs to be retrained using a significant amount of data. Updating a machine learning model can take time because the machine learning model's parameters need to be retrained using data, and this retraining cannot be performed in real time, such as after the session for the request has ended (e.g., the video broadcast has ended). Rather, the machine learning model may have to be taken offline, retrained, and then reinstalled with new parameter values. Therefore, the machine learning model may not accurately predict QoS performance as data changes over time.

[0011]

[0019] In some embodiments, the agents can be updated in real time using the results of allocations for current requests. For example, a request for media content from a client may be received, and QoS performance scores for multiple content delivery networks may be predicted. The allocation engine may then select a content delivery network to service the request. The delivery of the content for the request may then be observed. QoS performance may be measured, for example, performance of playback of the content may be observed, the delivery of the content may be observed for any rebuffering instances that occur, etc. Feedback from the delivery of the content may then be used to tune the agents in real time. For example, the CDN selection system uses the feedback to adjust one or more parameter values ​​of the agents used to perform the QoS performance score prediction. In some examples, the CDN selection system may adjust parameters of the respective agents for the selected content delivery network based on whether rebuffering occurred. In some embodiments, the CDN selection system may perform updates in real time because updates may be lightweight, such as requiring one or more operations, such as one or more addition operations, to update the parameters of the agents, as opposed to having to perform retraining of a machine learning model.

[0012]

[0020] System Overview

[0013]

[0021] FIG. 1 illustrates a simplified system 100 for selecting a content delivery network 106, according to some embodiments. The system 100 includes content delivery networks (CDNs) 106-1 through 106-N, clients 102, an origin server 104, and a server system 108. A content provider operates the server system 108 to provide a video distribution service that allows entities to request and receive media content from a library of media content, such as on-demand and live video. The content provider may use the server system 108 to coordinate the distribution of media content to the client 102. The media content may include different types of content, such as video, audio, or other types of content information that can be rendered on a computing device. Video may be used for purposes of discussion, although other types of content may be recognized. An instance of the media content may be the instance requested for a request. For example, a first instance of media content may be for a first show, a second instance of media content may be for a second show, a third instance may be for the first show, etc.

[0014]

[0022] The content delivery network 106 may distribute instances of media content for a video distribution service. The content delivery network 106 may include one or more computing devices, such as servers, that distribute instances of media content to clients 102. A content provider may operate its own content delivery network in addition to contracting with multiple third-party content delivery networks to assist in the distribution of its media content to clients 102. While an instance of a client 102 is shown, the content delivery network 106 may distribute media content to one or more instances of a client 102. The content delivery network 106 may also include multiple computing devices, such as edge nodes, which may be geographically distributed. An edge node may receive segments of media content from an origin server 104 and distribute the segments to different clients 102. Any server associated with the content delivery network 106 may perform the functions described herein. Also, while a server system 108 and a content delivery network 106 are described, the functions performed by either may be distributed between the two entities or performed by other entities.

[0015]

[0023] The client 102 may include a computing device such as a smartphone, a set-top box, a tablet device, a living room device, a game console, etc. The client 102 includes a media player 114 capable of playing content. The client 102 may use one of the content delivery networks 106, such as content delivery network #2, to begin playing an instance of the media content on the media player 114.

[0016]

[0024] The server system 108 may process requests for instances of media content from the clients 102. Although not shown, the server system 108 may be processing requests from multiple clients 102, both sequentially and simultaneously. For example, the clients 102 may be using a video streaming service to request videos simultaneously or within a period of time.

[0017]

[0025] The origin server 104 may store representations of the media content in the storage device 112. The representations may include different playback characteristics for the content, such as different representations of the media content encoded at different bit rates or qualities (e.g., resolutions). For example, video may be encoded at different bit rates. In some examples, the client 102 may be able to request an instance of the media content from each content delivery network 106, and each content delivery network 106 may be able to independently respond with a representation of the instance of the media content. For example, each content delivery network 106 may include one or more servers that can request representations of the media content from the origin server 104 and receive representations of the media content from the origin server 104. Each content delivery network 106 can then deliver the instances of the media content to the client 102.

[0018]

[0026] The server system 108 is configured to select one of the content delivery networks 106 to process a request for an instance of the media content. A content delivery network (CDN) selection system 110 may select a content delivery network for the request and provide selection data representing the selection of the content delivery network 106 to the client 102. The selection data may be provided to the selected content delivery network 106 either in addition to or instead of being provided to the client 102. The client 102 may then interact with the selected content delivery network 106, such as content delivery network #2 106-2, to request a segment of the instance of the media content. Content delivery network #2 106-2 may then deliver the segment to the client 102, which displays the segment on the media player 114.

[0019]

[0027] The clients 102 may experience different playback conditions during playback of an instance of media content. For example, the playback conditions may vary across content delivery networks 106. This may result in the instance of media content being delivered to the clients 102 with different values ​​of a performance metric. The performance metric may be referred to as quality of service, which may be a measurement of the performance of the service delivering the instance of media content from the content delivery network 106 to the client 102. Some metrics that may be used to measure the quality of service performance metric may relate to network performance, such as connection-induced rebuffering, packet loss, bit rate, throughput, transmission delay, network availability, jitter, etc. The performance metric may be based on one or more metrics.

[0020]

[0028] The CDN selection system 110 may select a content delivery network 106 for the request based on a selection process. For example, the CDN selection system 110 may use an analysis of performance metrics for the content delivery networks 106 to select a content delivery network 106 for the request. For example, the CDN selection system 110 may predict a QoS performance score for the content delivery network 106. In some embodiments, the QoS performance score may measure the likelihood of rebuffering. For example, connection-induced rebuffering may be used. Connection-induced rebuffering may reflect network conditions between the client device 102 and the respective content delivery network 106. That is, connection-induced rebuffering experienced due to a connection may indicate a problem with the network connection, such as connection-induced rebuffering due to network delivery of the content. This may differ from rebuffering that may occur due to a lack of computer processing resources on the client 102. Connection-induced rebuffering may be measured based on the amount of data received and stored in a buffer. If not enough data is received to support playback of the content, the client 102 may determine that connection-induced rebuffering has occurred. Another rebuffer may occur when there is enough data in the buffer to support playback, and the client 102 may not consider this a connection-induced rebuffer. In other embodiments, rebuffer occurrences may simply be measured, since most rebuffers may be caused by network problems. In general, a higher predicted QoS performance score may indicate a lower likelihood of experiencing rebuffers.

[0021]

[0029] In some embodiments, the CDN selection system 110 may perform the selection in real time upon receiving a request for an instance of media content. That is, the CDN selection system 110 receives a request for an instance of media content (e.g., a program) and selects one of the content delivery networks #1-#N. The CDN selection system 110 may then send an identification of the selected content delivery network 106 to the client 102, which may then request the instance of the media content from the selected content delivery network 106. Other methods for communicating the selection of a content delivery network 106 in response to a request may also be recognized. For example, the CDN selection system 110 may send a request to the selected content delivery network 106, which may then communicate with the client 102. The selection process is described in more detail below, beginning with FIG. 2.

[0022]

[0030] Upon delivery of an instance of media content from a selected content delivery network 106 to a client 102, performance metrics may be measured. Measurement of the performance metrics may be performed using different methods, such as the client 102 sending information used to measure the performance metrics, the content delivery network 106 sending information, etc. For example, if connection-induced rebuffering is used, measurements may be made to determine whether one or more rebuffers occur, or a ratio of the number of rebuffers based on the amount of data delivered may be used. As described above, rebuffering may be caused by different factors. Connection-induced rebuffering may be measured. In other embodiments, rebuffering occurrence may simply be measured, since most rebuffering may be caused by network issues. While measuring rebuffering occurrence is described, other factors, such as video startup time or throughput, may also be measured. Feedback from the performance metrics may be used to select a content delivery network 106 for subsequent requests for media content from this client 102 or other clients 102 after the feedback is received. For example, the CDN selection system 110 may use measured performance metrics to update parameter values ​​of agents, which are described in more detail below, starting with FIG.

[0023]

[0031] The following describes the CDN selection system 110 in more detail, followed by a description of the QoS prediction and allocation processes.

[0024]

[0032] CDN Selection System

[0025]

[0033] 2 illustrates a more detailed example of a CDN selection system 110, according to some embodiments. The CDN selection system 110 receives a request for an instance of media content from a client 102. Although not shown, the CDN selection system 110 may receive multiple requests for instances of media content from multiple clients 102. The requests may be for different media content or the same media content. For example, a first request may be for a first program, a second request may be for a second program, and a third request may be for the first program.

[0026]

[0034] The prediction engine 202 receives input for the request and outputs a QoS performance score for the content delivery network. In some embodiments, features may be selected as input to the prediction engine 202. The input for the request includes values ​​for features associated with characteristics of the request. In some embodiments, the following features from Table I may be used, although other features may also be recognized:

[0027] [Table 1]

[0028]

[0035] In Table I, the features may include device platform, region, city, internet service provider, specified trade area, network type, geographic region, time of day, etc. The features may be categorical features, and the number of category values ​​may vary. For example, for the city feature, there may be 20,000 cities in the United States and 5,000 cities in Canada. The value associated with the feature may be determined based on the received request. For example, if a request is received from client device 102 and client device 102 is in Santa Monica, the GeoIP value may be associated with Santa Monica or a zip code for Santa Monica. Similarly, if the time is 2:00 PM, that time is entered for the time feature.

[0029]

[0036] The prediction engine 202 may generate predictions for content delivery networks. For example, there may be three content delivery networks that may be able to service the request. The prediction engine 202 may then generate a QoS performance score for each content delivery network. For example, predicted scores for QoS performance may be [0.43, 0.55, and 0.54]. The scores may be ranked such that higher scores are ranked higher. In some embodiments, a higher predicted QoS performance score may indicate that the respective content delivery network may deliver better QoS performance to the request. For example, a higher value may indicate a lower likelihood of experiencing rebuffering during video playback. In other embodiments, the score value may indicate other likelihoods, such as a lower score indicating a lower likelihood of experiencing rebuffering. In this case, a lower score is ranked higher.

[0030]

[0037] After outputting the QoS performance values, the allocation engine 204 may select a content delivery network to service the request. The allocation engine 204 may use different methods to select a content delivery network. In some embodiments, the allocation engine 204 may select the content delivery network with the highest predicted QoS performance score. For example, the allocation engine selects content delivery network #2 because its QoS performance score of 0.55 is the highest among three values ​​[0.43, 0.55, and 0.54]. However, other methods, such as weighted random selection, functions, etc., may be used. Weighted random selection may weight content delivery networks using scores, but may also randomly select content delivery networks using weights. However, selecting the highest-ranked content delivery network may improve performance because the content delivery network least likely to experience poor performance, such as rebuffering, is selected. In weighted random selection, a content delivery network may be selected based on a random variable, but a more highly weighted content delivery network may have a greater chance of being selected. After selecting a content delivery network, the allocation engine 204 outputs the selected content delivery network.

[0031]

[0038] The CDN selection system 110 may receive feedback from the delivery of media content to the client device 102. As discussed above, different feedback metrics may be used and are discussed below in FIG.

[0032]

[0039] The prediction process will now be explained in more detail.

[0033]

[0040] Forecasting Process

[0034]

[0041] The prediction engine 202 uses agents to generate values ​​for QoS performance. The agents may receive values ​​for features associated with the request and output a QoS performance score for the content delivery network. In some embodiments, each content delivery network may be associated with an agent. That is, each agent for a content delivery network may include different parameter values ​​based on the performance of the respective content delivery network. However, the agents for each content delivery network may be implemented in different ways. For example, a single agent may include different portions that generate a QoS performance score for each content delivery network.

[0035]

[0042] 3 illustrates a simplified flowchart 300 of a method for performing QoS prediction, according to some embodiments. At 302, CDN selection system 110 receives a request. The request may be for a video to be played in a playback session. At 304, CDN selection system 110 receives values ​​for features for the request. As discussed above, the values ​​for the features may be based on information associated with the request, such as information based on the client that sent the request, the network type being used by the client, and the context of the playback session.

[0036]

[0043] At 306, the CDN selection system 110 processes the values ​​for the features. Processing the values ​​may include using agents to generate QoS performance scores based on the values. For example, at 308, agent #1 uses the features to generate a prediction for content delivery network #1. Similarly, at 310, agent #2 uses the features to generate a prediction for content delivery network #2, and at 312, agent #N uses the features to generate a prediction for content delivery network #N. That is, predictions are generated for each of the N content delivery networks using the same feature values. As discussed above, each respective agent for a content delivery network may have different values ​​for the parameters. Therefore, each respective agent may generate a prediction for each content delivery network based on the respective parameter values ​​and the same feature values. This may result in predictions that may have different values ​​or may have similar values ​​if the agents include the same parameter values. An example of values ​​generated by the agents may include [0.43, 0.55, and 0.54] for content delivery networks #1, #2, and #N, respectively.

[0037]

[0044] After performing the prediction, the process then proceeds to perform the allocation process.

[0038]

[0045] allocation

[0039]

[0046] 4 illustrates a simplified flowchart 400 of a method for selecting a content delivery network using an allocation method, according to some embodiments. At 402, the CDN selection system 110 receives QoS performance scores for the content delivery networks. For example, the CDN selection system 110 receives the QoS performance scores for each content delivery network. Then, at 404, the CDN selection system 110 selects a content delivery network based on the prediction. For example, using a selection process, the CDN selection system 110 may select one of the content delivery networks based on its respective QoS performance score. As discussed above, if the three scores are [0.43, 0.55, and 0.54], the CDN selection system 110 may select content delivery network #2, which has the highest QoS performance score of 0.55.

[0040]

[0047] At 406, the CDN selection system 110 may initiate a session with the selected content delivery network. For example, if content delivery network #2 is selected, information is sent to the client device 102 using the selected content delivery network. The client device 102 may then initiate a playback session to play the requested content using content delivery network #2.

[0041]

[0048] At 408, the CDN selection system 110 receives feedback about performance metrics of the selected content delivery network during the session. For example, the CDN selection system 110 may receive information about the delivery of content using content delivery network #2. The feedback may be the same metrics predicted as QoS performance, such as whether rebuffering occurred. However, other information may also be received, such as the number of rebuffers that occurred, playback start time, etc. The CDN selection system 110 may then use the feedback to update the agent for the content delivery network at 410. For example, the CDN selection system 110 updates one or more parameter values ​​for agent #2 of content delivery network #2. In some embodiments, if no rebuffering is experienced, the parameters may be updated to predict higher QoS performance in the next session, or if rebuffering is experienced, the parameters may be updated to predict lower QoS performance.

[0042]

[0049] The following describes the agent in more detail. Note that the agent may use different methods to generate predictions. The following method is used, although other methods may be recognized:

[0043]

[0050] agent

[0044]

[0051] In some embodiments, the agent may generate a QoS prediction score using an upper confidence limit. The upper confidence limit may define the point below which a certain percentage of the distribution lies. For a random variable, if there is a 95% chance that the value is less than 10, then the upper confidence limit at a 95% confidence level is 10. FIG. 5 illustrates a graph illustrating an upper confidence limit, according to some embodiments. Graph 500 illustrates the difference between the mean and the upper confidence limit. Curve 502 illustrates the distribution of random variable A, and curve 504 illustrates the distribution of random variable B. The two curves share the same mean but have different variances. The mean is the average of the values. As for variance, the distribution of values ​​may vary differently. As is typically observed, curve 502 includes more of the area between values ​​2 and 4, while curve 504 includes a wider range of values ​​between values ​​1 and 5.

[0045]

[0052] At 506 and 508, the means of curves 502 and 504, respectively, are the same value, "3." At 510, an upper confidence limit for curve 502 is shown, and at 512, an upper confidence limit for curve 504 is shown. Variable B has a larger variance in curve 504 and therefore includes a larger upper confidence limit; that is, values ​​distributed below curve 504 vary more than values ​​below curve 502.

[0046]

[0053] A traditional machine learning model may predict the mean value of a random variable, in which case the machine learning model may predict the same value for both curves. If the mean value represents poor QoS performance for a content delivery network and there are few QoS historical observations for the content delivery network, the content delivery network may not be selected frequently by the allocation engine. However, the content delivery network may be capable of providing good performance if selected. In contrast, a confidence upper bound may reflect both the mean value and the variance. The mean value may be associated with the exploitation of the variable, and the variance may be associated with the exploration of the variable. Utilizing both the mean value and the variance may form a prediction of the expectation and potential of the variable. In FIG. 5 , variable B with its curve 504 has a higher confidence upper bound 512, which may reflect that its curve 504 has a higher potential to deliver better performance than variable A with its curve 502 and confidence upper bound 510.

[0047]

[0054] 6 illustrates a simplified flowchart 600 of a method for performing content delivery network selection using a confidence upper bound, according to some embodiments. At 602, the CDN selection system 110 initializes parameter values ​​for each agent for the content delivery network 106. In some embodiments, the agent calculates a matrix A∈R d×d and vector b∈R d where R is a set of real numbers and d illustrates the dimension of the embedding in the embedding space. The parameters are responsible for storing information about the historical observations, where matrix A records the features and vector b records the features and observations in different ways. Under the assumption that the confidence upper bound is linear in its features, the agent can predict the confidence upper bound using different methods such as ridge regression. The agent can then calculate the confidence upper bound using:

[0048]

number

[0049] Here, the first component represents exploitation by predicting expectation (e.g., mean value), the second component represents exploration (e.g., variance), and α is a hyperparameter that controls the tradeoff between exploitation and exploration. s " represents the feature embedding for the session, "s" is the session, "T" illustrates the transpose of the matrix, and "-1" illustrates the inverse of the matrix. "c" indicates the content delivery network; for example, each of the upper confidence bounds is calculated separately for the content delivery network. In general, when the number of observations is small, the value of the second component may be higher. However, as the number of observations increases, the value of the second component may decrease. The values ​​for the parameters may be stored in a storage device as a state for the agent, and the agent uses the values ​​to generate the upper confidence bound score.

[0050]

[0055] At 604, the CDN selection system 110 generates embeddings from the features. For example, the embeddings may be representations of values ​​for the features in an embedding space. At 606, the CDN selection system 110 uses the embeddings and associated parameter values ​​for each respective agent to generate values ​​for confidence upper bounds for all content delivery networks. For example, the CDN selection system 110 retrieves values ​​for the parameters from storage and applies the embeddings to the values. The CDN selection system 110 may perform operations such as multiplication and addition to determine the confidence upper bounds. At 608, the CDN selection system 110 selects a content delivery network based on the confidence upper bound values.

[0051]

[0056] After selecting a content delivery network, at 610, the CDN selection system 110 determines which content delivery network to monitor for feedback. For the selected content delivery network, at 612, the CDN selection system 110 monitors the delivery of the content and generates a reward based on performance. In some embodiments, the CDN selection system 110 may determine a reward value based on feedback received from the delivery of the content. In some embodiments, if a rebuffer occurs during the session, the reward may be set as a first value, such as "0," or if no rebuffer is experienced, the reward may be set as a second value, such as "1." A value of "1" may be a more valuable, higher reward indicating that the session was successful compared to a value of "0." At 614, the CDN selection system 110 updates the agent based on the reward. For example, the update may involve an operation such as addition. For example, the CDN selection system 110 performs two matrix additions to update the agent's parameter values ​​and stores the new values ​​in storage. Compared to retraining the parameters for a machine learning model, updates are faster and lighter weight because additions can be performed in real time without the need to retrain the model. In some embodiments, agents are updated as follows:

[0052]

number

[0053] That is, the session embedding is used to update matrix A, which is a matrix of all historical sessions that record feature information.

[0054]

number

[0055] The vector b is updated using the reward and the embedding of a particular session. The vector b is a sum of the e for positive history sessions that records the information of the positive session features.s For example, the reward value multiplied by the embedding is the vector b i , so a positive reward is added to the vector b i A reward value of 0 will make the second term a value of 0, so a negative reward will increase the value of the vector b i The values ​​for .times. ...

[0056]

[0057] The following describes an example illustrating the generation of an upper confidence limit score.

[0057]

[0058] example

[0058]

[0059] FIG. 7 illustrates an example of a state calculated using an agent, according to some embodiments. The CDN selection system 110 is selecting from three content delivery networks, which may be named content delivery network #1, content delivery network #2, and content delivery network #3. Three agents, Agent #1, Agent #2, and Agent #3, may be initialized. The parameters of each Agent #i may be A#i and B#i. For each session, two features, region and DMA, may be used, although other features may also be recognized. The hyperparameter α is equal to 0.5, which may equally balance the two components of exploitation and exploration. If either the exploitation or exploration component should be more prioritized, the value of the hyperparameter may be changed. The feature values ​​are encoded as embeddings for the agents. Once the predicted QoS values ​​for the confidence upper bounds are output, the CDN selection system 110 may select the maximum value as the content delivery network to service the request. After delivering the content to the client 102, feedback about the playback session is received and the CDN selection system 110 updates the stored values ​​for each agent associated with the selected content delivery network.

[0059]

[0060] The following describes the state of values ​​for the agent's parameters across multiple sessions. Table 700-1 shows the initial state of the parameters. As described above, parameter A is a matrix, and parameter b is a vector. These parameters store information reflecting historical observations when a content delivery network was selected. The parameter values ​​for content delivery network #1 are shown as 702-1, the parameter values ​​for content delivery network #2 are shown as 702-2, and the parameter values ​​for content delivery network #3 are shown as 702-3. The matrix for parameter value A is a 2x2 matrix with values ​​"1 0" in the first row and values ​​"0 1" in the second row. The matrix may be a 2x2 identity matrix, which contains values ​​"1" on the main diagonal and values ​​"zero" elsewhere, as shown. The value for vector b is 0 0. The vector may be a two-dimensional vector. The initial states of A and b are both empty states. The sizes of A and b correspond to the size of the embedding and may be different if more features are used instead of two.

[0060]

[0061] For session #1, the embedding may be [-1, 0], which represents a value of -1 for region and a value of 0 for DMA. Agents #1, #2, and #3 each calculate the confidence upper bound as [0.5, 0.5, 0.5]. In this case, the CDN selection system 110 may select content delivery network #1 even though the values ​​for the confidence upper bound for each content delivery network are the same. In other examples, content delivery network #2 or #3 may also be selected. In some embodiments, the CDN selection system 110 may use a tiebreaker when the highest values ​​for the confidence upper bound are the same. Different methods for tiebreaking may be recognized, such as random selection, selecting a content delivery network that is selected less than other content delivery networks, selecting a content delivery network based on the cost of delivering the content, etc. After selecting content delivery network #1, a playback session for content delivery network #1 is observed, and rebuffering occurs. In this case, the reward is equal to 0. Table 700-2 shows:

[0061]

number

[0062] 7 shows the updated state of values ​​after session #1, as defined by: In 704 for agent #1, the value of matrix A in the first row is changed from 0010 to 0020 to reflect the selection of content delivery network #1. The value for vector b is not changed because there was no rebuffering. The values ​​for other agents #2 and #3 are not changed because no associated content delivery network for the other agents was selected. However, in other examples, the values ​​for these agents may be adjusted based on feedback for agent #1. For example, the values ​​for agents #1 and #2 may be adjusted to reflect that a more positive prediction of QoS performance should be generated due to the rebuffering occurring in content delivery network #1.

[0063] In session #2, the embedding is [1, 0.5]. Agents #1, #2, and #3 calculate the confidence upper bound as [0.43, 0.55, 0.55]. The CDN selection system 110 selects content delivery network #2. Although content delivery network #2 and content delivery network #3 have the same predicted confidence upper bound, content delivery network #2 is selected. During content delivery using content delivery network #2, there is no rebuffering during the session. Therefore, the reward value is equal to 1. In table 700-3, in 706, matrix A#2 is updated to the indicated value, and in 708, vector b#2 is updated to the indicated value. For example, in matrix A#2, the first row is updated from

[0010] to [2.01 0.35] and the second row is updated from

[0001] to [0.35 3.5] to reflect that content delivery network #2 has been selected. Vector b#2 is updated from

[0000] to [1 0.5] to show the effect of positive rewards.

[0064] For session #3, the embedding is [-0.1, 1.5] and the upper confidence limit is [0.750, 1.0, 0.751]. The CDN selection system 110 selects content delivery network #2 because it has the highest upper confidence limit score of 1.0. After observing the playback session for content delivery network #2, there is a rebuffer and the reward is equal to 0. In table 700-4, in 710, matrix A#2 is updated to the values ​​shown to reflect the selection of content delivery network #2, and vector b#2 remains the same because the reward was 0. For example, matrix A#2 has its first row updated from [1.01 0.35] to [2 0.5] and its second row updated from [0.35 3.5] to [0.5 0.125].

[0065] In the above example, when the hyperparameter alpha is set to 0.5, when session #3 occurs, the upper confidence bound is the sum of two parts: UCB#1=0+0.5*1.501 UCB#2=0.28+0.5*1.4 UCB#3=0+0.5*1.503

[0066] The first component reflects utilization, and the second component reflects variance. Agent #2 has twice the number of observations and updates, which leads to the lowest variance of 1.4 (e.g., the second component is the variance of the hyperparameter of 0.5 * 1.4). Agent #3 was not selected and therefore received no observations, suggesting a greater potential for exploring value. This leads to the highest variance of the group at 1.503. Agent #1 has one observation with a rebuffer, and Agent #2 has one observation with a rebuffer and one other observation without a rebuffer. From the selected historical observations, Agent #2 has a larger expected value of 0.28 compared to the expected value of 0 for Agent #1. If the mean were simply used without the variance, the rebuffering experienced by Content Delivery Network #1 could reduce its chances of being selected for subsequent requests, as negative observations could cause poor QoS performance scores to be predicted. However, there may still be the potential for content delivery network #1 to provide acceptable performance (e.g., without rebuffering). The variance of 1.501 may still allow the potential performance of content delivery network #1 to be explored.

[0067]

[0066] Conclusion

[0068] Some embodiments select a content delivery network based on predicted QoS performance. This can lead to better network connectivity because a request can be assigned a content delivery network predicted to be least likely to experience poor network conditions, such as rebuffering. Predictions are also calculated using an improved process by allowing feedback to be incorporated in real time after a session with a selected content delivery network is completed and the feedback is received. Updates can be performed faster compared to machine learning models.

[0069]

[0068] Compared with previous solutions, this solution can consider the performance difference between different content delivery networks using exploitation and exploration. This can improve the overall quality of service performance and user experience. By adjusting the hyperparameter α, the trade-off that balances exploration and exploitation can also be adjusted. Using a parameter that explores potential capabilities, the potential of a content delivery network may not be ignored when there are few observations for each content delivery network, or when there may be a small amount of bad observations for the content delivery network.

[0070]

[0069] System

[0071]

[0070] Features and aspects as disclosed herein may be implemented in conjunction with a video streaming system 800 in communication with multiple client devices over one or more communications networks, as shown in Figure 8. Aspects of the video streaming system 800 are described merely to provide an example application for enabling distribution and delivery of content prepared in accordance with the present disclosure. It should be appreciated that the present technology is not limited to streaming video applications and may be adapted to other applications and delivery mechanisms.

[0072] In one embodiment, a media program provider may include a library of media programs. For example, the media programs may be aggregated and provided through a site (e.g., a website), an application, or a browser. A user may access the media program provider's site or application and request a media program. The user may be limited to requesting only media programs provided by the media program provider.

[0073] In system 800, video data may be obtained from one or more sources, such as video source 810, for use as input to video content server 802. The input video data may comprise raw or edited frame-based video data in any suitable digital format, such as Moving Pictures Experts Group (MPEG)-1, MPEG-2, MPEG-4, VC-1, H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), or other formats. Alternatively, the video may be provided in a non-digital format and converted to a digital format using a scanner or transcoder. The input video data may comprise various types of video clips or programs, such as television episodes, movies, and other content produced as the primary content of interest to consumers. The video data may also include audio, or audio alone may be used.

[0074] The video streaming system 800 may include one or more computer servers or modules 802, 804, and 807 distributed across one or more computers. Each server 802, 804, 807 may include or be operatively coupled to one or more data stores 809, such as databases, indexes, files, or other data structures. The video content server 802 may access a data store (not shown) of various video segments. The video content server 802 may provide the video segments as directed by a user interface controller in communication with client devices. As used herein, a video segment refers to a distinct portion of frame-based video data, such as may be used in a streaming video session to view a television episode, a movie, a recorded live performance, or other video content.

[0075] In some embodiments, the video ad server 804 may access a data store of relatively short videos (e.g., 10-second, 30-second, or 60-second video ads) configured as advertisements for particular advertisers or messages. The ads may be offered for advertisers in exchange for some type of payment, or may comprise promotional messages, public service messages, or some other information for the system 800. The video ad server 804 may serve video ad segments as directed by a user interface controller (not shown).

[0076]

[0075] The video streaming system 800 may also include a CDN selection system 110.

[0077]

[0076] The video streaming system 800 may further include an integration and streaming component 807 that integrates video content and video advertisements into streaming video segments. For example, the streaming component 807 may be a content server or a streaming media server. A controller (not shown) may determine the selection or configuration of advertisements in the streaming video based on any suitable algorithm or process. The video streaming system 800 may include other modules or units not shown in FIG. 8, such as a management server, a commerce server, a network infrastructure, an advertisement selection engine, etc.

[0078] The video streaming system 800 may be connected to a data communications network 812. The data communications network 812 may comprise a local area network (LAN), a wide area network (WAN), such as the Internet, a telephone network, a wireless network 814 (e.g., a wireless cellular telecommunications network (WCS)), or some combination of these or similar networks.

[0079] One or more client devices 820 may be in communication with the video streaming system 800 via the data communications network 812, the wireless network 814, or another network. Such client devices may include, for example, one or more laptop computers 820-1, desktop computers 820-2, “smart” mobile phones 820-3, tablet devices 820-4, network-enabled televisions 820-5, or combinations thereof, via a router 818 for a LAN, a base station 817 for the wireless network 814, or some other connection. In operation, such client devices 820 may send or receive data or instructions to the system 800 in response to user input or other input received from a user input device. In response, the system 800 may provide video segments and metadata from the data store 809 to the client device 820 in response to a media program selection. The client device 820 may output video content from streaming video segments in a media player using a display screen, projector, or other video output device and receive user input for interacting with the video content.

[0080]

[0079] Distribution of audiovisual data can be accomplished in various ways, such as using streaming, from the streaming component 807 to remote client devices through computer networks, telecommunications networks, and combinations of such networks. In streaming, a content server continuously streams audiovisual data to a media player component running at least partially on the client device, and the media player component can play the audiovisual data simultaneously as it receives the streaming data from the server. Although streaming is discussed, other distribution methods can be used. The media player component can begin playing the video data immediately after receiving the first portion of data from the content provider. Traditional streaming techniques use a single provider that delivers a stream of data to a set of end users. Delivering a single stream to a large audience can require high bandwidth and processing power, and as the number of end users increases, the provider's required bandwidth can increase.

[0081] Streaming media can be delivered on-demand or live. Streaming allows for instant playback at any point in a file. End users can skip a media file to begin playback or change playback to any point in the media file. Thus, end users do not have to wait for a file to download incrementally. Typically, streaming media is delivered via dedicated devices that accept requests for video files from a small number of dedicated servers with high bandwidth capabilities and deliver only the amount of data needed to play the video, along with information about the format, bandwidth, and structure of those files, at the speed needed to play it. The streaming media server may also take into account the transmission bandwidth and capabilities of the media player on the destination client. The streaming component 807 can communicate with the client device 820 using control and data messages to adapt to changing network conditions as the video is played. These control messages can include commands to enable control functions such as fast-forwarding, fast-rewinding, pausing, or seeking to a specific part of the file at the client.

[0082]

[0081] Because the streaming component 807 transmits video data only as needed and at the rate needed, precise control over the number of streams provided can be maintained. A viewer would not be able to watch high-data-rate video over a lower-data-rate transmission medium. However, a streaming media server (1) provides users with random access to video files, (2) allows monitoring of who is watching what video programs and for how long, (3) uses transmission bandwidth more efficiently because only the amount of data needed to support the viewing experience is transmitted, and (4) video files are not stored on the viewer's computer but are discarded by the media player, thereby allowing more control over the content.

[0083] The streaming component 807 may use TCP-based protocols such as HTTP (HyperText Transfer Protocol) and RTMP (Real Time Messaging Protocol). The streaming component 807 can also deliver live webcasts and perform multicasting, which allows more than one client to tune into a single stream, thereby saving bandwidth. Streaming media players may not rely on buffering the entire video to provide random access to any one point in a media program. Instead, this is achieved using control messages sent from the media player to the streaming media server. Other protocols used for streaming are HLS (HTTP live streaming) or DASH (Dynamic Adaptive Streaming over HTTP). The HLS and DASH protocols deliver video over HTTP via a playlist of small segments, typically made available at various bitrates from one or more content delivery networks (CDNs). This allows the media player to switch both bitrate and content source on a segment-by-segment basis. Switching helps compensate for network bandwidth variations and infrastructure failures that may occur during video playback.

[0084]

[0083] Delivery of video content via streaming can be accomplished under a variety of models. In one model, users pay to view video programs, for example, by paying a fee for access to a library of media programs or a limited portion of a media program, or by using a pay-per-view service. In another model, widely adopted by television broadcasting shortly after its inception, sponsors pay for the presentation of media programs in exchange for the right to present advertisements during or adjacent to the presentation of the program. In some models, advertisements are inserted at predetermined times during the video program, which may be referred to as "ad slots" or "ad breaks." With streaming video, media players can be configured to prevent client devices from playing the video without also playing predetermined advertisements during designated ad slots.

[0085] 9, a diagram of an apparatus 900 for viewing video content and advertisements is illustrated. In selected embodiments, the apparatus 900 may include a processor (CPU) 902 operably coupled to a processor memory 904, which holds binary-coded functional modules for execution by the processor 902. Such functional modules may include an operating system 906 for handling system functions such as input / output and memory access, a browser 908 for displaying web pages, and a media player 910 for playing videos. The memory 904 may hold additional modules not shown in FIG. 9, such as modules for performing other operations described elsewhere herein.

[0086]

[0085] A bus 914 or other communication components may support communication of information within the device 900. The processor 902 may be a specialized or dedicated microprocessor configured or operable to perform particular tasks in accordance with the features and aspects disclosed herein by executing machine-readable software code that defines those tasks. Processor memory 904 (e.g., random access memory (RAM) or other dynamic storage device) may be connected to the bus 914 or directly to the processor 902 and may store information and instructions executed by the processor 902. The memory 904 may also store temporary variables or other intermediate information during the execution of such instructions.

[0087] A computer-readable medium in storage device 924 is connected to bus 914 and may store static information and instructions for processor 902, for example, storage device (CRM) 924 may store modules for operating system 906, browser 908, and media player 910 when device 900 is powered off, from which the modules may be loaded into processor memory 904 when device 900 is powered on. Storage device 924 may include a non-transitory computer-readable storage medium that holds information, instructions, or some combination thereof, for example, instructions that, when executed by processor 902, configure or enable device 900 to perform one or more operations of the methods as described herein.

[0088] A network communication (comm.) interface 916 may also be connected to the bus 914. The network communication interface 916 may optionally provide or support two-way data communication between the device 900 and one or more external devices, such as the streaming system 800, via a router / modem 926 and a wired or wireless connection 925. Alternatively or additionally, the device 900 may include a transceiver 918 connected to an antenna 929, through which the device 900 may communicate wirelessly with a base station for a wireless communication system or with the router / modem 926. Alternatively, the device 900 may communicate with the video streaming system 800 via a local area network, a virtual private network, or other network. In another alternative, the device 900 may be incorporated as a module or component of the system 800 and communicate with other components via the bus 914 or by some other modality.

[0089] Device 900 may be connected (e.g., via bus 914 and graphics processing unit 920) to a display unit 928. Display 928 may include any suitable configuration for displaying information to an operator of device 900. For example, display 928 may include or utilize a liquid crystal display (LCD), a touchscreen LCD (e.g., a capacitive display), a light emitting diode (LED) display, a projector, or other display device for presenting information to a user of device 900 in a visual display.

[0090] One or more input devices 930 (e.g., an alphanumeric keyboard, microphone, keypad, remote control, game controller, camera, or camera array) may be connected to bus 914 via user input port 922 for communicating information and commands to apparatus 900. In selected embodiments, input device 930 may provide or support control over cursor positioning. Such cursor control devices, also referred to as pointing devices, may be configured as a mouse, trackball, trackpad, touchscreen, cursor direction keys, or other devices for receiving or tracking physical movements and converting the movements into electrical signals indicative of cursor movement. The cursor control device may be incorporated into display unit 928, for example, using a touch-sensitive screen. The cursor control device may communicate directional information and command selections to processor 902 and control cursor movement on display 928. The cursor control device may have two or more degrees of freedom, for example, allowing the device to specify a cursor position in a plane or three-dimensional space.

[0091] Some embodiments may be implemented in a non-transitory computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform methods described by some embodiments. The computer system may include one or more computing devices. The instructions, when executed by one or more computer processors, may be configured or operable to perform those described in some embodiments.

[0092] As used in this description and throughout the claims that follow, the words "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Also, as used in this description and throughout the claims that follow, the meaning of "in" includes "in" and "on," unless the context clearly dictates otherwise.

[0093]

[0092] The above description illustrates various embodiments, along with examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be considered the only embodiments, but have been presented to illustrate the flexibility and advantages of some embodiments as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations, and equivalents may be used without departing from the scope of the invention as defined by the claims. The inventions described in the claims of the present application as originally filed are set forth below. [C1] 1. A method comprising: determining, by the computing device, input values ​​for a request for content; deriving, by the computing device, a plurality of states for a plurality of content delivery networks, wherein the states include variance-based components; generating, by the computing device, a plurality of scores for the plurality of content delivery networks based on the input values ​​and the plurality of states, wherein the scores for the content delivery networks are generated using parameter values ​​for the components and the input values; selecting, by the computing device, a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; outputting, by the computing device, information about the selected content delivery network to have the selected content delivery network service the request to deliver the content; A method comprising: [C2] generating the plurality of scores comprises: generating the score for each content delivery network in the plurality of content delivery networks based on the input values ​​and a respective state for the content delivery network; The method of C1, comprising: [C3] the parameter value comprises a first parameter value; the score is generated using the first parameter value and a second parameter value, the second parameter value being based on historical observations of distribution of content to the respective content distribution networks; The method described in C1. [C4] The method of C3, wherein the first parameter value decreases as the number of historical observations for the respective content delivery network increases. [C5] The method of C1, wherein the score is based on an upper bound prediction. [C6] The method of C1, wherein the request for content is received from a client, and a content delivery network in the plurality of content delivery networks is capable of delivering the content to the client. [C7] The method of C1, wherein the input values ​​comprise a set of input values ​​based on a context associated with the request. [C8] The method of C7, wherein the context is associated with a client that sends the request and receives the delivery of the content. [C9] generating the plurality of scores comprises: The method of C1, comprising using a plurality of agents to generate the plurality of scores, each content delivery network being associated with a respective agent. [C10] The method of C9, wherein a state is stored for each of the plurality of agents. [C11] receiving feedback from the distribution of the content by the selected content distribution network; adjusting the parameter values ​​based on the feedback; and The method of C1, further comprising: [C12] the feedback is based on performance metrics measured based on the delivery of the content by the selected content delivery network; The method described in C11. [C13] The method of C12, wherein the feedback comprises whether rebuffering occurred during the delivery of the content by the selected content delivery network. [C14] The method of C13, wherein the parameter value is changed to a value that predicts a higher score when no rebuffering occurs. [C15] The method of C11, wherein adjusting the parameter value is performed in real time as the feedback is received. [C16] The method of C11, wherein adjusting the parameter value is performed by an operation for adjusting the parameter value from a first value to a second value using the feedback. [C17] selecting the content delivery network comprises: The method of claim 1, comprising selecting a highest ranked score from the plurality of scores. [C18] A non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a computing device, cause the computing device to: determining input values ​​for a request for content; deriving a plurality of states for a plurality of content delivery networks, wherein the states include a variance-based component; generating a plurality of scores for the plurality of content delivery networks based on the input values ​​and the plurality of conditions, wherein the scores for the content delivery networks are generated using the parameter values ​​for the components and the input values; and selecting a content delivery network from the plurality of content delivery networks based on the scores for the content delivery networks. outputting information about the selected content delivery network to have the selected content delivery network service the request to deliver the content; 1. A non-transitory computer-readable storage medium operable to perform the steps of: [C19] the parameter value comprises a first parameter value; the score is generated using the first parameter value and a second parameter value, the second parameter value being based on historical observations of distribution of content to the respective content distribution networks; A non-transitory computer-readable storage medium as described in C18. [C20] 1. An apparatus comprising: one or more computer processors; A computer-readable storage medium The computer-readable storage medium comprises: determining input values ​​for a request for content; deriving a plurality of states for a plurality of content delivery networks, wherein the states include a variance-based component; generating a plurality of scores for the plurality of content delivery networks based on the input values ​​and the plurality of states, wherein the scores for the content delivery networks are generated using parameter values ​​for the components and the input values; selecting a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; outputting information about the selected content delivery network to have the selected content delivery network service the request to deliver the content; 20. An apparatus comprising: instructions for controlling the one or more computer processors to be operable to:

Claims

1. 1. A method comprising: determining, by the computing device, input values ​​for a request for content; retrieving, by the computing device, a plurality of states for a plurality of content delivery networks, wherein the states include a second parameter and a first parameter based on historical observations of content delivery for each content delivery network; generating, by the computing device, a plurality of scores for the plurality of content delivery networks based on the input values ​​and the plurality of states, wherein the scores for the content delivery networks are generated using a first component representing a utilization value from the first parameter and the second parameter, a second component representing a variance from the first parameter and the second parameter, and the input values, and the scores are based on a prediction of an upper confidence limit; selecting, by the computing device, a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; outputting, by the computing device, information about the selected content delivery network to have the selected content delivery network service the request to deliver the content; A method comprising:

2. generating the plurality of scores comprises: generating the score for each content delivery network in the plurality of content delivery networks based on the input values ​​and a respective state for the content delivery network; The method of claim 1 , comprising:

3. The method of claim 1 , wherein the request for content is received from a client, and a content delivery network in the plurality of content delivery networks is capable of delivering the content to the client.

4. The method of claim 1 , wherein the input values ​​comprise a set of input values ​​based on a context associated with the request.

5. The method of claim 4 , wherein the context is associated with a client that sends the request and receives the delivery of the content.

6. generating the plurality of scores comprises: The method of claim 1 , comprising using multiple agents to generate the multiple scores, each content delivery network being associated with a respective agent.

7. The method of claim 6 , wherein a state is stored for each of the plurality of agents.

8. receiving feedback from the distribution of the content by the selected content distribution network; adjusting the first parameter or the second parameter based on the feedback; and The method of claim 1 further comprising:

9. the feedback is based on performance metrics measured based on the delivery of the content by the selected content delivery network; The method of claim 8.

10. The method of claim 9 , wherein the feedback comprises whether rebuffering occurred during the delivery of the content by the selected content delivery network.

11. The method of claim 10 , wherein the first parameter or the second parameter is changed to a value that predicts a higher score when no rebuffering occurs.

12. The method of claim 8 , wherein adjusting the first parameter or the second parameter is performed in real time as the feedback is received.

13. 9. The method of claim 8, wherein adjusting the first parameter or the second parameter is performed by an operation for adjusting the first parameter or the second parameter from a first value to a second value using the feedback.

14. selecting the content delivery network comprises: The method of claim 1 , comprising selecting a highest ranked score from the plurality of scores.

15. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a computing device, cause the computing device to: determining input values ​​for a request for content; deriving a plurality of states for a plurality of content delivery networks, wherein the states include a second parameter and a first parameter based on historical observations of content delivery for each content delivery network; generating a plurality of scores for the plurality of content delivery networks based on the input values ​​and the plurality of states, wherein the scores for the content delivery networks are generated using a first component representing a utilization value from the first parameter and the second parameter, a second component representing a variance from the first parameter and the second parameter, and the input values, and the scores are based on a prediction of an upper confidence limit; selecting a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; outputting information about the selected content delivery network to have the selected content delivery network service the request to deliver the content; 1. A non-transitory computer-readable storage medium operable to perform the steps of:

16. 1. An apparatus comprising: one or more computer processors; A computer-readable storage medium The computer-readable storage medium comprises: determining input values ​​for a request for content; deriving a plurality of states for a plurality of content delivery networks, wherein the states include a second parameter and a first parameter based on historical observations of content delivery for each content delivery network; generating a plurality of scores for the plurality of content delivery networks based on the input values ​​and the plurality of states, wherein the scores for the content delivery networks are generated using a first component representing a utilization value from the first parameter and the second parameter, a second component representing a variance from the first parameter and the second parameter, and the input values, and the scores are based on a prediction of an upper confidence limit; selecting a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; outputting information about the selected content delivery network to have the selected content delivery network service the request to deliver the content; 20. An apparatus comprising: instructions for controlling said one or more computer processors to be operable to:

17. the first parameter represents a selection history of a content delivery network among the plurality of content delivery networks; The method of claim 1 , wherein the second parameter is based on a value for a metric.

18. The method of claim 1 , wherein the utilization value is based on an average of the first parameter and the second parameter.

Images