Quality awareness profile filtering in content delivery

Abstract

This invention provides a content delivery method, apparatus, and non-temporary computer-readable storage medium that allow selection of target video quality. [Solution] The method includes determining an instance of content, which includes multiple segments associated with multiple profiles, to be delivered to a client device; determining a target metric and bandwidth allocation; making multiple selectable top profile decisions for multiple segments; determining the performance of the target metric for the multiple top profile decisions based on the bandwidth allocation; selecting a top profile decision from the multiple top profile decisions based on the performance of the target metric for the multiple top profile decisions; and using the top profile decisions to determine a target quality used to select a top profile for requesting segments during playback of the instance.

Description

[Background technology]

[0001]

[0001] In adaptive bitrate streaming, content is encoded in multiple profiles that include different bitrates, qualities, or a combination of bitrates and qualities. The media player can then adaptively switch between different profiles during playback, depending on criteria such as available bandwidth. However, not all profiles are necessary for all types of devices. For example, on some devices, such as smartphones with limited computing resources or small screens, users may not notice the quality improvements that come with selecting a very high bitrate profile. However, very high bitrates increase the risk of rebuffering if there is not enough content available to match the media player's playback speed, and can also increase costs for the user or for the company transferring the data due to the delivery of cellular data. [Overview of the project]

[0002]

[0002] The included drawings are for illustrative purposes only and serve only to provide examples of possible configurations and operations of the disclosed systems, apparatus, methods, and computer program products of the present invention. These drawings do not limit any modifications of shape and detail that can be made by those skilled in the art without departing from the spirit and scope of the disclosed embodiments. [Brief explanation of the drawing]

[0003] [Figure 1]

[0003] A diagram showing a simplified system for performing profile filtering according to several embodiments. [Figure 2]

[0004] A diagram showing a simplified flowchart of a method for determining the top profile according to several embodiments. [Figure 3A]

[0005] A diagram showing examples of video quality intervals according to several embodiments. [Figure 3B]

[0006] A figure illustrating examples of reduction representations in several embodiments. [Figure 4]

[0007] A diagram showing a simplified flowchart of a method for predicting the performance of top profile determination according to several embodiments. [Figure 5]

[0008] A diagram showing a simplified flowchart of a method for selecting a target video quality according to several embodiments. [Figure 6]

[0009] A diagram showing an example of a computing device according to several embodiments. [Modes for carrying out the invention]

[0004]

[0010] This specification describes techniques for content delivery systems. The following description includes numerous examples and specific details for illustrative purposes to provide a thorough understanding of several embodiments. Some embodiments defined by the claims may include some or all of the features in these examples, either individually or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

[0005]

[0011] System Overview

[0012] In some embodiments, the system uses a profile filtering process to exclude certain profiles available to a client device. Content such as video, audio, video and audio, or other multimedia content may be encoded in multiple profiles corresponding to different levels, which may differ in bitrate, quality, or bitrate and quality. Profiles may be contained within a profile ladder.

[0006]

[0013] The system can receive adaptable parameter settings that meet specified objectives, such as quality of service objectives. In some embodiments, the system may use objectives for maximum allowable quality loss or minimum required data savings, but other objectives based on the performance metrics used may be used. The objective values ​​may be varied, and for example, either absolute values ​​or percentages may be used. The maximum allowable quality loss may be the maximum amount of quality lost by removing a profile from the profile ladder. The minimum required data savings may be the minimum data savings based on the data saved by removing one of the profiles. These parameters may be adjusted, and based on their values, the process may analyze profiles of multiple segments to select a target video quality for the content. The target video quality may be sent to a client device. The client device then uses the target video quality to perform profile filtering on the profile ladder. Profile filtering may remove profiles from the profile ladder. Therefore, the adaptive bitrate algorithm may not be able to select these profiles during content playback.

[0007]

[0014] Selecting target video quality using parameters can improve the content delivery process. For example, removing certain profiles may ensure that content delivery optimally meets the desired target quality of service. The profile filtering process can analyze possible combinations of top profiles for a segment to determine the optimal top profile for that segment. This determination is then used to select the target video quality. This process allows for the selection of target video qualitys where profiles can be removed on client devices that can meet the target quality of service when the content is delivered. This can result in resource savings or more efficient use. For example, removing profiles that may be unnecessary for certain types of devices can reduce the amount of data transmitted over the network. Delivery can also be more efficient by reducing rebuffering that may occur due to profiles that could potentially cause rebuffering.

[0008]

[0015] system

[0016] Figure 1 shows a simplified system 100 for performing profile filtering in several embodiments. System 100 includes a server system 102 and client devices 104. The server system 102 may include one or more computing devices capable of calculating a target video quality. The target video quality may be calculated based on receiving requests to play an instance of content, such as a request to play a video from a content delivery service. The content may be encoded with multiple profiles in a profile ladder available for download in adaptive bitrate streaming. The client device 104 receives the target video quality and uses it to remove profiles from the profile ladder. Although one client device 104 is shown, multiple client devices 104 may be used and capable of receiving the target video quality. In some embodiments, the server system 102 can dynamically determine the target video quality for different client devices 104. That is, a first client device may receive a first target video quality, and a second client device may receive a second target video quality for the same video. Different videos can also have different target video quality levels for the same client device.

[0009]

[0017] The client device 104 includes a profile filtering system 112, an adaptive bitrate (ABR) algorithm 114, and a media player 116. The profile filtering system 112 receives the target video quality from the server system 102. The generation of the target video quality is described in more detail below. Next, the profile filtering system 112 determines the highest profile that the ABR algorithm 114 can select. The highest profile is sometimes called the top profile. As discussed above, the profile ladder includes profiles that can be used to request different versions of content. This profile is associated with one or more characteristics, such as bitrate characteristics, quality characteristics, or bitrate and quality characteristics. Bitrate can be the number of bits per second associated with a segment, such as the bitrate at which the segment is encoded. Quality characteristics can indicate the quality of the video (e.g., resolution or other quality metrics). The resolution of a video can be based on the number of pixels in a single frame, and higher resolution is associated with higher quality. Higher bitrate is also associated with higher quality. Profiles can be organized into levels such as Level 0 to 10 in a profile ladder. The level of a profile in the profile ladder can be determined based on how the video is converted to that level. For example, a Level 0 profile might be encoded at 200kbps, and a Level 1 profile might be encoded at 400kbps. A resolution of 640x360 (360p) pixels might be associated with a Level 0 profile, and a resolution of 1280x720 (720p) pixels might be associated with a Level 1 profile. Resolution can be expressed in terms of the number of pixels horizontally and vertically. In general, lower-level profiles in the profile ladder may be considered lower quality profiles compared to higher-level profiles.

[0010]

[0018] The profile filtering system 112 uses target video quality to select the top profile for each segment of content. For example, content may be divided into parts called segments. Certain limitations are stored for the adaptive bitrate algorithm 114 so that it cannot select a profile associated with a bitrate that exceeds the top profile (for example, exceeding the bitrate of the top profile).

[0011]

[0019] During playback, the adaptive bitrate algorithm 114 can analyze the bandwidth available while delivering the content segment. The adaptive bitrate algorithm 114 can select one of the available profiles (the available ones do not include any removed profiles) and request the segment with that selected profile. Once the media player 116 receives the content, it can display the content of the segment.

[0012]

[0020] In the server system 102, the parameter adaptation system 105 can generate target video quality values. While target video quality is discussed, if non-video content is used, the target quality can be based on that type of content, such as target audio quality. The top profile determination generator system 106 can generate feasible top profile determinations. The top profile determination can be a combination of top profiles for segments based on each video quality of the profile. This process is explained in more detail in Figure 2.

[0013]

[0021] The performance prediction system 108 can predict performance in terms of metrics such as video quality loss and data saving rate for each top profile determination. The performance prediction system 108 receives bandwidth allocations used to predict performance. Bandwidth allocations can be probability distributions of download bandwidth that may be experienced during the delivery of an instance of content. The probability distribution is used to determine the probability that a given download bandwidth is used for a segment of content. In some embodiments, bandwidth allocations can be calculated based on historical bandwidth information collected from content delivery. In some embodiments, historical bandwidth allocations can be based on a specific client device 104 that requested an instance of content. Historical bandwidth information may also be from other client devices. Using historical information from a specific client device 104, the generation of target video quality can be personalized to the circumstances experienced by that specific client device 104. Other historical bandwidth information from other client devices may be used to add context from other circumstances being experienced.

[0014]

[0022] The target video quality selection system 110 can receive all possible top profile determinations and the performance predicted by those top profile determinations. The target video quality selection system 110 receives a metric selection and outputs a target video quality. The metric selection may be from different metrics that can be used to select the top profile of a segment. In some embodiments, maximum video quality loss or minimum data savings rate may be selected. A combination of these two metrics may be used, or other metrics may be used. The maximum video quality loss may be the absolute delta between two video quality values. The video quality values ​​may be evaluated by metrics such as video multi-scheme evaluation fusion (VMAF) peak signal-to-noise ratio (PSNR) or other metrics. A maximum video quality loss of a certain value, for example 2, means that by limiting the top profiles that can be selected, the average video quality perceived by the user with that metric is expected to decrease by a maximum of 2 units when measured by VMAF. The minimum data savings rate may be a percentage threshold of data saved. For example, a minimum data saving rate of 5% means that by limiting the top profile that can be selected, client device 104 is expected to save at least 5% of the downloaded data of the content by not having a profile with a higher bitrate than the top profile available. Although a rate has been discussed, the metric does not have to be a rate; it can be based on the amount of data saved. In general, saved data metrics are based on the amount of data saved.

[0015]

[0023] Using the selected metrics, the target video quality selection system 110 evaluates the top profile determination to determine the optimal top profile determination. Then, the target video quality selection system 110 selects an appropriate target video quality to achieve this top profile determination.

[0016]

[0024] A simplified example might involve four segments, each containing three profiles associated with three bitrates for instances of content. A feasible top profile determination might include:

[0017] 1. [0,0,0,0], when the target quality is between -∞ and 20. The position in the array indicates the top profile of each segment to which each position is associated. Here, there are four segments: 1, 2, 3, and 4. This decision means that the maximum selectable profile (e.g., the top profile) for all four segments is profile 0 (the lowest).

[0018] 2. [0,1,2,0], when the target quality is between 20 and 40. In this decision, the top profile of the first segment is profile 0 (lowest profile), the second segment is profile 1 (medium profile), the third segment is profile 2 (highest profile), and the fourth segment is profile 0.

[0019] 3. [1,2,2,0], when the target quality is between 40 and 80.

[0020] 4. [2,2,2,2], when the target quality is between 80 and infinity.

[0021]

[0025] In the above, [w,x,y,z] represents the top profile determination, where w is the top profile of the first segment, x is the top profile of the second segment, y is the top profile of the third segment, and z is the top profile of the fourth segment. The values ​​of w,x,y, andz can be profile levels, with higher numbers associated with higher bitrates or quality. More segments can be added as values ​​in the array.

[0022]

[0026] The following can be input to the parameter-adaptive system 105. In this example, the minimum data saving rate is selected as the target metric, and the value is 5% data saving. The bandwidth allocation can be 100 kbit / s (kbps): 10%, 2000 kbps: 20%, and 5000 kbps: 80%. Here, 100 kbps: 10% means that there is a 10% probability that the download bandwidth is 100 kbps. Depending on the bandwidth allocation, the predictive performance of each achievable top profile determination can be seen in Table I.

[0023] [Table 1]

[0024]

[0027] The associated target quality interval is the interval of quality associated with the profile selected in the top profile determination. Small brackets can indicate that the endpoints are excluded, and square brackets can indicate that the endpoints are included. For example, [(20,40)] indicates that 20 is not included, but 40 is included. The predicted video quality loss is 5 units, 3 units, 2 units, and 0 units for each top profile determination based on the metric used (e.g., VMAF). The predicted data savings rate is 10%, 8%, 5%, and 0% for each top profile determination. The target video quality selection system 110 can select a top profile determination from the available top profile determinations based on evaluating the received metric. In this example, the third top profile determination [1,2,2,0] is selected because it not only satisfies the predicted data savings rate of 5% but also minimizes the video quality loss to a value of "2". In contrast, the top [0,0,0,0] In the profile determination, the predicted data saving rate may be 10%, but the predicted video quality loss may be a value of "5". The top profile determination [0,1,2,0] includes a predicted data saving rate of 8%, but includes a predicted video quality loss of "3". In these top profile determinations, the predicted video quality loss is greater than the predicted quality loss of 2 for the third top profile determination. Here, the target video quality selection system 110 determines that the third top profile determination not only satisfies the data saving rate target but also minimizes the predicted video quality loss, and therefore it is considered optimal to select this top profile determination. The top profile determination is used to select a target video quality, such as a target video quality in the range of 40 to 80, which is sent to the client device 104.

[0025]

[0028] On client device 104, to ensure that segment quality is consistent, the top profile is selected to be close to a given target video quality, rather than using random selection. The following describes an example of selecting the top profile based on target video quality. The video has N segments, which are encoded into N profiles, and the variable q ij q is the video quality of segment i containing profile j. Video quality can be measured by any quantitative video quality metric, such as VMAF, PSNR, etc. VMAF is used as an example. ij The higher the value of this parameter, the higher the quality of segment i when profile j is selected and played back.

[0026]

[0029] The top profile determination process requires the target video quality as input. This is Q t This is sometimes expressed as follows: For each segment i, when the video quality of profile j is closest to the target quality compared to the quality of other profiles, profile j is selected as the top profile, for example, by using the following formula.

[0027]

number

[0028]

[0030] The adaptive bitrate algorithm 114 is not allowed or is restricted from selecting a profile higher than the top profile when segment i is requested for playback. Assume that the video has three segments, encoded into four profiles associated with four bitrates, and the video quality of each segment for each profile is as follows:

[0029] [Table 2]

[0030]

[0031] If the target video quality is 80, the top profiles for the three segments are Profile 2, Profile 1, and Profile 2 for Segment 0, Segment 1, and Segment 2, respectively. These segments have the video quality closest to 80. For example, Profile 2 has a video quality of 85 for Segment 0, Profile 1 has a video quality of 80 for Segment 1, and Profile 2 has a video quality of 80 for Segment 2. These video qualities can be the closest to a video quality of 80 compared to the video qualities of the other profiles.

[0031]

[0032] Next, we will explain the target video quality selection process in more detail below.

[0032]

[0033] Top Profile Determined

[0034] Figure 2 shows a simplified flowchart 200 of a method for determining a top profile determination according to several embodiments. The target metric value may be set to an arbitrary value, but the number of feasible top profile determinations may be finite. Each feasible top profile determination is a combination of profiles for a segment and may be associated with a video quality interval. When the target video quality falls within this video quality interval, the profile in the associated top profile determination will be selected by the client 104 as the top profile for the segment. The top profile determination generator system 106 can generate all top profile determinations for a single segment and determine the associated video quality interval for that top profile determination. Next, the top profile determination generator system 106 can reduce the interval for a single segment to a representation that includes top profile determinations for multiple segments and the associated video quality interval. From there, the top profile determination generator 106 can find all feasible top bitrate determinations, and the target video quality selection system 110 first selects the optimal top bitrate determination from among them before it finally determines the target video quality.

[0033]

[0035] In this process, at 202, the top profile determination generator system 106 determines the video quality interval of the profile for each segment. The top profile changes when the target quality enters a different interval. In particular, when client 104 is determining the top profile for each segment i, the following is given:

[0034] section

[0035]

number

[0036] Given any target quality, the top profile would be profile 0 (for example, the lowest profile).

[0037] section

[0038]

number

[0039] Given any target quality, the top profile would be profile m-1 (for example, the highest profile).

[0040] section

[0041]

number

[0042] ,

[0043]

number

[0044] Given any target quality, the top profile will be profile j (where j is neither 0 nor m-1).

[0045]

[0036] As above, q ij is the video quality of segment i containing profile j. Generally, intervals can be determined as midpoints between consecutive video quality profiles. That is, the video quality is divided into m consecutive intervals, each interval associated with one profile. When the target video quality falls within an interval, the associated profile is selected as the top profile for this segment.

[0046]

[0037] Figure 3A shows examples of video quality segments according to several embodiments. In Figure 3A, m consecutive segments can be divided by m-1 boundaries. For example, if there are three segments, there are two boundaries. i If it refers to a subdivision of segment i, then T iis bounded, for example, by T i =[t i0 ,t i1 ,...,t i(m-2) , and here

[0047]

Number

[0048] is. In particular, if there is only one profile (for example, m = 1), T i = [], indicating only one interval (-∞, ∞).

[0049]

[0038] When there are three profiles for one segment, the intervals associated with the profiles that can be selected as the top profile can be calculated for segment 0 in this example. The video qualities of profile 0, profile 1, and profile 2 can be listed as 40, 60, and 80 at 302-0, 302-1, and 302-2. The interval partition can be represented as T = [50, 70] for these three profiles. When profile 0 is selected as the top profile, the target quality should fall in the interval (-∞, 50], profile 1 in the interval (50, 70], and profile 2 in the interval (70, ∞). The boundary can be determined by the median value between the quality values. For example, the interval of 50 is determined as the value between the quality values of 40 and 60, and the value of 70 is determined as the value between the quality values of 60 and 80. By using -∞ and ∞, the possible lowest video quality and the possible highest video quality are indicated.

[0050]

[0039] Returning to Figure 2, at 204, the top profile determination generator system 106 reduces the video quality section of multiple segments. When there are multiple segments, the section containing those segments is reduced to a single representation. Figure 3B shows examples of the reduced representation 306 according to several embodiments. There are only two segments, segment 0 in 304-1 and segment 1 in 304-2, but the reduction can be performed on more than two segments. In some embodiments, all segments of the video may be reduced to a single representation, while in other embodiments, only a portion of the segments, such as 20 segments at a time, may be reduced to their respective representations.

[0051]

[0040] 306 shows a reduced representation of video quality intervals. For each interval, the top profile determination for these segments is shown. For example, the interval could be T0=[50,70] for segment 0 and T1=[41,80] for segment 1. As seen in the reduced representation, the interval is based on the quality interval values ​​of both segments. Next, the top profile determination for all reduced intervals is shown. In 308-1, the top profile determination can be [0,0]. Each position in the array may represent the selected top profile for each segment. Position 0 shows the selected top profile for segment 0, and position 1 shows the top profile for segment 1. Here, the first value of 0 means that profile 0 is selected as the top profile for segment 0, and the second value of 0 means that profile 0 is selected for segment 1. If there was a third segment that was reduced, the third value would represent the top profile selected for segment 2, such as [0,0,0]. By reducing the intervals, all possible top bitrate determinations are found. If there are m profiles and n segments, then m of the profiles nThere are several possible combinations as top profile determinations, but only some of them are executable by client 104. For example, top profile determination [1,0] is not shown. This is because the determination requires a target quality of over 50 to select profile 1 as the top profile for segment 0, which contradicts the target quality requirement of less than 41 for selecting profile 0 for segment 1, given that the target quality should be the same when the client is requesting a segment of a single video.

[0052]

[0041] In 308-2, the top profile determination for interval (41, 50) is shown as [0, 1]. Here, the selected top profile for profile 0 is shown for segment 0, and the selected top profile for profile 1 is shown for segment 1. Segment 0 selects profile 0 as the top profile, and segment 1 selects profile 1 when the target quality falls within this interval. In 308-3, the top profile determination is [1, 1]. Here, segment 0 selects profile 1, and segment 1 selects profile 1. In 308-4, the top profile determination is [2, 1], and in 308-5, the top profile determination is [2, 2]. These top profile determinations are similarly determined upon request.

[0053]

[0042] The following describes embodiments that may be used to determine the top profile. When there are multiple segments, the segment divisions of these segments can be reduced. D L Let be the decision list of all top profile decisions for the first L segments. Lk =[b0,b1,...,b L-1 ]∈D L is an L-element vector pointing to the k-th top profile determination, where b i This is the top profile of segment i. LLet be a bounding list showing reduced intervals for the first L segments. List D has K elements. L When the element is in a certain state, there are K-1 boundaries representing K intervals associated with that element. L It is located at S L =[t0,...,t K-2 In the case of ], decision d Lk The associated interval is (t (k-1) ,t k ]. In particular, this interval is (-∞, t0) when k=0, and (t) when k=K-1. K-2 ,∞). When K=1, S L =[] represents only one interval (-∞,∞). Once all notations are determined, the reduction methods for creating a reduced representation are as follows:

[0054]

[0043] When L=1, there are only m possible top profile determinations, for example, d 1j =[j], and the video quality interval is precisely the interval division for segment 0, for example, S1=T0.

[0055]

[0044] Decision list D of the first L-1 segments L-1 and section S (L-1) And the segment T of segment L-1 (the Lth segment) L-1 Considering this, the top profile determination generator system 106 uses this process to determine the first L segments of the list D L and section S L It is possible to calculate this.

[0056] 1. S L-1 There are K intervals, T L-1 Suppose there are H intervals. Therefore, each interval has K-1 and H-1 interval boundaries.

[0057] 2. Merge all of these K+H-2 interval boundaries and obtain the first L intervals S L =[t0,...,t K+H-3Sort in ascending order to obtain the interval partitions for ].

[0058] 3. When x=0, the top profile determination associated with the x-th interval (-∞, t0) is d Lx =d (L-1)0 It is +[0].

[0059] 4.0 <x<K+H-2のとき、S L-1 i boundaries from and T L-1 The j boundaries from S L The first x boundaries (for example, {t0,...,t}) (x-1) Assume it is inside}. Boundary t x and the x-th interval (t x-1 ,t x For ], the associated top profile determination d Lx is d (L-1)i +[j] is the (L-1) element vector d (L-1)i This is a vector of L elements calculated by adding a new element j to the original vector.

[0060] 5. When x = K + H - 2, the x-th interval (t K+H-3 The associated top profile determination d (infinity) Lx is, d (L-1)(K-1) It is +[m-1].

[0061]

[0045] Returning to Figure 2, at 206, the top profile determination generator system 106 outputs the top profile determination for the quality interval in a representation. The top profile determination can be output to the performance prediction system 108. Subsequently, performance prediction is performed after the top profile determination has been made.

[0062]

[0046] Performance prediction

[0047] Figure 4 shows a simplified flowchart 400 of a method for predicting the performance of top profile determination according to several embodiments. Performance prediction may include the steps of: evaluating the probability of selecting each profile for each segment; predicting average video quality and data usage with and without top profile constraints; and calculating metrics such as video quality loss and data savings rate.

[0063]

[0048] In 402, the performance prediction system 108 evaluates the probability of selecting each profile for each segment. The performance prediction system 108 receives bandwidth allocation to evaluate the probability of selecting each profile for each segment. Bandwidth allocation may be determined based on historical bandwidth information collected from client devices 104. As discussed above, historical bandwidth information may be the available bandwidth experienced while downloading content. Historical information may be based solely on bandwidth information from a specific client 104 that requested an instance of content. Historical bandwidth information may also be used based on other client devices.

[0064]

[0049] In some cases, the bandwidth allocation may be as shown in Table III below.

[0065] [Table 3]

[0066]

[0050] In Table III, the bandwidth can be 200kbps, 1200kbps, and 2000kbps, and the probability of experiencing each bandwidth can be 30%, 60%, and 10%, respectively. This means that there is a 30% probability that the bandwidth experienced by client device 104 is 200kbps, a 60% probability that the bandwidth experienced is 1200kbps, and a 10% probability that the bandwidth experienced is 2000kbps. Therefore, the bandwidth most likely to be experienced is 1200kbps, and the bandwidth least likely is 2000kbps.

[0067]

[0051] If an instance of content is encoded into three different versions with three different profiles, the following may be the actual profile of segment i and the probability of selecting that profile.

[0068] [Table 4]

[0069]

[0052] In Table III, P = {p ij} and here, p ij b refers to the probability of selecting profile j in segment i, and b ij is the bitrate of that profile. Here, the three version quality levels are 360p, 720p, and 1080p for profiles 0, 1, and 2, respectively. The probability of selecting profile 0 is 30%, the probability of selecting profile 1 is 70%, and the probability of selecting profile 2 is 0%. The probability of selecting profile 1, which has an associated bitrate of 1000kbps, is determined by the probability that the bandwidth falls within [1000kbps, 4500kbps]. This probability can be 70%, since the available bandwidth is expected to be 1200kbps with an expected 60% probability and 2000kbps with an expected 10% probability. The probability of selecting profile 2 is 0%. This is because the probability is determined by the probability that the bandwidth is ≥4500kbps, which is not possible with the given bandwidth allocation.

[0070]

[0053] In some embodiments, the following process may be used. The performance prediction system 108 receives bandwidth allocation to evaluate the probability of selecting each profile for each segment. This bandwidth allocation can be implemented by sending download information to the server system 102, and the server system 102 calculates the bandwidth allocation. Let f(r) be the probability density function of the download bandwidth. Therefore

[0071]

number

[0072] P = {p ij} and here, p ij Σ refers to the probability of selecting profile j for segment i, with the constraint Σ j p ij = 1

[0073]

[0054] Assuming that one profile is selected when the download bandwidth exceeds the bitrate associated with the profile but falls below the bitrate of the next highest profile, the probability of selecting bitrate j for segment i is:

[0074]

number

[0075] And here, LB ij is the lower limit of the integral, UP ij This is the upper limit. From here,

[0076]

number

[0077] and

[0078]

number

[0079] Here, j refers to the profile index when all profiles are sorted in ascending order, and b i,j This refers to the bitrate value of the j-th profile.

[0080]

[0055] In 404, the performance prediction system 108 predicts the average video quality and data usage with and without top profile constraints. Using top profile constraints may restrict the selection of available profiles, while not having top profile constraints may not restrict the selection of profiles at all. The performance prediction system 108 can determine unconstrained predicted video quality and unconstrained data usage. The performance prediction system 108 can also determine constrained predicted video quality and constrained predicted data usage. Performance can be determined using both metrics, even if one of the metrics is designated as the target. This is because top profile determination can use both metrics when determining the top profile based on the target.

[0081]

[0056] The following examples of segments may be used.

[0082] [Table 5]

[0083]

[0057] In this example, the following can be assumed.

[0084] The video has n segments and is encoded into m profiles.

[0085] d=[b0,...,b n] is the top profile determination for n-segment video, and b i b3=0 is the profile index of the top profile in segment i. For example, b3=0 means that the top profile in segment 3 is the lowest. In other words, the media player is not allowed to select a profile higher than the lowest profile.

[0086] P={p ij} is the estimated probability of selecting profile j for segment i.

[0087] q ij This is the video quality of segment i containing profile j. As mentioned above, this video quality can be measured by any quantitative video quality metric such as VMAF.

[0088] s ij This is the size (in bytes) of segment i containing profile j.

[0089]

[0058] The estimated selection probabilities can be 30% for profile 0, 40% for profile 1, and 30% for profile 2. If there is no top profile constraint, for example, if the media player 116 can select any of the three profiles,

[0090] The expected video quality for this segment is 30% × 60 + 40% × 85 + 30% × 90 = 79.

[0091] The estimated data usage for this segment is 30% × 2.44 + 40% × 6.25 + 30% × 25.75 = 10.957 MB.

[0092]

[0059] When the top profile of this segment is profile 1,

[0093] The expected video quality for this segment is 30% × 60 + 70% × 85 = 77.5.

[0094] The estimated data usage for this segment is 30% × 2.44 + 70% × 6.25 = 5.107 MB.

[0095]

[0060] In some embodiments, the video quality and data usage of a single segment can be calculated as follows: For segment i,

[0096] constraints

[0097]

number

[0098] No predicted video quality.

[0099] constraints

[0100]

number

[0101] Predicted data usage without a forecast.

[0102] constraints

[0103]

number

[0104] Predicted video quality.

[0105] constraints

[0106]

number

[0107] Predicted data usage.

[0108]

[0061] When there are multiple segments, the average video quality metric is the average of the video quality of all segments, and the data usage metric is the sum of the data usage of all segments, as follows:

[0109] constraints

[0110]

number

[0111] No predicted average video quality.

[0112] constraints

[0113]

number

[0114] Predicted data usage without a forecast.

[0115] constraints

[0116]

number

[0117] Predicted average video quality.

[0118] constraints

[0119]

number

[0120] Predicted data usage.

[0121]

[0062] For example, if there are two identical segments in the video from a previous example, the performance prediction system 108 will

[0122]

number

[0123] ,

[0124]

number

[0125] Next, we calculate Q(P,d) = 77.5 and U(P,d) = 10.214 MB.

[0126]

[0063] After calculating the average video quality and data usage with and without constraints, the performance prediction system 108 calculates performance values ​​for the metrics of video quality loss and data saving rate. In some embodiments, two metrics may be used to evaluate the performance of top profile determination for video quality loss and data saving rate.

[0127] Video Quality Loss (VQL): Predicted mean video quality with constraint Q, and constraint

[0128]

number

[0129] The delta between the predicted average video quality and the actual quality, for example,

[0130]

number

[0131] .

[0132] Data Savings Rate (DSR): Predicted data usage with constraints U, and constraints

[0133]

number

[0134] The relative delta between predicted data usage and no predicted data usage, for example.

[0135]

number

[0136] .

[0137]

[0064] Input (Top profile determination d=[b0,...,b n Given the bandwidth allocation f(r) and the performance prediction system 108 outputs the predicted video quality loss and the data saving rate. Next, the target video quality selection system 110 calculates the target video quality according to several embodiments.

[0138]

[0065] Select target video quality

[0066] Figure 5 shows a simplified flowchart 500 of a method for selecting a target video quality according to several embodiments. In this process, all feasible top profile determinations may be evaluated, and the target video quality selection system 110 determines the optimal top profile determination and selects an appropriate target video quality to achieve this determination.

[0139]

[0067] In 502, the target video quality selection system 110 receives the selected target metrics for use in evaluation. For example, the target metrics could be maximum video quality loss or minimum data saving rate. A value may be received for the target metrics, such as maximum video quality loss being set to 10. This value is the data saving rate DSR. t Lower limit or video quality loss VQL t It can be any of the upper limits, where t is the target.

[0140]

[0068] In 504, the target video quality selection system 110 receives the top profile determination. The top profile determination may be generated by the top profile determination generator system 106.

[0141]

[0069] In 506, the target video quality selection system 110 analyzes the target metric for the top profile determination in order to select the optimal top profile determination. In some examples, maximum video quality loss may be selected as the metric and set to a value of 10. If there are five viable top profile determinations labeled A, B, C, D, and E, their predicted performance is listed below in Table V.

[0142] [Table 6]

[0143]

[0070] Predicted video quality loss and predicted data saving rates are calculated by the target video quality selection system 110 and included for each top profile determination. The following examples may be presented.

[0144] Case 1: Both top profile decisions A and B meet the target metric (because their video quality loss is less than or equal to 10 of the maximum video quality loss). Top profile decision B is better than top profile decision A because it has a higher data saving rate.

[0145] Case 2: Neither top profile determination C nor D meets the target metric (because their video quality loss exceeds 10). Since their distance to the target is the same (|10-12|=2 for both top profile determinations C and D), and top profile determination D has a higher data saving rate, top profile determination D is better than top profile determination C.

[0146] Case 3: Neither top profile determination D nor E meets the target metric, but top profile determination E is better than top profile determination E because the distance to the target (=5) is longer than top profile determination E (=2).

[0147] Case 4: Top profile decisions A and B satisfy the objective, while top profile decisions C, D, and E do not. Therefore, all of top profile decisions {A and B} are better than top profile decisions {C, D, and E}.

[0148]

[0071] Here, the optimal top profile determination is top profile determination B because the predicted video quality loss satisfies the target of 10, and this top profile determination includes a 10% higher data saving rate than top profile determination A.

[0149]

[0072] In summary, the following may be used, where D is the set of all viable top profile decisions: all top profile decisions d k For ∈D, the target video quality selection system 110 first determines the video quality loss VQL for this top profile determination. k and data saving rate DSR k The system then calculates the metric. Next, the target video quality selection system 110 determines the current top profile d k By comparing the metric for (where k is the profile determination index) with that for the best top profile determination to date, the best top profile determination can be updated.

[0150]

[0073] Below, the top profile is determined d * This refers to the best decision, and video quality loss (VQL) * and data saving rate DSR * This refers to its performance. The specified target metric is the maximum acceptable video quality loss, e.g., VQL. t Let's assume that this is the case.

[0151] Top Profile Determination d k and d * Both are target metrics (for example, VQL k ≤VQL t and VQL * ≤VQL tWhen the condition is satisfied, the top profile determination d k is the DSR k > DSR * only when it is the best decision.

[0152] The top profile determination d k is also the top profile determination d * is also the target metric (e.g., VQL k > VQL t and VQL * > VQL t is not satisfied, 〇 When both of these are equally close to the target metric (e.g., |VQL k - VQL t | = |VQL * - VQL t |), the top profile determination d k is the DSR k > DSR * only when it is the best decision.

[0153] 〇 Otherwise, the top profile determination d k is the best decision only when it is close to the target metric (e.g., |VQL k - VQL t | < |VQL * - VQL t |).

[0154] Otherwise, the top profile determination d k is the best decision as long as it satisfies the target metric (e.g., VQL k ≦ VQL t ).

[0155]

[0074] When the selected metric is the minimum data saving rate, the following can be implemented. Assume that the minimum data saving rate is set to 5%. If there are five achievable top profile determinations and the following predicted performances are presented in Table VI.

[0156]

Table 7

[0157]

[0075] Here, the top profile determinations are again A, B, C, D, and E. The predicted video quality loss and predicted data saving rate for each from the target video quality selection system 110 are shown. The following examples may be determined by the target video quality selection system 110.

[0158] Case 1: Both top profile determinations A and B satisfy the target metric (data savings rate ≥ minimum data savings rate of 5%). Top profile determination B is better than top profile determination A because it has lower video quality loss.

[0159] Case 2: Neither top profile determination C nor D meets the target metric (by having a data saving rate of less than 5%). Since these are the same distance from the target (|3%-5%|=2% for both top profile determinations C and D), and top profile determination C has lower video quality loss, top profile determination C is better than top profile determination D.

[0160] Case 3: Neither top profile determination D nor E meets the target metric, but top profile determination E is better than top profile determination E because its distance to the target (=5%) is greater than that of top profile determination E (=2%).

[0161] Case 4: Top profile decisions A and B satisfy the objective, while top profile decisions C, D, and E do not. Therefore, both top profile decisions {A and B} are better than top profile decisions {C, D, and E}.

[0162]

[0076] Here, the target video quality selection system 110 selects the optimal top profile decision as top profile decision B because both top profile decisions A and B satisfy the target metric (by having an 8% data saving rate), but top profile decision B has a lower video quality loss of 10 compared to 12.

[0163]

[0077] In summary, the specified QoS goal is the minimum data saving rate (e.g., DSR) t When this is the case, the following may be used to determine the top profile:

[0164] Top Profile Determination d k and d * Both are target metrics (for example, DSR) k ≥DSR t and DSR * ≥DSR t If the conditions are met, the top profile is determined d k VQL k <VQL * It will only be the best decision in that situation.

[0165] Top Profile Determination d k d * Also target metrics (for example, DSR) k <DSR t and DSR * <DSR t If the conditions are not met, 〇 Both of these are equally close to the target metric, for example, |DSR k -DSR t |=|DSR * -DSR t | In this case, the top profile is determined d k VQL k >VQL * It will only be the best decision in that situation.

[0166] ○ Otherwise, determine the top profile d k For example, |DSR k-DSR t |<|DSR * -DSR t It is only the best decision when |

[0167] Otherwise, the top profile is determined d k For example, DSR satisfies the top profile determination. k ≥DSR t As long as that's the case, it will be the best decision.

[0168]

[0078] In step 508, the target video quality selection system 110 outputs a target quality based on the optimal top profile determination.

[0169]

[0079] The target video quality selection system 110 determines the associated video quality interval for optimal top profile determination. That is, optimal top profile determination d * After determining the target video quality, the target video quality selection system 110 can find its associated quality interval f, which is (a, b). The selected target quality can also be any value within this interval. The following may be used, but are not limited to, the interval being:

[0170] If a = -∞ and b is not infinite, then the best target quality can be b.

[0171] If b = ∞ and a is not ∞, the highest target quality can be a + 1.

[0172] If a = -∞ and b = ∞, the best target quality can be any value, for example, 10.

[0173] Otherwise, the best target quality is the median of the interval, for example.

[0174]

number

[0175] It can be done this way.

[0176]

[0080] In the following example, the associated target video quality interval for top profile determination from the previous example is as follows:

[0177] [Table 8]

[0178]

[0081] If top profile determination A is the optimal choice, the target video quality can be set to any value within the range (-∞, 20). In some embodiments, a target video quality of 20 is sent to the client because this is the highest quality within that range, although other quality values ​​from that range may also be sent. If top profile determination E is the optimal choice, the target quality can be set to any value within (80, ∞). In some embodiments, a target video quality of 81 is sent because the target video quality selection system 110 may not want to send a quality value that is too high. In some embodiments, the target video quality selection system 110 determines that top profile determination B is the optimal choice, so the actual target quality can be set to any value within the range (20, 40). In some embodiments, the target video quality selection system 110 uses the median of that range, such as sending a value of 30 (=(20+40) / 2) when one of the values ​​is not infinite.

[0179]

[0082] After determining the target video quality, the target video quality selection system 110 sends the target video quality to the client device 104. The client device 104 may then use the target video quality to perform adaptive bitrate streaming on the requested content instance. As discussed above, the client device 104 uses the target video quality to remove some profiles from the profile ladder if necessary.

[0180]

[0083] Conclusion

[0084] The parameter adaptation system 105 may provide a process for predicting the performance of metrics for top profile determination. This process provides flexibility in setting quality of service objectives, such as the ability to set maximum quality loss or minimum data savings rate based on different preferences. The process can be adapted in real time to dynamically respond to changes in network conditions, content characteristics, and user preferences. This can improve the efficiency of content delivery. Furthermore, the data savings rate and maximum quality loss used can be understood in relevant terms that have practical and physical meaning, which allows for understanding the tuning of metrics. Consequently, the resulting video download performance can be evaluated.

[0181]

[0085] System

[0086] Figure 6 shows an example of a computing device according to several embodiments. According to various embodiments, a system 600 suitable for implementing the embodiments described herein may include a processor 601, memory 603, storage device 605, interface 611, and bus 615 (e.g., PCI bus or other interconnect fabric). System 600 can operate as various devices such as server system 102, or as any other device or service described herein. Although specific configurations are described, various alternative configurations are possible. The processor 601 can perform operations such as those described herein. Instructions for performing such operations may be embodied in memory 603, one or more non-temporary computer-readable media, or any other storage device. Various specially configured devices can also be used instead of or in addition to the processor 601. Memory 603 may be random access memory (RAM) or other dynamic storage devices. The storage device 605 may include a non-temporary computer-readable storage medium that holds information, instructions, or any combination thereof (for example, instructions that, when executed by processor 601, set or enable processor 601 to perform one or more operations of the methods described herein). Bus 615 or other communication components may support the transmission of information within system 600. Interface 611 may be connected to bus 615 and configured to send and receive data packets over a network. Examples of supported interfaces include, but are not limited to, Ethernet®, High Speed ​​Ethernet, Gigabit Ethernet, Frame Relay, Cable, Digital Subscriber Line (DSL), Token Ring, Asynchronous Transfer Mode (ATM), High Speed ​​Serial Interface (HSSI), and Fiber Distributed Data Interface (FDDI). These interfaces may include ports suitable for communication with appropriate media.These may also include a separate processor and / or volatile RAM. The computer system or computing device may include, or communicate with, a monitor, printer, or other suitable display for providing the user with any of the results referred to herein.

[0182]

[0087] Any of the disclosed embodiments can be embodied in various types of hardware, software, firmware, computer-readable media, and combinations thereof. For example, some of the techniques disclosed herein can be at least partially implemented by non-temporary computer-readable media, including program instructions, state information, etc., for configuring a computing system to perform the various services and operations described herein. Examples of program instructions include both machine code, such as that generated by a compiler, and higher-level code, which can be executed via an interpreter. Instructions can be embodied in any suitable language, such as Java®, Python, C++, C, HTML, or any other markup language, JavaScript®, ActiveX, VBScript, or Perl. Examples of non-temporary computer-readable media include, but are not limited to, magnetic media such as hard disks and magnetic tapes; optical media such as flash memory, compact discs (CDs) or digital versatile discs (DVDs); magneto-optical media; and other hardware devices such as read-only memory ("ROM") devices and random access memory ("RAM") devices. Non-temporary computer-readable media can be any combination of such storage devices.

[0183]

[0088] In the foregoing specification, various techniques and mechanisms are sometimes described in the singular for clarity. However, it should be noted that some embodiments involve multiple iterations of a technique or multiple instantiations of a mechanism, unless otherwise specified. For example, a system uses a processor in various situations, and unless otherwise specified, multiple processors may be used while remaining within the scope of this disclosure. Similarly, various techniques and mechanisms are sometimes described as involving a connection between two entities. However, since various other entities (e.g., bridges, controllers, gateways, etc.) may exist between the two entities, the connection does not necessarily mean a direct, unhindered connection.

[0184]

[0089] Some embodiments may be implemented on a non-temporary computer-readable storage medium for use in or in connection with an instruction execution system, device, system, or machine. This computer-readable storage medium contains instructions for controlling a computer system to carry out the methods described in some embodiments. This computer system may include one or more computing devices. The instructions may be configured or operable to carry out what is described in some embodiments when executed by one or more computer processors.

[0185]

[0090] As used throughout this specification and the entirety of the subsequent claims, “a,” “an,” and “the” include the plural form unless otherwise specified by the context. Also, as used throughout this specification and the entirety of the subsequent claims, “in” includes the meanings of “inside” and “on top” unless otherwise specified by the context.

[0186]

[0091] The above description illustrates various embodiments, along with examples of how aspects of several embodiments may be carried out. The above examples and embodiments should not be considered as the only embodiments, but are presented to illustrate the applicability and advantages of several embodiments as defined by the appended claims. Based on the above disclosure and the appended claims, other configurations, embodiments, and equivalents may be used without departing from the scope of this disclosure as defined by the claims.

Claims

1. Determining an instance of content to be delivered to a client device, wherein the instance of content includes multiple segments associated with multiple profiles, Determining the target metric and bandwidth allocation, Determining multiple top profile determinations for the multiple segments, wherein the top profile determination lists the top profiles that can be selected for each of the multiple segments. Based on the bandwidth allocation, the performance of the target metric is determined for the multiple top profile determinations, Selecting a top profile determination from the multiple top profile determinations based on the performance of the target metric for the multiple top profile determinations, The use of the top profile determination to determine the target quality, wherein the target quality is used to select the top profile for requesting a segment within the multiple segments during playback of the instance of the content, A method for providing this.

2. The method according to claim 1, wherein the target metric is selected from a first metric based on saved data and a second metric based on quality loss.

3. The method according to claim 1, wherein the bandwidth allocation is based on the download bandwidth probability.

4. The method according to claim 3, wherein the bandwidth allocation is based on information from the client device.

5. The method according to claim 1, wherein determining the target metric comprises receiving a target value for the target metric.

6. The method according to claim 5, wherein the target value of the target metric is adaptable for each instance of content or client device.

7. Determining the multiple top profile determinations is Determining the metric value of the quality metric of the profile within the aforementioned plurality of profiles, The method according to claim 1, comprising determining interval partitions based on the metric values ​​of the quality metric of a segment, wherein the interval partitions list a set of metric values ​​of the quality metric to form intervals associated with each profile.

8. Determining the multiple top profile determinations is Determining the division of segments for multiple segments within the aforementioned multiple segments, To determine a representation that includes a single segment representing the segment division of the plurality of segments. The method according to claim 7, comprising:

9. The above expression includes multiple intervals, Each section lists the top profile of each segment within multiple segments. The method according to claim 8, wherein the top profile is determined for each interval.

10. Determining the performance of the target metric for the multiple top profile determinations is To evaluate the probability of selecting a profile from the aforementioned multiple profiles, The method of claim 1, comprising predicting a first value of the target metric based on the performance of the metric using constraints relating to the top profile from the top profile determination, and a second value of the target metric without using the constraints, wherein the first value and the second value are used to determine a third value of the performance of the target metric used to select the top profile determination.

11. Determining the performance of the target metric for the multiple top profile determinations is Predicting the performance of the second metric using and without the constraints on the top profile derived from the top profile determination, The method of claim 10, comprising predicting a fourth value of the second metric based on the performance of the metric using the constraints relating to the top profile from the top profile determination, and a fifth value of the second metric without using the constraints, wherein the fourth and fifth values ​​are used to determine a sixth value of the performance of the second metric used to select the top profile determination.

12. The method according to claim 11, wherein the target metric is used to select the top profile determination based on the second metric.

13. Selecting the aforementioned top profile determination The method according to claim 1, comprising comparing the target value of the target metric with respect to each of the target profile determinations of the plurality of top profile determinations in order to select the top profile determination.

14. The method according to claim 13, wherein the top profile determination includes a target value of the target metric that is closer to the target metric than another top profile determination in the plurality of top profile determinations.

15. Comparing the target values ​​of the aforementioned target metrics is The method according to claim 13, further comprising using a target value of a second metric to select the top profile determination.

16. Using the top profile determination to determine the target quality, The method according to claim 1, further comprising selecting a target quality value based on the interval of quality values ​​associated with the top profile determination.

17. The method according to claim 1, wherein the top profile determinations in the plurality of top profile determinations are associated with different intervals of quality values.

18. A non-temporary computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are executed by a computing device, the computing device... Determining an instance of content to be delivered to a client device, wherein the instance of content includes multiple segments associated with multiple profiles, Determining the target metric and bandwidth allocation, Determining multiple top profile determinations for the multiple segments, wherein the top profile determination lists the top profiles that can be selected for each of the multiple segments. Based on the bandwidth allocation, the performance of the target metric is determined for the multiple top profile determinations, Selecting a top profile determination from the multiple top profile determinations based on the performance of the target metric for the multiple top profile determinations, The use of the top profile determination to determine the target quality, wherein the target quality is used to select the top profile for requesting a segment within the multiple segments during playback of the instance of the content, A non-temporary computer-readable storage medium that becomes operational for that purpose.

19. Determining the performance of the target metric for the multiple top profile determinations is To evaluate the probability of selecting a profile from the aforementioned multiple profiles, A non-temporary computer-readable storage medium according to claim 18, comprising predicting a first value of the target metric based on the performance of the metric using constraints relating to the top profile from the top profile determination, and a second value of the target metric without using the constraints, wherein the first value and the second value are used to determine a third value of the performance of the target metric used to select the top profile determination.

20. One or more computer processors, A device comprising a computer-readable storage medium having instructions for controlling one or more computer processors, wherein the one or more computer processors Determining an instance of content to be delivered to a client device, wherein the instance of content includes multiple segments associated with multiple profiles, Determining the target metric and bandwidth allocation, Determining multiple top profile determinations for the multiple segments, wherein the top profile determination lists the top profiles that can be selected for each of the multiple segments. Based on the bandwidth allocation, the performance of the target metric is determined for the multiple top profile determinations, Selecting a top profile determination from the multiple top profile determinations based on the performance of the target metric for the multiple top profile determinations, The use of the top profile determination to determine the target quality, wherein the target quality is used to select the top profile for requesting a segment within the multiple segments during playback of the instance of the content, A device that becomes operational for that purpose.

Images