System and method for an adaptive bitrate streaming ladder optimization

Abstract

A method to optimize an adaptive bitrate (ABR) ladder includes receiving a first rendition and a second rendition from an ABR ladder of a rendition encoded video asset, where the second rendition and the first rendition are an adjacent rendition pair. The method further includes estimating a rendition spacing ratio for the second rendition as a ratio between a bitrate of the first rendition and a bitrate of the second rendition, and optimizing the ABR ladder by removing at least one of the first rendition or the second rendition when the rendition spacing ratio is less than or equal to a rendition spacing ratio threshold.

Description

SYSTEM AND METHOD FOR AN ADAPTIVE BITRATE STREAMING LADDER OPTIMIZATIONCROSS-REFERENCE TO RELATED APPLICATIONS|O00] ] This application claims the benefit of U.S. provisional application Serial No. 63 / 571,538 filed March 29, 2024, the disclosure of which is hereby incorporated in its entirety by reference herein.TECHNICAL FIELD

[0002] The present disclosure generally relates to adaptive bitrate streaming (ABR) for streaming media content, such as videos.BACKGROUND

[0003] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.10004] Adaptive bitrate (ABR) streaming refers to a streaming system where the bit rate of a stream is dynamically adjusted to match bandwidth availability, customer settings, or other streaming criteria. ABR streaming may be used to deliver live or on-demand videos to devices such as, but not limited to, a gaming system, a television, a cinema projector and / or a streaming software application executed on a computing device (e.g., a laptop, tablet, and / or a smart phone).

[0005] To allow for the adaptation to be performed, ABR streaming utilizes multiple versions of the same video that differ in resolution and / or bitrate. Each version of the multiple versions of the video with a different resolution and bitrate is referred to herein as a rendition.

[0006] Enhancements to making ABR more efficient is often necessary to get acceptable quality improvements of video assets streamed from a source location through an end-to-end system to a viewing device. ABR enhancement methods can be used in conjunction with existing ABRalgorithms. Disclosed are methods and systems that further optimize the perceptual quality of an ABR ladder where transmission bitrates are limited.SUMMARY

[0007] In some aspects, the present disclosure is directed to a method to optimize an adaptive bitrate (ABR) ladder. The method includes receiving a first rendition and a second rendition from an ABR ladder of a rendition encoded video asset, where the second rendition and the first rendition are an adjacent rendition pair. The method further includes estimating a rendition spacing ratio for the second rendition as a ratio between a bitrate of the first rendition and a bitrate of the second rendition, and optimizing the ABR ladder by removing at least one of the first rendition or the second rendition when the rendition spacing ratio is less than or equal to a rendition spacing ratio threshold.

[0008] In another aspect, the present disclosure is directed to enhancing a system for optimizing an adaptive bitrate (ABR) ladder. The system includes one or more hardware computing devices configured to: receive a first rendition and a second rendition from an ABR ladder of a rendition encoded video asset, the second rendition and the first rendition being an adjacent rendition pair; estimate a rendition spacing ratio for the second rendition as a ratio between a bitrate of the first rendition and a bitrate of the second rendition; and optimizing the ABR ladder by removing at least one of the first rendition or the second rendition when the rendition spacing ratio is less than or equal to a rendition spacing ratio threshold.BRIEF DESCRIPTION OF THE DRAWINGS10009] In order that the disclosure may be well understood, there will now be described various forms thereof, given by way of example, reference being made to the accompanying drawings, in which:

[0010] FIG. 1 A illustrates an example system for streaming videos in accordance with the present disclosure;

[0011] FIG. IB is an example block diagram of the system of FIG. 1A;

[0012] FIG. 2 is a block diagram of an ABR system in accordance with the present disclosure;

[0013] FIG. 3 is a flowchart of an example ABR ladder optimization routine in accordance with the present disclosure; and

[0014] FIG. 4 illustrates an example of a computing device for use in the determination of an ABR ladder for streaming video content.

[0015] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.DETAILED DESCRIPTION

[0016] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.

[0017] Referring to FIGS. 1A and IB, an end-to-end video delivery system 100 is configured to stream live or on-demand media content from a pre-recorded source 101, such as a source that has Hollywood cinematic content. The system 100 includes a video streaming system 102 configured to obtain video content 103 (e.g., a video file) from one or more video service servers (e.g., box 120 of FIG. IB) for managing a plurality of live and / or pre-saved video content 103. The video streaming system 102 performs a pre-delivery processing (e.g., box 122 of FIG. IB) to provide the video content 103 to multiple player devices 104 of the system 100 that are in communication with the video streaming system 102 using, as a delivery network 124, wireless communication (e.g., cellular network, Wi-Fi, satellite, BLUETOOTH, and / or among other communication techniques) or other networks that have bandwidth limitations such was wire (e.g., twisted pair) or cable (e.g., coax cable) or combinations that could involve wireless, and wire and cable and fiber. The video streaming system 102 can be integrated as an upgrade into a pre-delivery processor that does the pre-delivery process 122, the pre-delivery process can be located at the location of the video source providing the video asset or the location that provides video content to the delivery network 124.

[0018] The video content 103 may be live video feeds from current events, pre-recorded shows or movies, and advertisements or other clips to be inserted into other video feeds. The content 103 may include just video in some examples, but in many cases the video content 103 further includes additional content such as audio, subtitles, and metadata information descriptive of the content and / or format of the video. The video content 103 may comprise a series of one or more frames to be displayed in succession, which may be encoded in any of various resolutions (e.g., 720p, 1080p, WUXGA, 2K, ultra-high definition (UHD), cinema 4K, 8K, etc.), frame rates (e.g., 24fps, 30fps, 60 fps, etc.), dynamic ranges (e.g., standard dynamic range (SDR), high dynamic range (HDR), and color spaces (Y’Cb’Cr’, RGB, etc.).

[0019] In one form, the video streaming system 102 includes an adaptive bitrate streaming (ABR) system 203 configured to split video content 103 into small segments (e.g., of 2 to 20 seconds lengths). An ABR ladder refers to a set of renditions of the video, with each rendition having multiple time aligned segments. To facilitate the ABR streaming, the media content is encoded at each of the different renditions defined by the ABR ladder. During playback, a player device 104 of the end-to-end system requests the streamed video files of a rendition, choosing the best quality rendition that has a bitrate which is within the available bandwidth of the player device. The player device 104 in the end-to-end streaming system can be a device that is able to interface with the delivery network, process a received video asset from the delivery network (e.g., post delivery processing at box 126 of FIG. IB), and display the received video asset for viewing. The player device 104 can be multiple devices, for example the first device for receiving the video asset from the network and processing the received video asset before outputting the video asset to a second device that is a viewing device for displaying the video asset. Accordingly, the term “player device” may be interpreted to include a viewing device and a viewing device is to be interpreted to include a player device.

[0020] The ABR system 203 may be further configured to process each segment in a plurality of renditions (e.g., video segment renditions 110-1, 110-2, 110-3, ... ,110-N; collectively video segment renditions 110) having different bitrate, and in some instances, different resolutions. Furthermore, a rendition can be encoded content based on a specific video encoder configuration. A video encoder configuration refers to a set of encoding parameters that influence the content being encoded and can include but is not limited to bitrate, resolution, video filtering, and codec.

[0021] In another form, the video streaming system 102 may encode an entire video sequence in several renditions and then segment each rendition such that each segment for each rendition represents the same period in time for playback.

[0022] A user is able to select a rendition from among a plurality of available renditions for streaming the video segment renditions 110. In a non-limiting example, the user is able to check an available internet bandwidth using a desired player device 104 and selects the appropriate rendition. FIG 2. indicates rendition 110a specific to a player device 104-1 and rendition 110b specific to player device 104-2. In one form, the player device 104 may include, but is not limited to, smartphone, tablet, laptop, a cinema projector, and / or television system with streaming capabilities.

[0023] Referring to FIG. 2, in one form, the video streaming system 102 includes a video rendition encoder 202 and the ABR system 203 having a bitrate optimizer array 204 including a plurality of rendition segment selector processes 206-1, 206-2, 206-3, ... ,206-N (collectively segment selector 206) and an ABR ladder optimization (ABR-LO) module 208.

[0024] In one form, the video rendition encoder 202 is configured to generate a plurality of initial renditions having fixed bitrate of the video content 103 (e.g., a fixed bitrate rendition) that may be based on a selected ABR standard, such as Apple HLS. In a non-limiting example, Table 1 illustrates nine (9) different renditions of a video, which are listed in descending order based on bitrate (e.g., rendition R1 has a bitrate of 7,184,190 bps and rendition R9 has a bitrate of 173,850 bps). The video rendition encoder 202 outputs fixed video bitrate renditions 210-1, 210-2, 210-3, ... , 210-N (e.g., collectively fixed renditions 210) to the ABR system 203. The bitrate for each rendition is calculated based on the entire rendition of the video asset 103. In a non-limiting example, the video rendition encoder 202 includes various hardware and / or software implementing a series of algorithms to encode the video contents 103 into separate renditions.Table 1: Example Fixed Video Rendition

[0025] In one form, the bitrate optimizer array 204 is configured to estimate quality characteristics such as perceptual quality characteristics of fixed video renditions. The perceptual quality characteristics are quality characteristics that can be perceived by the human vision system (HVS). The perceptual quality characteristics is a subset of all possible quality characteristics that could be used in measures for scoring quality. In one embodiment a reference perceptual quality score may be determined on a quality score that measures the perceptual quality difference of perceptual quality characteristics between a source video 103 and the content being rendition encoded for each rendition segment (210-1 to 210-N). A quality score may be an encoder performance score (EPS) or a banding score. An encoder performance score (EPS), as used herein, is a measure of the encoder’s ability to retain the perceptual source video quality. The banding score is a measurement of the amount of color banding or color contouring present within each video rendition segment. In another embodiment, if the video asset 103 is not available within the bitrate optimizer array 204, a no-reference perceptual quality score can be determined on a score that estimates the difference of perceptual quality characteristics between the received video before rendition encoding and the content after being rendition encoded. Additionally, the quality scoremay be the absolute perceptual quality of the video content. Quality may be measured by a metric that represents either a subjective or objective value, where the objective metric may be modeled by gathering mean opinion score (MOS) of subjects. Other types of quality scores that may be used to estimate the quality characteristics of the various renditions may include, but is not limited to: structural similarity index measure (SSIM), video multimethod assessment fusion (VMAF), peak signal-to-noise ratio (PSNR), multi-scale SSIM (MS-SSIM), and variance inflation factor (VIF). Additionally or alternatively to use of the banding score, contrast-aware multiscale banding index (CAMBI) may be used as a measure of banding artifacts.

[0026] In one embodiment, the bitrate optimizer array 204 may be configured to compress the bitrate renditions of the video rendition encoder 202 to generate adjusted bitrate video renditions 212-1, 212-2, ... ,and 212-N (collectively adjusted bitrate renditions 212), which have a reduced bitrate while maintaining video quality. Stated differently, in some aspects, a plurality of encoders are used to provide a plurality of optimized renditions (e.g., adjusted bitrate video renditions), which are then analyzed to further optimize the ABR ladder and define an enhanced optimized ABR ladder. In a non-limiting example, the bitrate optimizer array 204 and more particularly, the rendition segment selection process 206 may be applied to existing ABR optimization processes to maintain perceptible quality while reducing bitrate. In another embodiment, the bitrate optimizer array 204 may be applied to at least a portion of the fixed renditions 210 by selecting rendition segments where the lowest bitrate rendition that has similar perceptual quality of the adjacent higher bitrate rendition.

[0027] For example, based on the fixed video rendition of Table 1, the rendition segment selection process 206 is applied to the first four fixed renditions (Rl, R2, R3 and R4) of Table 1 and Table 2, shown below, provides the reduced bitrate of the first four renditions after the rendition segment selection process 206 has been applied to produce Rl *, R2*, R3* and R4*, The produced renditions Rl*, R2*, R3* and R4* are adjusted fixed bitrate renditions outputted in FIG. 2 as 212-1 .... 212-4. In addition, to the video rendition encoders 202, the bitrate optimizer array 204 may be configured to estimate quality characteristics of the adjusted bitrate renditions. Examples of quality characteristics may be, but not limited to, the EPS, the banding score, the film grain quality score, etc. In table 2, bitrate reduction is limited to four renditions of the 9 renditions in which bitrate compression algorithms may be applied to renditions Rl, R2, R3 and R4 toproduce adjusted bitrate renditions Rl*, R2*, R3* and R4* respectively. The largest gains to be had in reducing a rendition bitrate may occur with higher bitrate and higher resolution renditions. For renditions with low bitrates and low resolutions may not provide the same level of optimization as the renditions with high bitrate and high resolution due to the large scaling ratio of source content 103 and rendition resolution. For example, in the segment selection process 206 the just noticeable difference (JND) is a dynamic measure dependent on the EPS level. The EPS JND is above 2 when the EPS is over 50, the EPS JND approaches 1 when the EPS is below 50 and the EPS JND approaches 0 when the EPS is below 30. For lower resolution renditions the EPS JND can be very low which gives the lower resolution renditions a reduced effectiveness for bitrate savings.Table 2: Example Renditions After Bitrate Reduction

[0028] In one embodiment, the renditions generated (210-1 .... 210-N) by the video rendition encoder 202 and / or the renditions generated (212-1 .... 212-N) by the bitrate optimizer array 204 may become an initial ABR ladder for streaming the segment, or may be applied to the ABR-LOmodule 208 for further optimization. The ABR-LO module 208 may be configured to reduce the number of renditions based on bitrate spacing between renditions. As described herein, the ABR- LO module 208 is configured to identify narrow rendition spacings between adjacent renditions as candidates for rendition removal.

[0029] In one form, the ABR-LO module 208 is configured to include a rendition spacing comparer 220, a bitrate rendition reducer 222, and an ABR ladder generator 224. Referring to FIG. 3, an example ABR ladder optimization routine 300 is provided and executed by the ABR system 203 having the ABR-LO module 208 of the present disclosure. At operation 301, the ABR system 203 receives an initial ABR ladder. If the initial ABR ladder is not received an initial ABR ladder may be generated from a received video asset 103. In the following example, the scores used to evaluate the renditions include EPS and banding, however, other suitable scores may be employed, some of which are identified above.

[0030] At operation 302, the rendition spacing comparer 220 is configured to estimate a rendition spacing (RS) between an adjacent pair of bitrate renditions of an adjusted initial ABR ladder (e.g. between R1 & R2 or between R2 & R3 etc.). The rendition spacing is a metric that indicates a difference or differential between the adjacent rendition pair of bitrate renditions. In one form, the rendition spacing is provided as a rendition ratio for the adjacent rendition pair of bitrate renditions. In non-limiting example, Table 2 provides an initial ABR ladder having nine bitrate renditions, and the rendition spacing (RS) is indicated by the rendition ratio. As provided, the rendition ratio between renditions R1 and R2 is approximately 1.25 (e.g., R1 / R2 = 3,952,869 / 3,166,293 ~ 1.25.

[0031] Once obtained, the rendition spacing comparer 220 is configured to determine whether a rendition from among the plurality of bitrate renditions may be removed / eliminated, at operation 304. Specifically, the rendition spacing comparer 220 is configured to identify bitrate renditions that are too close or, stated differently, that are too narrowly spaced. In one form, the rendition ratio is selected based on a recommended rendition ratio and more specifically, less than a recommended minimum rendition spacing. For example, if the rendition ratio is recommended to be between 1.3 to 2.5, the rendition ratio threshold could be a value less than 1.3.

[0032] Referring to Table 2, with the rendition ratio threshold less than 1.3, the rendition ratio 0.82 is less than 1.3 indicating that the adjacent rendition pair of bitrate renditions R4* and R5 arespaced close together (e.g., R4*= 1,608,504 and R5 = 1,969,907). However, when rendition R5 is removed the updated spacing is now between rendition R4* and rendition R6 which has an updated rendition spacing ratio of 1.44. In the following, a rendition ratio that is less than or equal to the rendition ratio threshold, is referenced as a narrow rendition ratio.

[0033] In response to detecting the narrow rendition ratio, the bitrate rendition reducer 222 is configured to remove a selected bitrate rendition from the initial ABR ladder to define an optimized ABR ladder. Specifically, the bitrate rendition reducer 222 is configured to select a bitrate rendition from among the adjacent rendition pairs of bitrate renditions associated with the narrow rendition ratio based on a perceptual quality characteristic associated with each bitrate rendition, such as but not limited to the quality score (e.g., EPS) and / or the banding score. That is, since a rendition is being eliminated, the quality of the videos from the ABR ladder should not be affected, and therefore, the bitrate rendition to be eliminated is based on video quality. The quality score used for the rendition elimination algorithm in the bitrate rendition reducer 222 must be calculated for the rendition in its entirety and not just the individual segments that compose the rendition. Specifically, quality characteristic scores, such as other quality scores and banding scores, provide a summary metric assessing an overall perceptual quality of experience a viewer has after watching at least a portion of the video.|0034[ To eliminate a rendition the bitrate rendition reducer 222 is configured to calculate an EPS differential, which is an example of a quality differential, between the quality scores associated with the adjacent rendition pair of bitrate renditions, at operation 306. In a non-limiting example, the EPS differential is provided as a difference between the EPSs associated with the adjacent rendition pair of bitrate renditions.

[0035] At operation 308, the bitrate rendition reducer 222 determines if the difference between the EPSs, as the EPS differential, is greater than or equal to an EPS threshold, which may also be referred to as an EPS differential threshold, (e.g., EPS threshold = 1). In one form, the EPS threshold may be selected as a user input where the EPS has been defined through testing for certain content and then leveraged via video asset content classification. Such as having a database that has been trained with the objective of specifying different EPS threshold for each differentcontent scenes or other meaningful classification. Classifications can include scenes with different textures, or motions, or color, or actions, etc.

[0036] If the EPS differential is greater than or equal to the EPS threshold, the bitrate rendition reducer 222 is configured to select the bitrate rendition having the lower EPS from the adjacent rendition pair of bitrate renditions as the selected bitrate rendition to be removed, in operation 310. For example, in Table 2, the EPS differential for the adjacent rendition pair R4* & R5 with a bitrate rendition score of 0.82 is equal to 5.59 (e.g., 76.27 -70.68), which is greater than the EPS threshold of one (1). In this example, the bitrate rendition reducer 222 would remove the rendition R5, which has the lower EPS. If R5 had the higher EPS value between R4* and R5, then R4* would be removed. An example of the optimized ABR ladder is provided in Table 3 below.Table 3: Example Final ABR Ladder

[0037] The bitrate rendition reducer 222 may be configured to use two or more quality characteristics such as perceptual quality characteristics to estimate a quality score for each perceptual quality characteristic for each rendition. A combined quality score using each of the estimated perceptual quality scores for a rendition may be determined by weighting each estimate perceptual quality score for the rendition and summing all of the weighted estimate perceptualquality scores to get an estimate weighted quality characteristic score for the rendition. Based on the estimate weighted quality characteristic score of each rendition a weighted quality characteristics differential score can be determined for each adjacent rendition pair. By determining and applying a weighted quality characteristic differential threshold criteria to the weighted quality characteristics differential score the removal of the rendition can be determined.

[0038] For example if the EPS differential is less than the EPS threshold, the bitrate rendition reducer 222 may be configured to estimate a weighted quality characteristics (WQC) for each adjacent rendition pair of the bitrate renditions, at operation 312 and the bitrate rendition having the lower WQC score is selected to be removed from the initial ABR ladder, at operation 314. In one form, the weighted quality characteristic is estimated using the EPSs and the banding scores of the adjacent rendition pair of the bitrate renditions. In a non-limiting example, the EPS is weighted 3.2x and added to the banding score and may be represented as a WQC score = (3.2 *EPS)+banding score. Although R4* and R5 meet the EPS differential inquiry, for the purpose of explanation, R4* may have WQC score of approximately 272.86 (WQS = 3.2*76.27+28.8 = 272.86) and R5 has a WQC score of approximately 258.65 (WQS = 3.2*70.68+32.47 = 258.65). Accordingly, R5 would be removed from the ABR ladder. It should be readily understood that the WQC score may be determined using other suitable weighted algorithms using quality characteristics and should not be limited to the example provided.

[0039] In the event that the rendition spacing comparer 220 detects multiple narrow rendition ratios, the bitrate rendition reducer 222 is configured to iteratively evaluate the narrow rendition ratios until no spacing is below the rendition elimination threshold, and the remaining renditions form the optimized ABR ladder to be employed. For example, at operation 316, the ABR-LO module 208 is configured to determine if all of the adjacent rendition pairs of the initial ABR ladder were evaluated. If not, the ABR-LO module 208 goes to the next pair of adjacent renditions and returns to operation 304. For example, the next adjacent rendition is between R4* and R6.

[0040] If all of the adjacent rendition pairs are evaluated, the ABR ladder generator 224 is configured to output a plurality of video segment renditions, such as video segment renditions 110, to be streamed using the optimized ABR ladder, at operation 320.

[0041] If the rendition spacing comparer 220 does not identify any narrow rendition ratios (i.e., all of the rendition spacings is greater than the rendition spacing threshold), the bitrate rendition reducer does not eliminate any renditions, and the initial ABR ladder having the plurality of bitrate renditions as the optimized ABR ladder.

[0042] The ABR-LO routine 300 is just one example of ABR ladder optimization routine, and the routine may be configured to in other suitable ways will with the scope of the present disclosure. The following provides some example variations that may be employed separately or in combination.

[0043] In some variations, the ABR-LO module 208 is configured to analyze selected pairs, and is not required to analyze all of the adjacent rendition pairs.

[0044] While FIG. 2 illustrates the ABR-LO module 208 as being performed after the bitrate optimizer array 204, the ABR-LO module 208 may be executed at other suitable locations. For example, the rendition encoding 202 or the rendition encoding 202 and processes of the ABR system 203 could be done at the source location providing the video asset or at another location where pre-delivery processing with the video streaming system 102 is executed for preparing the video asset 103 to be sent via the delivery network 124. In some variations, the ABR-LO module 208 is configured to process at least a portion or all of the fixed bitrate renditions from the video rendition encoder 202. For example, all of the fixed bitrate renditions except for the highest fixed bitrate rendition may be processed by the ABR-LO module 208 to eliminate narrow rendition ratios, and the highest fixed bitrate rendition along with at least a portion or all of the remaining renditions from the ABR-LO module 208 may be provided to the bitrate optimizer array 204 to generate the optimized ABR ladder. In this variation, the ABR-LO module 208 is configured to determine the number of renditions to be kept from the video rendition encoder 202. For renditions that have not been processed by the ABR-LO module 208 and have not been yet encoded by the video rendition encoder 202 a target bitrate may be used where the target bitrate is based on knowledge from processing other renditions.

[0045] In one variation of the processes described above, when a player device 104 requests the adjusted ABR ladder from the delivery network 124, the player device may provide viewing device parameters 230 (e.g. 230-1 and 230-1 collectively described as 230) that may describe but is notlimited to the resolution of the viewing device, the physical size of the viewing device, as well as the brightness of the device. The delivery network 124 would forward the viewing device parameters 230 to the ABR-LO module 208. The ABR-LO module 208 may use the viewing device parameters 230 to augment the EPS value and other quality scores that may be used in routine 300. The rendition spacing comparer 220 and bitrate rendition reducer 222 may adjust the estimate EPS value and modify the EPS threshold to a value that more accurately represents the quality score for the specific viewing device parameters 230. In addition to the estimate EPS value being adjusted, the rendition spacing comparer 220 and bitrate rendition reducer 222 may adjust other estimate quality scores including the banding score and the film grain quality score based on the viewing device parameters 230 to more accurately represent these quality scores for the specific viewing device parameters 230. Routine 300 will run until completion and a device adapted optimized ABR ladder will be provided to player device 104 for playback of a specific rendition.

[0046] While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.

[0047] In this application, the features of the video streaming system 102 may be provided at a single server or distributed across multiple servers and / or computing devices.

[0048] FIG. 4 illustrates an example 400 of a computing device 402 for use in defining the ABR ladder and streaming the video content 103. Referring to FIG. 4, and with reference to FIGS. 1-3, the devices and modules discussed herein may be examples of such computing devices 402. For instance, the operations performed by the video encoder 202 and the ABR system 203 having the bitrate optimizer array 204 with segment selectors 206 and the ABR-LO module 208 with the rendition spacing comparer 220, the bitrate rendition reducer 222, and the ABR ladder generator 224, etc., as well as the operations discussed in routine 300 may be performed by such computing devices 402. As shown, the computing device 402 includes a processor 404 that is operatively connected to a storage 406, a network device 408, an output device 410, and an input device 412.It should be noted that this is merely an example, and computing devices 402 with more, fewer, or different components may be used.

[0049] The processor 404 may include one or more integrated circuits that implement the functionality of a central processing unit (CPU) and / or graphics processing unit (GPU). In some examples, the processors 404 are a system on a chip (SoC) that integrates the functionality of the CPU and GPU. The SoC may optionally include other components such as, for example, the storage 406 and the network device 408 into a single integrated device. In other examples, the CPU and GPU are connected to each other via a peripheral connection device such as peripheral component interconnect (PCI) express or another suitable peripheral data connection. In one example, the CPU is a commercially available central processing device that implements an instruction set such as one of the x86, ARM, Power, or microprocessor without interlocked pipeline stage (MIPS) instruction set families.

[0050] Regardless of the specifics, during operation the processor 404 executes stored program instructions that are retrieved from the storage 406. The stored program instructions, accordingly, include software that controls the operation of the processors 404 to perform the operations described herein. The storage 406 may include both non-volatile memory and volatile memory devices. The non-volatile memory includes solid-state memories, such as not and (NAND) flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when the system is deactivated or loses electrical power. The volatile memory includes static and dynamic random-access memory (RAM) that stores program instructions and data during operation of the system 102.1 051 ] The GPU may include hardware and software for display of at least two-dimensional (2D) and optionally 3D graphics to the output device 410. The output device 410 may include a graphical or visual display device, such as an electronic display screen, projector, printer, or any other suitable device that reproduces a graphical display. As another example, the output device 410 may include an audio device, such as a loudspeaker or headphone. As yet a further example, the output device 410 may include a tactile device, such as a mechanically raisable device that may, in an example, be configured to display braille or another physical output that may be touched to provide information to a user.

[0052] The input device 412 may include any of various devices that enable the computing device 402 to receive control input from users. Examples of suitable input devices that receive human interface inputs may include keyboards, mice, trackballs, touchscreens, voice input devices, graphics tablets, and the like.

[0053] The network devices 408 may each include any of various devices that enable the devices to send and / or receive data from external devices over networks. Examples of suitable network devices 408 include an Ethernet interface, a Wi-Fi transceiver, a cellular transceiver, or a BLUETOOTH or Bluetooth Low Energy (BLE) transceiver, ultra-wideband (UWB) transceiver, or other network adapter or peripheral interconnection device that receives data from another computer or external data storage device, which can be useful for receiving large sets of data in an efficient manner.

[0054] The processes, methods, or algorithms disclosed herein can be deliverable to / implemented by a processing device, controller, or computer, which can include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, the processes, methods, or algorithms can be stored as data and instructions executable by a controller or computer in many forms including, but not limited to, information permanently stored on non- writable storage media such as read-only memory (ROM) devices and information alterably stored on writable storage media such as floppy disks, magnetic tapes, compact discs (CDs), RAM devices, and other magnetic and optical media. The processes, methods, or algorithms can also be implemented in a software executable object. Alternatively, the processes, methods, or algorithms can be embodied in whole or in part using suitable hardware components, such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), state machines, controllers or other hardware components or devices, or a combination of hardware, software, and firmware components.

[0055] While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms encompassed by the claims. The words used in the specification are words of description rather than limitation, and it is understood that various changes can be made without departing from the spirit and scope of the disclosure. As previously described, the features of various embodiments can be combined to form further embodiments ofthe invention that may not be explicitly described or illustrated. While various embodiments could have been described as providing advantages or being preferred over other embodiments or prior art implementations with respect to one or more desired characteristics, those of ordinary skill in the art recognize that one or more features or characteristics can be compromised to achieve desired overall system attributes, which depend on the specific application and implementation. These attributes can include, but are not limited to strength, durability, life cycle, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. As such, to the extent any embodiments are described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics, these embodiments are not outside the scope of the disclosure and can be desirable for particular applications.

Claims

WHAT IS CLAIMED IS:

1. A method to optimize an adaptive bitrate (ABR) ladder, comprising: receiving a first rendition and a second rendition from an ABR ladder of a rendition encoded video asset, the second rendition and the first rendition being an adjacent rendition pair; estimating a rendition spacing ratio for the second rendition as a ratio between a bitrate of the first rendition and a bitrate of the second rendition; and optimizing the ABR ladder by removing at least one of the first rendition or the second rendition when the rendition spacing ratio is less than or equal to a rendition spacing ratio threshold.

2. The method of claim 1, wherein the first rendition and the second rendition are fixed bitrate renditions.

3. The method of claim 1, wherein the rendition spacing ratio threshold is set to one.

4. The method of claim 1, further comprising: generating an adjusted first rendition that is the first rendition with a compressed bitrate and an adjusted second rendition that is the second rendition with a compressed bitrate; and estimating a rendition spacing ratio for the second rendition as a ratio between a bitrate of the adjusted first rendition and a bitrate of the adjusted second rendition.

5. The method of claim 1, further comprising: estimating a first encoder performance score (EPS) for the first rendition; estimating a second EPS for the second rendition; calculating an EPS differential of the adjacent rendition pair; and removing at least one of the first rendition or the second rendition based on the EPS differential and an EPS differential threshold in response to the rendition spacing ratio being less than or equal to the rendition spacing ratio threshold.

6. The method of claim 5, wherein when the EPS differential is less than or equal to the EPS differential threshold, a selected rendition having the lower EPS value is removed, the selected rendition is selected from among the first rendition and the second rendition.

7. The method of claim 5, wherein the estimating of the first EPS and the estimating of the second EPS is based on a device parameter of a viewing device.

8. The method of claim 5, wherein the EPS differential threshold is based on a device parameter of a viewing device.

9. The method of claim 1, further comprising: estimating at least two quality characteristic scores of the first rendition; estimating at least two quality characteristics of the second rendition; determining a first weighted quality characteristic (WQC) score for the first rendition using a weighting algorithm to combine the at least two quality characteristic scores of the first rendition; determining a second WQC score for the second rendition using the weighting algorithm to combine the at least two quality characteristic scores of the second rendition; calculating a WQC differential score for the second rendition using the first WQC score and the second WQC score; and removing at least one rendition based on a WQC differential score threshold and the WQC differential score in response to the rendition spacing ratio being less than or equal to the rendition spacing ratio threshold.

10. The method of claim 9, the rendition with the lower WQC score value is removed in response to the WQC differential score being less than or equal to the WQC differential score threshold.

11. The method of claim 9, wherein the quality characteristic scores are based on a perceptual quality characteristic that is at least one of a color banding, a film grain quality, or an encoder performance.

12. The method of claim 1, further comprising streaming a plurality of video segment renditions within an end-to-end video system using the optimized ABR ladder.

13. A pre-delivery video processor configured to perform the method of claim 1 as part of an end-to-end system for delivering a video.

14. A non- transitory computer- readable medium comprising instructions for defining the ABR ladder that, when executed by one or more hardware computing devices cause the one or more hardware computing devices to perform the method of claim 1.

15. A system for defining an adaptive bitrate (ABR) ladder, comprising: one or more hardware computing devices configured to: receive a first rendition and a second rendition from an ABR ladder of a rendition encoded video asset, the second rendition and the first rendition being an adjacent rendition pair; estimate a rendition spacing ratio for the second rendition as a ratio between a bitrate of the first rendition and a bitrate of the second rendition; and optimizing the ABR ladder by removing at least one of the first rendition or the second rendition when the rendition spacing ratio is less than or equal to a rendition spacing ratio threshold.

16. The system of claim 15, wherein the first rendition and the second rendition are fixed bitrate renditions.

17. The system of claim 15, wherein the rendition spacing ratio threshold is one.

18. The system of claim 15, wherein the one or more hardware computing devices is further configured to: generate an adjusted first rendition that is the first rendition with a compressed bitrate and an adjusted second rendition that is the second rendition with a compressed bitrate; andestimate a rendition spacing ratio for the second rendition as a ratio between a bitrate of the adjusted first rendition and a bitrate of the adjusted second rendition.

19. The system of claim 15, wherein the one or more hardware computing devices is further configured to: estimate a first encoder performance score (EPS) for the first rendition; estimate a second EPS for the second rendition; calculate an EPS differential of the adjacent rendition pair; and remove at least one of the first rendition or the second rendition based on the EPS differential and an EPS differential threshold in response to the rendition spacing ratio being less than or equal to the rendition spacing ratio threshold.

20. The system of claim 19, wherein when the EPS differential is less than or equal to the EPS differential threshold, a selected rendition having the lower EPS value is removed, the selected rendition is selected from among the first rendition and the second rendition.

21. The system of claim 19, wherein the estimating of the first EPS and the estimating of the second EPS is based on a device parameter of a viewing device.

22. The system of claim 19, wherein the EPS differential threshold is based on a device parameter of a viewing device.

23. The system of claim 15, wherein the one or more hardware computing devices is further configured to: estimate at least two quality characteristic scores of the first rendition; estimate at least two quality characteristics of the second rendition; determine a first weighted quality characteristic (WQC) score for the first rendition using a weighting algorithm to combine the at least two quality characteristic scores of the first rendition; determine a second WQC score for the second rendition using the weighting algorithm to combine the at least two quality characteristic scores of the second rendition;calculate a WQC differential score for the second rendition using the first WQC score and the second WQC score; and remove at least one rendition based on a WQC differential score threshold and the WQC differential score in response to the rendition spacing ratio being less than or equal to the rendition spacing ratio threshold.

24. The system of claim 23, wherein, the rendition with the lower WQC score value is removed in response to the WQC differential score being less than or equal to the WQC differential score threshold.

25. The system of claim 23, wherein the quality characteristic scores are based on a perceptual quality characteristic that is at least one of a color banding, a film grain quality, or an encoder performance.

26. The system of claim 23, wherein: the one or more hardware computing devices defines at least a portion of a pre-delivery process of an end-to-end video delivery system, and the one or more hardware computing devices is configured to receive a viewing device parameter from a viewing device.

Images