Adaptive bitrate method for live broadcasting
Adaptive bitrate techniques with shifted distribution layers and dynamically generated layers address the challenges of startup latency and rebuffering in live media streaming, providing seamless and high-quality streaming experiences.
Patent Information
- Application Number
- JP2024020633
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2018-04-26
- Filing Date
- 2024-02-14
- Publication Date
- 2025-10-06
- Estimated Expiration
- 2039-04-26
AI Technical Summary
Existing live media streaming technologies face challenges in maintaining continuity and minimizing startup latency and rebuffering due to varying network conditions, leading to suboptimal user experiences.
The implementation of adaptive bitrate techniques with shifted distribution layers and dynamically generated layers to ensure seamless streaming by providing multiple keyframes at different time positions, allowing clients to seamlessly switch between bitrates based on network conditions.
This approach reduces startup latency and rebuffering, ensuring high-quality live media streaming across diverse network conditions, enhancing user experience.
Smart Images

Figure 0007749724000001 
Figure 0007749724000002 
Figure 0007749724000003
Abstract
Description
[Technical Field]
[0001] The disclosed teachings relate to live media streaming over packet-switched networks, and more particularly to adaptive rate selection for live streaming media content over such networks. [Background technology]
[0002] Streaming video content over the Internet is rapidly gaining popularity as a way of displaying video content. One such form of streaming video content is live video content such as live action sports, device-to-device video calls, and live news reports. Electronic devices (e.g., smartphones, computers, tablets) can access (or "subscribe to") video streams over the Internet.
[0003] An important criterion for streaming video content over the Internet is the quality level, which indicates the quality of the streamed video. This quality level can be determined by the bit rate selected to transmit the video content over the Internet. A higher bit rate indicates that more information in the video content is encoded and the reproduction of the original video content will be more accurate. In contrast, a lower bit rate indicates that less information is encoded and the reproduction of the original video content may be less accurate.
[0004] The bit rate for transmitting video content depends on the network speed of the client device. the desired startup latency (i.e., the latency experienced when first initializing video playback) delays) and tolerance to glitches / rebuffering (i.e., video content This depends on several factors, such as the tolerance for video playback stopping due to missing data. The client device can make a selection based on the video content. Start-up latency or rebuffering during streaming is undesirable. , while streaming higher bitrate video streams. It is desirable to minimize or eliminate start-up latency and rebuffering of video content. It's nice. Summary of the Invention
[0005] This disclosure relates to techniques for maintaining continuity in live media streams. The media stream has at least one key frame at a first time in the time domain. The first distribution layer at the first bitrate corresponds to the first keyframe. This media stream also contains a set of delta frames. The distribution layer distributes the set of delta frames following the key frame to other layers. Many additional disks containing at least one keyframe that is time-shifted relative to the A contribution layer may also be included.
[0006] The streaming server takes the first distribution layer of the media stream. The streaming server can obtain a time-shifted stream from the first time. Generate an additional distribution layer of media streams with keyframes Additional distribution tiers can be The generated image may be generated based on identifying an instruction representing a request to generate the image. A media stream can be sent to at least one client device. do.
[0007] The client device is a first distribution layer or You can subscribe to one of the additional distribution tiers. The device subscribes to an additional distribution layer of the media stream at a second time. Live, minimize startup time, and handle media content of varying complexity and dynamic This allows for optimal quality of experience across a wide range of network conditions.
[0008] Other aspects of the technology will become apparent from the accompanying figures and detailed description.
[0009] This Summary presents a selection of concepts in a simplified form that are further described below in the Detailed Description. This Abstract is provided to identify key or essential features of the claimed subject matter. is not intended to be specific and is used to limit the scope of the claimed subject matter; is not intended either. [Brief explanation of the drawings]
[0010] [Figure 1] 1 illustrates a system for streaming live media content to multiple client devices, according to various embodiments. [Figure 2] 1 illustrates a streaming server architecture according to various embodiments. [Figure 3] 1 illustrates the architecture of a client device, according to various embodiments. [Figure 4] 1 illustrates an encoded media stream according to various embodiments. [Figure 5] 1 illustrates an encoded media stream with a set of distribution layers according to various embodiments. [Figure 6]1 illustrates an encoded media stream with adaptive bitrate according to various embodiments. [Figure 7A] 1 illustrates adaptive bitrate streaming for live broadcasting with multiple subscribers, according to various embodiments. [Figure 7B] 1 illustrates adaptive bitrate streaming for live broadcasting in dynamic network conditions, according to various embodiments. [Figure 8A] 1 illustrates a set of encoded media data with shifted distribution layers according to various embodiments. [Figure 8B] 1 illustrates a set of encoded media data with multiple bitrate shifted distribution layers according to various embodiments. [Figure 9] 1 illustrates encoded streaming data with dynamically generated distribution layers and key frames according to various embodiments. [Figure 10] 1 illustrates a set of distribution layers and dynamically generated layers for an encoded media stream according to various embodiments. [Figure 11] 1 illustrates a set of distribution layers and bridge distribution layers for an encoded media stream according to various embodiments. [Figure 12A] 1 illustrates an architecture for dynamically generating a distribution tier at an origin streaming server, according to various embodiments. [Figure 12B] 1 illustrates an architecture for dynamically generating distribution layers along a data path, according to various embodiments. [Figure 13] 1 illustrates a signaling process for maintaining continuity of a media stream, according to various embodiments. [Figure 14] FIG. 1 is a block diagram illustrating an example of a processing system capable of performing at least some of the operations described herein. DETAILED DESCRIPTION OF THE INVENTION
[0011] The drawings depict various embodiments for purposes of illustration only. Those skilled in the art will readily appreciate that deviations from the principles of the technology may occur. It will be recognized that alternative embodiments may be employed without departing from the spirit and scope of the present invention. Although embodiments are shown in the drawings, the technology lends itself to various modifications.
[0012] The embodiments described below will enable those skilled in the art to implement the embodiments and will be described in detail below. The following information is provided to illustrate the best mode of carrying out the embodiment. Upon reading the description below, one skilled in the art will understand the concepts of the present disclosure and will be able to understand the concepts specifically addressed herein. These concepts and applications are provided in accordance with the present disclosure and accompanying The invention is included in the scope of the claims.
[0013] The embodiments may refer to specific computer programs, system configurations, networks, etc. However, those skilled in the art will appreciate that these functions can be implemented in other computer programs. Recognize that it is equally applicable to RAM types, system configurations, network types, etc. For example, the term "Wi-Fi network" describes a network. Although the present invention can be used for other types of networks, related embodiments may also be used for other types of networks. It can also be expanded.
[0014] Additionally, the disclosed technology may involve special purpose hardware (e.g., circuitry), software, and / or programmable circuits suitably programmed with firmware, or special It can be implemented using a combination of objective hardware and programmable circuitry. Thus, embodiments may be implemented to inspect video content generated by electronic devices. It identifies elements in the video content and applies classification models to take appropriate actions. and take appropriate action. , base station or networked computer server) The method may include a machine-readable medium having instructions that can
[0015] (term) The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the scope of the present disclosure. Where the context permits, the use of the singular or plural forms is Each word includes the plural or singular form.
[0016] As used herein, unless otherwise specified, "processing," "operation," "computation," " Terms such as "determine," "display," and "generate" refer to the physical representation of data in the memory or registers of a computer. (electron) quantities, and manipulates data in computer memory, registers, or other such storage medium, transmission, or display device, may be represented as a physical quantity. The application of a computer or similar electronic computing device that converts data into other data Refers to actions and processes.
[0017] As used herein, terms such as "connected" and "coupled" mean two or more This refers to any direct or indirect connection or coupling between the above elements. The method may be logical, logical, or a combination thereof.
[0018] References to "one embodiment" or "one embodiment" do not necessarily imply that the particular features, functions, or aspects being described are accurate. It means that the particular feature, structure, or characteristic is included in at least one embodiment. Appearances of phrases do not necessarily refer to the same embodiment, nor are they necessarily interchangeable. Nor do they refer to mutually exclusive alternative embodiments.
[0019] Unless the context clearly requires otherwise, the terms "comprise" and "include" are exclusive. in an inclusive sense (i.e., "including but not limited to"), rather than in an exclusive or exhaustive sense. should be interpreted as meaning "not applicable").
[0020] The term "based on" is also to be interpreted in an inclusive sense, rather than in an exclusive or exhaustive sense. Therefore, unless otherwise specified, the term "based on" should be interpreted as meaning "at least "based at least in part on"
[0021] The term "module" refers to a software component, a hardware component, A module generally refers to a specified A functional component that can generate useful data or other output based on inputs provided to it. A module can be self-contained. A computer program is , can include one or more modules. Thus, a computer program can Multiple modules that allow you to complete different tasks or multiple tasks It may include a single module capable of
[0022] The term "live media stream" is used broadly to describe a live media stream that is broadcast while an event is occurring. This refers to broadcasting an event over the network to the source device. generated by the user, transmitted over a network, and received by the user of the receiving device. No latency / delay perception (also known as "near real-time") ) broadcasts content that is rendered by receiving devices. Examples of such latencies that are not perceptible to the user of the device include 10 It can include 0 ms, 300 ms, 500 ms, etc. Therefore, broadcast Users of client devices in the group can view the live media stream in near real-time. real-time (i.e., delay) for the users of the client devices in the broadcast group. The user can interact or respond in a manner that is transparent to the user.
[0023] When used in reference to a list of multiple items, the word "or" shall be construed as follows: Any item in the list, all items in the list, and any of the items in the list It is intended to cover all combinations of
[0024] The sequence of steps performed in any of the processes described herein is illustrative only. However, steps may be performed in various orders and combinations consistent with physical feasibility. For example, you can add a step to the process described here. You may be able to add or remove steps from the process. Therefore, any process description is subject to constraints. It is intended to be limitless.
[0025] (overview) Streaming media content over packet-switched networks such as the Internet Content is rapidly becoming a way of consuming various media (e.g., video, audio). Electronic devices (e.g., smartphones, computers, tablets) Connect to a network (e.g., the Internet) and receive various video and audio data. For example, multiple electronic devices (or The client device receives the live stream of the video call generated by the calling device. In another example, a client device may subscribe to a stream. You can subscribe to live streams of sports events. All client devices that subscribe to a program are collectively called a "broadcast group." It can be called a "loop."
[0026] With the increasing popularity of streaming media content, high-quality content is increasingly being delivered over the Internet. bandwidth and computing resources to deliver quality video and audio streaming There is also an increasing demand from users for faster video playback due to the delay that occurs when initializing video playback. The quality of the live media stream may be affected, as this may result in a degradation of the user experience. Based at least in part on the startup latency of video playback. Stream quality may be subject to rebuffering or glitches in video playback. This can also result in a poor user experience.
[0027] To provide streaming media content to a variety of client devices, A streaming device generates a media stream and transmits it over a network. It is possible to transmit a media stream to a client device. A stream must contain at least one keyframe (i.e., the frame the client device is encoding). data that can be loaded and used to render a complete frame), and many predictions ( or "delta" frames (i.e., data representing the differences from the key frames). This media stream is then encoded as a bitrate, which represents the number of bits that represent a video frame. The increase in bit rate may include a It represents higher image quality.
[0028] In some embodiments, the new client device is a streaming media controller. New client devices can subscribe to the content. To subscribe to streaming media content, a client device first: The keyframes can be processed to render video frames. Once the new client device is connected, it processes the subsequent predicted frames to stream the video. A series of constituent frames can be rendered and output. A series of keyframes may be placed at various points along the media stream. If a new client device enters the media stream at a time between key frames, If you subscribe to a stream, new clients will not be able to access it until the next keyframe arrives. The device is unable to process the media stream and render the video. This may result in increased startup latency for new client devices. may occur.
[0029] Similarly, in some embodiments, a network associated with the client device Network conditions may change. For example, the client device may experience a decrease in bandwidth. , if the client device cannot handle the media stream at the current bitrate This allows the client device to process and render the video properly. This can cause glitches / interruptions in the video. This allows the client device to stream media at a higher bit rate than the current bit rate. However, the client device may be able to process the stream. , until a new keyframe arrives for the larger bitrate stream. You may not be able to subscribe to the bitrate stream (this is because the bandwidth is not sufficient) (This can lead to underutilization and poor video quality.) Changing network conditions The client device that experiences the streaming media may have different bit rates. There is a possibility to subscribe to alternative streams of content. As with subscribers, client devices are the ones that receive the streaming media content. Wait until a new keyframe to render the content at a different bitrate. It may be necessary to
[0030] Thus, startup latency and rebuffering / glitches are minimized or eliminated, To optimize the quality of live media streaming, various streaming technologies are used. Additionally, to optimize the quality of live media streaming, Media quality is maintained in the context of dynamic or heterogeneous network conditions. Media content should be transmitted and reconstructed without interruption. This includes selecting the quality level at which to encode the video.
[0031] This disclosure utilizes adaptive bitrate techniques for live media streaming. and dynamically generating a distribution layer for media streams. The encoded media stream is then sent to the client device for streaming. To reduce startup latency in processing and rendering media being streamed one or more alternative streams (or can be generated by the "distribution layer"). The media stream distribution layer is designed to allow new client devices to Whether or not a client device subscribes to a media stream or not Dynamically delivers notifications to client devices based on their decision to experience changes in network conditions. This allows existing media data streams to be generated by the subscriber. This reduces the startup time of the client device that is streaming the media. This improves the quality of the content and results in a better experience for all clients in the broadcast group. This can improve the user experience of the Ant device.
[0032] FIG. 1 illustrates a video content distribution system for multiple client devices 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110A, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 1 14a-c show a system 100 for streaming. The media content may be encoded and transmitted over a network 112 such as the Internet. One or more streaming servers 110 configured to record and stream In some embodiments, the streaming server 110 may include The streaming server can be called a "network-accessible server system 110." The server 110 receives data from an external node (e.g., a client device, an external server, a video camera). The streaming server 110 can receive media data from the The client device 114 in the broadcast group encodes the data. It can be transferred to a~c.
[0033] In some embodiments, the media data may include a live video stream. As an example, a live video stream can be generated by an external node and The data is then transferred over the network by the messaging server 110 and then to the client device. The client devices 114a-c may be connected to the In some embodiments, the live video can be rendered in a It can contain video that is almost real-time. Users who are watching live video can , live video so that you can interact with and respond to live video. The video can be transmitted to client devices included in the broadcast group. Cut.
[0034] As an example, the smartphone 114a may receive live video (e.g., video capture game Generates a video (capturing the user's movements) and sends it to a streaming server. The streaming server 110 can transmit the received live video to the and then broadcast the encoded video to other clients in the group. client devices (e.g., laptop computer 114b, computer 114c) In this example, other client devices in the broadcast group can The device provides minimal buffering and imperceptible time delays (e.g., 300 ms). You can receive encoded live video in less than a second.
[0035] Continuing the above example, laptop computer 114b receives live video. The second live video can then be transmitted and a second live video can be generated. The streaming server 110 can transmit the , encode the second live video, and broadcast it to other classes in the group. This is transmitted to the client device (smartphone 114a, computer 114c). In the example, devices in a broadcast group are broadcasting a live video stream. This allows for interactivity with imperceptible delay / latency to the users of the devices 114a-c. It is possible to communicate with the
[0036] The streaming server 110 receives the encoded media stream (e.g., stream 116) (collectively referred to as broadcast group 114a-c) The data can be transmitted to the client devices 114a-c. In this embodiment, the streaming server 110 receives streams from the client devices 114a-c. 118, and based on receiving the stream request 118, the encoded media stream. 116 can be transmitted to the client devices 114a-c. The stream 118 allows users of the client devices 114a-c to view encoded media. Indicate that you have already requested to subscribe to Astream 116 can be done.
[0037] For example, a user of laptop computer 114b may access the internet over network 112. By transmitting a stream request to the streaming server 110, the sports You can subscribe to the live video stream of the event. Upon receiving the request, the streaming server 110 sends the encoded video content to The content can then be transferred to the laptop computer 114b. The pop computer 114b decodes the encoded video data and You can output a live video stream of the event to a display.
[0038] FIG. 2 illustrates the architecture of a streaming server 210, according to various embodiments. The streaming server 210 encodes the encoded media stream. and configured to transmit over a network to one or more client devices. In some embodiments, the electronic device may include one or more electronic devices. The streaming server 210 includes an interconnect configured to transmit data between the servers. It is part of a series of streaming servers.
[0039] As mentioned above, the streaming server 210 may receive data from external devices (e.g., video cameras, A live stream 212 can be received from a streaming server (external server). The server 210 receives live media content over a network interface 216. The network interface 216 may be configured to provide suitable wired and / or wireless communication. It can interface with external devices via a communication protocol.
[0040] In some embodiments, the network interface 216 is The encoded stream of media content 210 is received from the encoding server. As shown in Figure 12 below, multiple streaming servers are deployed along the data path. The encoded media stream is generated and then passed to the data processor. along the route to a client device (e.g., client devices 114a-c). It is possible.
[0041] Once the data is received by the network interface 216, the streaming server 110 can determine whether the received data is encoded. If the received data is encoded, the streaming data is sent to the decoder 218. In some embodiments, the received data may be encoded In this case, the live stream may be a live stream 212 that is not The transcoder 220 then uses the appropriate encoding protocol to encode this data. You can encode a live stream into an encoded media stream. Cut.
[0042] The decoder 218 decodes the received encoded data using an appropriate decoding protocol. It can be configured to decode the media streams that are being decoded. The resulting media stream can be forwarded to the transcoder 220. The encoder 220 encodes the received data using the appropriate encoding protocol. , can be encoded into an encoded media stream.
[0043] The encoded media content is transferred to the network interface 222. The network interface 222 may be any suitable wired and / or wireless The wired communication protocol transmits the encoded media content 214 over the network. can be transferred to various client devices via
[0044] In some embodiments, the network monitor 224 may monitor received media streams. Monitors the decoding and encoding of media streams and The received information can be used to receive information related to the coding and encoding. Based on this, the network monitor 224 identifies errors in the encoded stream. It is possible.
[0045] The network monitor 224 may modify or monitor other components within the streaming server 210. can transmit a request to adjust to the adjuster 226. The adjuster 226 adjusts the video The stream and / or parts of the transcoder 220 can be adjusted / modified. Such adjustments may include, for example, transcoding techniques or protocol modifications. Cut.
[0046] In some embodiments, advertisements 228 are added to the video stream. This may include video content configured to be supplemented with audio streams. In this example, the streaming server 210 transmits the encoded media stream at various times. You can add ads to your playlist.
[0047] FIG. 3 illustrates the architecture of a client device 314, according to various embodiments. The client device 314 may be a smartphone, a computer, a tablet, a game console, or the like. This includes network-accessible electronic devices such as computers, smart home devices, etc. It is possible.
[0048] As mentioned above, the client device 314 may be connected to a network such as the Internet. Stream request 3 is sent via network interface 330 connected to the network. 18 to a streaming server (e.g., streaming server 110 in FIG. 1). The stream request 318 is used to subscribe to a live video stream. , and to receive the encoded media stream from the streaming server. This may include a request for
[0049] The network interface 330 of the client device 314 The encoded stream 316 can be received from the encoding server. The resulting media stream 316 can be forwarded to a decoder 332. 332 decodes the encoded media stream and renders it. transmits the video / audio content to an output component such as a display 334 Other exemplary output components are speakers, a headset, a The display 334 may include a touch screen. Stream to a resolution / bitrate that corresponds to the bitrate of the encoded media stream. It can be displayed in high quality.
[0050] The service monitor 336 monitors the encoded and decoded streams. For example, if you are decoding an encoded stream, If there is an error in the queue, the service monitor identifies the error or You can modify the client device.
[0051] (Summary of encoded media stream) FIG. 4 illustrates an encoded media stream according to various embodiments. loaded media stream (e.g. chunk-based stream S1, encoded The stream S2) carries media content, e.g., a live video stream. It can contain data representing
[0052] The encoded media stream is a series of key frames (e.g., key frames 402-1 to 402-3, 404-1 to 404-3) and the subsequent predicted frames (e.g., A set of key frames (e.g., a first key frame, a second key frame, a third key frame, a fourth key frame, a fifth key frame, a fifth key frame, a sixth ... An i-frame 402-1) (or "i-frame") represents a full frame of an image in a video. In one example, a key frame can be an intraframe in VP8 or an MPEG It can be similar to a keyframe.
[0053] During operation, a client device (e.g., client device 314 of FIG. 3) receives a first The key frames 402-1, 404-1 are processed to generate corresponding media, such as video frames. The keyframes can be used to render media content. In this case, the decoder can decode the frame without referring to the decoder frame. Reconstruct such frames starting from a "default" state. In the video stream, keyframes are random access (or seeking) points. We can provide you with the following information.
[0054] The encoded media stream also contains multiple frames that follow the key frame in time. The predicted frames 406, 408 (or "delta frames") may also be included. The frames 406, 408 can represent the difference between the key frames. This allows for a reduction in the amount of data required to render a frame represented by a frame. In one example, the predicted frames 406, 408 are VP8 interframes or MP They can be analogous to P-frames in EG terminology. Predicted frames 406, 408 are frames that are the result of a previous The frames can be encoded with reference to each other, and in some embodiments, all All previous frames, up to and including the most recent keyframe, can be encoded. In many cases, the correct decoding of the predicted frames 406, 408 is It depends on the correct decoding of the keyframe and all subsequent predicted frames. As a result, the decoding algorithm does not tolerate dropping of key frames. In environments where frames may be dropped or corrupted, it may take a while for a keyframe to be received correctly. Therefore, it may not be possible to perform correct decoding.
[0055] The media stream is divided into chunks (i.e., overlapping segments of the media stream). Use encoding techniques such as chunked transfer encoding to create a The chunk-based stream S1 in Figure 4 can be encoded using An example of a chunk-based stream can be represented as: Chunks of the program can be transmitted and received independently of each other. In this embodiment, both the receiver and sender of a chunk-based stream are It does not need to know the data stream outside the chunk being processed.
[0056] FIG. 5 illustrates an encoding process with a set of distribution layers, according to various embodiments. The encoded media stream is referred to as the "displayed" media stream. It may contain multiple alternative streams called "streaming layers": LD, SD, HD. As shown in Figure 5, the encoded media stream is divided into multiple bits. Rates 508-1, 508-2, 508-3 (or "Multi Bit Rate" (MBR) It can be streamed using "i bit-rate" encoding. The bitrate of a distribution layer can represent the number of bits in a video frame. For example, a higher bit rate correlates to a higher quality video. The first distribution layer LD includes a low-quality bitrate 508-1, such as The second distribution layer, SD, is standard definition with a bit rate of 508-2 and a third distribution tier, HD, for high-quality bitrates. 508-3.
[0057] In some embodiments, the encoded media content may be various bits (or is the "encoding") rate, and the segments or chunks (e.g., as shown in Figure 4) In these embodiments, if ,Network conditions change (i.e., data transmission and processing capacity increases or decreases) ), the subscribing device can switch between different chunks to read different bits. Subscribe to an alternate version of a media stream encoded at a rate It is possible.
[0058] MBR encoding often uses a constant bitrate approach to ensure each Encode the media stream using constant bitrate encoding instead. When encoding a replacement stream, the resulting media output quality is inconsistent. The results may be undesirable, for example, if the encoded media A video, represented as a media stream, contains parts (or "scenes") of varying visual complexity. In this example, constant bitrate encoding includes a variety of qualities. These video segments cannot be efficiently encoded because they are fixed bitrates. For trait encoding, it is necessary to encode low-complexity video segments. takes too many bits, and too few bits to encode high complexity video segments. This is because there may be a shortage.
[0059] In many cases, MBR encoding is used to determine the final display resolution of the client device. Requires a fixed display resolution. Using a fixed display resolution will result in multi-bit Decodes a standard video stream and scales it to a fixed display resolution. This will minimize glitches in the media you view. Using splay resolution, various alternative media streams can be displayed at, for example, several megabits per second. It can have a wide range of bit rates, from 100 Mbps to several kilobits per second. In this case, the problem faced when streaming to a fixed display resolution is The goal is to match the bitrate of multiple video streams to the various bitrates of the multiple video streams. Multi-bitrate encoding technology utilizes predefined encoding resolutions. This allows for a wide variety of client devices on different networks, and for a variety of video scene complexity. However, it is not well suited to the dynamic network conditions of the device.
[0060] In many cases, the encoded media stream splits the media content into multiple video streams. You can use adaptive bitrate streaming to encode at the desired bitrate. FIG. 6 illustrates adaptive bitrate encoded media in accordance with various embodiments. Indicates a stream that uses adaptive bitrate encoded media streams. The client device then downloads the encoded media based on its available resources. You can switch between streaming the content at different bit rates. For example, If the client device is configured to stream at a higher bitrate than the currently being streamed If it is possible to stream at the coding rate bitrate, the client The device can switch to other bit rates with higher coding rates. do.
[0061] In some embodiments, the source media content may be streamed in a variety of formats at a variety of bit rates. The distribution layer encodes and segments the media content into parts. Streaming clients can receive streams at various bit rates. It can make available streams known, and streaming clients can The stream can be segmented by a manifest file.
[0062] For example, upon starting / initialization (at time T1), the client device subscribe to the first stream LD of the target 608-1 (or If the client device At time T2, the first stream has a higher bit rate than the second stream, which has a lower bit rate. If the client device determines that it can handle the stream, The device subscribes to a second stream SD at a higher bit rate 608-2 Similarly, the client device may Based on the determination that it can handle a stream with a bit rate of 608-3, You can subscribe to a third stream HD with T3.
[0063] Conversely, if the download speed or processing speed of the client device is faster than the current subscription If the bitrate is lower than the bitrate of the stream being played, the client device , a lower bitrate segment can be subscribed to. In this case, the client device may experience a degradation in network throughput or is based on a determination that the processing power of the client device is already below a threshold level. , a lower bitrate segment can be subscribed to.
[0064] FIG. 7A illustrates a live broadcast with multiple subscribers according to various embodiments. As shown in Figure 7A, adaptive bitrate streaming is A number of subscribers (e.g., subscribers S1, S2) are As an example, Subscriber 1 (S1) can subscribe to the stream. , the encoded media stream can be subscribed to at time T0. Subscriber 2 (S2) can subscribe at time T1.
[0065] In some embodiments, if the encoded media stream is When using rate streaming, each distribution tier of the stream is However, for live broadcasting If so, subscribe to an encoded media stream that is already being broadcast. A new client that subscribes (e.g., Subscriber 2 (S2)) will receive a A new keyframe (i.e., the distribution of the stream associated with a particular resolution) In most cases, new client S2 requests a new token in the time domain. For the region, at the subscription time before the time T2 when the new key frame arrives, The encoded media stream can be requested. Between the subscription time T1 and the time T2 when the first frame is rendered The time interval may include a start-up latency T3.
[0066] FIG. 7B illustrates a live broadcasting system under dynamic network conditions, in accordance with various embodiments. 1 illustrates adaptive bitrate streaming for network In response to changing network conditions, the client adapts to the changing network conditions. To do this, you can request a change in the bit rate of the encoded media stream. This can be done.
[0067] As an example, as shown in FIG. 7B, the client may select a higher bandwidth at a first time T1. It can be determined that more bandwidth is available. It can be determined that more bandwidth is available. The thing is, available bandwidth allows for higher bitrate encoded media. This may include the client identifying itself as capable of processing the stream. Therefore, clients may choose to use higher bitrates to improve the quality of the representative media. You can request to subscribe to a distribution tier with a can.
[0068] However, subscribing to a distribution tier with a larger bitrate To do this, the client must create a new key for the requested distribution layer. It may be necessary to wait until time T2 when the frame arrives. The time interval between time T1 and time T2 represents a time T3 during which the bandwidth is underutilized. This waiting time T3 (i.e., "time to first frame") during which the bandwidth is underutilized indicates that the bitrate of the decoded media is higher than the bitrate of the source stream. This results in a poor client experience because the quality is lower than expected. There is a possibility.
[0069] Furthermore, as shown in FIG. 7B, the client experiences a drop in bandwidth at time T4. Bandwidth degradation may occur when the client is unable to encode at the current bitrate. The available bandwidth is reduced because the device is unable to handle the downgraded media stream. In this example, the client may choose to stream at a lower bit rate. and new keyframes at this lower bitrate will be , arrives at a later time (e.g., the first keyframe time T5). The duration of the bandwidth between the time T5 of the keyframe and the rendering discontinuity duration T6 During this rendering discontinuity, the resulting media , may be interrupted / glitched, or may be unable to view the media.
[0070] Broadcast groups subscribe to encoded media streams In some cases, if a block One client in the broadcast group can send the encoded media stream If the client cannot maintain the desired bitrate, it will be dropped from the stream. New broadcasters who are removed or subscribed to lower bitrate streams In the second case, if the One client can maintain the bit rate of the encoded media stream. If this is not possible, all clients in the broadcast group will The encoded media stream is then sent to the In either case, a single client or may result in a decrease in overall quality for any of the broadcast groups. do.
[0071] In some embodiments, the present invention provides a method for efficiently utilizing computing and network resources while minimizing the impact of network congestion on the cluster. Optimizing the client experience and quality involves several steps. The group can broadcast content to multiple clients in a broadcast group. The quality level at which content is encoded so that it can be transferred and reconstructed without any hassle The second step may include selecting a dynamic network condition. New subscribing to encoded media streams while maintaining their quality in text The method may include reducing the time to the first frame.
[0072] Shifted Distribution Tiers for Live Broadcasting (SD L: shifted distribution layers)) FIG. 8A illustrates a system with a shifted distribution layer, in accordance with various embodiments. Indicates a set of encoded media data. Encoded Media Stream The distribution layers LD, LD+, and LD++ provide keyframes at various time positions. (which can be called the "shifted distribution layer") It can include.
[0073] As an example, in FIG. 8A, the first distribution layer LD is A series of key frames 802-1 to 802-8 with the first key frame 802-1 at position T1 The second distribution layer, LD+, is related to the time domain. The image may include a series of keyframes 804-1 to 804-3 that are shifted in this order. Here, the first key frame 804-1 is at the second time position T2. The distribution layer LD++ generates a series of keyframes shifted in the time domain. The first key frame 806-1 may include: The keyframe is located at the second time position T3. By using alternate versions of the stream where the frames are shifted in time, Latency is reduced, streaming interruptions are avoided, and network conditions change Improved resource utilization under high performance.
[0074] FIG. 8B illustrates a multi-bitrate distribution tier according to various embodiments. 8B shows a set of shifted encoded media data. The encoded media data is then sent to various bit rates or coding rates. It can contain a distribution layer (e.g., LD, SD, HD). The encoded media data of the distribution layer LD, SD, and HD is The first keyframe may be included at time T1.
[0075] The distribution layer of the encoded media stream is the shifted data A distribution layer may be included, where the key frames are For example, as shown in Figure 8, the shifted distribution The layers LD+, SD+, and HD+ are shifted in time to a second time T2. Similarly, the shifted distribution layers LD++, SD++, and HD++ includes the first keyframe shifted in time to a third time T3. The distribution layer is a time-shifted distribution layer that can be used to generate arbitrary time positions in the time domain. It may contain key frames.
[0076] New subscribers to the encoded media stream will receive the The distribution layer with the key frame closest in time to the application time is subscribed. This allows for shifting in the encoded media stream. Utilizing a distributed layer can reduce startup latency. For example, a new client device may receive encoded media at time TA. Subscribe to MediaStream to receive HD bitrate distribution tiers. In this example, the first time shifted to a third time T3 can be requested. The key frame of is the shortest distance to the current time TA, so the new client Client devices can subscribe to the distribution layer HD++. Therefore, the shifted distribution layer of the encoded data stream is By subscribing, new clients can access the unshifted distribution. No waiting for the next keyframe in the view layer (HD) reduces startup latency can be shortened.
[0077] (Shifted distribution layer and dynamically generated layer (DGL; dynamicall y generated layers)) In some embodiments, the shifted distribution layers and key frames are , can be dynamically generated. FIG. 9 illustrates a dynamically generated data Encoded streaming data with distribution layer and key frames This indicates the data.
[0078] The shifted distribution tier can be generated dynamically. At time TA, the new client starts standard definition (SD) encoding. You can subscribe to media streams encoded at loading speed. However, if a new client joins the distribution layer at time TA, When SD is subscribed, between the time TA and the time of the next key frame 906-2, There may be startup latency. As mentioned above, the encoded media stream The client experience and quality of media received from the stream is affected by the low startup latency. There is a possibility that it will go down.
[0079] Continuing the example above, a new client might request an encoded media stream. Based on the determination that you are already subscribed to the new distribution Layers can be dynamically generated. The dynamically generated SD* 908 key The key frame is the start latency between the subscription time TA and the key frame time. Shorten.
[0080] In some embodiments, to reduce startup latency, the encoded media For each new client that subscribes to the stream, a new DGL is generated. If the time interval between the current time (e.g., TA) and the next keyframe is , a new DGL can be generated when the threshold time interval is exceeded. A specific number of predicted frames to the next keyframe, or a time interval to the next keyframe can be expressed as:
[0081] As an example, if this threshold time interval is the current time and the distribution stratum (e.g. , SD) is 5 predicted frames between the current time and the next keyframe. If the next keyframe in the distribution layer is frame 7, DGL( Conversely, if you want to use the current time and distribution The time interval between the next keyframe of the application layer is shorter than a threshold time interval (e.g., 3 frames). If not, no DGL (e.g., SD*) will be generated.
[0082] FIG. 10 illustrates a distribution of encoded media streams in accordance with various embodiments. 10 shows a set of view layers and dynamically generated layers. The layers LD*, SD*, and HD* are dynamically generated layers (DGL) at various bit rates. By using the shifted distribution layer and DGL, This improves the quality of experience for stream subscribers and reduces resource demands. It can be done.
[0083] As an example, a new DGL for a given bitrate may be used to encode media. New clients subscribe to the media stream or changes in network conditions It can be dynamically generated based on the client's experience. In an embodiment, the streaming server may provide network Detecting changes in state or sending messages to clients indicating changes in network state dynamically generated layers or shifted layers based on either receiving from the remote device A distribution layer can be created.
[0084] (Distribution layer and associated bridge distribution layer (BDL: bridge distribution layers) FIG. 11 illustrates a distribution of encoded media streams according to various embodiments. The set of the distribution layer and the bridge distribution layer is shown in Figure 11. The layers LD, SD, and HD represent the distribution of the encoded media stream. The layers LD**, SD**, and HD** represent the distribution layers of the encoded media. Represents the bridge distribution layer of DearStream.
[0085] The Bridge Distribution Layer (BDL) is a dynamic layer with a finite number of frames. It can contain generated distribution layers. BDL (e.g., LD**, SD**) , HD**) keyframes, and the current time and next key of the distribution layer For example, SD** can be used to indicate the number of predicted frames between the time Here, the BDL can be generated at time T3 by using the key frame 1110 at time T3. and the corresponding distribution layer (SD) keyframe to the next keyframe It includes a number of predicted frames 1112 up to the time.
[0086] If a client subscribes to a BDL (e.g., SD**), the client The corresponding distribution is generated when the time matches the time of the next keyframe (e.g., T4). In some embodiments, the BD L (e.g., HD**) is switched from BDL at time T4. can be abandoned.
[0087] In some embodiments, the streaming server may, based on a trigger event, One such trigger event is to stop the generation of DL predicted frames. may contain the arriving keyframes of the corresponding distribution layer. In this case, the streaming server will receive the keyframe for this corresponding distribution layer. The second trigger event is ,predict the duration between the current time (e.g., T3) and the time of the next keyframe (e.g., T4). Based on this estimated time interval, the The rendering server must be able to generate the appropriate number of predicted frames after the BDL key frame. can.
[0088] FIG. 12A illustrates a distribution system at an origin streaming server according to various embodiments. We present an architecture for dynamically generating the encoding layer. One or more media streams that assist in transferring downloaded media streams to client devices. As shown in Figure 12A, the source streaming server The server 1210 sends the encoded media stream to the intermediate streaming server 12 12a-b can be transmitted to a client device via a network. .
[0089] In the embodiment shown in FIG. 12A, the source streaming server 1210 Distribution tiers LD, SD, HD and shifted distribution tiers LD+, S Generating encoded media streams having both D+ and HD+ The encoded media stream can be sent to various intermediate streaming servers. 212a-b and to various clients that are part of the broadcast group. This configuration allows the quality of experience to be improved. Generates encoded media stream distribution layer while degrading This reduces the resource demands on the client device when You have to wait a longer wait time for the next keyframe of the attribution layer. The quality of the experience may be diminished.
[0090] FIG. 12B illustrates a distribution layer along a data path, according to various embodiments. As shown in Figure 12B, the source stream The streaming server 1210 is an engine having distribution layers LD, SD, and HD. The intermediate streaming server 1 can generate coded media streams. 212a-b receive the encoded media stream and can be forwarded to the client devices 1214a-b. The intermediate client devices 1212a-b receive the shifted distribution Generate layers LD+, SD+, HD+, and clear the shifted distribution layers The data can be transmitted to the client device 1214a-b. 1212a-b indicate that the network conditions have changed or the client device is new. Distribution tier requests shifted due to being a subscriber Based on receiving the The signal can be transmitted to the devices 1214a-b.
[0091] Multiple intermediate streaming servers deliver the shifted data to the client device. The distribution layer can be generated dynamically, so the first frame can be generated from the current time. This allows for a shorter time to the client device before the In addition to the origin streaming server, multiple encoders are running on both ends of the data path. It can also be done.
[0092] FIG. 13 illustrates a signaling scheme for maintaining continuity of a media stream, according to various embodiments. As shown in Figure 13, one or more servers (e.g., The server 1312 and intermediate server 1314 transmit the encoded label along the data path. The live media stream can be transmitted to a client device 1316 .
[0093] The client device 1316 sends a new stream request 1301 to The new stream request 1 can be transmitted along the stream to the intermediate server 1314. 301 can represent a request to subscribe to a live media stream. Upon receiving a new stream request 1301, the intermediate server 1314 The message 1301 can be forwarded along a data path to the originating server 1312 .
[0094] An origin server 1312 can generate a live media stream 1302 In some embodiments, the origin server 1312 receives the new stream request 13 Based on the .01 standard, media streams can be generated at one or multiple bit rates. The origin server 1312 can send the media stream 1303 to the broadcast group. to multiple client devices (e.g., client device 1316) in a group. In some embodiments, the origin server 1312 may include a The media stream 1303 can be transmitted to an intermediate server 1314. The server 1314 forwards the media stream 1303 to the client device 1316. Upon receiving the media stream 1303, the client device 13 16 to process and render associated live media content can be done.
[0095] In some embodiments, the client device 1316 may process the request 1304. The update request 1304 can be transmitted to the server 1314 and updated. new distribution layers of media streams at different bit rates. For example, the client device 1316 may determines that it can process the media stream at a higher bit rate As another example, the client device 1316 may determine that it has this current bit At this rate, you can now stream media without experiencing glitches or interruptions in the media stream. It may decide that it cannot process the media stream.
[0096] In some embodiments, the intermediate server 1314 receives the data from the client device 1316. Upon receiving the update request 1304, it generates a second distribution layer 1305. The second distribution layer 1305 can distribute the data to client devices. The updated bit rate is different from the bit rate of the media stream 1303 that was originally sent at 1316. The bit rate specified in the new request 1304 can be included. Once the distribution layer 1305 is created, the intermediate server 1314 creates a second distribution layer. The second data layer 1306 can be transmitted to the client device 1316. The distribution layer 1306 provides a first distribution of the media stream. It may contain keyframes that are shifted in time relative to the keyframes in the image layer. come.
[0097] In some embodiments, the client device 1316 receives update requests on the data path. The intermediate server 1314 can then send the request 1307 to the intermediate server 1314. This request 1307 can be forwarded to the source server 1312. Request 1 In response to receiving 307, origin server 1312 sends a second distribution tier 1 The origin server 1312 can generate a second distribution request 308. The data layer 1310 can be transmitted along a data path to an intermediate server 1314. The server 1314 distributes the second distribution layer 1310 to the client device 1312. It can be forwarded to 316.
[0098] (Processing System) FIG. 14 illustrates a processing system capable of performing at least some of the operations described herein. 14 is a block diagram illustrating an example of a processing system 1400. For example, some of the processing systems 1400 The components include a streaming server (e.g., streaming server 210 in FIG. 2) or hosted on a client device (e.g., client device 314 in FIG. 3). It is possible.
[0099] The processing system 1400 includes one or more central processing units (“processors”) 140 2, main memory 1406, non-volatile memory 1410, network adapter 1412 ( e.g., network interface), video display 1418, input / output devices a keyboard and pointing device; a control device 1422 (e.g., a keyboard and pointing device); A drive unit 1424 including a storage medium 1426 and a The bus 1416 may include a signal generating device 1430. One or more physical buses and / or devices connected by a network, adapter, or controller or point-to-point connections. The bus 1416 is a system bus, a PCI (Peripheral Component Interconnect) bus, or a PCI bus. CI-Express bus, HyperTransport or industry standard architecture (ISA: industry standard architecture) bus, small computer system interface small computer system interface (SCSI) bus, Universal Serial Bus USB (universal serial bus), IIC (I2C) bus, or electrical and electronic technology Institute of Electrical and Electronics Engineers (IEEE) Standard 139 It can contain 4 buses (i.e., "Firewire").
[0100] The processing system 1400 may be a desktop computer, a tablet computer, a mobile Information terminals (PDAs: personal digital assistants), smartphones, game consoles , music players, wearable electronic devices (e.g., watches or fitness trackers), Network-connected (“smart”) devices (e.g., televisions or home assistant devices) devices), virtual / augmented reality systems (e.g., head-mounted displays), or processing systems A sequence of instructions (sequential or otherwise) specifying actions to be performed by the system 1400. a computer processor architecture, such as another electronic device capable of executing a sequence The architecture can be shared.
[0101] Main memory 1406, non-volatile memory 1410, and storage medium 1426 ("machine-accessible"). Although shown as a single medium, the term "machine-readable medium" and The term "storage medium" refers to a single medium that stores one or more sets of instructions 1428. or multiple media (e.g., centralized / distributed databases and / or associated caches) "Machine-readable medium" and "storage medium" The term "instruction" may also refer to a sequence of instructions for execution by the processing system 1400. shall be construed to include any medium capable of storing, encoding, or transporting Not possible.
[0102] Generally, the routines executed to implement the embodiments of the present disclosure are A system or a specific application, component, program, object, or module a module, or sequence of instructions (collectively called a "computer program") A computer program can be implemented as part of a computer program. One or more settings may be stored at different times in various memories and storage devices within the operating device. It has several instructions (e.g., instructions 1404, 1408, 1428). When read and executed by the processor 1402, the instructions cause the processing system 1400 to The actions are performed to implement elements that comprise various aspects of the present disclosure.
[0103] Furthermore, the embodiments have been described thus far in the context of a fully functional computing device. Although described, those skilled in the art will appreciate that various embodiments may be distributed as program products in various forms. It will be understood that the present disclosure may be used to actually distribute This applies regardless of the particular type of machine or computer readable medium used.
[0104] Further examples of machine-readable storage media, machine-readable media, or computer-readable media include: Volatile and non-volatile memory devices 1410, floppy and other removable disks, Hard disk drives, optical disks (e.g., compact disk read-only memory (C D-ROM), Digital Versatile Disc (DVD), and digital and analog communication links This includes recordable media such as transmission media such as links.
[0105] The network adapter 1412 allows the processing system 1400 to and the processing system via any communication protocol supported by the external entity. Brokering data within network 1414 with entities external to system 1400 The network adapter 1412 can be a network adapter card, a wireless Network interface cards, routers, access points, wireless routers, switches Multi-layer switches, protocol converters, gateways, bridges, bridge routers, It may include a hub, a digital media receiver, and / or a repeater.
[0106] The network adapter 1412 is used to access / transmit data within a computer network. Determine and / or manage permissions to proxy and share data across different machines and / or applications This can include a firewall that tracks different levels of trust between applications. A wall is a specific set of machine-to-application, machine-to-machine, and and / or between applications to provide a set of predefined access rights (e.g. to regulate the flow of traffic and resource sharing between these entities Any combination of hardware and / or software components that can The firewall may be any number of modules with individual the rights to access and manipulate objects by people, machines, and / or applications; and managing access control lists detailing permissions, including the circumstances under which permission exists; and and / or have access to it.
[0107] The techniques presented here involve programmable circuits (e.g., one or more microprocessors) dedicated hardwired (i.e., processor), software and / or firmware, may be implemented by non-programmable (non-reprogrammable) circuits, or combinations of such forms. The dedicated circuitry may consist of one or more application-specific integrated circuits (ASICs). ecific integrated circuits), programmable logic devices (PLDs) e logic devices), field programmable gate arrays (FPGAs), The gate arrays can be of various formats.
[0108] In some embodiments, any suitable encoding protocol may be utilized. For example, the protocols H.264 or VP9 can be used and It can be applied to any combination of such protocols.
[0109] (remarks) The foregoing description of various embodiments of the claimed subject matter has been presented for purposes of illustration and description. Restrictive enumeration or limiting claimed subject matter to the precise form disclosed is not permitted. Many modifications and variations will be apparent to those skilled in the art. The configurations were chosen and described in order to best explain the principles of the invention and its practical application. This allows those skilled in the relevant art to easily understand the claimed subject matter, various embodiments, and various modifications suitable for particular applications can be appreciated.
[0110] While the detailed description of the invention describes specific embodiments and the best mode contemplated, No matter how detailed the description of the invention appears, the technology may be practiced in many ways. Although embodiments are included in the specification, the details of their implementation are not readily apparent. When used to describe particular features or aspects of various embodiments, Certain terms are used to limit the scope of a particular feature, characteristic, or aspect of the technology to which they relate. This should not be construed as meaning that the term "antibody" is being redefined herein. Terms used in the claims are intended to be illustrative and not restrictive unless those terms are expressly defined herein. However, the present disclosure should not be construed as limiting the technology to the specific embodiments disclosed herein. Thus, the actual scope of the technology is limited to the disclosed embodiments and any modifications or variations thereof. The present invention also encompasses all equivalent methods of carrying out the method.
[0111] The language used herein has been chosen primarily for ease of reading and explanation. It has not been chosen to delineate or limit the subject matter. The claims issued for the application based on this, rather than the detailed description of the invention Accordingly, the disclosure of various embodiments is intended to be limited by the following: It is intended to illustrate, but not limit, the scope of the claimed technology. It is not intended to be.
[0112] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on the patent application filed on April 26, 2018 for "Live Broadcasting U.S. Provisional Patent Application Serial No. 62 entitled "Adaptive Bit Rate Method for Audiovisual Recording," / 663,182, which is incorporated herein by reference in its entirety. It is being done.
Claims
1. obtaining a live media stream including a first distribution layer, the first distribution layer including a first set of key frames and, for each of a plurality of key frames in the first set of key frames, a first set of delta frames following the key frame to a next key frame in the first set of key frames; receiving, at a client device, instructions to modify the rendering of the live media stream; In response to receiving the instruction, communicating a second distribution tier of the live media stream to the client device, the second distribution tier including a second set of key frames and, for each of a plurality of key frames in the second set of key frames, a second set of delta frames following the key frame, the second set of key frames including fewer key frames than the first set of key frames, and the second distribution tier being different and distinct from the first distribution tier; and enabling the client device to maintain continuity of the live media stream by switching from rendering one of the first set of delta frames of the first distribution layer to rendering the second distribution layer starting at a particular key frame of the second set of key frames and continuing with the second set of delta frames following the particular key frame. A method for providing the above.
2. an origin server generating the first distribution tier; and generating, by the origin server, the second distribution tier based on receiving, from the client device, the instruction to modify the rendering of the live media stream; The method of claim 1 further comprising:
3. obtaining the live media stream, an intermediate server receiving the first distribution layer from an origin server, the first distribution layer being generated by the origin server; generating, by the intermediate server, the second distribution tier based on receiving, from the client device, the instruction to modify the rendering of the live media stream; and the intermediate server communicating both the first distribution layer and the second distribution layer to the client device. The method of claim 1 , comprising:
4. 2. The method of claim 1, wherein the first distribution tier comprises a first bit rate and the second distribution tier comprises a second bit rate that is different from the first bit rate.
5. The method of claim 1 , further comprising generating the second distribution tier based on a determination that the client device is a new subscriber to the live media stream.
6. generating the second distribution layer includes: determining that a time interval between a time of subscription of the live media stream by the client device and a time of a next key frame in the first set of key frames is greater than a threshold time interval; and generating the second distribution tier in response to determining that the time interval is greater than the threshold time interval. The method of claim 5 , comprising:
7. 1. A device for streaming live media content, comprising:
1. A memory containing instructions for maintaining continuity of a live media stream, the instructions, when executed by a processor, causing the processor to perform the following steps: obtaining a live media stream including a first distribution layer, the first distribution layer including a first set of key frames and, for each of a plurality of key frames in the first set of key frames, a first set of delta frames following the key frame to a next key frame in the first set of key frames; receiving, at a client device, instructions to modify the rendering of the live media stream; generating a second distribution tier of the live media stream to the client device, the second distribution tier including a second set of key frames and, for each of a plurality of key frames in the second set of key frames, a second set of delta frames following the key frame, the second set of key frames including fewer key frames than the first set of key frames, and the second distribution tier being different and separate from the first distribution tier; communicating the live media stream including the second distribution layer to the client device; and enabling the client device to maintain continuity of the live media stream by switching from rendering one of the first set of delta frames of the first distribution layer to rendering the second distribution layer starting at a particular key frame of the second set of key frames and continuing with the second set of delta frames following the particular key frame; Execute the Devices that include:
8. The instructions further cause the processor to: receiving live streaming data from a source client device; and encoding the live streaming data to generate the live media stream including the first distribution layer and the second distribution layer; The device of claim 7.
9. The device of claim 7 , wherein the instructions further cause the processor to receive the first distribution layer from a source device.
10. obtaining the live media stream, an intermediate server receiving the first distribution layer from an origin server, the first distribution layer being generated by the origin server; generating, by the intermediate server, the second distribution tier based on receiving the instruction from the client device to modify the rendering of the live media stream; and the intermediate server communicating both the first distribution layer and the second distribution layer to the client device. The device of claim 7, comprising:
11. 8. The device of claim 7, wherein the first distribution tier comprises a first bit rate and the second distribution tier comprises a second bit rate that is different from the first bit rate.
12. The device of claim 7 , further comprising generating the second distribution tier based on a determination that the client device is a new subscriber to the live media stream.
13. generating the second distribution layer includes: determining that a time interval between a subscription time of the live media stream by the client device and a time of a next key frame in the first set of key frames is greater than a threshold time interval; and generating the second distribution tier in response to determining that the time interval is greater than the threshold time interval. The device of claim 7 further comprising:
14. A non-transitory computer-readable storage medium storing computer program instructions, comprising: The computer program instructions, when executed by a processor, cause the processor to perform the following steps: obtaining a live media stream including a first distribution layer, the first distribution layer including a first set of key frames and, for each of a plurality of key frames in the first set of key frames, a first set of delta frames following the key frame to a next key frame in the first set of key frames; receiving, at a client device, instructions to modify the rendering of the live media stream; and and in response to receiving the instruction, communicating a second distribution tier of the live media stream to the client device, the second distribution tier including a second set of key frames and, for each of a plurality of key frames in the second set of key frames, a second set of delta frames following the key frame, the second distribution tier being different and distinct from the first distribution tier, the second set of key frames including fewer key frames than the first set of key frames, and a particular key frame in the second set of key frames enabling the client device to switch from rendering one of the first set of delta frames of the first distribution tier to rendering the second distribution tier starting from the particular key frame. A non-transitory computer-readable storage medium that causes the
15. The computer program instructions further cause the processor to: receiving live streaming data from a source client device; and encoding the live streaming data to generate the live media stream including the first distribution layer and the second distribution layer; 15. The non-transitory computer-readable storage medium of claim 14.
16. The computer program instructions further cause the processor to: generating the first distribution layer at an origin server; and causing the origin server to generate the second distribution tier based on receiving the instruction from the client device to modify the rendering of the live media stream; 15. The non-transitory computer-readable storage medium of claim 14.
17. The step of acquiring the live media stream comprises: an intermediate server receiving the first distribution layer from an origin server, the first distribution layer being generated by the origin server; generating, by the intermediate server, the second distribution tier based on receiving the instruction from the client device to modify the rendering of the live media stream; and the intermediate server communicating both the first distribution layer and the second distribution layer to the client device; 15. The non-transitory computer-readable storage medium of claim 14, comprising:
18. 15. The non-transitory computer-readable storage medium of claim 14, wherein the first distribution tier comprises a first bit rate and the second distribution tier comprises a second bit rate that is different from the first bit rate.
19. 15. The non-transitory computer-readable storage medium of claim 14, further comprising generating the second distribution tier based on a determination that the client device is a new subscriber to the live media stream.
20. generating the second distribution layer includes: determining that a time interval between a subscription time of the live media stream by the client device and a time of a next key frame in the first set of key frames is greater than a threshold time interval; and generating the second distribution tier in response to determining that the time interval is greater than the threshold time interval.
20. The non-transitory computer-readable storage medium of claim 19, comprising:
Citation Information
Patent Citations
High-speed starting for digital video stream
JP2005006339A
Distributed encoding of a video stream
US20160381367A1