HLS global synchronization and multi-viewer viewing party

By calculating the time difference between the video encoder and UTC, the position of the playback head is dynamically adjusted. By utilizing a JavaScript synchronization module and an NTP server, the infrastructure dependency problem of the HLS synchronization method is solved, enabling multi-user video synchronization and network adaptability, thus improving the user experience.

CN121646922APending Publication Date: 2026-03-10NAPSTER LLC
View PDF 1 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2026-03-10

AI Technical Summary

Technical Problem

Existing HLS synchronization methods require dedicated infrastructure to start client playback at the same point in time, cannot effectively handle network-related errors, result in multiple streams from different clients being out of sync, and lack quality-of-life features for multi-user viewing.

Method used

By calculating the time difference between the video encoder's date/time stamp and the UTC time server, the playback head position of HTML media elements is controlled, and the playback head position is dynamically adjusted to maintain the time offset. Multiple real-time HLS streams are synchronized using a JavaScript synchronization module and an NTP server, making it suitable for both single-client and multi-client environments.

Benefits of technology

It enables video synchronization across multiple clients without requiring additional infrastructure, providing a multi-user viewing experience for gatherings. It also supports automatic resynchronization in the event of changes in network conditions, thus enhancing the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121646922A_ABST
    Figure CN121646922A_ABST
Patent Text Reader

Abstract

Synchronization is performed for a plurality of real-time hypertext transfer protocol (HTTP) real-time streaming (HLS) streams in a hypertext markup language (HTML) media element. A time difference between the video encoder date / time tag and a current date / time retrieved from a universal time (UTC) time server is calculated. The head position of the HTML media element is controlled to maintain the calculated time offset value, which is defined as the current date / time minus the predefined time offset. Media element source content and head position are dynamically controlled while maintaining video synchronization with a predefined time offset.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 514,590, filed July 20, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to synchronous real-time streaming digital video delivery, which can be used with the Hypertext Transfer Protocol (HTTP) Real-Time Streaming (HLS) protocol and multi-user social viewing party content viewing. Background Technology

[0004] HLS synchronization methods, such as those defined in US 10,397,289 B2, require specialized infrastructure to start client playback at the same point in time. Several open-source HLS players can be configured to start playback of “X” seconds or several segments from the live edge. The latter approach does not account for different clients retrieving the playlist at different times, resulting in different segments constituting the live edge. None of these methods account for network-related errors after playback begins, which could lead to local desynchronization of one or more streams, nor do they provide viewers with any “quality-of-life” features to effectively watch multiple streams. Summary of the Invention

[0005] In one or more illustrative examples, a method for synchronizing multiple real-time Hypertext Transfer Protocol (HTTP) real-time streaming (HLS) streams in Hypertext Markup Language (HTML) media elements includes: calculating the time difference between a video encoder date / time tag and the current date / time obtained from a Universal Time (UTC) time server; controlling the playhead position of the HTML media element to maintain a calculated time offset value, the calculated time offset value being defined as the current date / time minus a predefined time offset; and dynamically controlling the media element content and the playhead position while maintaining video synchronization relative to the predefined time offset.

[0006] In one or more illustrative examples, multiple live HLS streams are segmented, and the UTC date / timestamp of the segments is written to the playlist or catalog of the multiple live HLS streams.

[0007] In one or more illustrative examples, perform one or more of the following: use the application programming interface (API) of HTML media elements to perform the method, or use the API of the HLS video player to perform the method.

[0008] In one or more illustrative examples, the method also includes using a search function to adjust the playhead position to a specified location.

[0009] In one or more illustrative examples, the method further includes up-modulating the playhead speed to adjust the playhead position to a specified location.

[0010] In one or more illustrative examples, the method also includes down-modulating the playback speed to adjust the playhead position to a specified location.

[0011] In one or more illustrative examples, the method further includes checking the actual time offset over the interval and performing an action to adjust the playhead position in response to the actual time offset indicating that it exceeds a specified drift.

[0012] In one or more illustrative examples, HTML media elements are destroyed and recreated in a way that restores UTC synchronization using new HTML media elements.

[0013] In one or more illustrative examples, the time value calculated before one of the HTML media elements is destroyed is reindexed to restore the state of the HTML media element when it is recreated.

[0014] In one or more illustrative examples, the time increment between the previous UTC date / time tag value and the new UTC date / time tag value obtained when the element is recreated is calculated.

[0015] In one or more illustrative examples, the time increment value is used to calculate and search for the playhead position of the source content.

[0016] In one or more illustrative examples, the method also includes synchronously receiving multiple HLS streams from the client device.

[0017] In one or more illustrative examples, the method also includes synchronizing multiple HLS streams across multiple client devices to achieve global synchronization of video playback for a viewing party consisting of multiple viewers.

[0018] In one or more illustrative examples, the audience's geographical location varies.

[0019] In one or more illustrative examples, the audience communicates via a voice chat mechanism.

[0020] In one or more illustrative examples, viewers communicate via a video chat mechanism.

[0021] In one or more illustrative examples, the audience communicates via a text chat mechanism.

[0022] In one or more illustrative examples, a system for synchronizing multiple real-time HLS streams in an HTML media element, the multiple real-time HLS streams being segmented into a manifest marked with a UTC date / timestamp, the system including a client hardware device executing a web browser application to perform operations including: calculating the time difference between the UTC date / timestamp and the current date / time obtained from a UTC time server; controlling the playhead position of the HTML media element to maintain a calculated time offset value, the calculated time offset value being defined as the current date / time minus a predefined time offset; and dynamically controlling the source content and playhead position of the media element while maintaining video synchronization across the multiple real-time HLS streams relative to the predefined time offset.

[0023] In one or more illustrative examples, a non-transient computer-readable medium includes instructions for synchronizing multiple real-time HLS streams in an HTML media element, the multiple real-time HLS streams being segmented into a manifest marked with a UTC date / timestamp, the manifest causing the client hardware device to perform operations when executed by a client hardware device executing a web browser application, the operations including: calculating the time difference between the UTC date / timestamp and the current date / time obtained from a UTC time server; controlling the playhead position of the HTML media element to maintain a calculated time offset value, the calculated time offset value being defined as the current date / time minus a predefined time offset; and dynamically controlling the source content and playhead position of the media element while maintaining video synchronization across the multiple real-time HLS streams relative to the predefined time offset. Attached Figure Description

[0024] Figure 1 A block diagram of a system for streaming digital video transmission is shown.

[0025] Figure 2 An example HLS list is shown;

[0026] Figure 3 An example process for streaming digital video transmission is shown;

[0027] Figure 4 An example user interface for a web-based player for streaming digital video is shown.

[0028] Figure 5 An example user interface for a web-based player for streaming digital video is shown.

[0029] Figure 6 An example user interface for a web-based player for streaming digital video is shown.

[0030] Figure 7An example user interface for a web-based player for streaming digital video is shown.

[0031] Figure 8 The example video-on-demand (VOD) query string is shown in the generic resource location (URL);

[0032] Figure 9 An example of VOD parameters compared to the current playback position is shown;

[0033] Figure 10 Further details of the initial synchronization are shown; and

[0034] Figure 11 Further details of the subsequent synchronization are shown. Detailed Implementation

[0035] Detailed embodiments of the invention are disclosed herein as needed; however, it should be understood that the disclosed embodiments are merely examples of the invention, which can be implemented in various and alternative forms. The drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show details of specific components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as a representative basis for teaching those skilled in the art to employ the invention in various ways.

[0036] Figure 1 A block diagram of a system for streaming digital video delivery is shown. As illustrated, the system includes a Content Delivery Network (CDN) 101, multiple streams 103, and a user session 105 executed by a client (such as a browser on a user's device). In the diagram, two sample streams 103 are provided from a content source to the user session 105 via the CDN 101. Streams 103 may, for example, have the same events but originate from different camera angles.

[0037] At a high level, HLS works by encoding video stream 103 into segments. These segments are uploaded as they become available to the origin server. The video is typically distributed to clients via CDN 101. Segments are usually aligned with Groups of Pictures (GOPs) starting with an "I" frame, which provides resilience in case of network errors, and are appended to the live manifest (e.g., live m3u8 format). In many cases, even when multiple streams 103 of the same live event are viewed from different camera angles from a single user session 105, the stream 103 may not be synchronized with one or more GOPs, which are typically 2–6 seconds long. This problem is complicated by the propagation of video segments and the live manifest via CDN 101, increasing the time of desynchronization between viewers.

[0038] A method is provided for synchronizing multiple HLS live streams 103 or a set of VOD streams 103 created from a previous HLS live stream 103. By design, HLS lacks features that allow simultaneous and synchronized viewing of multiple audio / video (AV) streams 103 produced by individual encoders.

[0039] This disclosure allows clients to simultaneously play back multiple HLS streams 103, synchronized to a common time (UTC) offset or a set of VOD streams 103 generated by previous events synchronized by time parameters. Therefore, regardless of geographic location, all viewers will be able to watch multiple “camera angles” of live events encoded to HLS, not only in sync within their browser sessions but also in sync with other viewers. This mechanism enables remote “watch parties” where geographically dispersed viewers can socially interact with each other via voice / video / text chat while simultaneously watching content at the same point in the broadcast, as if they were in the same physical location.

[0040] Figure 2 A sample HLS manifest is shown. This disclosure improves existing web-based HLS players by adding a JavaScript synchronization module that provides the functionality required to synchronize real-time HLS and VOD HLS. The real-time HLS functionality allows multiple real-time videos to be synchronized to an offset of “X” seconds from the current UTC time by reading the #EXT-X-PPOGRAM-DATE-TIME tag 201 written to the HLS manifest by the encoder and running a time increment algorithm software module to move the playhead to the appropriate time offset. (For high accuracy, the encoder should be configured to add this tag to each segment of the manifest.) The VOD HLS functionality parses timing parameters obtained via a URL query string or inter-client network transaction, which provide the data needed to calculate the playhead position. The real-time HLS implementation utilizes the timestamp provided by the encoder as a timing truth source, and the VOD HLS implementation queries a Public Network Time Protocol (NTP) time server during video loading, where the failover system time is used as the timing truth source. The same time synchronization method can be used with the Institute of Embedded Wired and Telecommunications Engineers (SCTE)-35 timecode, which allows the same method to be used with any container capable of carrying Stream 103 packets. The module runs within the client browser session and requires no additional infrastructure.

[0041] The real-time HLS algorithm creates a time increment value from the system's current UTC and the program date / time stamp retrieved from the HLS manifest. This increment is used to ensure that the playback head of each video is within 1 second of the offset time (+ / -0.5 seconds). The time difference array is a set of time offsets calculated by the synchronization module, with each real-time stream 103 having its own time offset, which seeks to move forward / backward to achieve + / -1 second of the UTC time obtained from the NTP server.

[0042] The VOD HLS algorithm has two timing modes. The static time offset mode provides an experience similar to live streaming or client-to-client streaming, where timing is generated by the initiating user's playhead time. The VOD static time offset method parses a URL query string that provides the start time (year / month / day / hour / minute / second), the VOD duration (in hours / minutes / seconds), and the calculated playhead position, as if the VOD were a live stream loop from its start time. The VOD client-to-client method transmits the initiating user's playback start time and current playhead time to "viewing party visitors" via a p2p connection.

[0043] Figure 3 An example data stream is shown for the operation of the real-time HLS synchronization algorithm module. As shown, a video source such as CDN101, providing HLS stream 103 to the synchronization module.

[0044] The real-time HLS synchronization algorithm module in the web player is initialized through three function calls. The first call initializes the source array 301, which contains the HLS URLs for playback. The second call initializes the HLS player 303 and uses the array to retrieve the HLS manifest and begin downloading segments. The HLS player 303 includes logic to provide compatibility with both Chromium and Webkit browsers. The third call initiates synchronization 305, calling the appropriate functions required by Chromium or Webkit. The synchronization function now obtains UTC from the NTP server and also from the extended date / time tag value of the first segment, and uses these two values ​​to calculate the time increment between them. This increment value is used by the forward / reverse (F / R) converter to perform initial synchronization. The initialization of the source array 301, the HLS player 303, and the synchronization 305 can be wrapped in a single JavaScript function or called separately depending on the requirements of the front-end software stack. Functions 301, 303, and 305 are designed to allow the initialization of one or more video elements at a time.

[0045] For each video element, the element object, index, and tolerance value are passed to the synchronization function 305. The #EXT-X-PROGRAM-DATE-TIME tag 201 for each element is saved to an array, the current playhead time is stored in an array, and the two are added together to produce a new datetime object representing the current UTC time of the playhead, which is stored in a new date array. A new UTC datetime object representing the current UTC time is created, and the time difference is calculated by subtracting the index value of the new date array from the current UTC time object 307, and the resulting time difference is stored in array 309. Several validation checks 311 are performed to ensure that the values ​​returned in the previous steps are of the correct type passed as input to the algorithm; if they pass, the main logic of the synchronization function begins at 313. The time difference array value 309 for the corresponding video element is checked to determine whether it is less than or greater than half the delay minus the tolerance value, or, as shown at 315, whether it is greater than or less than half the delay plus the tolerance value. The result of check 311 awaits a decision, and one of four actions will occur to satisfy the configured valid edge offset value. These four actions include (i) searching the video to the calculated playhead position, (ii) increasing the playback speed to catch up with the offset, (iii) decreasing the playback speed to meet the offset, or (iv) setting the playback speed to normal when the offset has been met.

[0046] Once a video element has met its offset, within 1 second (+ / - 0.5 seconds) of the offset target 317, the drift value is applied to its tolerance, and the synchronization function sleeps, checking the interval to determine if the drift tolerance 319 has been exceeded. The drift value prevents unnecessary logic from running, such as when the video stabilizes near the edge of the initial synchronization value. If the drift exceeds the specified limit, the main logic of the synchronization function executes at 321 and returns the playback head position to the specified offset. Network errors, bandwidth congestion, recoverable video errors, or any event that moves playback out of its specified drift range will trigger resynchronization.

[0047] When a "VOD" or "EVENT" playlist type is detected, the VOD HLS synchronization routine is initialized. It provides an end-user viewing experience similar to pre-programmed scheduled programs. First, real-time synchronization is disabled to prevent unintended speed modulation. Then, the URL is parsed based on the presence of the "vod" query string, such as... Figure 8As shown, if it is the first or only URL parameter, the query string is delimited by "?" and the URL, or by "&" if it is a subsequent parameter. The "vod" parameter provides timing information to the VOD synchronization function in two parts: start date / time and VOD duration. The start date / time string provides the year, month, day, hour, minute, and second at which the VOD program content will begin its playback loop. The second field, delimited by "-", provides the duration of the VOD in hours, minutes, and seconds. In the example, each value of both fields is represented by two numeric characters. Once parsed, the VOD synchronization function calculates the current playhead position relative to the start time provided in the first parameter field, taking into account the content duration in the second field. This is in Figure 9 As shown in the diagram. The result is a looping sequence of content that appears to the end user as live playback. In contrast to live looping channel types, the advantage of VOD time offset is a significant reduction in infrastructure costs, as it eliminates the need to run an encoder pipeline and provides static content via CDN 101.

[0048] Figure 10 Further details of the initial synchronization are shown. As shown in operation 1001, the encoder encodes segments of one or more video streams 103, adding #EXT-X-PROGRAM-DATE-TIME information. In the example, each segment can be approximately three seconds long. As shown, these segments are pushed to CDN 101. It should be noted that multiple streams 103 can exist for video, such as streams 103 obtained from different locations, views, or perspectives of the same event. These segments can be encoded and pushed to CDN 101 simultaneously. However, it is possible that the header of one stream 103 is ahead or behind the header of another stream 103 by several seconds. This can be caused by various factors, such as how the manifest is read and the speed at which the segments propagate through CDN 101.

[0049] At operation 1003, user session 105, executed by a client application (such as a browser on a user device wishing to receive stream 103), pulls the segment. The client application also makes a time request to the NTP server and receives the current time back from the NTP server. The client application then creates an increment between the date-time stamp indicated by the encoder in the #EXT-X-PROGRAM-DATE-TIME information and the current time received from the NTP time server. At operation 1005, this time difference is saved to the date-time array 309.

[0050] At operation 1007, the client application reads the old incremental time value from the array. As discussed herein, the old DT value refers to the time difference currently being used by stream 103, as described above, and the new DT value is the time difference being tested. It should be noted that during the first iteration of this process, the new DT value and the old DT value refer to the same value.

[0051] At operation 1009, the client application calculates the difference between the new DT value and the old DT value to check the synchronization of stream 103. For the first iteration, this difference is, of course, zero. At operation 1011, this difference is compared with a synchronization tolerance value. The synchronization tolerance value should generally be less than the segment length. For example, if the segment length is three seconds, a one-second synchronization tolerance can be applied.

[0052] At operation 1013, if the difference exceeds the synchronization tolerance value, the playhead position is updated based on the time difference between DT new and UTC time. At operation 1015, if the difference is within the synchronization tolerance, the playhead position is updated to continue streaming the current segment at the correct time offset. At operation 1017, if the difference is outside the synchronization tolerance (if necessary), a find is used to adjust synchronization.

[0053] Figure 11 Further details of the subsequent synchronization are shown. Similar to operation 1001, in operation 1101, the encoder encodes video segments, adding #EXT-X-PROGRAM-DATE-TIME information. As shown, these segments are pushed to CDN 101. At operation 1103, similar to operation 1103, the user session 105 of the client application (such as the browser of a user's device that wants to stream content) continues to pull segments.

[0054] In operation 1105, the client application sends another time request to the NTP server and receives the current time back from the server. In operation 1107, the client application again determines the time difference between the current UTC time from the NTP server and #EXT-X-PROGRAM-DATE-TIME to check the synchronization of stream 103.

[0055] At operation 1109, this difference is set as a time offset variable. At operation 1111, the time offset variable can be compared with a synchronization tolerance value to determine if stream 103 has drifted far enough in time offset to require resynchronization. For example, if the difference exceeds the synchronization tolerance value, the playhead position is updated based on the time difference between DT new and UTC time. At operation 1113, the playhead position is then updated to continue streaming the current segment at the correct time offset.

[0056] Figure 4An example user interface is shown displaying multiple video elements 401 for video selection and a larger video element 403. This user interface offers several advantages when watching multiple live streams 103 simultaneously, including the ability to refocus any smaller sub-video element 401 onto the larger main element 403. This refocusing can be achieved, for example, by tapping or clicking on a smaller video element 401 to bring that video element 401 into the larger video element 403. Although not shown, it is also conceivable that separate screens could be synchronized with the same and different related content, rather than an array of screens on a page.

[0057] Figure 5 An example user interface is shown displaying multiple video elements 501 and a larger video element 403 for audio selection. Here, the main video element 501 is muted, while one of the child video elements 503 is unmuteed. The user interface provides the ability to unmute the smaller video element 503, which can be performed through various selections, such as hovering the mouse over 501 (desktop) or long-pressing (moving). Enabling the sound of one of the smaller video elements 503 may also include muting the main video element 501.

[0058] Return to reference Figure 4 The ability to refocus one of the child video elements 403 onto the main element 401 is novel and non-obvious because, within the context of the browser's Document Object Model (DOM), the initial state of the element to be moved must first be destroyed. When the element is destroyed and reloaded, without... Figure 3 In the case of the module discussed in detail, there is no guarantee that the user interface will load the playhead back to its previous position. In the case of this module, the pre-destruction time increment allows newly created elements to quickly find their way back to the synchronized playhead offset, and the algorithm performs fine-tuning as needed. These mechanisms provide viewers with a way to efficiently watch multiple video streams simultaneously 103 with less screen space, as we see when watching on a mobile device.

[0059] Example Implementation Plan

[0060] While there are several ways to implement an HLS synchronization module, some preferred implementations employ HTML5 media elements with an open-source HLS player for Chromium-based browsers (many options are available) or a native HLS player backend for Webkit-based browsers. The synchronization module (e.g., as per [reference to...]) Figure 3 The discussed implementation is imported from the front-end stack and calls three initialization functions 301, 303, and 305, specifying the source array 301 and the video element array, and then calls the corresponding synchronization function for the detected browser core. One of two visual layouts is suggested for this implementation (e.g., as...). Figures 4 to 7 (as shown in the image), but many are possible and left to the judgment of the web designer.

[0061] The first proposed implementation best highlights the features of the module. The larger main video element 601, hosting the mainstream 103 or program feed, is effective for utilizing the entire feature set as described, where the smaller sub-video elements 603 surround... Figure 6 The surrounding layout is as follows. All elements are initialized for online playback and start in a muted state. Upon loading, the module function will synchronize all streams 103 in time. The click event of the main element provides the necessary user input to enable audio and unmute the main element. A long press while hovering or moving the mouse over the child video element 603 will mute the main element and unmute the target child element, while moving the mouse away will mute the child element 603 and unmute the main element 601. The click event of the child element 603 provides one of two recommended actions: the child element 603 refocuses on the position of the main element 601, or the child element 601 and the main element 603 will swap positions. When the design requires all videos to behave as child elements 603 with static positions, the first implementation is appropriate, where only the main element 601 will be destroyed and recreated upon click. When child element 603 does not require a static position, the second implementation scenario is appropriate. When clicked, both the main element 601 and the target child element 603 will be destroyed. The main element 601 will be recreated in the position of child element 603, and the child element 603 will be recreated in the position of the main element 601.

[0062] In either implementation scenario, when an element is destroyed and recreated, the associated time difference variable is moved to the corresponding index in its array. When the element is loaded into its new position, a new #EXT-X-PROGRAM-DATE-TIME time tag 201 is stored in the variable, and a time increment is created between the old and new tags. Subtracting the difference between the two tags from the current time array index of the target element, the newly created element "finds" the adjusted playhead position, and if fine adjustments are needed, speed modulation is performed.

[0063] The second proposed implementation scheme consists of "multiple boxes" of video elements 701 of equal size, such as Figure 7 As shown, all elements are initialized for online playback and begin in a muted state. Similar to the first implementation, a click event on video element 701 will enable audio and unmute the main (usually the top left of the multi-box) video element. Mouse hover over child element 701 will perform the same actions described previously, but in this case, the click functionality is unnecessary and will be disabled. This type of implementation is more suitable when working with front-end frameworks that experience issues with destroying and recreating elements, but still utilizes multi-stream synchronization capabilities.

[0064] Purpose and advantages

[0065] The proposed method provides the ability to synchronize multiple live and on-demand HLS video streams 103 without requiring external infrastructure, utilizing only the HLS protocol's built-in tagging system and simple software modules. It is capable of automatic resynchronization in the event of network / bandwidth issues, a behavior not observed in any other HLS player's 303 function set. It also provides additional "quality of life" functionality for dynamically controlling audio state and refocusing Hypertext Markup Language (HTML) media elements across different positions on a webpage, which would be practically impossible without the aforementioned synchronization features, thus improving the user experience for watching multiple live and / or on-demand streams 103 simultaneously.

[0066] It should be noted that the steps are the same for both single-client multi-stream and multi-client X number of streams (103). The source of the time is always UTC time obtained from a globally synchronized NTP server. Therefore, all viewers of the event, whether in a shared viewing party or watching individually, are at playhead positions + / - 1 seconds apart.

[0067] Typically, the processes, methods, or algorithms disclosed herein can be transmitted to / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, processes, methods, or algorithms can be stored in various forms as data and instructions executable by a controller or computer, including but not limited to information permanently stored on non-writable storage media (such as read-only memory (ROM) devices) and information variablely stored on writable storage media (such as floppy disks, magnetic tapes, optical discs (CDs), random access memory (RAM) devices, and other magnetic and optical media). Processes, methods, or algorithms can also be implemented in a software executable object. Alternatively, the processes, methods, or algorithms can be embodied, in whole or in part, using suitable hardware components (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices) or a combination of hardware, software, and firmware components.

[0068] While exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms covered by the claims. The language used in this specification is descriptive rather than restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form other embodiments of the invention that may not be explicitly described or shown. While various embodiments may have been described as providing advantages or superiority over other embodiments or prior art embodiments in one or more desired characteristics, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system properties, depending on the specific application and embodiment. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, etc. Therefore, any embodiment described as less desirable than other embodiments or prior art embodiments in one or more characteristics is not outside the scope of this disclosure and may be desirable for a particular application.

Claims

1. A method for synchronizing multiple live Hyper Text Transfer Protocol (HTTP) Live Streaming (HLS) streams in a Hyper Text Markup Language (HTML) media element, the method comprising: computing a time difference between a video encoder date / time stamp and a current date / time taken from a Universal Time (UTC) time server; controlling a playhead position of the HTML media element to maintain a computed time offset value defined as the current date / time minus a predefined time offset; and dynamically controlling media element source content and the playhead position while maintaining video synchronization relative to the predefined time offset. The multiple live HLS streams are segmented, and a UTC date / time stamp of a segment is written into a manifest or playlist of the multiple live HLS streams.

2. The method of claim 1, wherein, One or more of the following are performed:

3. The method of claim 1, wherein, The method is performed using an application programming interface (API) of the HTML media element, or The method is performed using an API of an HLS video player. A seek function is used to adjust the playhead position to a specified position.

4. The method of claim 3, further comprising: The playhead speed is modulated up to adjust the playhead position to the specified position.

5. The method of claim 4, further comprising: The play speed is modulated down to adjust the playhead position to the specified position.

6. The method of claim 4, further comprising: An actual time offset over an interval is checked, and an action is performed to adjust the playhead position in response to the actual time offset indicating an excess specified drift.

7. The method of claim 1, further comprising: Destroying and recreating the HTML media element in a manner that resumes UTC synchronization with a new HTML media element.

8. The method of claim 1, wherein, A time value computed prior to destroying one of the HTML media elements is re-indexed to restore a state of the HTML media element upon recreating the HTML media element.

9. The method of claim 8, wherein, A time increment value is computed between a previous UTC date / time stamp value and a new UTC date / time stamp value taken upon element recreation.

10. The method of claim 8, wherein, The time increment value is used to compute and seek the playhead position of the source content.

11. The method of claim 10, wherein, Synchronizing multiple HLS streams received by a client device.

12. The method of claim 1, further comprising: Synchronizing multiple HLS streams across multiple client devices enabling global synchronization of video playback of a viewing party composed of multiple viewers.

13. The method of claim 1, further comprising: The viewers are geographically dispersed.

14. The method of claim 13, wherein, The viewers communicate through a voice chat mechanism.

15. The method of claim 14, wherein, The viewers communicate through a video chat mechanism.

16. The method of claim 14, wherein, The viewers communicate through a text chat mechanism.

17. The method of claim 14, wherein, 18. A system for synchronizing multiple live HLS streams in an HTML media element, the multiple live HLS streams being segmented with a UTC date / time stamp in a manifest, the system comprising: a client hardware device executing a web browser application to perform operations including: computing a time difference between the UTC date / time stamp and a current date / time taken from a UTC time server; controlling a playhead position of the HTML media element to maintain a computed time offset value defined as the current date / time minus a predefined time offset; and ​ ​ controlling media element source content and the playhead position dynamically while maintaining video synchronization across the multiple live HLS streams relative to the predefined time offset.

19. A non-transitory computer-readable medium comprising instructions for synchronizing multiple live HLS streams in an HTML media element, the multiple live HLS streams being tagged with a UTC date / time stamp in a manifest, the manifest, when executed by a client hardware device executing a web browser application, causing the client hardware device to perform operations comprising: calculating a time difference between the UTC date / time stamp and a current date / time taken from a UTC time server; controlling a playhead position of an HTML media element to maintain a calculated time offset value defined as the current date / time minus a predefined time offset; and controlling media element source content and the playhead position dynamically while maintaining video synchronization across the multiple live HLS streams relative to the predefined time offset.

Citation Information

Patent Citations

  • HTTP live streaming (HLS) video client synchronization

    US10397289B2