Live stream playing method and device based on HLS protocol, electronic equipment and product

Abstract

The invention relates to a live stream playing method and device based on an HLS protocol, electronic equipment and a product. The method comprises the following steps: in response to receiving same-channel live streams produced by different sources, a main playing link and a standby playing link respectively generate data files based on an HLS protocol, and the data files comprise live streams needing to be played; synchronizing the data file generated by the main playing link with the data file generated by the standby playing link; and in response to the failure of the main playing link, distributing the data file to the player through the content distribution network by the standby playing link, and seamlessly playing the live stream by the player. According to the method and the device, the data files generated by the main playing link and the standby playing link are synchronized, so that seamless and continuous playing of the live stream can be realized.

Description

Technical Field

[0001] This disclosure generally relates to the field of live streaming playback, and in particular to a live streaming playback method, apparatus, system, electronic device and product based on the HLS protocol. Background Technology

[0002] With the rapid development of internet TV live streaming services, the stability and reliability of live streaming services have become core competitive advantages for platforms. Any node failure can lead to broadcast incidents; therefore, related technologies require a complete disaster recovery system to ensure high availability of live streaming services under various failure scenarios. However, in practice, when the primary node fails and switches to a backup node, the terminal player may experience playback abnormalities due to the asynchronous sequence number (EXT-X-MEDIA-SEQUENCE, or sequence for short) in the HTTP Live Streaming (HLS) stream. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a live streaming playback method, apparatus, system, electronic device and product based on the HLS protocol.

[0004] According to some embodiments of this disclosure, a live stream playback method based on the HLS protocol is provided, comprising: in response to receiving a live stream of the same channel produced by different sources, generating a data file based on the HLS protocol by a main playback link and a backup playback link respectively, the data file containing the live stream to be played; synchronizing the data file generated by the main playback link and the data file generated by the backup playback link; and in response to a failure of the main playback link, distributing the data file to a player via a content delivery network by the backup playback link, and having the player seamlessly play the live stream.

[0005] In some embodiments, the generation of HLS-based data files by the primary playback link and the backup playback link includes: generating a preset number of slice files from the live stream in chronological order, wherein the slice files are used to represent the segments of the live stream in chronological order; and determining an index file that has a mapping relationship with the preset number of slice files, and using the index file and the preset number of slice files as the data file.

[0006] In some embodiments, determining the index file that has a mapping relationship with the preset number of slice files includes: determining the sequence number and the corresponding absolute sampling time of the slice file in chronological order, wherein the absolute sampling time is the absolute timestamp of the first sampled frame in the corresponding slice file, the interval between the absolute sampling times of two adjacent slice files is the slice duration of the slice file, and the slice duration of each slice file is equal, wherein the slice duration is the playback duration of each segment of the live stream; and using a file containing the sequence number, slice duration, and absolute sampling time corresponding to the slice file as the index file.

[0007] In some embodiments, synchronizing the data file generated by the main playback link with the data file generated by the backup playback link includes: determining a historical baseline time, wherein the historical baseline time is world time; determining a first absolute sampling time for each slice file generated by the main playback link, and determining a second absolute sampling time for each slice file generated by the backup playback link, wherein the time difference between the first absolute sampling time and the corresponding second absolute sampling time is less than the slice duration; and determining a first sequence number for each slice file generated by the main playback link based on the historical baseline time, the first absolute sampling time, and the slice duration, and determining a second sequence number for each corresponding slice file generated by the backup playback link based on the historical baseline time, the second absolute sampling time, and the slice duration, wherein the first sequence number and the second sequence number have an overlapping portion.

[0008] In some embodiments, the step of distributing the data file from the backup playback link to the player via a content delivery network, and then having the player seamlessly play the live stream, includes: the player requesting the server to distribute the index file locally; the server returning an updated index file to the backup playback link based on the newly recorded live stream; the updated index file being distributed to the player via the content delivery network; and the player seamlessly playing the newly recorded live stream based on the updated index file.

[0009] According to some embodiments of this disclosure, a live stream playback device based on the HLS protocol is provided, comprising: a data generation unit, configured to, in response to receiving a live stream of the same channel produced by different sources, generate a data file based on the HLS protocol by a main playback link and a backup playback link respectively, the data file containing the live stream to be played; a synchronization unit, configured to synchronize the data file generated by the main playback link with the data file generated by the backup playback link; and an execution unit, configured to, in response to a failure of the main playback link, distribute the data file to a player via a content delivery network by the backup playback link, and have the player seamlessly play the live stream.

[0010] According to some embodiments of this disclosure, a live streaming playback system based on the HLS protocol is provided, including: a main playback link, a backup playback link, a content distribution network, and a player; wherein: the main playback link is used to receive a first live stream and generate a data file based on the HLS protocol from the first live stream; the backup playback link is used to receive a second live stream and generate a data file based on the HLS protocol from the second live stream; wherein the first live stream and the second live stream are live streams from the same channel produced from different sources, and the data file generated by the main playback link and the data file generated by the backup playback link are synchronized data files; the content distribution network is used to distribute the data file generated by the main playback link to the player, and when the main playback link fails, to distribute the data file generated by the backup playback link to the player; and the player is used to seamlessly play the live stream according to the data file generated by the main playback link or the data file generated by the backup playback link.

[0011] According to some embodiments of this disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable computer programs or instructions; wherein the processor is configured to execute the computer programs or instructions to implement the live streaming playback method based on the HLS protocol in any of the above embodiments.

[0012] According to some embodiments of this disclosure, a storage medium is provided that stores a computer program or instructions, which, when executed by a processor of an electronic device, enable the electronic device to perform the live streaming playback method based on the HLS protocol in any of the above embodiments.

[0013] According to some embodiments of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the live stream playback method based on the HLS protocol in any of the above embodiments.

[0014] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: By synchronizing the HLS-based data files generated by the main playback link and the backup playback link, this disclosure enables the player to seamlessly play live streams of the same channel produced by different sources even when the main playback link fails. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of this disclosure, the embodiments of this disclosure will be further explained and described with reference to the following drawings. These drawings are only used to more conveniently and specifically describe the embodiments of this disclosure and are not intended to limit this disclosure.

[0016] Figure 1 This is a flowchart illustrating a live stream playback method based on the HLS protocol, according to some embodiments of this disclosure;

[0017] Figure 2 This is a flowchart illustrating a method for generating a data file based on the HLS protocol, according to some embodiments of the present disclosure;

[0018] Figure 3 This is a flowchart illustrating a method for determining an index file according to some embodiments of the present disclosure;

[0019] Figure 4 This is a schematic diagram illustrating the generation of data files between the primary playback link and the backup playback link, based on some embodiments of this disclosure;

[0020] Figure 5 This is a schematic diagram illustrating, according to some embodiments of the present disclosure, that the sequence numbers of the index files generated by the primary playback link and the backup playback link are not completely consistent;

[0021] Figure 6 This is a block diagram illustrating a live streaming playback device based on the HLS protocol, according to some embodiments of this disclosure;

[0022] Figure 7 This is a block diagram illustrating a live streaming playback system based on the HLS protocol, according to some embodiments of this disclosure;

[0023] Figure 8 This is a block diagram illustrating an apparatus for playing a live stream, according to some embodiments of the present disclosure; and

[0024] Figure 9 This is another block diagram illustrating an apparatus for playing live streams according to some embodiments of the present disclosure. Detailed Implementation

[0025] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become readily apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become readily apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0026] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0027] The present disclosure provides a live stream playback method based on the HLS protocol, which is used for continuous playback of live streams. By synchronizing the data files generated by the main playback link with the data files generated by the backup playback link, seamless playback of the live stream can still be achieved when the main node in the main playback link fails.

[0028] In related technologies, based on the idea of ​​"unique serial number source," the primary encoder and backup encoder at the encoding layer share a single sequence generator, thus naturally ensuring complete consistency of the EXT-X-MEDIA-SEQUENCE. This is typically achieved in two ways. One approach involves a shared slicer and shared storage. Both the primary and backup encoders are only responsible for outputting slice files. They simultaneously write the slice files to the same NAS (Network Attached Storage) or GPFS (General Parallel File System) directory. The generation of slice files and playlists (also known as the index) is handled by a single-instance sequence generator (packager, such as shaka-packager). If this sequence generator fails, high-availability (HA) software migrates the virtual IP to a backup node. However, only one sequence generator is writing the index at any given time, ensuring the EXT-X-MEDIA-SEQUENCE remains unique. The other approach uses a distributed, shared sequence number service. This service uses distributed locks and counters to uniformly generate sequence numbers. Regardless of which node generates the slice file, it consistently obtains the sequence number from the service and generates the index. The relevant technologies address the uniqueness of sequence numbers at the encoding layer, ensuring sequence number consistency across live streams during distribution. This solution is effective for self-built live stream sources. However, when integrating third-party live stream sources, the sequence numbers of third-party streams are highly likely to be inconsistent with self-built streams on the same channel, or even between different third-party streams on the same channel, causing stuttering or skipping during playback. Furthermore, because these technologies rely on a unique sequence number source, data exchange between the primary and backup nodes is inevitable during master / slave switching, potentially leading to playback delays.

[0029] In view of this, some embodiments of this disclosure provide a live stream playback method based on the HLS protocol, which enables the player to seamlessly play the live stream even when the main playback link fails by synchronizing the data files generated by the main playback link and the backup playback link.

[0030] Figure 1 This is a flowchart illustrating a live stream playback method based on the HLS protocol, according to some embodiments of this disclosure. Figure 1 As shown, the method 100 includes steps S102, S104 and S106.

[0031] In step S102, in response to receiving live streams from different sources on the same channel, the main playback link and the backup playback link respectively generate data files based on the HLS protocol. The data files contain the live streams to be played.

[0032] In this embodiment of the disclosure, in a live streaming system based on the HLS protocol, it is common to receive live streams from different sources on the same channel. For example, when an end user watches an online live educational program, different television program producers may record the online live educational program. To ensure smooth playback of the live stream, two playback links are typically required: a primary playback link and a backup playback link. The primary and backup playback links convert the live streams from different sources into data files based on the HLS protocol. The HLS protocol in this embodiment is essentially a "divide and conquer" video transmission strategy, which can cut a large video stream into many small video segments; therefore, the data file contains the live stream to be played.

[0033] In step S104, the data file generated by the main playback link is synchronized with the data file generated by the backup playback link.

[0034] In step S106, in response to a failure of the main playback link, the backup playback link distributes the data file to the player via the content delivery network, and the player seamlessly plays the live stream.

[0035] In this embodiment, the data files generated by the primary playback link and the backup playback link need to be synchronized. This can be achieved through the Network Time Protocol (NTP) to ensure that the timestamps of the primary and backup playback links are consistent when generating the data files. This ensures that the corresponding small video segments generated by the two playback links are aligned in time, and that each video segment has the same playback duration. During normal playback of the live stream, the primary playback link distributes the data files to the player via the Content Delivery Network (CDN), and the player plays the continuous live stream. When the primary playback link fails, such as when the master node of the distribution layer fails, a primary / backup node switch can be performed. The backup playback link then distributes the synchronized data files to the player via the CDN, enabling the player to play the live stream seamlessly and continuously, without the end user being aware of this switch.

[0036] In this embodiment, by synchronizing the data files generated by the primary playback link with those generated by the backup playback link, a rapid switch to the backup playback link can be achieved when the primary playback link fails. This ensures the continuity and stability of the live stream, avoids live stream interruptions caused by playback link failures, and improves the reliability of the live streaming system. Furthermore, since the data files of the primary and backup playback links are already synchronized, the switching process is imperceptible to the user, achieving seamless playback of the live stream.

[0037] Figure 2 This is a flowchart illustrating a method for generating HLS-based data files according to some embodiments of this disclosure, such as... Figure 2 As shown, the method 200 includes steps 202 and 204.

[0038] In step 202, a preset number of slice files are generated from the live stream in chronological order, wherein the slice files are used to represent the various segments of the live stream in chronological order.

[0039] In step 204, an index file that has a mapping relationship with a preset number of slice files is determined, and the index file and the preset number of slice files are used as data files.

[0040] In this embodiment, based on the HLS protocol, the server first encodes the recorded live stream and then divides it into multiple small video segments, referred to as slice files, in chronological order. Therefore, slice files can be used to represent the various video segments of the live stream in chronological order, and slice files are typically in .ts format. Simultaneously, an index file is generated that maps to multiple slice files, and the index file is typically in .m3u8 format. The index file indicates the position and playback order of these slice files. The terminal (i.e., the player) continuously requests and downloads the latest index file and slice files to achieve continuous playback of the live content. An example of an index file is shown below:

[0041] #EXTM3U

[0042] #EXT-X-VERSION:3

[0043] #EXT-X-TARGETDURATION:10

[0044] #EXT-X-MEDIA-SEQUENCE:100

[0045] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:00.000Z

[0046] #EXTINF:10.0,

[0047] segment100.ts

[0048] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:10.000Z

[0049] #EXTINF:10.0,

[0050] segment101.ts

[0051] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:20.000Z

[0052] #EXTINF:10.0,

[0053] segment102.ts

[0054] The "segment100.ts", "segment101.ts", and "segment102.ts" represent the names of the three segment files and their playback order in the live stream, such as sequence numbers 100, 101, and 102. This index file has a mapping relationship with these three segment files. This index file and the corresponding segment files can be used as data files generated by the main playback link and the backup playback link.

[0055] Figure 3 This is a flowchart illustrating a method for determining an index file according to some embodiments of this disclosure. Figure 3 As shown, the method 300 includes steps 302 and 304.

[0056] In step S302, the sequence number and the corresponding absolute sampling time of the slice file are determined in chronological order. The absolute sampling time is the absolute timestamp of the first sampled frame in the corresponding slice file. The interval between the absolute sampling times of two adjacent slice files is the slice duration of the slice file. The slice duration of each slice file is equal. The slice duration is the playback duration of each segment of the live stream.

[0057] In this embodiment, the HLS protocol has three tags: slice duration (represented by EXTINF), sequence number (represented by sequence), and absolute sampling time (represented by EXT-X-PROGRAM-DATE-TIME). Each slice file corresponds to its own sequence number and absolute sampling time. The sequence number is numbered according to the recording time order of the live stream to indicate the playback order of the slice files. The absolute sampling time is the absolute timestamp of the first sampled frame in the corresponding slice file. Since each slice file represents a small video segment, it contains multiple video image frames, also known as sampled frames. The absolute timestamp corresponding to the first sampled frame during the production of the slice file is the absolute sampling time of that slice file. The absolute timestamp is a timestamp relative to UTC, and the absolute sampling time indicates the position of the corresponding slice file on the timeline of the live stream. For example, in the index file above, "EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:10.000Z,#EXTINF:10.0,segment101.ts" indicates that segment file with sequence number 101 has an absolute sampling time of "2023-10-27T19:00:10.000Z". The absolute sampling time of segment file "segment102.ts" with sequence number 102 is "2023-10-27T19:00:20.000Z". The interval between the absolute sampling times of two adjacent segment files is the segment duration. The segment duration can be understood as the playback duration of the corresponding video segment in the live stream. For example, if the playback duration of a segmented video is 10 seconds, then the segment duration is 10 seconds. In this embodiment of the disclosure, it is necessary to ensure that the segment duration of each segment file generated by the main playback link and each segment file generated by the backup playback link are the same, which can be a fixed value. Assuming the above index file is generated by the main playback link, where "EXTINF:10.0, segment100.ts" indicates that the segment duration of the segment file "segment100.ts" is 10 seconds, and "EXTINF:10.0, segment102.ts" indicates that the segment duration of the segment file "segment102.ts" is 10 seconds, similarly, the playback duration of each segment file generated by the backup playback link is also 10 seconds.

[0058] In step 304, the file containing the sequence number, slice duration, and absolute sampling time corresponding to the slice file is used as an index file.

[0059] As illustrated in the examples above in this disclosure, the sequence number 100, slice duration 10 seconds, and absolute sampling time "2023-10-27T19:00:00.000Z" correspond to the slice file "segment100.ts". Similarly, the sequence number 101, slice duration 10 seconds, and absolute sampling time "2023-10-27T19:00:10.000Z" correspond to the slice file "segment101.ts", and the sequence number 102, slice duration 10 seconds, and absolute sampling time "2023-10-27T19:00:20.000Z" correspond to the slice file "segment102.ts". Files containing these parameters are used as index files. Since the index files are continuously updated, the current index file contains the identifier of the starting slice file. For example, "EXT-X-MEDIA-SEQUENCE:100" in the above index file indicates that the starting slice file's sequence number is "100". Furthermore, a preset number of three slice files are mapped to this index file.

[0060] In this embodiment of the disclosure, in order to quickly switch to the backup playback link when the main playback link fails and to enable seamless and continuous playback of the live stream, it is necessary to synchronize the data files generated by the main playback link and the data files generated by the backup playback link. Figure 4 This is a schematic diagram 400 illustrating the generation of data files between the primary playback link and the backup playback link, based on some embodiments of this disclosure. Figure 4 As shown, a historical baseline time t0 is selected, where t0 is world time, such as Greenwich Mean Solar Time or Coordinated Universal Time (UTC). The absolute sampling time of each slice file generated by the main playback link is used as the first absolute sampling time, and the absolute sampling time of each slice file generated by the backup playback link is used as the second absolute sampling time. Figure 4 As shown, the segment durations of the segment files generated by the main playback link and the backup playback link are the same. The first absolute sampling times of the segment files 1.ts, 2.ts, 3.ts, 4.ts, and 5.ts in the main playback link are t1, t2, t3, t4, and t5, respectively. The second absolute sampling times of the segment files 1.ts, 2.ts, 3.ts, 4.ts, and 5.ts in the backup playback link are t1', t2', t3', t4', and t5', respectively. The time difference between the first absolute sampling time of each segment file in the main playback link and the second absolute sampling time of the corresponding segment file in the backup playback link is less than the segment duration. For example, for segment file 1.st, the time difference between the first absolute sampling time t1 of the main playback link and the second absolute sampling time t1' of the backup playback link is within one segment duration.

[0061] In this embodiment of the disclosure, taking the main playback link as an example, each segment file has a first absolute sample time, denoted by ti, and a historical baseline time t0 is selected. The difference between the absolute sample time and the historical baseline time of each segment file is Ti = (ti - t0), and the difference between each segment file and its previous segment file Ti-1 is a segment duration, i.e., Ti - Ti-1 = duration. Each time a new segment file is added, a new segment duration is added. Since duration = Ti - Ti-1, i.e., duration = (ti - t0) - (ti-1 - t0) = ti - ti-1, Ti / duration can be used as the sequence number of each corresponding segment file. Thus, each time a new segment file is added, its sequence number will also increase by 1. The formula is as follows:

[0062] Serial number = Ti / duration = (ti-t0) / duration. The serial numbers of each segment file in the backup playback link are determined in the same way. Therefore, the first serial number of each segment file generated by the main playback link can be determined based on the historical baseline time, the first absolute sampling time, and the segment duration; similarly, the second serial number of each corresponding segment file generated by the backup playback link can be determined based on the historical baseline time, the second absolute sampling time, and the segment duration. For example... Figure 4 As shown, the sequence numbers of the slice files in the main playback link are 1, 2, 3, 4, and 5, and the sequence numbers of the slice files in the backup playback link are also 1, 2, 3, 4, and 5. Therefore, the sequence numbers corresponding to the same slice files in the main and backup playback links are the same.

[0063] In this embodiment, based on the rules of the HLS protocol, a preset number of slice files are generated in chronological order. Therefore, the index file also corresponds to the preset number of slice file sequence numbers. For example, if the preset number is five, the index file always maintains the sequence numbers of the five slice files. If the generated slice file sequence numbers are 3, 4, 5, 6, and 7, when a slice file with sequence number 8 is generated, the slice file sequence numbers in the index file are updated to 4, 5, 6, 7, and 8. Sequence number 4 serves as the starting sequence number of the current index file, and the earliest slice file sequence number 3 is removed. Therefore, as slice files are continuously generated, the index file continuously "rolls over" to update the slice file sequence numbers, always maintaining the preset number of sequence numbers. Therefore, the existence of overlapping slice file sequence numbers in the current index files of the primary and backup playback links can be considered as the data files generated by the primary playback link and the data files generated by the backup playback link being synchronous data files.

[0064] In this embodiment, when playing live streams from different sources on the same channel, the live streams collected by the main playback link and the live streams collected by the backup playback link are produced by different sources. During live stream playback, the player requests a URL from the server, such as "http: / / example.com / cctv1 / live.m3u8", requesting the distribution of an index file in ".m3u8" format to the local player. Because the index file continuously updates the sequence numbers of the segment files, the server generates a new index file based on the newly recorded live stream and returns the updated index file to both the main and backup playback links. During normal playback, the updated index file from the main playback link is distributed to the player via CDN. The index file shown below is the index file for the main and backup playback links (taking a preset number of 3 segment files as an example):

[0065] #EXT-X-VERSION:3

[0066] #EXT-X-TARGETDURATION:10

[0067] #EXT-X-MEDIA-SEQUENCE:100

[0068] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:00.000Z

[0069] #EXTINF:10.0,

[0070] segment100.ts

[0071] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:10.000Z

[0072] #EXTINF:10.0,

[0073] segment101.ts

[0074] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:20.000Z

[0075] #EXTINF:10.0,

[0076] segment102.ts

[0077] The player downloads the segment file segment100.ts and begins playing this 10-second content (which may be the opening theme and the host's introduction). While segment100.ts is playing, segment101.ts is being downloaded in the background. After segment100.ts finishes playing, segment101.ts begins playing, while segment102.ts is downloaded in the background. After segment101.ts finishes playing, segment102.ts continues playing. After the player is distributed to the index file, it begins polling for new index files while playing the current segment file; for example, it begins requesting a new index file while playing segment100.ts. Since the segment duration is 10 seconds, segment103.ts is not generated during the playback of segment100.ts, and the index file is not updated. When playback reaches the 10th second, the segment file segment103.ts has been generated, and the index file has been updated, as shown in the second index file below, which is the index file updated by the primary and backup playback links:

[0078] #EXT-X-VERSION:3

[0079] #EXT-X-TARGETDURATION:10

[0080] #EXT-X-MEDIA-SEQUENCE:101

[0081] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:10.000Z

[0082] #EXTINF:10.0,

[0083] segment101.ts

[0084] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:20.000Z

[0085] #EXTINF:10.0,

[0086] segment102.ts

[0087] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:30.000Z

[0088] #EXTINF:10.0,

[0089] segment103.ts <-- [Newly added slice file]

[0090] The player requests and obtains the new index file, and continues playing segment103.ts after segment102.ts has finished playing.

[0091] In this embodiment, if the primary playback link fails and cannot update the index file while the player is playing segment102.ts, but the backup playback link can update the index file normally, the player polls for a new index file during the playback of segment102.ts. When playback reaches the 20th second, the backup playback link produces the segment file segment104.ts, and the index file has been updated. The index file shown below (index file three) is the index file updated by the backup playback link.

[0092] #EXT-X-VERSION:3

[0093] #EXT-X-TARGETDURATION:10

[0094] #EXT-X-MEDIA-SEQUENCE:102

[0095] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:20.000Z

[0096] #EXTINF:10.0,

[0097] segment102.ts

[0098] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:30.000Z

[0099] #EXTINF:10.0,

[0100] segment103.ts

[0101] #EXT-X-PROGRAM-DATE-TIME:2023-10-27T19:00:40.000Z

[0102] #EXTINF:10.0,

[0103] segment104.ts <-- [Newly added slice file]

[0104] The player requests and obtains the new index file for the backup playback link, and continues playing segment104.ts after segment103.ts has finished playing.

[0105] In this embodiment of the disclosure, considering that the main playback link and the backup playback link will take a certain amount of time in terms of the production, transmission and related delay strategies of the slice files, resulting in a time delay, there may be serial numbers that are not completely consistent in the current index files of the main playback link and the backup playback link. Figure 5 This is a schematic diagram 500 illustrating, according to some embodiments of this disclosure, that the sequence numbers of the index files generated by the primary playback link and the backup playback link are not completely consistent. Figure 5 As shown, the current index file generated by the main playback link contains segment file sequence numbers 5, 6, 7, 8, and 9, while the current index file of the backup playback link contains sequence numbers 3, 4, 5, 6, and 7. Segment files with the same sequence number have identical content. Assuming a segment duration of 10 seconds, if the main playback link fails while the player is playing segment file with sequence number 7, and the player has a 30-second buffer time while playing segments with sequence numbers 7, 8, and 9, the backup playback link can generate segment files with sequence numbers 8, 9, and 10. Therefore, after the player finishes playing segment file with sequence number 9, it can directly obtain the new index file from the backup playback link and seamlessly play segment file with sequence number 10. Thus, even when there are overlapping sequence numbers in the current index files of the main and backup playback links, it can still be ensured that if the main playback link fails, the backup playback link can obtain the segment file with the next adjacent sequence number, guaranteeing continuous and seamless playback when receiving live streams from multiple sources. In this disclosure, the main playback link and the backup playback link can operate independently, and the serial numbers of both can be synchronized. This avoids the interaction waiting time overhead caused by using the "shared serial number generator" method, and also avoids the risk of failure caused by interaction.

[0106] Based on the same concept, this disclosure also provides a live streaming playback device 600 based on the HLS protocol. It is understood that the live streaming playback device 600 provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. Combining the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0107] Figure 6 This is a block diagram illustrating a live streaming playback device 600 based on the HLS protocol, according to some embodiments of this disclosure. (Refer to...) Figure 6 The device includes a data generation unit 602, a synchronization unit 604, and an execution unit 606.

[0108] The data generation unit 602 is configured to, in response to receiving a live stream from a different source on the same channel, generate a data file based on the HLS protocol from the main playback link and the backup playback link respectively. The data file contains the live stream to be played. The synchronization unit 604 is configured to synchronize the data file generated by the main playback link with the data file generated by the backup playback link. The execution unit 606 is configured to, in response to a failure of the main playback link, distribute the data file to the player via the content delivery network through the backup playback link, and the player seamlessly plays the live stream.

[0109] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0110] Figure 7 This is a block diagram illustrating a live streaming system 700 based on the HLS protocol, according to some embodiments of this disclosure. Figure 7 As shown, the system 700 includes a main playback link 702, a backup playback link 704, a content distribution network 706, and a player 708.

[0111] The main playback link 702 is used to receive the first live stream and generate a data file based on the HLS protocol. The main playback link 702 includes a video acquisition module 7020, an encoding module 7021, a slicing module 7022, and a master node 7023. The video acquisition module 7020 receives the live signal from the first source, the encoding module 7021 encodes the received live signal, the slicing module 7022 generates an index file in .m3u8 format and slice files in .ts format from the encoded live stream according to the HLS protocol, and the master node 7023 distributes the index files to the content delivery network 706.

[0112] The backup playback link 704 is used to receive the second live stream and generate a data file based on the HLS protocol. The structure of the backup playback link 704 is similar to that of the main playback link 702, also including a video acquisition module 7040, an encoding module 7041, a slicing module 7042, and a backup node 7043, but it receives live streams from different sources on the same channel. The first and second live streams are live streams from different sources on the same channel. In this embodiment, the encoding module 7021 and slicing module 7022 of the main playback link 702 and the encoding module 7041 and slicing module 7042 of the backup playback link 704 can be referred to as the encoding layer, and the main node 7023 of the main playback link 702 and the backup node 7043 of the backup playback link 704 can be referred to as the distribution layer.

[0113] In this embodiment, the data file generated by the primary playback link 702 and the data file generated by the backup playback link 704 are synchronized data files. To ensure synchronization between the data files generated by the primary and backup playback links, the system also includes a time synchronization module (not shown in the figure). This time synchronization module ensures the consistency of timestamps between the primary and backup playback links through the NTP protocol, so that the slice files generated by the two links are aligned within the time error range. The time synchronization module periodically sends synchronization signals to the primary and backup playback links to adjust their respective time bases, ensuring that the time series in the generated .m3u8 index files remains consistent.

[0114] Content delivery network 706 distributes data files generated by the main playback link 702 to player 708, and distributes data files generated by backup playback link 704 to player 708 when the main playback link 702 fails. Content delivery network 706 includes edge nodes, central nodes, and a load balancer. Edge nodes are distributed in different geographical locations to cache data files and provide services to end users; central nodes are responsible for obtaining the latest data files from the main or backup playback links and distributing them to the edge nodes; the load balancer monitors the status of the main and backup playback links and automatically switches to the backup playback link data source when a failure of the main playback link is detected.

[0115] Player 708 is used to seamlessly play live streams based on data files generated by the main playback link or backup playback link. Player 708 requests and parses index files in sequence, ensuring smooth playback by playing segmented files seamlessly and continuously in both time and content.

[0116] Through the design of the primary and backup playback links and the seamless switching mechanism described above, the playback system significantly improves the reliability and stability of the live streaming service. Even if the primary playback link fails, the continuous playback of the live streaming content can be guaranteed, effectively avoiding the problem of live streaming interruption or stuttering caused by a single point of failure. Moreover, the end user is unaware of the switching between the primary and backup playback links, thus improving the user experience.

[0117] Figure 8 This is a block diagram illustrating an apparatus 800 for live streaming playback according to some embodiments of the present disclosure. The apparatus 800 can be provided as a terminal. For example, the apparatus 800 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0118] Reference Figure 8 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0119] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0120] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0121] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.

[0122] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0123] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0124] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0125] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0126] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0127] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0128] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0129] Figure 9 This is another block diagram illustrating an apparatus 900 for live stream playback according to some embodiments of the present disclosure. For example, apparatus 900 may be provided as a server. See also... Figure 9 The device 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by a memory 932 for storing instructions, such as application programs, that can be executed by the processing component 922. The application programs stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 922 is configured to execute instructions to perform the aforementioned antenna tuning method.

[0130] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 can operate on an operating system stored in memory 932, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0131] In some embodiments of this disclosure, a storage medium is provided, which may be a non-transitory computer-readable storage medium.

[0132] In some embodiments of this disclosure, when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the live stream playback method based on the HLS protocol described above.

[0133] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the live stream playback method based on the HLS protocol involved in any of the above embodiments.

[0134] In this description, "multiple" means at least two, referring to two or more, such as two, three, etc., unless otherwise explicitly specified. Other quantifiers are similar. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, unless otherwise specified or clearly indicated from the context, the articles "a" and "an" used in this application and the appended claims are generally understood to mean "one or more."

[0135] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of two or more; "and / or" describes the association relationship between related objects, indicating that three relationships may exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Similarly, "at least one in "..." includes any one of the related listed items and any combination of two or more.

[0136] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0137] Furthermore, the term "exemplary" is used herein to indicate that it serves as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term "exemplary" is intended to present concepts in a concrete manner. As used herein, the term "or" is intended to indicate an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X applies A or B" is intended to indicate any of the natural inclusive arrangements. That is, if X applies A; X applies B; or X applies both A and B, then applying A or B is satisfied under any of the foregoing instances.

[0138] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the use of "comprising," "owning," "having," "having," or variations thereof in this disclosure, such terms are intended to be inclusive in a manner similar to the term "including."

[0139] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0140] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for playing live streams based on the HLS protocol, characterized in that, include: In response to receiving a live stream from the same channel produced by different sources, the main playback link and the backup playback link respectively generate a data file based on the HLS protocol, the data file containing the live stream to be played; Synchronize the data file generated by the main playback link with the data file generated by the backup playback link; as well as In response to a failure of the main playback link, the backup playback link distributes the data file to the player via the content delivery network, and the player seamlessly plays the live stream.

2. The method according to claim 1, characterized in that, The generation of HLS-based data files by the primary playback link and the backup playback link respectively includes: The live stream is used to generate a preset number of slice files in chronological order, wherein the slice files are used to represent various segments of the live stream in chronological order; and An index file is determined that has a mapping relationship with the preset number of slice files, and the index file and the preset number of slice files are used as the data file.

3. The method according to claim 2, characterized in that, The index file that determines the mapping relationship with the preset number of slice files includes: The sequence number and absolute sampling time of each slice file are determined chronologically, where the absolute sampling time is the absolute timestamp of the first sampled frame in the corresponding slice file. The interval between the absolute sampling times of two adjacent slice files is the slice duration of the slice file, and the slice duration of each slice file is equal, where the slice duration is the playback duration of each segment of the live stream; and The file containing the sequence number, slice duration, and absolute sampling time corresponding to the slice file is used as the index file.

4. The method according to claim 3, characterized in that, The step of synchronizing the data file generated by the main playback link with the data file generated by the backup playback link includes: Determine the historical baseline time, wherein the historical baseline time is UTC; Determine the first absolute sampling time of each segment file generated by the main playback link, and determine the second absolute sampling time of each segment file generated by the backup playback link, wherein the time difference between the first absolute sampling time and the corresponding second absolute sampling time is less than the segment duration; and The first sequence number of each slice file generated by the main playback link is determined based on the historical baseline time, the first absolute sampling time, and the slice duration. The second sequence number of each slice file generated by the backup playback link is determined based on the historical baseline time, the second absolute sampling time, and the slice duration. The first sequence number and the second sequence number have an overlapping portion.

5. The method according to claim 2, characterized in that, The step of distributing the data file from the backup playback link to the player via the content delivery network, and then having the player seamlessly play the live stream, includes: The player requests the server to distribute the index file to its local machine; The server returns an updated index file to the backup playback link based on the newly recorded live stream. The updated index file is distributed to the player via the content delivery network; and The player seamlessly plays the newly recorded live stream based on the updated index file.

6. A live stream playback device based on the HLS protocol, characterized in that, include: The data generation unit is used to respond to receiving live streams from different sources on the same channel by generating data files based on the HLS protocol from the main playback link and the backup playback link respectively. The data files contain the live streams to be played. A synchronization unit is used to synchronize the data file generated by the main playback link with the data file generated by the backup playback link; as well as An execution unit is configured to, in response to a failure of the main playback link, distribute the data file to the player via the backup playback link through the content delivery network, and have the player seamlessly play the live stream.

7. A live streaming playback system based on the HLS protocol, characterized in that, include: The main playback link, backup playback link, content distribution network, and player; among which: The main playback link is used to receive the first live stream and generate a data file based on the HLS protocol from the first live stream. The backup playback link is used to receive the second live stream and generate a data file based on the HLS protocol from the second live stream; wherein the first live stream and the second live stream are live streams from the same channel produced by different sources, and the data file generated by the main playback link and the data file generated by the backup playback link are synchronized data files; The content delivery network is used to distribute data files generated by the primary playback link to the player, and, in the event of a failure of the primary playback link, to distribute data files generated by the backup playback link to the player; and The player is used to seamlessly play the live stream based on the data file generated by the main playback link or the data file generated by the backup playback link.

8. An electronic device, characterized in that, include: processor; Memory for storing computer programs or instructions executable by the processor; The processor is configured to execute the computer program or instructions to implement the live stream playback method based on the HLS protocol as described in any one of claims 1-5.

9. A storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by the processor of an electronic device, enable the electronic device to perform the live stream playback method based on the HLS protocol as described in any one of claims 1-5.

10. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the live stream playback method based on the HLS protocol as described in any one of claims 1-5.

Images