A method for fixed-delay playback and seamless switching of live streams based on absolute time stamps.

Abstract

This invention discloses a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps, comprising the following steps: S1: Receive an HTTP-FLV data stream, append an absolute time stamp (absolute_time) to each Tag object, and store the Tag objects in a circular buffer queue; S2: Determine the playback time and locate the Tag object to be sent based on the absolute time stamp (absolute_time); S3: Rewrite the timestamp of the Tag object to be sent, encapsulate it, and send it to the player client; S4: Upon receiving a switching command to switch the current playback stream to the target playback stream, seamlessly switch to the target playback stream for playback. This invention's method for fixed-delay playback and seamless switching of live streams based on absolute time stamps can achieve precise fixed-delay playback of live streams and seamless switching between multiple live sources without full-stream transcoding.

Description

Technical Field

[0001] This invention belongs to the field of streaming media processing technology, and in particular relates to a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps. Background Technology

[0002] In scenarios such as live streaming monitoring and delayed live sports broadcasts, it is necessary to delay the playback of the live stream for a fixed duration and switch between different signal sources. Current existing technologies mainly include: client-side buffering and server-side transcoding alignment.

[0003] Client-side buffering (e.g., Chinese patent application CN102752193A):

[0004] The principle is to set a large buffer on the player side, or maintain a circular buffer queue based on sequence numbers (Seq) on the server side. The focus is on using arrays to construct the circular queue and Seq indices to improve buffer access speed. The goal is "fast buffering," while the timestamp still depends on the source stream.

[0005] It has the following drawbacks: latency accuracy is greatly affected by network jitter, and cannot guarantee an "absolutely fixed" latency (for example, the latency may change after refreshing the page); it only solves the "fast buffering" problem, but does not solve the "timeline continuity" problem when switching between multiple sources. Since it does not involve modifying timestamps, it cannot solve the problem of timeline jumps when switching between multiple sources.

[0006] Server-side transcoding alignment methods (such as Chinese patent application CN117768687A):

[0007] The principle is as follows: all live sources are uniformly transcoded on the server side, and the image group (GOP) structure and timestamp are forcibly aligned to ensure that the keyframes and timestamps are consistent when switching.

[0008] It has the following drawbacks: extremely high computational cost, requiring real-time transcoding of all streams; requiring all streams to be processed synchronously at the transcoding end, resulting in poor flexibility; and inability to handle situations where the timestamps of the source streams are inconsistent. Relying on server-side transcoding to forcibly align the GOPs and timestamps of different streams requires all streams to pass through a unified transcoder, consuming huge computational resources and resulting in high latency. Summary of the Invention

[0009] To address the problems in the prior art, this invention aims to provide a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps. This method can achieve precise fixed-delay playback of live streams and seamless switching between multiple live stream sources without full-stream transcoding.

[0010] To achieve the above objectives, this invention provides a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps, comprising the following steps:

[0011] S1: Stream access and buffering steps: Receive an HTTP-FLV data stream, parse the Tag objects of the HTTP-FLV data stream one by one, attach an absolute time stamp (absolute_time) to the Tag objects, and store the Tag objects in a circular buffer queue.

[0012] S2: Extended retrieval steps: Determine the playback time and locate the Tag object as the Tag object to be sent based on the absolute time marker absolute_time;

[0013] S3: Virtual timeline rewriting and output steps: rewrite the timestamp of the Tag object to be sent and encapsulate it to send it to a player client;

[0014] S4: Seamless switching logic steps: When a switching stream instruction is received to switch the current playback stream to the target playback stream, seamless switching to the target playback stream is performed.

[0015] As one implementation, step S1 further includes the step of:

[0016] S11: Connection establishment step: The client connects to an original live stream address and receives the HTTP-FLV data stream;

[0017] S12: Read and parse the tag header of each Tag object in the HTTP-FLV data stream sequentially, and extract the type and original timestamp of the Tag object;

[0018] S13: Add the absolute time marker absolute_time to the Tag object, obtain the current system time, and assign it to the absolute time marker absolute_time;

[0019] S14: Encapsulate the Tag object and append it to the tail of the circular buffer queue corresponding to the HTTP-FLV data stream;

[0020] S15: After each new Tag object is stored, check the absolute time stamp (absolute_time) of the head Tag object of the circular buffer queue; if the difference between the current time and the assigned value of the absolute time stamp (absolute_time) is greater than the preset maximum cache duration, remove the head Tag object and use the second Tag object of the circular buffer queue as the new head Tag object.

[0021] As one implementation, the circular buffer queue is a double-ended circular buffer queue.

[0022] In one implementation, step S13 further adds a keyframe identifier and a global sequence number to the Tag object.

[0023] As one implementation, step S2 further includes the step of:

[0024] S21: Calculate the absolute time point T_{target} corresponding to the content that should be played at the current moment;

[0025] S22: Search for a Tag object in the circular buffer queue corresponding to the absolute time point T_{target} assigned to the absolute time tag absolute_time in the current HTTP-FLV data stream; use this Tag object as the starting frame;

[0026] S23: Starting from the position of the starting frame in the circular buffer queue, sequentially take the subsequent Tag objects in the circular buffer queue as the Tag objects to be sent, and perform subsequent processing steps.

[0027] As one implementation method, in step S21, the absolute time point T_{target} is calculated according to formula (1):

[0028] T_{target}=T_{current\_system}-T_{delay} (1);

[0029] Where T_{current_system} represents the server's current system clock time; T_{delay} represents the preset delay duration.

[0030] As one implementation, step S3 further includes the following steps:

[0031] S31: Define a raw timestamp variable TS_{raw}, a first timestamp offset Offset, an output timestamp TS_{out}, and an output timestamp TS_{last\_out} for the timestamp of the Tag object to be sent;

[0032] S32: Calculate the output timestamp TS_{out} according to formula (2):

[0033] TS_{out}=TS_{raw}-Offset (2);

[0034] S33: Determine whether the output timestamp TS_{out} is less than the output timestamp TS_{last\_out} of the previous sent tag. If so, forcibly correct the output timestamp TS_{out} according to formula (3):

[0035] TS_{out}=TS_{last\_out}+\Delta (3);

[0036] Where \Delta represents a tiny increment;

[0037] S34: The tag header and tag data of the Tag object with the rewritten timestamp are recombined into binary data and encapsulated to form playback data, which is then sent to the player client via an HTTP connection;

[0038] S35: The player client plays the HTTP-FLV data stream as the current playback stream based on the playback data.

[0039] As one implementation, the value of the tiny increment Δ includes 10ms.

[0040] As one implementation, step S4 further includes the following steps:

[0041] S41: A switching stream instruction to switch the current playback stream to the target playback stream has been received;

[0042] S42: Calculate the absolute time point T_{target} corresponding to the content that the target playback stream should play;

[0043] S43: Search for a Tag object in the circular buffer queue corresponding to the absolute time point T_{target} assigned to the absolute time tag absolute_time in the target playback stream; starting from this Tag object, search for the first Tag object with the keyframe identifier True after the circular buffer queue and use it as the first keyframe Tag;

[0044] S44: Calculate the second timestamp offset Offset_B of the target playback stream according to formula (4):

[0045] Offset_B=TS_{key\_B}-TS_{last\_A}-10ms (4);

[0046] Where TS_{key_B} represents the original timestamp of the first keyframe tag; TS_{last_A} represents the output timestamp of the last tag output by the current playback stream;

[0047] S45: Starting from the position of the first keyframe tag in the circular buffer queue of the target playback stream, extract the script tag and sequence header of the tag one by one and synthesize a new tag as the target tag;

[0048] S46: Copy the data of the target tag, calculate and rewrite the output timestamp TS_{out_new} of the target tag using the second timestamp offset Offset_B, and send the target tag to the player client first.

[0049] As one implementation method, the output timestamp TS_{out_new} of the target tag is calculated according to formula (5):

[0050] TS_{out\_new}=TS_{last\_A}+1ms (5).

[0051] Because the present invention adopts the above technical solution, it has the following beneficial effects:

[0052] This invention introduces an absolute time stamp (absolute_time) to establish a mapping between physical time and streaming content, thereby achieving fixed latency accurate to milliseconds, unaffected by changes in the source stream timestamp. By rewriting the timestamps, a virtual timeline is constructed, solving the timeline jump problem during multi-source switching. Direct manipulation of the FLV Tag container layer eliminates video decoding / re-encoding, resulting in extremely low CPU consumption. Pre-alignment of the source streams is unnecessary. By calculating offsets, the timelines of different source streams are dynamically "stitched" together at the output. Regardless of whether the source stream timestamps are in the tens or hundreds of millions, they can all be mapped to the same continuous 0~N timeline. This application has the following beneficial effects:

[0053] 1. Extremely low cost: No GPU cluster is required for transcoding; a single machine can handle hundreds of high-concurrency, low-latency streams.

[0054] 2. Excellent user experience: When switching live stream sources, the player is completely unaware of the changes, with no black screen, no buffering animation, and no progress bar rewinding.

[0055] 3. High reliability: By retransmitting the sequence header and aligning the keyframes, the common problems of "screen flickering" and "decoding errors" during FLV switching are completely solved. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps, as described in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Please see Figure 1 An embodiment of the present invention provides a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps, comprising the following steps:

[0060] S1: Stream access and buffering steps: Receive an HTTP-FLV data stream, parse the Tag objects of the HTTP-FLV data stream one by one, attach an absolute time stamp (absolute_time) to the Tag objects, and store the Tag objects in a circular buffer queue.

[0061] Step S1 further includes the following steps:

[0062] S11: Connection establishment step: The client connects to an original live stream address and receives the HTTP-FLV data stream;

[0063] S12: Read and parse the tag header of each Tag object in the HTTP-FLV data stream sequentially, and extract the type and original timestamp of the Tag object;

[0064] S13: Add the absolute time marker absolute_time to the Tag object, obtain the current system time, and assign it to the absolute time marker absolute_time;

[0065] S14: Encapsulate the Tag object and append it to the tail of the circular buffer queue corresponding to the HTTP-FLV data stream;

[0066] S15: After each new Tag object is stored, check the absolute time stamp (absolute_time) of the head Tag object of the circular buffer queue; if the difference between the current time and the assigned value of the absolute time stamp (absolute_time) is greater than the preset maximum cache duration, remove the head Tag object and use the second Tag object of the circular buffer queue as the new head Tag object.

[0067] In this embodiment, the circular buffer queue is a double-ended circular buffer queue.

[0068] In this embodiment, in step S13, a keyframe identifier and a global sequence number are also added to the Tag object.

[0069] This invention defines an extended FLV Tag data structure in system memory. It should be noted that the "extended fields" below are not fields inherent in the original HTTP-FLV stream, but rather internal metadata calculated in real-time by the stream access module and appended to the memory object after the system receives the original stream data.

[0070] The structure is defined as follows:

[0071] Class FLVTag:

[0072] # --- Raw FLV protocol fields, directly parse self-streaming data ---

[0073] tag_type: int # Integer type, indicating the tag type: 8, audio / 9, video / 18, script;

[0074] timestamp: int # Integer type, the relative timestamp in the raw stream, in milliseconds;

[0075] data: bytes # Byte stream, audio and video payload data;

[0076] # --- The newly added system extension fields in this invention are generated and appended by this system---

[0077] absolute_time: datetime # Absolute system time at the time of receipt, core index key;

[0078] is_keyframe: bool # Whether it is a keyframe, used for GOP location;

[0079] sequence_number: int # Global sequence number used for sorting;

[0080] The meanings of the fields added in this application are as follows:

[0081] 1. Absolute time stamp (absolute_time):

[0082] Type: Datetime object

[0083] Meaning: Records the precise physical time when the tag was received by the system, for example: 2025-12-05 12:00:00.123.

[0084] Function: As the core index key, it is used to map relative streaming time to an absolute physical time axis, enabling delayed retrieval based on absolute time.

[0085] 2. Keyframe identifier is_keyframe:

[0086] Type: Boolean (bool, True / False)

[0087] Meaning: Indicates whether the current video tag contains keyframes or I-frames. The system determines this by parsing the first byte (AVC Packet Type) of the tag data.

[0088] Purpose: Used for "seamless switching" logic. The video decoder must start decoding from the keyframe, so when switching streams, the system must find a tag with is_keyframe=True as the entry point; otherwise, it will cause screen glitches.

[0089] 3. Global sequence number: sequence_number

[0090] Type: Integer (int)

[0091] Meaning: The system assigns a unique, monotonically increasing number to each received tag.

[0092] Function: To ensure the strict order of tags in the buffer and prevent out-of-order issues during multi-threaded processing; also to assist in determining the boundaries of group of images (GOP).

[0093] In this embodiment, this step processes a continuous HTTP-FLV data stream.

[0094] 1. Establish connection: The system, as a client, connects to the original live stream address and begins receiving data.

[0095] 2. Parse Tags One by One: An HTTP-FLV stream consists of an FLV Header and numerous consecutive Tag units. Each Tag contains a Tag Header and Tag Data. The system reads and parses the Tag Header of each Tag sequentially, extracting its type (Audio / Video / Script) and raw timestamp.

[0096] 3. Attach absolute time: After parsing the Tag object, i.e. the Python object in memory, the system immediately obtains the current system time, such as 2025-12-05 12:00:00.123, and assigns it to the absolute time tag absolute_time of the Tag object.

[0097] Note: This operation only modifies the memory object and does not modify the original FLV binary data.

[0098] Store in a circular buffer: Append the encapsulated Tag object to the tail of the deque corresponding to the stream.

[0099] Memory eviction: After each new tag is stored, check the absolute time stamp (absolute_time) of the oldest tag at the head of the queue.

[0100] If the current time minus the header tag time is greater than the maximum cache duration, then the header tag is removed (Pop left).

[0101] After removal, the second tag in the original queue automatically becomes the new head tag, thus maintaining a dynamic balance between queue length and memory usage.

[0102] S2: Extended retrieval steps: Determine the playback time and locate the Tag object as the Tag object to be sent based on the absolute time marker absolute_time;

[0103] Step S2 further includes the following steps:

[0104] S21: Calculate the absolute time point T_{target} corresponding to the content that should be played at the current moment;

[0105] S22: Search for a Tag object in the circular buffer queue corresponding to the absolute time point T_{target} assigned to the absolute time tag absolute_time in the current HTTP-FLV data stream; use this Tag object as the starting frame;

[0106] S23: Starting from the position of the starting frame in the circular buffer queue, sequentially take the subsequent Tag objects in the circular buffer queue as the Tag objects to be sent, and perform subsequent processing steps.

[0107] As one implementation method, in step S21, the absolute time point T_{target} is calculated according to formula (1):

[0108] T_{target}=T_{current\_system}-T_{delay} (1);

[0109] Where T_{current_system} represents the server's current system clock time; T_{delay} represents the preset delay duration.

[0110] In this step of the embodiment, the delayed retrieval includes the following steps:

[0111] 1. Determine the playback time: The output thread runs periodically. First, it calculates the absolute time point T_{target} corresponding to the content that should be played at the current time.

[0112] Formula: T_{target}=T_{current\_system}-T_{delay};

[0113] Where T_{current_system} is the server's current system clock time; T_{delay} is the administrator's preset delay duration (e.g., 30 seconds).

[0114] 2. Locate the playback point: In the current stream's buffer, find the tag whose absolute time marker (absolute_time) is closest to T_{target}.

[0115] This tag is the "starting frame" that needs to be sent to the player.

[0116] Subsequently, the system will start from that position and process the subsequent tags in the buffer sequentially.

[0117] S3: Virtual timeline rewriting and output steps: rewrite the timestamp of the Tag object to be sent and encapsulate it to send it to a player client;

[0118] Step S3 further includes the following steps:

[0119] S31: Define a raw timestamp variable TS_{raw}, a first timestamp offset Offset, an output timestamp TS_{out}, and an output timestamp TS_{last\_out} for the timestamp of the Tag object to be sent;

[0120] S32: Calculate the output timestamp TS_{out} according to formula (2):

[0121] TS_{out}=TS_{raw}-Offset (2);

[0122] S33: Determine whether the output timestamp TS_{out} is less than the output timestamp TS_{last\_out} of the previous sent tag. If so, forcibly correct the output timestamp TS_{out} according to formula (3):

[0123] TS_{out}=TS_{last\_out}+\Delta (3);

[0124] Where \Delta represents a tiny increment;

[0125] S34: The tag header and tag data of the Tag object with the rewritten timestamp are recombined into binary data and encapsulated to form playback data, which is then sent to the player client via an HTTP connection;

[0126] S35: The player client plays the HTTP-FLV data stream as the current playback stream based on the playback data.

[0127] In this embodiment, the value of the minute increment Δ includes 10ms.

[0128] In this embodiment, the virtual timeline rewriting and output steps are as follows:

[0129] In order for the player to think that it is playing a continuous stream, the timestamp of each tag to be sent must be rewritten.

[0130] Define variables:

[0131] TS_{raw}: The raw timestamp carried in the Tag object, parsed from the source stream.

[0132] Offset: The timestamp offset of the current stream, a state variable maintained by the system.

[0133] TS_{out}: After rewriting, write the final FLV binary data and send the output timestamp to the player.

[0134] TS_{last\_out}: Records the output timestamp of the last sent tag.

[0135] Processing logic:

[0136] 1. Basic calculation: TS_{out} = TS_{raw} - Offset;

[0137] 2. Monotonically Increasing Correction: Due to potential timestamp resets or jitter in the source stream, the calculated TS_{out} may be less than or equal to the timestamp of the previous packet, i.e., TS_{out} ≤ TS_{last_out}. To prevent player errors, a forced correction must be made: TS_{out} = TS_{last_out} + Delta.

[0138] Delta represents a small increment, typically 10ms.

[0139] After TS_{out} is corrected, the new value is used directly without returning and re-evaluating.

[0140] 3. Encapsulation and Sending: The rewritten timestamp tag header and tag data are recombined into binary data and sent to the player client via an HTTP connection.

[0141] S4: Seamless switching logic steps: When a switching stream instruction is received to switch the current playback stream to the target playback stream, seamless switching to the target playback stream is performed.

[0142] Step S4 further includes the following steps:

[0143] S41: A switching stream instruction to switch the current playback stream to the target playback stream has been received;

[0144] S42: Calculate the absolute time point T_{target} corresponding to the content that the target playback stream should play;

[0145] S43: Search for a Tag object in the circular buffer queue corresponding to the absolute time point T_{target} assigned to the absolute time tag absolute_time in the target playback stream; starting from this Tag object, search for the first Tag object with the keyframe identifier True after the circular buffer queue and use it as the first keyframe Tag;

[0146] S44: Calculate the second timestamp offset Offset_B of the target playback stream according to formula (4):

[0147] Offset_B=TS_{key\_B}-TS_{last\_A}-10ms (4);

[0148] Where TS_{key_B} represents the original timestamp of the first keyframe tag; TS_{last_A} represents the output timestamp of the last tag output by the current playback stream;

[0149] S45: Starting from the position of the first keyframe tag in the circular buffer queue of the target playback stream, extract the script tag and sequence header of the tag one by one and synthesize a new tag as the target tag;

[0150] S46: Copy the data of the target tag, calculate and rewrite the output timestamp TS_{out_new} of the target tag using the second timestamp offset Offset_B, and send the target tag to the player client first.

[0151] Calculate the output timestamp TS_{out_new} of the target tag according to formula (5):

[0152] TS_{out\_new}=TS_{last\_A}+1ms (5).

[0153] It should be noted that the "calculation using Offset_B" mentioned in step S46 means that the entire target stream, including key frames and initialization tags, is in the same new virtual time axis system, which is defined by $Offset_B$. The formula (5) TS_{out_new}=TS{last_A}+1ms is the final output result for the initialization tag, the script tag (Script Tag) / sequence header (Sequence Header), which is a special object. Since initialization tags are usually metadata synthesized or extracted by the system, they need to be inserted into the tiny gap between "the end of the old stream" and "the first frame of the new stream". Formula (5) precisely defines this insertion position, that is, 1ms after the end of the old stream. According to the previous formula, the timestamp of the first frame of the new stream, that is, the key frame, is calculated as: TS_{new_key}=TS_{last_A}+10ms, which is derived based on Offset_B. The initialization tag must be played before the key frame. The value given by formula (5), TS_{last_A} + 1ms, exactly satisfies this order TS_{last_A}< TS_{out_new}<TS_{new_key}. Therefore, although formula (5) does not explicitly include Offset_B, the result it describes meets the continuity requirements of the new stream time axis determined by Offset_B and is the most accurate definition of the timestamp of the synthesized data. Formula (5) is the "result", while "using Offset_B" is the "cause" of the new stream time axis.

[0154] In this embodiment, the seamless switching logic includes:

[0155] When receiving a switching instruction to switch from stream A to stream B, the system performs the following operations:

[0156] 1. Find the entry point: In the buffer of stream B, start searching backward from the time point corresponding to T_{target} and find the first video tag with is_keyframe being True.

[0157] Purpose: Ensure that the first frame after switching is the key frame I-frame to avoid screen flickering.

[0158] 2. Calculate the new offset: To splice the time axis, we need to calculate the new offset Offset_B of stream B so that the timestamp of the first frame of stream B is immediately after the last frame of stream A.

[0159] Let: TS_{key_B} be the original timestamp of the key frame at the entry point of stream B.

[0160] Let: TS_{last_A} be the output timestamp of the last tag output by stream A.

[0161] Formula derivation: TS_{out\_new}\approx TS_{last\_A}+10ms, which means: TS_{key\_B}-Offset_B=TS_{last\_A}+10ms. Therefore: Offset_B=TS_{key\_B}- TS_{last\_A}-10ms.

[0162] The system updates the current session's Offset to Offset_B.

[0163] 3. Retransmit initialization data: Before sending the media data of stream B, the system extracts the latest metadata (script tags) and encoding parameters (sequence header) from the buffer of stream B and immediately synthesizes a new tag:

[0164] Operation: Copy the data of these tags, calculate their output timestamps using the new Offset_B, usually set to a small increment such as TS_{last_A}+1ms, and then send them to the player first.

[0165] Purpose: To inform the player of new video resolution, encoding format, and other information.

[0166] Switch output: Starting from the keyframe found in step 1, use the new Offset_B to execute the rewrite and output logic in "step 3" for each subsequent tag of stream B.

[0167] An embodiment of the present invention provides a method for fixed-delay playback and seamless switching of live streams based on absolute time stamps. The system mainly comprises three core modules:

[0168] Stream access and tagging module: Receives the raw FLV stream, parses the tags, adds an absolute time stamp (absolute_time) to the stream, and places it into a circular buffer in memory.

[0169] Delayed retrieval module: Based on the configured delay duration D$, calculate the target physical time $T_{target} = T_{now}-D, and quickly retrieve the corresponding tag from the buffer.

[0170] Virtual Timeline Output Module: This is the core of this application. It maintains a virtual, continuously increasing playback timeline. Regardless of which stream is switched to, the output tag timestamps are rewritten and mapped onto this virtual timeline, thus deceiving the player into believing that the same continuous stream is always playing.

[0171] This invention introduces an absolute time stamp (absolute_time) to establish a mapping between physical time and streaming content, thereby achieving fixed latency accurate to milliseconds, unaffected by changes in the source stream timestamp. By rewriting the timestamps, a virtual timeline is constructed, resolving the timeline jump problem during multi-source switching. Direct manipulation of the FLV Tag container layer eliminates video decoding / re-encoding, resulting in extremely low CPU consumption. No pre-alignment of source streams is required. By calculating offsets, the timelines of different source streams are dynamically "stitched" together at the output. Regardless of whether the source stream timestamps are in the tens or hundreds of millions, they can all be mapped to the same continuous 0~N timeline. This application also boasts extremely low cost, eliminating the need for GPU clusters for transcoding, and enabling a single machine to handle hundreds of high-concurrency, low-latency streams; excellent user experience, with no player awareness during live source switching, no black screens, no buffering animations, no progress bar rollback, and high reliability. By retransmitting the sequence header and aligning keyframes, it completely solves the common "screen tearing" and "decoding errors" problems during FLV switching.

[0172] The embodiments of this invention can be applied to scenarios including delayed live sports broadcasts, seamless multi-camera switching, and time-shifted live playback. In the delayed live sports broadcast scenario, a fixed delay is provided for content security review. In the seamless multi-camera switching scenario, the control room switches between different camera feeds, providing viewers with a smooth experience similar to television broadcasting. In the time-shifted live playback scenario, an instant playback function based on a circular buffer is implemented.

[0173] The offset calculation and switching logic of this invention embodiment is explained in detail below:

[0174] # Pseudocode: Offset calculation when switching streams:

[0175] def switch_stream(new_stream_buffer, last_output_ts):

[0176] Functionality: Calculates the timestamp offset required when switching to a new stream and sends initialization data.

[0177] Input parameters:

[0178] - new_stream_buffer: A circular buffer for the new stream, containing a list of tags to be played;

[0179] - last_output_ts: The output timestamp of the last tag sent by the old stream to the player before switching, used to ensure continuity.

[0180] illustrate:

[0181] 1. Find the entry point in the new stream buffer, the first keyframe.

[0182] Explanation: Traverse the new stream buffer and find the first tag where is_keyframe is True.

[0183] Objective: To ensure that the decoder starts decoding from a complete I-frame and avoids screen tearing.

[0184] first_keyframe = find_first_keyframe(new_stream_buffer).

[0185] 2. Calculate the new offset

[0186] Variable definition:

[0187] - first_keyframe.timestamp: The original timestamp of the new stream ingress keyframe in the source stream.

[0188] - new_offset: The new offset we need to calculate.

[0189] - last_output_ts: Playback timestamp at the end of the old stream.

[0190] Calculation logic:

[0191] The playback time of the first frame of the new stream, new_timestamp, immediately follows the old stream, that is:

[0192] new_timestamp = last_output_ts + 10ms, where 10ms is a preset small interval;

[0193] At the same time, according to the rewritten formula:

[0194] new_timestamp=first_keyframe.timestamp-new_offset;

[0195] By combining the two equations, we can obtain:

[0196] first_keyframe.timestamp-new_offset=last_output_ts+10;

[0197] Rearranging terms to derive the core formula:

[0198] new_offset=first_keyframe.timestamp-last_output_ts–10.

[0199] 3. Resend the initialization header using the new offset.

[0200] explain:

[0201] send_metadata: Sends a Script Tag containing metadata such as resolution and frame rate.

[0202] send_sequence_header: Sends the AVC Sequence Header, which contains SPS / PPS decoding parameters.

[0203] offset=new_offset: Instructs the send function to rewrite the timestamps of these initialization tags using the newly calculated offset.

[0204] effect:

[0205] Ensure that the player receives the correct decoding parameters before receiving new video frames, and that the timestamps of these parameters are in the correct positions on the virtual timeline.

[0206] send_metadata(new_stream.metadata,offset=new_offset)

[0207] send_sequence_header(new_stream.sps_pps,offset=new_offset).

[0208] 4. Return Results

[0209] Explanation: Return the calculated new_offset to the main control loop.

[0210] Purpose: The main loop will use this new_offset to process all subsequent tags in the new stream.

[0211] return new_offset.

[0212] Detailed explanation of the logic guaranteeing monotonically increasing timestamps:

[0213] # Pseudocode: Timestamp processing in the output loop:

[0214] def process_tag(tag, offset, last_output_ts):

[0215] Functionality: Handles timestamp rewriting for a single tag and corrects possible rollbacks.

[0216] Input parameters:

[0217] -tag: The FLV Tag object to be processed, containing the original timestamp tag.timestamp.

[0218] -offset: The offset of the current timestamp.

[0219] -last_output_ts: Output timestamp of the last sent tag.

[0220] Step 1: Basic Calculations

[0221] Calculate the expected output timestamp using the standard formula.

[0222] # new_ts: The timestamp of the current tag that is scheduled to be sent to the player.

[0223] new_ts=tag.timestamp–offset.

[0224] Step 2: Correct rollback, monotonically increasing check

[0225] Problem background: The timestamp of the source stream may experience slight backtracking or jitter due to encoder issues.

[0226] Or, at the moment of switching, the calculated new_ts may occasionally be less than last_output_ts.

[0227] Judgment logic: If the current calculated value new_ts <= the previous output value last_output_ts;

[0228] if new_ts <= last_output_ts;

[0229] Forced correction:

[0230] Discard the calculated value and directly set the current timestamp to the previous timestamp + 10ms.

[0231] This artificially creates a tiny positive increment, forcibly ensuring that the time axis flows forward.

[0232] new_ts = last_output_ts + 10.

[0233] Step 3: Return the final result:

[0234] Return the corrected new_ts as the timestamp of the final write to the FLV data packet.

[0235] return new_ts.

[0236] It should be noted that while the preferred embodiments of the present invention are provided in the specification and accompanying drawings, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are not intended to impose additional limitations on the content of the present invention; their purpose is to provide a more thorough and comprehensive understanding of the disclosure of the present invention. Furthermore, the above-described technical features can be combined with each other to form various embodiments not listed above, all of which are considered to be within the scope of the present invention specification. Moreover, those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

[0237] The present invention has been described in detail above with reference to the accompanying drawings and embodiments. Those skilled in the art can make various modifications to the present invention based on the above description. Therefore, certain details in the embodiments should not be construed as limiting the present invention, and the scope of protection of the present invention shall be defined by the appended claims.

Claims

1. A method for fixed-delay playback and seamless switching of live streams based on absolute time stamps, comprising the following steps: S1: Stream access and buffering steps: Receive an HTTP-FLV data stream, parse the Tag objects of the HTTP-FLV data stream one by one, attach an absolute time stamp (absolute_time) to the Tag objects, and store the Tag objects in a circular buffer queue. S2: Extended retrieval steps: Determine the playback time and locate the Tag object as the Tag object to be sent based on the absolute time marker absolute_time; S3: Virtual timeline rewriting and output steps: rewrite the timestamp of the Tag object to be sent and encapsulate it to send it to a player client; S4: Seamless switching logic steps: When a switching stream instruction is received to switch the current playback stream to the target playback stream, seamlessly switch to the target playback stream for playback; Step S3 further includes the following steps: S31: Define a raw timestamp variable TS_{raw}, a first timestamp offset Offset, an output timestamp TS_{out}, and an output timestamp TS_{last\_out} for the timestamp of the Tag object to be sent; S32: Calculate the output timestamp TS_{out} according to formula (2): TS_{out}=TS_{raw}-Offset (2); S33: Determine whether the output timestamp TS_{out} is less than the output timestamp TS_{last\_out} of the previous sent tag. If so, forcibly correct the output timestamp TS_{out} according to formula (3): TS_{out}=TS_{last\_out}+\Delta (3); in, Delta represents a tiny increment; S34: The tag header and tag data of the Tag object with the rewritten timestamp are recombined into binary data and encapsulated to form playback data, which is then sent to the player client via an HTTP connection; S35: The player client plays the HTTP-FLV data stream as the current playback stream based on the playback data.

2. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 1, characterized in that, Step S1 further includes the following steps: S11: Connection establishment step: The client connects to an original live stream address and receives the HTTP-FLV data stream; S12: Read and parse the tag header of each Tag object in the HTTP-FLV data stream sequentially, and extract the type and original timestamp of the Tag object; S13: Add the absolute time marker absolute_time to the Tag object, obtain the current system time, and assign it to the absolute time marker absolute_time; S14: Encapsulate the Tag object and append it to the tail of the circular buffer queue corresponding to the HTTP-FLV data stream; S15: After each new Tag object is stored, check the absolute time stamp (absolute_time) of the head Tag object of the circular buffer queue; if the difference between the current time and the assigned value of the absolute time stamp (absolute_time) is greater than the preset maximum cache duration, remove the head Tag object and use the second Tag object of the circular buffer queue as the new head Tag object.

3. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 2, characterized in that, The circular buffer queue is a double-ended circular buffer queue.

4. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 2, characterized in that, In step S13, a keyframe identifier and a global sequence number are also added to the Tag object.

5. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 4, characterized in that, Step S2 further includes the following steps: S21: Calculate the absolute time point T_{target} corresponding to the content that should be played at the current moment; S22: Search for a Tag object in the circular buffer queue corresponding to the absolute time point T_{target} assigned to the absolute time tag absolute_time in the current HTTP-FLV data stream; use this Tag object as the starting frame; S23: Starting from the position of the starting frame in the circular buffer queue, sequentially take the subsequent Tag objects in the circular buffer queue as the Tag objects to be sent, and perform subsequent processing steps.

6. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 5, characterized in that, In step S21, the absolute time point T_{target} is calculated according to formula (1): T_{target}=T_{current\_system}-T_{delay} (1); Where T_{current_system} represents the server's current system clock time; T_{delay} represents the preset delay duration.

7. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 5, characterized in that, The value of the tiny increment Δ includes 10ms.

8. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 5, characterized in that, Step S4 further includes the following steps: S41: A switching stream instruction to switch the current playback stream to the target playback stream has been received; S42: Calculate the absolute time point T_{target} corresponding to the content that the target playback stream should play; S43: Search for a Tag object in the circular buffer queue corresponding to the absolute time point T_{target} assigned to the absolute time tag absolute_time in the target playback stream; Starting from the tag object, find the first tag object with the keyframe identifier True after the circular buffer queue and use it as the first keyframe tag. S44: Calculate the second timestamp offset Offset_B of the target playback stream according to formula (4): Offset_B=TS_{key\_B}-TS_{last\_A}-10ms (4); Where TS_{key_B} represents the original timestamp of the first keyframe tag; TS_{last_A} represents the output timestamp of the last tag output by the current playback stream; S45: Starting from the position of the first keyframe tag in the circular buffer queue of the target playback stream, extract the script tag and sequence header of the tag one by one and synthesize a new tag as the target tag; S46: Copy the data of the target tag, calculate and rewrite the output timestamp TS_{out_new} of the target tag using the second timestamp offset Offset_B, and send the target tag to the player client first.

9. The method for fixed-delay playback and seamless switching of live streams based on absolute time stamps according to claim 8, characterized in that, Calculate the output timestamp TS_{out_new} of the target tag according to formula (5): TS_{out\_new}=TS_{last\_A}+1ms (5).

Images