HLS online splicing method, system, equipment and medium

Abstract

The invention discloses an HLS (HTTP Live Streaming) online splicing method, system, equipment and medium, and belongs to the technical field of streaming media. According to the method, a time addressable mechanism is introduced, and media splicing logic is bound to a unified time axis instead of being indexed according to a URI path or a serial number traditionally; and in combination with a'dynamic object mapping 'mechanism, a to-be-spliced fragment list is dynamically generated according to a playing context, so that seamless splicing of cross sources and cross video objects is realized. According to the method, decoding, rendering or multi-track synthesis is not carried out on the media, the byte segments of the media are extracted according to the time range based on the container-level structure, the HLS fMP4 fragments are generated online, and the method is different from an existing frame-by-frame processing method. According to the method, standard HLS output can be generated, the client does not need to have a special rendering capability, the online splicing efficiency and stability are remarkably improved, and the method is suitable for scenes such as rapid video combination and accurate positioning playing.

Description

Technical Field

[0001] This invention relates to the field of streaming media playback and processing technology, specifically to an HLS online splicing method, system, device, and medium, aiming to address the shortcomings of existing HLS splicing methods in terms of accurate splicing, cross-source compatibility, and efficiency. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] With the explosive growth of online audio and video content, users have placed higher demands on video editing, splicing, and on-demand playback. The ability to quickly stitch together multiple segments from different sources or locations into a continuous playback stream without downloading the entire video is a core requirement for many application scenarios (such as rapid combination of multiple clips, replay of live stream highlights, and UGC editing and publishing). Currently, common technical solutions for achieving HLS splicing include: HLS splicing based on the #EXT-X-DISCONTINUITY tag: The HLS protocol provides the #EXT-X-DISCONTINUITY tag to identify discontinuous transition points in a playlist. When TS segments from different sources or with different encodings are listed sequentially in a single M3U8 playlist and this tag is inserted, the HLS player recognizes it as a discontinuous jump in the stream and continues playing subsequent segments. However, this method has significant limitations: First, discontinuous tags usually require alignment at segment boundaries during transitions; otherwise, decoder errors or screen tearing may occur. Second, in many actual player implementations, frequent use of the #EXT-X-DISCONTINUITY tag can lead to playback instability. For example, reports indicate that some players reset the playback process when encountering this tag, causing stuttering and limitations on dragging progress. Furthermore, the discontinuous splicing method still relies on pre-cut fixed-length segments, making it difficult to accurately locate the disaligned time points, thus limiting the accuracy of the splicing.

[0004] TAMS (Time-Addressable Media Store) is a media storage method developed by the BBC. It stores media content in time-based segments and provides a unified open API, allowing applications to request media data segments based on time coordinates. This method enables "on-demand clipping," meaning that independent files can be dynamically combined and output based on the timecode of the source content without pre-generating separate files. For example, the TAMS API can efficiently extract segments with arbitrary start and end times from long videos for rapid splicing and publishing without copying or transcoding the entire file. Vendors like AWS have integrated this idea into their media services, allowing users to obtain corresponding media segments and automatically generate playlists by specifying a time range. However, the TAMS method primarily serves as a backend content storage and production tool, relying on specific server implementations and API protocols. Directly adopting TAMS in actual HLS online playback scenarios still faces challenges in compatibility with existing HLS clients, cross-domain access, security authorization, and real-time performance optimization. Therefore, effectively integrating the time-addressable principle into the standard HLS playback system still presents a significant technical hurdle.

[0005] Other splicing solutions include offline splicing of TS streams using transcoding tools (such as FFmpeg) or dynamic splicing of buffers on the client side via Media Source Extensions (MSE). These solutions either require significant preprocessing time or increase client-side complexity, making them less than ideal for large-scale online applications.

[0006] In summary, existing technologies for seamless online splicing of cross-source videos either suffer from deficiencies in splicing accuracy and stability (such as simple discontinuous tag splicing) or require complex customized infrastructure (such as a TAMS backend), lacking a sophisticated, controllable, efficient, and universal HLS splicing method. Therefore, it is necessary to provide a new technical solution that elevates media splicing logic to the timeline level and incorporates a dynamic mapping mechanism to overcome these shortcomings.

[0007] It should be noted that existing splicing technologies based on multimedia frame-by-frame processing (such as CN110691276B) rely on decoding media frames, creating multiple video tracks, and performing layer-level rendering and compositing. However, this invention does not involve frame-by-frame parsing, decoding, or rendering operations, nor does it generate edited video files. Instead, it directly extracts media segments by byte range based on the media container structure. Therefore, the technical principle and processing path are completely different from these existing technologies. Summary of the Invention

[0008] The purpose of this invention is to provide an HLS online splicing method, system, device, and medium to solve the problems of insufficient splicing accuracy, poor compatibility, and complex implementation in existing technologies for online splicing of multiple media segments. By changing the splicing control from the traditional segment sequence to direct addressing of the timeline, and combining it with a mechanism for dynamically selecting segment sources based on context, high-precision seamless splicing of streaming media content across different video sources is achieved, significantly improving splicing efficiency and playback stability.

[0009] The technical solution of the present invention is as follows: An HLS online stitching method includes: Establish a timeline address space: Create a unified timeline for the multiple media sources to be spliced, and map the video content of each source to the timeline position; Time-addressable splicing: Based on the start and end times of the target splicing segment on the timeline, the time range is mapped to the byte range of the underlying media file using the timestamp, sample table, and offset index inside the media container, and the data or address of the target splicing segment is obtained. The process of mapping the time range to the byte range of the underlying media file does not involve frame-level decoding. Dynamic object mapping: Dynamically selects and adjusts the list of segments to be spliced ​​based on playback context information, and maps the media objects corresponding to each time period in real time; Generate and output a playlist: Multiple media segments obtained through timeline positioning and object mapping are concatenated in chronological order to generate a continuously playable HLS playlist, and fMP4 segments are dynamically constructed based on the byte range when responding to playback requests.

[0010] Furthermore, the step of mapping the time range to the byte range of the underlying media file using the timestamp, sample table, and offset index within the media container includes: Based on the structure of the media container, the start and end times of the target splicing segment are accurately converted into the byte range of the underlying media file; the structure of the media container includes moov / stts / stsc / stco.

[0011] Furthermore, the dynamic object mapping includes: The source of the segment to be spliced ​​is dynamically adjusted according to the playback context; the playback context includes the user-requested combination of segments, user terminal capabilities, network conditions, and bitrate selection information; when the segment to be spliced ​​is missing or a source switch occurs, the media object corresponding to the time period of the segment to be spliced ​​is remapped statelessly.

[0012] Furthermore, when the target splicing segment comes from video sources with different encoding formats or different resolutions, an EXT-X-DISCONTINUITY tag or an EXT-X-MAP tag is inserted at the connection point of adjacent target splicing segments in the generated HLS playlist.

[0013] Furthermore, the duration of each target spliced ​​segment is determined by fMP4 segments dynamically generated based on the start and end times, rather than relying on pre-segmented fixed-length media segments.

[0014] Furthermore, the generation and output of the playlist employs a delayed calculation and caching mechanism; wherein, delayed calculation refers to completing the calculation on the server side by concatenating byte segments, without generating a complete media file or transmitting the edited frame sequence; the caching mechanism refers to caching the generated HLS playlist to respond to subsequent playback requests for the same combination of segments.

[0015] This invention also proposes an HLS online splicing system, comprising: a timeline address management module, a dynamic object mapping module, a list generation module, and a storage unit; The timeline address management module is used to create a unified timeline for multiple media sources to be spliced, map the video content of each source to the timeline position, and, based on the start and end times of the target splicing segment on the timeline, use the timestamp, sample table and offset index inside the media container to map the time range to the byte range of the underlying media file, and obtain the data or address of the target splicing segment. The mapping process does not involve frame-level decoding. The dynamic object mapping module is used to dynamically select and adjust the list of segments to be spliced ​​according to the playback context information, and to map the media objects corresponding to each time period in real time. The playlist generation module is used to splice multiple media segments obtained by time axis positioning and object mapping in chronological order to generate a continuously playing HLS playlist, and dynamically construct fMP4 segments based on the byte range when responding to playback requests. The storage unit is used to store the media source data and the generated HLS playlist.

[0016] Furthermore, the timeline address management module is specifically used for: Based on the structure of the media container, the start and end times of the target splicing segment are accurately converted into the byte range of the underlying media file; The dynamic object mapping module is specifically used for: The source of the segment to be spliced ​​is dynamically adjusted according to the playback context, and the media object of the time period corresponding to the segment to be spliced ​​is remapped statelessly when the segment to be spliced ​​is missing or the source is switched.

[0017] The present invention also proposes an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the method described above.

[0018] The present invention also proposes a computer-readable storage medium for storing instructions that, when executed, cause the above-described actions to be performed.

[0019] Compared with existing technologies, the advantages of this invention are: 1. High splicing accuracy and stability: Utilizing timeline addressing, it can precisely locate the video keyframe level for editing and splicing, avoiding the imprecise editing problems that may occur when splicing by segment number in traditional methods. Splicing points all fall on clear time boundaries, reducing the risk of audio-visual desynchronization or segment overlap. Simultaneously, by properly handling discontinuities (such as inserting EXT-X-DISCONTINUITY and adjusting PTS when necessary), it ensures smooth transitions between different segments, preventing player restarts or dragging failures that can occur with traditional methods. Therefore, the playback experience is much smoother.

[0020] 2. Flexible technical structure and strong compatibility: This invention places the main logic implementation on the server-side splicing server, adapting to various existing HLS players by outputting a standard HLS playlist file. Compared to solutions that require specially customized players or rely on special protocols (e.g., requiring a custom client to interface with the TAMS API), this method is transparent to the client and compatible with all players that conform to the HLS standard. Even in cases of cross-domain or cross-encoding format segments, the generated playlist follows the HLS protocol standard (including necessary tags and initialization information), ensuring that general players can recognize and play it correctly. Therefore, this solution is easy to integrate into existing streaming media distribution systems.

[0021] 3. High splicing efficiency and strong dynamic adaptability: Leveraging a dynamic object mapping mechanism, the splicing list can be generated instantly upon receiving a user request, eliminating the need for pre-combining video files and achieving true on-demand real-time splicing. This significantly reduces the waiting time for publishing multiple video segments, enabling "instant editing and playback." Furthermore, because it uses a "reference original content" splicing method, no media data copies are generated, reducing storage and bandwidth waste. All segments originate directly from the original storage, avoiding repeated transcoding and improving overall efficiency. Additionally, if the context changes during playback (e.g., the user jumps to another chapter, or the background detects an update to a segment), the system can dynamically adjust the subsequent splicing list, offering far greater flexibility than a static list.

[0022] 4. This invention combines timeline addressing and object mapping for HLS splicing, a solution unprecedented in existing technology. While TAMS provides the concept of time-addressed storage, applying it to online HLS splicing and outputting a standard HLS playlist for compatibility with any player requires a deep understanding of streaming media protocols and innovative design, something that non-professionals cannot easily conceive of. This solution differs from simple playlist splicing and pure backend video editing; it represents a revolutionary application of the HLS playback mechanism and possesses significant creative characteristics.

[0023] 5. This invention requires no media decoding or video track model. It relies solely on the structured index of the media file container to achieve a time range to byte range mapping and constructs HLS fMP4 segments in real time that can be directly decoded by the player. This media processing method belongs to the container-level online construction technology, rather than the frame-level editing / compositing technology, thus fundamentally different from traditional video editing and splicing methods.

[0024] In summary, this invention, through a novel combination of technologies, achieves a precise, efficient, and universal HLS online splicing method that can meet diverse audio and video splicing and playback needs and has broad application prospects. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0026] Figure 1This is a schematic diagram of the system architecture of the method of the present invention, illustrating the overall composition and data interaction process of the HLS stitching system based on time addressable and dynamic object mapping. (Please refer to the description of the specific implementation method below for the relationship between each module and step); Figure 2 This is a flowchart illustrating the processing steps of the method of the present invention, showing the main steps from input to output of the splicing request, including timeline establishment, object mapping, and manifest generation. (Please refer to the description of the specific implementation method below for the relationship between each module and step); Figure 3 Demonstrate how existing technology (similar patents) works; Figure 4 This demonstrates the completely different technical approach employed in this invention; through Figure 3 and Figure 4 The comparison clearly shows that there are fundamental differences between the two in terms of processing flow, data structure, and output format; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0028] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0029] Example 1 This embodiment proposes an online HLS stitching method, the core of which lies in two innovative mechanisms: "Time Addressable" and "Dynamic Object Mapping".

[0030] Specifically, this solution maps multiple video segments to be spliced ​​onto a unified timeline coordinate system: each segment is identified by its timestamp in the original video. When splicing segments from different sources, the system does not directly splice based on a file list or predetermined sequence number, but instead addresses the corresponding video content based on the expected playback time period. For example, to splice segments 10 to 20 seconds of video A and 0 to 5 seconds of video B, this method first defines two consecutive intervals on the timeline: [0s–10s] corresponding to the 10s segment of video A, and [10s–15s] corresponding to the 5s segment of video B. Then, through a time-addressable mechanism, it precisely extracts the 10–20 second segment from video A and the 0–5 second segment from video B, and aligns the latter's timeline offset to the 10-second position where the former ends. The entire process is based on the timeline, rather than relying on static TS file boundaries, thus achieving frame-level splicing control.

[0031] Meanwhile, the introduction of a "dynamic object mapping" mechanism means that the system can determine the list and source of media segments to use in real time based on the actual playback context. For example, when network conditions are poor, the system can dynamically select lower bitrate backup segments to insert into the list; when a segment from a certain source is missing or unavailable, it can be mapped to other backup sources to provide content for the same time period. Dynamic mapping is also reflected in its adaptability to playback requests: for playback needs customized for different terminals or users, the splicing list can be adjusted accordingly during generation, without the need for pre-editing all possible combinations. Through this dynamic mapping, this invention achieves flexible splicing across sources and across video objects: video streams from multiple heterogeneous sources can be seamlessly connected on a unified timeline, and the playback end can play them continuously without noticing. This cross-source splicing is very effective for scenarios such as multi-view switching, ad insertion, and content replacement.

[0032] In this embodiment, specifically, an HLS online stitching method includes the following steps: Establish a timeline address space: Create a unified timeline for the multiple media sources to be spliced, and map the video content of each source to the timeline position; Time-addressable splicing: Based on the start and end times of the target splicing segment on the timeline, the time range is mapped to the byte range of the underlying media file using the timestamp, sample table and offset index inside the media container, and the data or address of the target splicing segment is obtained. The process of mapping the time range to the byte range of the underlying media file does not involve frame-level decoding (i.e., it does not locate the segment through frame-level decoding). Dynamic object mapping: Dynamically selects and adjusts the list of segments to be spliced ​​based on playback context information, and maps the media objects corresponding to each time period in real time; that is, dynamically selects and adjusts the list of segments to be spliced ​​based on playback context information, including supporting the switching of segments between different sources or different video objects, and mapping the media objects corresponding to each time period in real time. Generate and output a playlist: Multiple media segments obtained through timeline positioning and object mapping are concatenated in chronological order to generate a continuously playing HLS playlist. When responding to a playback request, fMP4 segments are dynamically constructed based on the byte range. In other words, multiple media segments obtained through timeline positioning and object mapping are concatenated in chronological order to generate a continuously playing media stream playlist (M3U8 playlist file). When needed, fMP4 moof / mdat segments are dynamically constructed, rather than generating a locally edited new media file or performing image-level rendering. This playlist is then provided to the player for online playback.

[0033] In this embodiment, specifically, mapping the time range to the byte range of the underlying media file using the timestamp, sample table, and offset index within the media container includes: Based on the structure of the media container, the start and end times of the target splicing segment are accurately converted into the byte range of the underlying media file; the structure of the media container includes moov / stts / stsc / stco.

[0034] The "time-addressable" mechanism identifies segments by using timestamps in playlists or background services, enabling media segments to be retrieved by time position rather than file number. When segments with specific start and end times are needed, the system accurately calculates the byte range based on the media container's moov / stts / stsc / stco structure, without the need for frame-by-frame decoding or the creation of virtual video tracks.

[0035] In this embodiment, specifically, the dynamic object mapping includes: The source of the segment to be spliced ​​is dynamically adjusted according to the playback context; the playback context includes the user-requested combination of segments, user terminal capabilities, network conditions, and bitrate selection information; when the segment to be spliced ​​is missing or a source switch occurs, the media object corresponding to the time period of the segment to be spliced ​​is remapped statelessly.

[0036] The "dynamic object mapping" mechanism includes: dynamically adjusting the source of spliced ​​segments based on the current playback context (including but not limited to the combination of segments requested by the user, the user's terminal capabilities, network conditions, multi-track / multi-bitrate selection, etc.); and being able to remap the media objects corresponding to the time period statelessly when segments are missing or source switching occurs, without relying on the editing or compositing capabilities of the client player.

[0037] In this embodiment, specifically, when the target splicing segment comes from video sources with different encoding formats or different resolutions, an EXT-X-DISCONTINUITY tag or an EXT-X-MAP tag is inserted at the connection point of adjacent target splicing segments in the generated HLS playlist.

[0038] It should be noted that, for cases where the spliced ​​segments come from video sources with different encoding formats or different resolutions, this invention only controls the player initialization through HLS protocol tags, without performing image-level mixing, overlaying or rendering, nor creating multiple video tracks or timelines for frame-by-frame synthesis; if necessary, EXT-X-DISCONTINUITY or EXT-X-MAP tags are inserted to ensure compatibility.

[0039] In this embodiment, the duration of each target splicing segment is determined by the fMP4 segments dynamically generated based on the start and end times, rather than relying on pre-segmented fixed-duration media segments.

[0040] It should be noted that the generated spliced ​​playlist supports precise frame boundary time positioning based on the media file structure parsing. The duration of each segment is determined by the dynamically generated fMP4 segments based on the specified start and end times. There is no need to pre-segment fixed-length TS segments, nor does it rely on the player for editing or transition rendering operations.

[0041] In this embodiment, specifically, the generation and output of the playlist employs a delayed calculation and caching mechanism; wherein, delayed calculation refers to completing the calculation on the server side by concatenating byte segments, without generating a complete media file or transmitting the edited frame sequence; the caching mechanism refers to caching the generated HLS playlist to respond to subsequent playback requests for the same combination of segments.

[0042] The generation of the splicing list adopts a delayed calculation and caching mechanism: the calculation process is completed entirely on the server side in the form of byte segment splicing, without generating a complete media file or transmitting the edited frame sequence; the player only receives the standard HLS list.

[0043] Based on the same inventive concept, this embodiment proposes an HLS online splicing system, including: a timeline address management module, a dynamic object mapping module, a list generation module, and a storage unit; The timeline address management module is used to create a unified timeline for multiple media sources to be spliced, mapping the video content of each source to the timeline position; and based on the start and end times of the target splicing segment on the timeline, using the timestamp, sample table, and offset index inside the media container, mapping the time range to the byte range of the underlying media file, and obtaining the data or address of the target splicing segment. This mapping process does not involve frame-level decoding; that is, it is used to parse the index structure inside the media container and establish a time-to-byte mapping, rather than loading video frames or constructing a multi-track timeline. The dynamic object mapping module is used to dynamically select and adjust the list of segments to be spliced ​​according to the playback context information, and to map the media objects corresponding to each time period in real time; that is, it is used to select the corresponding media object from multiple media sources and extract the data corresponding to the byte range according to the playback request. The playlist generation module is used to concatenate multiple media segments obtained by timeline positioning and object mapping in chronological order to generate a continuously playable HLS playlist, and dynamically construct fMP4 segments based on the byte range when responding to a playback request; that is, it is used to generate an HLS playlist in chronological order according to byte segments or corresponding URLs, and can dynamically construct fMP4 segments without performing a decoding / encoding process. The storage unit is used to store the media source data and the generated HLS playlist.

[0044] In this embodiment, specifically, the timeline address management module is used for: Based on the structure of the media container, the start and end times of the target splicing segment are accurately converted into the byte range of the underlying media file; The dynamic object mapping module is specifically used for: The source of the segment to be spliced ​​is dynamically adjusted according to the playback context, and the media object of the time period corresponding to the segment to be spliced ​​is remapped statelessly when the segment to be spliced ​​is missing or the source is switched.

[0045] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the HLS online stitching method provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic equipment. Figure 5 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 5 The example used is the connection between the processor and memory via a bus. The bus... Figure 5The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 5 The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0046] In this embodiment of the invention, the memory stores instructions executable by at least one processor. By executing the instructions stored in the memory, the at least one processor can perform an HLS online stitching method as described above. The processor can implement... Figure 5 The functions of each module in the device shown.

[0047] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0048] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0049] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the HLS online stitching method disclosed in the embodiments of this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0050] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia cards, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), and electrically erasable programmable read-only memory (EPROM). Only memory (EEPROM), magnetic storage, magnetic disks, optical disks, etc. A memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in embodiments of this invention can also be a circuit or any other device capable of performing storage functions for storing program instructions and / or data.

[0051] By designing and programming the processor, the code corresponding to the HLS in-circuit stitching method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0052] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform an HLS online splicing method described above.

[0053] In some alternative embodiments, the present invention also provides that various aspects of the HLS online stitching method can also be implemented in the form of a program product, which includes program code that, when the program product is run on a device, causes the control device to perform the steps in an HLS online stitching method according to various exemplary embodiments of the present invention as described above.

[0054] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0055] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can be implemented in one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs) containing computer-usable program code. The form of a computer program product implemented on ROM, optical memory, etc.

[0056] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0058] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] In addition, in some embodiments, a computer program product is also proposed, which, when executed by a processor, implements the above-described HLS online stitching method.

[0062] Example 2 Example 2 is a specific application of the HLS online splicing method proposed in Example 1. In this example, it is assumed that there are two video source contents: Video A and Video B. This example demonstrates how to use the method of the present invention to seamlessly splice a 10-second segment from Video A starting at 10 seconds with a 5-second segment from the beginning of Video B, and then play the combined content through an HLS playback device.

[0063] System Architecture: like Figure 1As shown, the splicing system of this invention includes a splicing server, several media source storage units, and an HLS player connected to the terminal. The splicing server contains three core functional modules: a timeline-addressable splicing module, a dynamic object mapping module, and a playlist generation module. The timeline splicing module is responsible for maintaining the global timeline and determining the position of time segments according to requests; the object mapping module is responsible for selecting the correct video objects and segments according to playback requests; and the playlist generation module combines the segments to generate an M3U8 playlist conforming to the HLS standard. The media source storage can be a regular HLS content server or a media storage system conforming to a time-addressable interface (such as a storage service implementing the TAMS API) to provide raw video segment data. The HLS player sends a splicing playback request (containing information about several time segment segments to be played) to the splicing server via the network. Based on this, the splicing server obtains the required segment data or URLs from the corresponding media sources and dynamically generates an M3U8 playlist integrating all specified segments, returning it to the player. The player then requests the actual media data from each media source segment by segment according to this playlist to achieve continuous playback. The entire architecture design ensures that data transmission between the player and the splicing server / source storage is based on the standard HTTP protocol and HLS rules, without the need for special customization.

[0064] Process steps: like Figure 2 As shown, the specific splicing process includes the following steps: 1. Stitching Request Input: The client (or upper-layer application) sends a stitching request to the stitching server, specifying that the content to be played is composed of video segments from multiple sources. The request includes the source identifier of each segment, as well as the start and end times within that source video. For example, in this embodiment, the request includes: "segment [10s–20s] of video A" plus "segment [0s–5s] of video B". This request can be transmitted via the API interface, and the format can use timecode concatenation, such as Source=A,start=10,end=20; Source=B,start=0,end=5.

[0065] 2. Timeline Planning: After parsing the request, the timeline stitching module of the stitching server internally creates a unified playback timeline. In this example, assuming the final stitched stream starts playing from 0 seconds, the module first maps the 10-second segment of video A to the [0s–10s] interval of the output timeline, and then maps the 5-second segment of video B to the [10s–15s] interval of the output timeline. This determines the timing of the entire output stream. Each interval on this timeline is associated with the corresponding source video segment information (source ID, start and end times, etc.).

[0066] 3. Object Mapping and Segment Acquisition: After the timeline is determined, the dynamic object mapping module performs specific segment acquisition operations for each interval. For the [0–10s] interval, the module identifies its source as a 10-second segment of video A, and then retrieves the corresponding 10-second segment data from storage, starting at 10 seconds. If video A has been pre-sliced ​​into short segments, the TS segment list containing this time range can be directly located; if more precise positioning is required (e.g., the starting 10s is not the boundary of a TS segment), the module can request the backend storage to extract the media stream by time range (e.g., connecting to the TAMS API to request A: [10s_20s] to obtain the precise segment). Similarly, for the [10–15s] interval, the module retrieves the content of video B from 0s to 5s. During the retrieval process, the module can make dynamic adjustments based on the current context: for example, when network bandwidth is low, it can request a lower resolution segment; or if the original storage does not support precise time extraction, it can instead obtain the entire file covering this range and trim it on the server side. Dynamic object mapping ensures that each time period has an appropriate segment corresponding to it, regardless of the source, and can switch sources in real time to ensure continuity even when a source segment is missing.

[0067] 4. Segment Stitching and Playlist Generation: Once all required segments have been successfully acquired (or accessible URLs have been determined), the stitching server enters the playlist generation phase. The playlist generation module lists the URLs of each segment in a new M3U8 playlist file according to the playback order, based on the timeline sequence. To ensure seamless transitions between different segments, the module inserts control tags at necessary locations: for example, when it detects that the encoding parameters of video segment A are out of sync with those of video segment B, it inserts a #EXT-X-DISCONTINUITY tag at the junction to notify the player to reset the decoder state; if the encoding initialization information of cross-source segments is different (such as different decryption keys or encoding / decoding configurations), it inserts appropriate #EXT-X-KEY or #EXT-X-MAP tags to reinitialize. Since in this embodiment, it is assumed that video A and video B have the same encoding format, the playlist simply indicates the discontinuity at the junction. Subsequently, the module adds necessary header information (including target duration, sequence number, etc.) to the playlist, generating the final M3U8 text.

[0068] 5. Response Output: The generated spliced ​​playlist is returned to the client's HLS player via an HTTP response. Upon receiving the playlist, the player requests the actual media segments in the order listed. Because each segment URL in the playlist may point to a different source domain, the server needs to ensure that these sources support Cross-Origin Access (CORS) or adopt a strategy of proxying the segment back. This invention's method supports both directly having the player fetch segments from different sources (the playlist provides the complete URLs of each source) and having a splicing server act as an intermediary, aggregating segments from different sources and providing them uniformly. Regardless of the method, the media stream ultimately received by the player will sequentially contain selected segments from video A and selected segments from video B, presenting a continuous video playback. Users can smoothly transition from video A to video B without noticeable pauses or stuttering.

[0069] 6. Playback Control and Jumping: During playback, if the user performs a jump (seek) or other interactive operation, the method of this invention also applies. When jumping, the player may request a new playlist or segment corresponding to the target time, and the splicing server can recalculate the timeline mapping accordingly. For example, if the user jumps to the 12th second of the output stream, since this time belongs to... Figure 1 The output timeline shown, within the [10s–15s] interval, corresponds to the 2nd second of video B. The server will guide the player to directly request the corresponding segment from video B, thus enabling a quick jump without reloading from the beginning. Furthermore, if the business requires dynamically inserting new segments during playback (such as real-time insertion of content), the server can update the playlist, reflecting the new splicing arrangement in subsequent requests. This dynamic update capability is thanks to the flexible design of the timeline and object mapping, making playback control more flexible.

[0070] Detail optimization: To further improve the splicing effect, the following optimization measures can be adopted in actual implementation: Keyframe alignment: To avoid discontinuous decoding at splicing points, the system should try to segment at keyframe boundaries. During timeline addressing, the start time can be fine-tuned to the nearest I-frame timestamp. For example, if the user requests to start at 10.2 seconds, the module will find the next I-frame of video A close to 10.2 seconds (let's say 10.3 seconds) as the actual segment start point to ensure proper decoding. This adjustment is almost imperceptible to the user, but it prevents ghosting of the previous segment from appearing in the first frame after splicing.

[0071] Timestamp continuity or remapping: For scenarios requiring strict seamlessness, the PTS / DTS timestamps of subsequent segments can be remapped on the server side to connect with the last frame timestamp of the previous segment, thus achieving time continuity at the transport layer. However, this operation is relatively complex and requires re-encapsulation of the TS, and is usually only used in professional fields. Generally, HLS players automatically handle PTS reset issues upon encountering the #EXT-X-DISCONTINUITY tag to ensure normal playback of subsequent segments. This invention supports two modes: a simple mode relies on the player to handle discontinuities, while an enhanced mode has the server perform timestamp smoothing to meet higher requirements.

[0072] Caching and Prefetching: Since the playlist may be dynamically generated for each request, the system can cache the results of identical combinations. For example, if a popular combination is requested multiple times, the playlist is cached after the first generation, and subsequent requests directly return the cached data, reducing server load. Simultaneously, if it is known that a user may play a series of segments consecutively, the server can prefetch the next segment data or prepare the next playlist in advance, reducing waiting time and achieving a more immediate playback experience.

[0073] Security and Access Control: Dynamic splicing involves cross-source content, and the system can perform unified authentication and permission verification on the server side. For example, only authorized users can splice a specific video clip B into video A; the server needs to attach a valid token to access the protected source when retrieving the clip. URLs can also be signed or encrypted during manifest generation to prevent the leakage of unauthorized content.

[0074] Through the above steps and optimizations, this embodiment successfully and seamlessly splices two video clips from different sources on the timeline, outputting a standard HLS playlist for terminal playback verification. Test results show that the spliced ​​playback stream switches smoothly without noticeable stuttering; the order and duration of each clip perfectly match expectations; and it can be decoded and played normally on players that support HLS (including Web, mobile, and set-top box platforms), proving that the method of this invention has good compatibility and robustness.

[0075] In this embodiment, the splicing server does not perform decoding or rendering of the video content, does not generate new media frames, and does not create multiple video tracks or read synchronous frames by frame timestamp for compositing like a traditional video editor. The system only calculates the byte offset of the target segment based on the sample control structure in the container-level moov box, extracts the corresponding media data using Range-GET or equivalent byte reading methods, and assembles it into fMP4 segmented output.

[0076] It should also be emphasized that the method of this invention is not limited to splicing two segments, but also supports the concatenation of any number of segments, as well as more complex scenarios (such as alternating splicing of multiple segments, parallel multi-screen, etc., which only require extending the timeline model). After reading this specification, those skilled in the art can make various equivalent modifications and combinations based on the principles of this invention. For example, this method can be applied to splicing other streaming media protocols such as DASH, or the timeline addressing mechanism can be combined with an AI analysis and recommendation system to achieve intelligent content splicing. These modifications, as long as they do not depart from the spirit of this invention, should fall within the scope of protection claimed by this invention.

[0077] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0078] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for HLS online stitching, the method comprising: receiving a plurality of HLS segments; and stitching the plurality of HLS segments into a single HLS segment. The method comprises the following steps: Establishing a timeline address space: creating a unified timeline for multiple media sources to be spliced, and mapping video content of each source to a position on the timeline; Time-addressable splicing: according to the start and end times of a target splicing segment on the timeline, mapping a time range to a byte range of an underlying media file by using timestamps, sample tables and offset indexes inside a media container, and obtaining data or an address of the target splicing segment, wherein the process of mapping the time range to the byte range of the underlying media file does not perform frame-level decoding; Dynamic object mapping: dynamically selecting and adjusting a list of splicing segments according to playback context information, and mapping media objects corresponding to each time period in real time; Generating a playback list and outputting: splicing multiple media segments obtained through timeline positioning and object mapping in chronological order to generate an HLS playback list for continuous playback, and dynamically constructing fMP4 segments based on the byte range when responding to a playback request.

2. The HLS online stitching method of claim 1, wherein, The process of mapping a time range to a byte range of an underlying media file by using timestamps, sample tables and offset indexes inside a media container comprises: Based on the structure of the media container, accurately converting the start and end times of the target splicing segment to a byte interval of the underlying media file; the structure of the media container includes moov / stts / stsc / stco.

3. The HLS online stitching method of claim 1, wherein, The dynamic object mapping comprises: Dynamically adjusting the source of the splicing segment according to the playback context; the playback context includes user-requested segment combinations, user terminal capabilities, network conditions and code rate selection information; when the splicing segment is missing or source switching occurs, the media objects of the time period corresponding to the splicing segment are remapped without state.

4. The HLS online stitching method of claim 1, wherein, When the target splicing segment comes from video sources of different encoding formats or different resolutions, an EXT-X-DISCONTINUITY tag or an EXT-X-MAP tag is inserted at the connection between adjacent target splicing segments in the generated HLS playback list.

5. The HLS online stitching method of claim 1, wherein, The duration of each target splicing segment is determined by the fMP4 segment dynamically generated according to the start and end times, rather than relying on pre-cut fixed-duration media segments.

6. The HLS online stitching method of claim 1, wherein, The generation of the playback list and outputting adopts a delay calculation and caching mechanism; wherein delay calculation means that the calculation is completed in byte segment splicing manner on the server, without generating complete media files or transmitting edited frame sequences; the caching mechanism means that the generated HLS playback list is cached to respond to subsequent playback requests for the same segment combination.

7. An HLS online splicing system, characterized in that, The method comprises the following steps: A timeline address management module, a dynamic object mapping module, a list generation module and a storage unit; The timeline address management module is configured to create a unified timeline for a plurality of media sources to be spliced, map video content of each source to a position of the timeline, and map a time range to a byte range of a bottom-layer media file by using a timestamp, a sample table and an offset index in a media container, to obtain data or an address of the target spliced segment according to a start time and an end time of the target spliced segment on the timeline, wherein the mapping process does not perform frame-level decoding. The dynamic object mapping module is configured to dynamically select and adjust a list of segments to be spliced according to playback context information, and map media objects corresponding to each time period in real time. The manifest generation module is configured to splice a plurality of media segments obtained by timeline positioning and object mapping in a time sequence to generate an HLS playback manifest for continuous playback, and dynamically construct fMP4 segments based on the byte range in response to a playback request. The storage unit is configured to store the media source data and the generated HLS playback manifest.

8. The HLS online splicing system according to claim 7, characterized in that, The timeline address management module is specifically configured to: accurately convert the start time and the end time of the target spliced segment to a byte interval of the bottom-layer media file based on a structure of the media container. The dynamic object mapping module is specifically configured to: dynamically adjust a source of the segment to be spliced according to playback context, and remap media objects of a time period corresponding to the segment to be spliced in a stateless manner when the segment to be spliced is missing or source switching occurs.

9. An electronic device, comprising: The apparatus includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, performs the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store instructions that, when executed, cause the method of any one of claims 1-6 to be implemented.

Images