Mixed duration detection method for HLS

Abstract

The invention relates to the field of streaming media service and video processing, and discloses a mixed duration detection method for an HLS (HTTP Live Streaming), which comprises the following steps of: capturing a main index file and a slice file based on a video stream, and constructing a slice file index database without initial duration; extracting a first duration from the main index file and updating a slice file index database; for the slice files of which the duration is not obtained, comparing the serial number sequence of the slice files in the video stream, and if the sum of the starting time of the current slice and the preset slice duration is equal to the starting time of the slice file corresponding to the next serial number, determining that the duration of the current slice file, namely the second duration, is the preset slice duration; calling the streaming media processing tool to obtain a third duration for the slices of which the duration is still undetermined; according to the process, the constructed mixed duration detection algorithm gives consideration to both efficiency and accuracy, a complete index can still be rapidly constructed in a massive slice file scene, and the accuracy of slice file retrieval and playback is ensured.

Description

Technical Field

[0001] This invention relates to the field of streaming media services and video processing, and in particular to a method for detecting mixed durations in HLS. Background Technology

[0002] HLS, an HTTP-based streaming media protocol, is widely used in live video streaming and video-on-demand scenarios due to its advantages such as adaptive bitrate, broad compatibility, and robust reliability, providing users with a smooth viewing experience. Accurately obtaining the duration of video clips is crucial in HLS applications, especially in scenarios integrating video-on-demand and surveillance. It's necessary to associate recordings within a specific service time period of a video stream with specific business information, facilitating user-defined playback. However, existing technologies still have the following problems in detecting the duration of video clips: 1. The video stream slice file index lacks accurate duration information. Typically, real-time video streams are segmented into slice files based on a fixed set duration (e.g., 30 seconds), and HLS index files are output synchronously during video stream transmission. However, the actual duration of the generated slices is not strictly equal to the set duration; some slices may exceed the set value (e.g., 31 seconds, 32 seconds, etc.), while the last slice is often shorter than the set duration.

[0003] Currently, most methods for detecting video segment duration include: relying solely on filenames or video paths, or relying solely on calling tools like FFprobe to obtain the duration of each segment file. These methods lack multi-level, end-to-end detection mechanisms, resulting in incomplete duration detection mechanisms and simplistic implementations. Such a single method of detecting the duration of a segment file may fail to obtain the actual duration of the segment, thus making it impossible to accurately locate the corresponding segment file within the start-to-end time interval specified by the business logic.

[0004] 2. The conflict between index rebuilding performance and storage consumption If the segment duration is obtained solely by filename or video path, and the main index file is not updated in time due to network latency or server failure (e.g., newly added segments are not written to the index), the detected duration will lag behind the actual video stream, compromising the index's accuracy. If the duration is obtained by calling tools like FFprobe for each segment file and then written to the segment file index database, significant I / O and CPU overhead will occur. When a large number of segment files accumulate from a single stream, analyzing them one by one with FFprobe will generate massive decoding and disk read / write operations in a short period, impacting real-time service.

[0005] In summary, the main bottlenecks of existing technologies for detecting the duration of slice files based on HLS indexes are: the inability to obtain the precise duration of slice files, the imperfect duration detection mechanism, the single implementation method, and the difficulty in accurately locating slices within a specified time period; and the contradiction between index reconstruction performance and storage consumption. These shortcomings severely restrict the need to quickly build a complete index in scenarios with massive slice files, and cannot guarantee the accuracy of business retrieval and playback. Summary of the Invention

[0006] To address the aforementioned technical issues, this invention provides a hybrid duration detection method for HLS. Through multi-method collaboration, it achieves coverage of defects inherent in single-method approaches, ultimately adapting to the need for rapid construction of a complete index in scenarios with massive amounts of sliced ​​files, ensuring the accuracy of business retrieval and playback. Specific steps include: S1. Capture at least one HLS master index file and multiple slice files based on the video stream; S2. Construct an initial, time-limit-free slice file index database based on the aforementioned multiple slice files; S3. Based on the at least one HLS master index file, obtain the duration of the recorded slice file as the first duration, and update the obtained first duration to the slice file index database; S4. For a slice file whose duration has not been obtained, compare the sequence number order of the slice files in the video stream. If the start time of the current slice plus the preset slice duration equals the start time of the slice file corresponding to the next sequence number, then the duration of the current slice file is the preset slice duration and is used as the second duration. S5. For slice files whose duration has not yet been obtained, use streaming media processing tools to obtain the duration of the slice file as the third duration.

[0007] Preferably, in step S6, the changes in the HLS master index file in the video stream directory are monitored in real time, the newly added slice files are parsed, and steps S3-S5 are re-executed to achieve incremental updates.

[0008] Preferably, in step S7, the slice file index database is periodically scanned, and steps S3-S5 are performed on slice files that are not recorded in the slice file index database to complete the supplementary recording.

[0009] Preferably, the method for obtaining the HLS master index file includes: listening to the network hook callback events of the streaming media server through a sidecar proxy to perceive the file name, start time, and file path of the video stream.

[0010] Preferably, the processing method of the HLS master index file includes: renaming the master index file with the start time of the video as a marker.

[0011] Preferably, the method of obtaining the third duration using the streaming media processing tool includes: submitting the path of the segment file whose duration has not yet been obtained to the verification pool of the streaming media processing tool, executing the streaming media processing tool through an asynchronous thread pool to obtain the third duration, and writing the third duration into the segment file index database.

[0012] Preferably, after writing the second duration into the slice file index database, the second duration is then updated in the local cache.

[0013] Preferably, before using the streaming media processing tool, it is first checked whether there is already a duration record corresponding to the slice file path in the local cache; if so, the corresponding third duration is directly updated to the slice file index database.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention proposes a hybrid duration detection method for HLS, forming a hierarchical hybrid detection method of "path resolution → adjacent file verification → streaming media processing tools". This method effectively solves the problems of insufficient accuracy and low efficiency in the existing technology of duration detection. In terms of accuracy, multi-level detection eliminates the defects of single detection and significantly reduces the detection error rate. In terms of efficiency, the hierarchical strategy of lightweight detection as the main method and heavy tool verification as a supplement significantly reduces resource consumption and meets the real-time detection requirements of continuous video streaming. It can realize duration detection and rapid index construction of massive slice files. Experimental verification shows that when processing 100,000 slice files, the overall time is reduced from 12 hours of traditional methods to about 1 hour, and the duration extraction efficiency is improved by more than 95%.

[0015] 2. In this invention, by real-time monitoring of changes in the main index file and triggering the detection process for newly added slices, an instant response to dynamic HLS streams is achieved, reducing the detection latency from seconds to milliseconds. By periodically scanning the slice file index database and supplementing unrecorded slices, historical data vulnerabilities caused by network interruptions, server failures, and other anomalies are repaired, avoiding frequent writes to the slice file index database caused by scattered detection and improving the system throughput. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a flowchart of a mixed duration detection method provided in a preferred embodiment of the present invention; Figure 2 This is a flowchart of the slice file index construction process with added real-time monitoring provided in a preferred embodiment of the present invention; Figure 3 This is a flowchart illustrating the construction process of a slice file index with an introduction of a supplementary recording mechanism, provided in a preferred embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 In existing HLS streaming media transmissions, players often rely on the #EXTINF tag in the master index file (.m3u8) to determine the duration of each segment. However, on some platforms or in environments with weak signals, the master index may be incomplete, potentially missing some duration information, which limits the player's skipping, preloading, and playback positioning functions. Furthermore, relying entirely on the server's master index file cannot dynamically adapt to actual changes in the segment files.

[0020] In addition, some technical solutions rely solely on tools such as FFmpeg (e.g., the ffprobe command) to extract duration information from each slice file. Although this method has high accuracy, it has drawbacks such as low processing efficiency, high computational overhead, and is not suitable for large-scale real-time streaming. Especially in environments with a large number of slices or poor network conditions, frequent calls to external parsing tools may lead to system response delays and affect overall performance.

[0021] like Figure 1 As shown, this embodiment provides a method for mixed duration detection in HLS, including the following steps: S1. Capture at least one HLS master index file and multiple slice files based on the video stream.

[0022] In some preferred embodiments, the HLS master index file, i.e., m3u8, lists the names and partial duration information of the video slice files; the slice files, i.e., ts slice files, are used to store short-duration video segments, and each file is called a slice.

[0023] Obtain an m3u8 index file and multiple corresponding ts slice files from the video stream or server. The m3u8 index file contains the slice playback order and the #EXTINF tag information for some slices, while the ts slice files contain the actual media content.

[0024] S2. Construct an initial, time-limit-free slice file index database based on the aforementioned multiple slice files.

[0025] In some preferred embodiments, the slice file index database can be in the form of a database table, or it can be a reasonable design by those skilled in the art based on the actual situation or site requirements. The slice file index database explicitly records information such as the file name, start time, duration, and data source of each slice.

[0026] In some preferred embodiments, the slice file index database can be constructed sequentially by extracting the initial time and sequence number based on the file path; alternatively, it can be a reasonable design by those skilled in the art based on actual conditions or site requirements. The initially constructed slice index database includes the following fields: file name; Start time field; Serial number; File path.

[0027] S3. Based on the at least one HLS master index file, obtain the duration of the recorded slice file as the first duration, and update the slice file index database with the obtained first duration.

[0028] In some preferred embodiments, the first duration is the duration of the TS slice file directly read from the m3u8 file. The duration information contained in each "#EXTINF:" tag is parsed from the m3u8 index file, and this duration is used as the first duration and filled into the duration field of the corresponding slice. The m3u8 index file is read line by line; if "#EXTINF:x.xxxx" is detected, the next line containing the TS slice file name is read and a corresponding relationship is established.

[0029] S4. For slice files whose duration has not been obtained, compare the sequence number order of the slice files in the video stream. If the start time of the current slice plus the preset slice duration equals the start time of the slice file corresponding to the next sequence number, then the duration of the current slice file is the preset slice duration, which is used as the second duration.

[0030] In some preferred embodiments, the preset slice duration can be 10 seconds, or it can be a reasonable design by those skilled in the art based on actual conditions or site requirements. For slice files lacking #EXTINF tag information, if the slice's start time information is available (such as player recordings or timestamps), the duration of the current slice can be inferred by calculating the start time difference between adjacent slices. If the segment file is segment_ i The start time is T i segment_{ i The starting time of +1} is T i+1 The duration of the current slice is inferred to be T. i+1 -T i ; If the inferred duration of the current slice file is equal to the preset slice duration of 10 seconds, then the inferred result is updated in the slice file index database; if the inferred value is abnormal (such as being negative or greater than the threshold of 9 seconds or 11 seconds), then it is skipped.

[0031] S5. For slice files whose duration has not yet been obtained, obtain the duration of the slice file through the streaming media processing tool as the third duration; In some preferred embodiments, the third duration is the duration of the TS segment file obtained by the streaming media processing tool; for TS segment files skipped in step S4, the streaming media processing tool FFprobe is called to parse them and extract the third duration of the TS segment. An asynchronous thread pool is used to execute commands. This mechanism, by reusing threads and executing non-blockingly, can fully utilize system resources while ensuring the smooth operation of the main thread, making it particularly suitable for batch processing of file path verification needs.

[0032] In some preferred embodiments, the asynchronous thread pool executes commands as follows: ffprobe -v error -show_entries format=duration\ -of default=noprint_wrappers=1:nokey=1\ <ts_path> The meanings of its main parameters are explained below: The "-v error" field indicates that the log level is set to error, and only error messages will be output. The “show_entries format=duration” field indicates the metadata entries to be displayed, mainly the duration field in the format (i.e., file format information). This field is selected because it is used to detect the duration of sliced ​​files. In the field "-of default=noprint_wrappers=1:nokey=1", "-of default" indicates that the default output format is used, which is essentially a simplified INI-like structure; "noprint_wrappers=1" means that the header and footer wrapping information (such as "FORMAT" and " / FORMAT") are not printed; "nokey=1" means that the field name "duration=" is not displayed, and only the duration value itself is output. This can reduce redundant information, reduce parsing costs, and make pure numerical output easier to import into the slice file index database when processing massive slice files.

[0033] “<ts_path> "Field: Indicates the input file path, usually a .ts format video file, specifying the target file to be analyzed; The above command was executed successfully, and the output is a simple numerical value.

[0034] Example 2 like Figure 2 As shown, this embodiment adds real-time monitoring capabilities to the first embodiment, including the following: S6: Monitor changes in the HLS master index file in the video stream directory in real time, parse the newly added slice files, and re-execute steps S3-S5 to achieve incremental updates.

[0035] In some preferred embodiments, inotify can be used to continuously monitor the directory where the HLS master index file is located. Alternatively, those skilled in the art can design a suitable method based on the actual situation or site requirements to capture changes in the HLS master index file m3u8 in real time. Once a change in the m3u8 file is detected (such as adding paragraphs or refreshing content), steps S3-S5 are immediately re-executed to perform incremental parsing on the newly added slice file. For example, if three new slice files are added to the main index file, the following operations will be performed: Identify new slices; Add to the slice file index database; Execute the mixed duration detection method; This embodiment monitors the HLS main index file directory to detect newly added slice content in the video stream in a timely manner, thereby enabling automatic incremental updates of slice duration.

[0036] Example 3 like Figure 3 As shown, based on Embodiment 1 or 2, this embodiment introduces a supplementary recording mechanism for the slice file index database, and the implementation steps are as follows: S7: Periodically scan the slice file index database, and perform steps S3-S5 for slice files that are not recorded in the slice file index database to complete the recording.

[0037] The slice file index database is periodically scanned, and the recorded slice files are skipped directly. Steps S2-S5 are only performed on newly added or unrecorded slices. This embodiment significantly reduces the computational resources and processing time required for repeatedly analyzing slice files.

[0038] Example 4 Based on any one of Examples 1 to 3, this example is used to obtain the HLS master index file.

[0039] The HLS master index file is typically stored in the public path of the streaming media server. Common methods for obtaining the HLS master index file include polling or active querying. In these traditional methods, the client or processor often needs to periodically retrieve or brute-force scan the server directory to obtain newly generated index files. This results in a failure to respond in real-time to updates to the master index file, potentially leading to missed segment processing or delayed duration information. These methods suffer from low real-time performance, low efficiency in obtaining the HLS master index file, and high resource consumption. To address these issues, in some preferred embodiments, this invention uses a sidecar proxy to monitor network hook callback events of the streaming media server to perceive the video stream's filename, start time, and file path, thereby obtaining the HLS master index file.

[0040] In this embodiment, the HLS master index file can be obtained immediately when the network hook callback event occurs, which is more real-time and consumes relatively less resources.

[0041] Example 5 Based on any one of Examples 1 to 3, this example is used to process the HLS master index file to facilitate subsequent tracing and slice file index verification.

[0042] In existing technologies, HLS master index files typically use fixed filenames (such as index.m3u8) and are continuously overwritten and updated, lacking explicit time identifiers. Without processing this master index file, the system will struggle to determine the start time of the video stream corresponding to the master index file, thus affecting the accurate matching of slice durations with the playback timeline. Furthermore, if multiple master index files need to be stored for comparison, backtracking, or slice file index database management, fixed filenames can cause naming conflicts and index confusion; they may also lead to abnormal slice file sorting and incorrect durations, affecting the accuracy of slice duration detection and the reliability of the system.

[0043] In some preferred embodiments, the HLS master index file can be processed by renaming it with the start time of the video as a marker, or by a reasonable design by those skilled in the art based on the actual situation or on-site needs, to facilitate subsequent tracing and verification of the slice file index.

[0044] Example 6 Based on any one of Embodiments 1 to 3, this embodiment utilizes a streaming media processing tool to obtain the third duration.

[0045] In existing technologies, some techniques use MP4Box or MediaInfo tools to obtain the duration. MP4Box primarily targets files with .mp4 structures and has poor support for HLS segments of type .ts. During parsing, it often fails to obtain a valid duration due to a lack of complete encapsulation header information or keyframe indexes. While MediaInfo can display various audio and video metadata, it lacks the flexibility and robustness for programmatic systems, and has a high probability of parsing failure when dealing with segment files with encoding anomalies or missing frames. To address these issues, in some preferred embodiments, obtaining the third duration using the streaming media processing tool can be achieved by using the streaming media processing tool FFprobe to parse the segment files. First, the path of the segment file whose duration has not yet been obtained is submitted to the verification pool of the streaming media processing tool. Then, the streaming media processing tool is executed through an asynchronous thread pool to obtain the third duration. Finally, the third duration is written to the segment file index database.

[0046] This embodiment, by reusing threads and executing non-blockingly, can make full use of system resources while ensuring the smooth operation of the main thread, making it particularly suitable for batch processing of file path verification needs.

[0047] Example 7 Based on any one of Embodiments 1 to 3, this embodiment also performs a local caching operation on the third duration.

[0048] After the third duration is written to the slice file index database, if the third duration is not synchronously updated to the local cache, the following problem may occur: the system still needs to frequently read duration information from the slice file index database in subsequent processing, causing data access delays or wasted read and write resources. To solve the above problems, in some preferred embodiments, after the third duration is written to the slice file index database, the third duration can be updated to the local cache.

[0049] This embodiment can effectively reduce the frequency of reading and writing to the slice file index database, improve the overall efficiency of video stream processing, and avoid repeated processing of slice files for extracting the third duration.

[0050] Example 8 Based on any one of Embodiments 1 to 3, this embodiment introduces a local cache hit priority mechanism.

[0051] Typically, the duration of a segment file is obtained directly using streaming media processing tools for duration parsing. This approach does not check whether the duration record for the segment file already exists in the local cache. This could lead to situations where some segment files have already obtained their third duration in historical processing and are cached locally, but due to the lack of cache checks, they are still repeatedly submitted to the streaming media processing tool, resulting in computational redundancy and wasted computing resources. In some preferred embodiments, before the streaming media processing tool is used, the local cache is checked to see if a duration record corresponding to the segment file path already exists. If so, the corresponding third duration is directly updated to the segment file index database.

[0052] This embodiment saves resources by checking the local cache before calling the streaming media processing tool. For existing segment file duration records, the parsing step of the streaming media processing tool is skipped.

[0053] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A hybrid duration detection method for HLS, characterized in that, The method comprises the following steps: S1. Capturing at least one HLS master index file and a plurality of slice files based on a video stream; S2. Building an initial slice file index database without time length based on the plurality of slice files; S3. Obtaining a recorded time length of the slice file as a first time length based on the at least one HLS master index file, and updating the first time length obtained to the slice file index database; S4. For the slice file without the obtained time length, comparing the sequence number order of the slice file in the video stream, if the starting time of the current slice plus a preset slice time length is equal to the starting time of the slice file corresponding to the next sequence number, the time length of the current slice file is the preset slice time length as a second time length; S5. For the slice file still without the obtained time length, obtaining a time length of the slice file as a third time length through a streaming media processing tool.

2. The hybrid time length detection method for HLS according to claim 1, wherein S6. Real-time monitoring the change of the HLS master index file under the video stream directory, parsing the newly added slice file, and re-executing steps S3-S5 to realize incremental updating.

3. The hybrid time length detection method for HLS according to claim 1 or 2, wherein S7. Periodically scanning the slice file index database, and executing steps S3-S5 for the slice file not recorded in the slice file index database to realize supplementary recording.

4. The mixed duration detection method for HLS according to claim 1, wherein, The obtaining method of the HLS master index file comprises: sensing the file name, starting time and file path of the video stream by monitoring the network hook callback event of the streaming media server through the sidecar proxy mode.

5. The hybrid duration detection method for HLS according to claim 1, wherein, The processing method of the HLS master index file comprises: renaming the master index file with the starting time of the video as a mark.

6. The hybrid duration detection method for HLS according to claim 1, wherein, The third time length obtained through the streaming media processing tool comprises: submitting the path of the slice file still without the obtained time length to the verification pool of the streaming media processing tool, executing the streaming media processing tool through an asynchronous thread pool to obtain the third time length, and writing the third time length into the slice file index database.

7. The hybrid duration detection method for HLS according to claim 6, wherein, After writing the third time length into the slice file index database, the third time length is updated to the local cache.

8. The hybrid duration detection method for HLS according to claim 7, wherein, Before using the streaming media processing tool, it is checked whether there is a time length record corresponding to the path of the slice file in the local cache; if there is, the corresponding third time length is directly updated to the slice file index database.

Images