Method, apparatus, medium and program product for processing streaming media data

Abstract

The application provides a method, device, electronic equipment, computer readable medium and computer program product for processing streaming media data. The method comprises: obtaining monitoring processing information of streaming media data to be processed, the monitoring processing information comprising one or more screenshot monitoring modes and corresponding parameter information thereof, the screenshot monitoring mode referring to a series of preset screenshot configuration schemes according to different service requirements; starting an independent process for each screenshot monitoring mode, performing screenshot processing according to the corresponding screenshot monitoring mode in the process, and generating a screenshot file; based on the generated screenshot file, constructing one or more types of screenshot uploading tasks, and distributing each screenshot uploading task to a multi-level thread pool for parallel processing, the multi-level thread pool comprising a plurality of independently running thread pools, each thread pool being used to execute a certain type of screenshot uploading task. The application constructs an asynchronous parallel screenshot processing architecture based on a multi-level thread pool, greatly improves the processing efficiency and reduces the end-to-end processing delay, is different from the serial processing mode of the prior art, and effectively reduces the processing time.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to methods, apparatus, electronic devices, computer-readable media, and computer program products for processing streaming media data. Background Technology

[0002] Live stream screenshot technology plays a crucial role in various application scenarios such as content recognition, cover generation, and video preview. Currently, live stream screenshot technology is mainly based on two schemes: one is to use streaming media processing tools (such as FFmpeg) to directly read RTMP or HLS streams and extract image frames at set time intervals to save them as image files. This method requires starting a separate process for each live stream and configuring the relevant screenshot parameters through command-line parameters; the other is a synchronous processing scheme, which adopts a serial processing mode in the screenshot generation, file upload, and status reporting stages, and only starts processing the next screenshot after completing the entire processing flow of one screenshot.

[0003] These solutions have been widely used in streaming media processing systems, especially suitable for small to medium-sized live screenshot scenarios. However, with the continuous growth and diversification of business needs, existing technologies have certain limitations in terms of scalability, processing latency, and fault recovery capabilities. Summary of the Invention

[0004] This application aims to provide a method, apparatus, electronic device, computer-readable medium, and computer program product for processing streaming media data.

[0005] One aspect of this application provides a method for processing streaming media data, wherein the method includes: Acquire monitoring and processing information of streaming media data to be processed. The monitoring and processing information includes one or more screenshot monitoring modes and their corresponding parameter information. The screenshot monitoring mode refers to a series of screenshot configuration schemes preset according to different business needs. A separate process is started for each screenshot monitoring mode, and screenshot processing and screenshot files are generated in the corresponding screenshot monitoring mode. Based on the generated screenshot files, one or more types of screenshot upload tasks are constructed, and each screenshot upload task is assigned to a multi-level thread pool for parallel processing. The multi-level thread pool includes multiple independently running thread pools, each thread pool being used to execute a certain type of screenshot upload task.

[0006] In one aspect, this application provides an apparatus for processing streaming media data, wherein the apparatus includes: The mode information acquisition module is used to acquire monitoring and processing information of streaming media data to be processed. The monitoring and processing information includes one or more screenshot monitoring modes and their corresponding parameter information. The screenshot monitoring mode refers to a series of screenshot configuration schemes preset according to different business needs. The screenshot file generation module is used to start an independent process for each screenshot monitoring mode, and in the process, screenshot processing is performed according to the corresponding screenshot monitoring mode and screenshot files are generated. The screenshot file upload module is used to construct one or more types of screenshot upload tasks based on the generated screenshot files, and to allocate each screenshot upload task to a multi-level thread pool for parallel processing. The multi-level thread pool includes multiple independently running thread pools, each thread pool is used to execute a certain type of screenshot upload task.

[0007] In another aspect of this application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods of embodiments of this application.

[0008] In another aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor to implement the methods of the embodiments of this application.

[0009] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods of embodiments of this application.

[0010] The solution provided in this application significantly improves processing efficiency and reduces end-to-end processing latency by constructing an asynchronous parallel screenshot processing architecture based on a multi-level thread pool. This differs from the serial processing mode of existing technologies and effectively reduces processing time. The multi-bucket failover mechanism improves upload availability; when the primary bucket fails to upload, it automatically switches to a backup bucket, unlike the single storage solution of existing technologies, ensuring the continuity and stability of uploads. By adopting a memory file system, the disk I / O bottleneck is eliminated, achieving stable high throughput in high-concurrency scenarios and significantly improving the efficiency and reliability of live screenshots. The HTTP callback-driven event architecture achieves lower latency and higher efficiency event processing. The screenshot tool actively reports screenshot completion events via HTTP callbacks, triggering subsequent processing flows. Unlike the polling detection method of existing technologies, this achieves instant event response, significantly reducing processing latency and improving the overall efficiency and response speed of the system. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A flowchart for processing streaming media data according to an embodiment of this application is shown; Figure 2 A flowchart illustrating an exemplary method for generating and uploading screenshots of a live stream according to an embodiment of this application is shown; Figure 3 A schematic diagram of a device for processing streaming media data according to an embodiment of this application is shown; Figure 4 A schematic diagram of the structure of a device suitable for implementing the scheme in the embodiments of this application is shown.

[0013] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0016] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0017] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer program instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0018] Figure 1 A flowchart for processing streaming media data according to an embodiment of this application is shown. The method includes at least steps S101, S102, and S103.

[0019] In practical scenarios, the execution entity of the method can be a computer device or an application running on a computer device. The computer device includes user devices or network devices. User devices include, but are not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and smart bands. Network devices include, but are not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets. Here, the cloud consists of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a loosely coupled set of computers.

[0020] The terminology used in the embodiments of this application will be explained below.

[0021] HLS: HTTP Live Streaming, a streaming media transmission protocol based on HTTP, which divides video into multiple segments for transmission; M4S: Media Transmission Unit in the HLS protocol, traditionally composed of two parts: header and m4s; GOP: Group of Pictures (GOP) In video coding, a group of pictures refers to a group of consecutive frames that begin with a keyframe. Bit flags: refers to a programming technique that uses binary bits to represent multiple independent states.

[0022] It should be noted that all monitoring data acquired in this application was conducted in a private corporate network environment, and does not involve any personal privacy information or sensitive user data. Furthermore, the image acquisition process has been anonymized.

[0023] The following reference Figure 1 To explain, in step S101, monitoring and processing information of the streaming media data to be processed is obtained. The monitoring and processing information includes one or more screenshot monitoring modes and their corresponding parameter information.

[0024] The streaming media data includes, but is not limited to, real-time video streams, recorded video streams, or other multimedia content, which are typically transmitted using common streaming media protocols such as HLS, RTMP, and DASH.

[0025] Optionally, embodiments of this application support multiple streaming methods at the receiving end. Specifically, streaming media data can be pushed via various protocols such as RTMP, SRT (Secure Reliable Transport), and WHIP (Web High-performance Interactive Protocol). The receiving end will perform corresponding conversion processing on this data, ultimately converting it into HLS format for subsequent screenshot processing and distribution. This approach ensures the system's flexibility and adaptability, meeting the processing needs of streaming media data from different sources and formats.

[0026] The screenshot monitoring mode refers to a series of preset screenshot configuration schemes based on different business needs. These screenshot monitoring modes include, but are not limited to, regular screenshot mode and thumbnail screenshot mode, to adapt to diverse business scenarios. For example, regular screenshot mode is typically used to generate high-quality full-size images, suitable for scenarios requiring clear images, such as content recognition and cover generation. Thumbnail screenshot mode, on the other hand, focuses on generating smaller images, suitable for scenarios with specific image size requirements, such as video previews and thumbnail displays.

[0027] Optionally, each screenshot monitoring mode can be flexibly configured via a configuration flag value (e.g., pic_flag) to support different screenshot resolutions, screenshot intervals, keyframe modes, and other parameters. Through a bit flag mechanism, the configuration flag value is used to perform bitwise operations with a predefined mode mapping table to determine the set of screenshot modes to be activated. This enables precise control and flexible adjustment of screenshot tasks, and allows for rapid switching and configuration of screenshot modes according to different business needs, improving the efficiency and adaptability of screenshot processing.

[0028] According to one embodiment, the parameter information of the screenshot monitoring mode includes at least one of the following: 1) Screenshot interval: Defines the frequency of screenshots, such as capturing one frame per second.

[0029] 2) Image resolution: Specifies the resolution of the generated screenshot, such as 720P or 1080P. 3) Keyframe mode: Used to determine whether to capture only keyframes in order to optimize screenshot quality and processing efficiency.

[0030] 4) Frame Deviation Tolerance: This indicates the allowable deviation at the screenshot time point to accommodate the instability of streaming media data.

[0031] Optionally, depending on the data type of the streaming media data and the monitoring requirements, determine which screenshot monitoring mode or combination of screenshot monitoring modes to use.

[0032] Optionally, one or more screenshot monitoring modes can be set in addition to the standard screenshot mode to meet different monitoring needs. For example, for general live video, the standard mode can meet basic requirements, such as a configuration of 720P resolution, a 1-second screenshot interval, and capturing only key frames; while for video types that require special attention, such as videos involving high-risk content or special scenes, a screenshot monitoring mode with higher resolution and shorter screenshot intervals can be used to ensure the accuracy and timeliness of monitoring.

[0033] According to one embodiment, step S101 includes: parsing the stream identifier and screenshot configuration parameters based on the stream publishing event of the received streaming media data, and determining one or more screenshot monitoring modes that match the streaming media data through a bit flag mechanism; then, obtaining the parameter information corresponding to the one or more screenshot monitoring modes to obtain the monitoring and processing information of the streaming media data.

[0034] The process of determining one or more screenshot monitoring modes that match streaming media data through a bit flag mechanism includes: obtaining a predefined mapping relationship between bit flags and screenshot monitoring modes; when a stream publishing event is received, extracting a configuration identifier value from the event parameters, which encodes the combination of screenshot monitoring modes that need to be activated; then, based on the mapping relationship, performing a bitwise AND operation on the bit flags and configuration identifier values ​​of each mode, and if the result is non-zero, it indicates that the screenshot monitoring mode needs to be activated.

[0035] For example, at startup, a predefined mapping of bit flags to processing modes is loaded, where each binary bit corresponds to a processing mode. For instance, bit 0 (value 1) maps to a regular screenshot mode, configured for 720P resolution, 1-second intervals, and capturing only keyframes. When a stream publishing event is received, the stream publishing event from the streaming service is received, and the stream identifier (e.g., app / stream) and the processing mode configuration parameters (represented as pic_flag) are parsed from it. The pic_flag value encodes the combination of screenshot monitoring modes to be activated. Next, the mode mapping table is traversed, and a bitwise AND (&) operation is performed between the bit flags of each mode and pic_flag. If the result is non-zero, it indicates that the mode needs to be activated.

[0036] By using a bit-flag mechanism, adding a new processing mode only requires adding a single line of configuration to the mapping table, without modifying the core logic code, thus achieving high scalability of processing modes. The O(1) complexity of bit operations ensures the efficiency of mode parsing, significantly improving the system's flexibility and response speed.

[0037] In step S102, an independent process is started for each monitoring processing mode, and screenshot processing is performed and a screenshot file is generated in the corresponding screenshot mode.

[0038] The method involves starting an independent process for each mode and ensuring that these processes do not interfere with each other when performing screenshot tasks.

[0039] Specifically, a separate process is started for each type of monitoring process, and the streaming media data to be processed is captured according to the preset screenshot time interval and image resolution parameters of the corresponding screenshot mode.

[0040] Taking HLS streams as an example, based on a determined screenshot monitoring mode or a combination of multiple screenshot monitoring modes, an independent ffsnap process is launched for each mode. This process extracts image frames from the HLS stream at configured time intervals and generates screenshot files according to a preset resolution. For example, the ffsnap process extracts keyframes from the HLS stream and generates screenshot files according to a preset 720P resolution and a 1-second screenshot interval configuration.

[0041] In step S103, based on the generated screenshot file, multiple tasks required for uploading the screenshot file are constructed, and each task is assigned to a multi-level thread pool for parallel processing.

[0042] In this embodiment, the multi-level threading includes multiple independently running thread pools, each dedicated to executing a specific type of screenshot upload task. Furthermore, each thread pool operates independently without blocking or interfering with the others, ensuring that a backlog of one type of task does not affect the processing of other types of tasks.

[0043] According to one embodiment, the multi-level thread pool includes three thread pools: a general task pool, an image upload pool, and a video upload pool. The general task pool is responsible for handling common tasks such as starting the screenshot process and managing the stream state. The image upload pool is used to handle tasks of uploading screenshot files to the distributed object storage service, and the video upload pool is used to handle tasks of uploading video clip files to the distributed object storage service.

[0044] Optionally, the multi-level thread pool in this application embodiment allows the capacity configuration of each thread pool to be adjusted independently according to the processing capacity of each stage. For example, the dedicated image upload pool can dynamically adjust the number of threads according to the actual upload task volume, while the dedicated video upload pool can be optimized according to the upload requirements of video clip files.

[0045] According to one embodiment, step S103 includes steps S1031 and S1032.

[0046] In step S1031, based on the generated screenshot file, the screenshot metadata is parsed and multiple upload tasks are constructed.

[0047] Specifically, key information is extracted from the metadata, such as the screenshot file path, file size, screenshot timestamp, parameters corresponding to the screenshot mode (such as resolution and screenshot interval), and other upload-related configuration information. This information will be integrated into the configuration data for the upload task, ensuring that each task contains all the necessary information required to perform the upload operation.

[0048] Optionally, the process of building an upload task includes: Determine the task type: Based on the screenshot mode and information in the metadata, determine the task type, such as a screenshot file upload task or a video file upload task; Configure task parameters: Configure necessary parameters for each task, including file path, target storage location, upload priority, retry policy, timeout, etc. Set task metadata: Attach metadata such as screenshot timestamps and file size to the task so that it can be monitored and recorded during the upload process; Generate upload task instance: Encapsulate the above configuration and metadata into a task instance so that it can be submitted to the corresponding thread pool later.

[0049] In step S1032, tasks are assigned to corresponding thread pools according to task type, so that different upload tasks are executed in parallel in multiple thread pools.

[0050] Specifically, based on the pre-stored correspondence between task types and thread pools, the current task is assigned to the corresponding thread pool according to its type. For example, the ffsnap process startup task is submitted to the general task pool, the screenshot file upload task is submitted to the image upload dedicated pool, and the M4S file upload task is submitted to the M4S upload dedicated pool. This categorized submission method ensures resource isolation for different types of tasks and avoids the "head-of-queue blocking" problem in a single queue.

[0051] According to one embodiment, this application uses a work-stealing algorithm in a multi-level thread pool to achieve dynamic load balancing. The work-stealing algorithm allows idle threads to obtain tasks from the task queue of the busy thread pool, thereby effectively solving the problem of congestion in some task queues and achieving efficient utilization of resources.

[0052] Specifically, the number of queues in each thread pool is dynamically set based on the load. If one thread pool's queue is relatively idle while another thread pool's queue is busy, the work-stealing algorithm automatically redirects some tasks from the busy queue threads to the idle queue threads for execution, thus alleviating the pressure on the busy queue. This embodiment achieves dynamic load balancing through the work-stealing algorithm, improving overall throughput and ensuring efficient resource utilization.

[0053] According to one embodiment, the method employs an HTTP callback-driven event handling mechanism.

[0054] Specifically, after taking a screenshot in step S102, a notification is sent to the main service via an HTTP callback interface. Optionally, the notification carries necessary parameters for screenshot completion, such as the screenshot file path and timestamp. In response to receiving the HTTP callback indicating screenshot completion, step S103 is executed, where one or more types of screenshot upload tasks are constructed based on the generated screenshot file, and each screenshot upload task is allocated to a multi-level thread pool for parallel processing.

[0055] Optionally, the method registers the ` / on_new_image` interface on the HTTP server to receive callback notifications upon screenshot completion. The screenshot process sends a notification to this interface via an HTTP POST request, ensuring that subsequent processing is triggered immediately upon screenshot completion. This callback-driven mechanism decouples screenshot generation from subsequent processing, allowing the screenshot module to focus on generating screenshots, while the main service focuses on task allocation and execution.

[0056] Compared with the traditional polling detection method, the HTTP callback-driven event handling mechanism of this application embodiment realizes the immediate response to events, significantly reduces processing latency, and improves the overall efficiency and response speed of the system.

[0057] According to one embodiment, a failover mechanism using multiple storage buckets is employed to upload screenshot files. These storage buckets include a primary bucket and a backup bucket. Specifically, multiple storage buckets are used to upload screenshot files, and if uploading a screenshot file fails in the primary bucket, the system automatically switches to the backup bucket.

[0058] Optionally, a local proxy service can be used to connect to a distributed object storage service (such as S3 storage service), supporting multi-bucket configuration. When uploading screenshot files, the primary bucket is prioritized for file upload. Optionally, to manage the primary and backup buckets more efficiently, index numbers are created for the uploaders in the primary and backup buckets; for example, the primary bucket's number can be set to 0. If the primary bucket upload fails and the task is marked for retry, an uploader is randomly selected from the backup bucket list for retry until the screenshot file is successfully uploaded.

[0059] Taking S3 storage service as an example, the failover mechanism for multiple buckets follows these steps: At startup, multiple uploader instances for screenshot files are created. Each instance corresponds to a bucket and contains independent authentication information, endpoint addresses, and proxy configurations. Under normal circumstances, screenshot upload tasks are executed using the uploader with index 0 (i.e., the primary bucket). When a screenshot upload fails and the task is marked for retry, an uploader is randomly selected from the list of backup buckets for retry.

[0060] The failover mechanism of multiple storage buckets in this embodiment eliminates the risk of single point of failure. By randomly selecting backup buckets, it achieves load distribution in failure scenarios, which significantly improves the upload success rate and the overall availability of the system.

[0061] According to one embodiment, the method further includes steps S104 and S105.

[0062] In step S104, the screenshot upload status of each process is monitored to determine if any abnormalities have occurred. In step S105, if an abnormality is detected in a process, the process is restarted to re-process the screenshot.

[0063] Optionally, fault recovery can be achieved by periodically monitoring the timestamps of received screenshots based on Group of Pictures (GOPs). In video encoding and streaming transmission, a GOP represents a set of consecutive image frames in a video stream, and the GOP value reflects the interval period of keyframes. The size of a GOP is usually expressed in frames; for example, a GOP value of 12 means that each GOP contains 12 frames.

[0064] Specifically, the image group information (GOP) of the streaming media data is obtained. In step S104, the timestamps of each process uploading screenshot files are periodically monitored based on the image group information. If a process does not receive a new screenshot file within a time exceeding the sum of the GOP period and the fault tolerance threshold, it is determined that an anomaly has occurred, thereby triggering the restart of the process.

[0065] Optionally, each time a screenshot callback is received, the last screenshot timestamp (lastImageTimestamp) of the corresponding stream is updated, and the GOP size (gopSize, in milliseconds) is obtained through the HLS stream's metadata interface. If a process does not receive a new screenshot within a time exceeding the sum of the GOP period and the fault tolerance threshold, it is determined that the process has encountered an error.

[0066] Optionally, after restarting the process, wait for the initial screenshot callback to confirm successful recovery. If no callback is received within a specified time, trigger the restart process again.

[0067] The embodiments of this application are based on a dynamic threshold method of GOP period, which is more adaptable to live streams with different encoding configurations than a fixed threshold, reducing the possibility of false positives and false negatives. This automatic recovery mechanism realizes self-healing of faults and significantly improves the availability of the system.

[0068] According to one embodiment, the method stores screenshot files in a memory file system. The memory file system, such as / dev / shm, simulates a file system in memory, enabling high-speed file read and write operations.

[0069] Optionally, efficient uploading can be achieved through a local proxy mechanism, combined with a scheduled cleanup mechanism and fine-grained monitoring to ensure the system's efficient operation and high reliability.

[0070] Specifically, the storage and processing of screenshot files are as follows: First, the screenshot output directory is configured as a memory-based file system, allowing screenshot files to be written directly to memory, avoiding disk I / O overhead. Next, a local proxy service, such as a Unix Socket-based proxy, is used to connect to the upload service, replacing a direct network connection, reducing network overhead and facilitating unified management. Unix Socket is an inter-process communication mechanism that enables efficient data transfer between local processes through file system paths.

[0071] Optionally, the method initiates a timed cleanup task (e.g., executed every 3 minutes) to clean up screenshot files that have not been updated for a specified time, freeing up memory space and ensuring the efficient operation of the memory file system.

[0072] The method according to the embodiments of this application significantly improves processing efficiency and reduces end-to-end processing latency by constructing an asynchronous parallel screenshot processing architecture based on a multi-level thread pool. This differs from the serial processing mode of existing technologies and effectively reduces processing time. The multi-bucket failover mechanism improves upload availability; when the primary bucket fails to upload, it automatically switches to a backup bucket, unlike the single storage solution of existing technologies, ensuring the continuity and stability of uploads. By adopting a memory file system, the disk I / O bottleneck is eliminated, achieving stable high throughput in high-concurrency scenarios and significantly improving the efficiency and reliability of live screenshots. The HTTP callback-driven event architecture achieves lower latency and higher efficiency event processing. The screenshot tool actively reports screenshot completion events via HTTP callbacks, triggering subsequent processing flows. Unlike the polling detection method of existing technologies, this achieves immediate event response, significantly reducing processing latency and improving the overall efficiency and response speed of the system.

[0073] The following example will illustrate this point. Figure 2 A flowchart illustrating an exemplary method for generating and uploading screenshots of a live stream according to an embodiment of this application is shown. Figure 2 The process shown is executed by a system that provides live streaming services, and this system can efficiently handle screenshot tasks from the live stream. Figure 2 The series of modules shown work together to achieve a complete process from stream monitoring to automatic fault recovery.

[0074] The system mainly includes a stream event listening module, a screenshot mode configuration module, a screenshot generation module, a task scheduling module, an upload module, a notification module, and a monitoring module.

[0075] The streaming event listening module is responsible for listening to streaming publishing events from the streaming media service, parsing the streaming identifier and screenshot configuration parameters, and triggering the subsequent screenshot process.

[0076] The screenshot mode configuration module performs bitwise operations based on the configuration identifier value and the predefined mode mapping table to determine the set of screenshot modes to be activated, so as to support flexible mode combination configuration.

[0077] The screenshot generation module starts an independent ffsnap process for each screenshot mode, extracts image frames from the HLS stream at configured intervals, generates screenshot files, and notifies the main service via HTTP callback. The task scheduling module receives the HTTP callback from ffsnap, parses the screenshot metadata, constructs the upload task, and implements parallel scheduling of the task based on a three-level thread pool architecture.

[0078] This multi-level thread pool comprises three thread pools: a general task pool, a dedicated image upload pool, and a dedicated M4S upload pool. The general task pool handles common tasks such as screenshot process initiation and stream state management; the dedicated image upload pool specifically handles S3 upload tasks for screenshot files; and the dedicated M4S upload pool is responsible for S3 upload tasks for video clip files.

[0079] The upload module connects to a distributed object storage service through a local proxy service, supports multi-bucket configuration and failover, and enables high-speed uploading of images and M4S video clips.

[0080] The notification module will batch report screenshot information to the central service, supporting duplicate frame filtering and expired data filtering.

[0081] The monitoring module collects performance indicators for each stage and pushes them to Prometheus. It also performs timed fault detection and automatic recovery to ensure the stable operation of the system.

[0082] The system in this example efficiently executes screenshot upload tasks in parallel through a three-tier architecture of the task scheduling module, ensuring high performance and high reliability.

[0083] Figure 3 A schematic diagram of an apparatus for processing streaming media data according to an embodiment of this application is shown. The apparatus includes a mode information acquisition module 101, a screenshot file generation module 102, and a screenshot file upload module 103.

[0084] The mode information acquisition module 101 acquires the monitoring and processing information of the streaming media data to be processed, the monitoring and processing information including one or more screenshot monitoring modes and their corresponding parameter information.

[0085] The streaming media data includes, but is not limited to, real-time video streams, recorded video streams, or other multimedia content, which are typically transmitted using common streaming media protocols such as HLS, RTMP, and DASH.

[0086] Optionally, embodiments of this application support multiple streaming methods at the receiving end. Specifically, streaming media data can be pushed via various protocols such as RTMP, SRT (Secure Reliable Transport), and WHIP (Web High-performance Interactive Protocol). The receiving end will perform corresponding conversion processing on this data, ultimately converting it into HLS format for subsequent screenshot processing and distribution. This approach ensures the system's flexibility and adaptability, meeting the processing needs of streaming media data from different sources and formats.

[0087] The screenshot monitoring mode refers to a series of preset screenshot configuration schemes based on different business needs. These screenshot monitoring modes include, but are not limited to, regular screenshot mode and thumbnail screenshot mode, to adapt to diverse business scenarios. For example, regular screenshot mode is typically used to generate high-quality full-size images, suitable for scenarios requiring clear images, such as content recognition and cover generation. Thumbnail screenshot mode, on the other hand, focuses on generating smaller images, suitable for scenarios with specific image size requirements, such as video previews and thumbnail displays.

[0088] Optionally, each screenshot monitoring mode can be flexibly configured via a configuration flag value (e.g., pic_flag) to support different screenshot resolutions, screenshot intervals, keyframe modes, and other parameters. Through a bit flag mechanism, the configuration flag value is used to perform bitwise operations with a predefined mode mapping table to determine the set of screenshot modes to be activated. This enables precise control and flexible adjustment of screenshot tasks, and allows for rapid switching and configuration of screenshot modes according to different business needs, improving the efficiency and adaptability of screenshot processing.

[0089] The specific information included in the screenshot monitoring mode parameters has been described above and will not be repeated here.

[0090] Optionally, the mode information acquisition module 101 determines which screenshot monitoring mode or combination of screenshot monitoring modes to use based on the data type of the streaming media data and the monitoring requirements.

[0091] Optionally, one or more screenshot monitoring modes can be set in addition to the standard screenshot mode to meet different monitoring needs. For example, for general live video, the standard mode can meet basic requirements, such as a configuration of 720P resolution, a 1-second screenshot interval, and capturing only key frames; while for video types that require special attention, such as videos involving high-risk content or special scenes, a screenshot monitoring mode with higher resolution and shorter screenshot intervals can be used to ensure the accuracy and timeliness of monitoring.

[0092] According to one embodiment, the mode information acquisition module 101 parses the stream identifier and screenshot configuration parameters based on the stream publishing event of the received streaming media data, and determines one or more screenshot monitoring modes that match the streaming media data through a bit flag mechanism; then, it acquires the parameter information corresponding to the one or more screenshot monitoring modes to obtain the monitoring and processing information of the streaming media data.

[0093] The process of determining one or more screenshot monitoring modes that match streaming media data through a bit flag mechanism includes: obtaining a predefined mapping relationship between bit flags and screenshot monitoring modes; when a stream publishing event is received, extracting a configuration identifier value from the event parameters, which encodes the combination of screenshot monitoring modes that need to be activated; then, based on the mapping relationship, performing a bitwise AND operation on the bit flags and configuration identifier values ​​of each mode, and if the result is non-zero, it indicates that the screenshot monitoring mode needs to be activated.

[0094] By using a bit-flag mechanism, adding a new processing mode only requires adding a single line of configuration to the mapping table, without modifying the core logic code, thus achieving high scalability of processing modes. The O(1) complexity of bit operations ensures the efficiency of mode parsing, significantly improving the system's flexibility and response speed.

[0095] The screenshot file generation module 102 starts an independent process for each monitoring processing mode, and performs screenshot processing and generates screenshot files according to the corresponding screenshot mode in the process.

[0096] The method involves starting an independent process for each mode and ensuring that these processes do not interfere with each other when performing screenshot tasks.

[0097] Specifically, a separate process is started for each type of monitoring process, and the streaming media data to be processed is captured according to the preset screenshot time interval and image resolution parameters of the corresponding screenshot mode.

[0098] Taking HLS streams as an example, based on a determined screenshot monitoring mode or a combination of multiple screenshot monitoring modes, an independent ffsnap process is launched for each mode. This process extracts image frames from the HLS stream at configured time intervals and generates screenshot files according to a preset resolution. For example, the ffsnap process extracts keyframes from the HLS stream and generates screenshot files according to a preset 720P resolution and a 1-second screenshot interval configuration.

[0099] The screenshot file upload module 103 constructs multiple tasks required for uploading the screenshot file based on the generated screenshot file, and allocates each task to a multi-level thread pool for parallel processing.

[0100] In this embodiment, the multi-level threading includes multiple independently running thread pools, each dedicated to executing a specific type of screenshot upload task. Furthermore, each thread pool operates independently without blocking or interfering with the others, ensuring that a backlog of one type of task does not affect the processing of other types of tasks.

[0101] According to one embodiment, the multi-level thread pool includes three thread pools: a general task pool, an image upload pool, and a video upload pool. The general task pool is responsible for handling common tasks such as starting the screenshot process and managing the stream state. The image upload pool is used to handle tasks of uploading screenshot files to the distributed object storage service, and the video upload pool is used to handle tasks of uploading video clip files to the distributed object storage service.

[0102] Optionally, the multi-level thread pool in this application embodiment allows the capacity configuration of each thread pool to be adjusted independently according to the processing capacity of each stage. For example, the dedicated image upload pool can dynamically adjust the number of threads according to the actual upload task volume, while the dedicated video upload pool can be optimized according to the upload requirements of video clip files.

[0103] According to one embodiment, the screenshot file upload module 103 includes a screenshot parsing module and a task allocation module.

[0104] The screenshot parsing module parses the screenshot metadata based on the generated screenshot file and constructs multiple upload tasks.

[0105] Specifically, the screenshot parsing module extracts key information from the metadata, such as the screenshot file path, file size, screenshot timestamp, parameters corresponding to the screenshot mode (such as resolution and screenshot interval), and other upload-related configuration information. This information will be integrated into the configuration data for the upload task, ensuring that each task contains all the necessary information required to perform the upload operation.

[0106] Optionally, the process of constructing the upload task by the screenshot parsing module includes: Determine the task type: Based on the screenshot mode and information in the metadata, determine the task type, such as a screenshot file upload task or a video file upload task; Configure task parameters: Configure necessary parameters for each task, including file path, target storage location, upload priority, retry policy, timeout, etc. Set task metadata: Attach metadata such as screenshot timestamps and file size to the task so that it can be monitored and recorded during the upload process; Generate upload task instance: Encapsulate the above configuration and metadata into a task instance so that it can be submitted to the corresponding thread pool later.

[0107] The task allocation module assigns tasks to the corresponding thread pools based on the task type, thereby enabling different upload tasks to be executed in parallel across multiple thread pools.

[0108] Specifically, the task allocation module assigns the current task to the corresponding thread pool based on the pre-stored correspondence between task types and thread pools. For example, the ffsnap process startup task is submitted to the general task pool, the screenshot file upload task is submitted to the image upload dedicated pool, and the M4S file upload task is submitted to the M4S upload dedicated pool. This categorized submission method ensures resource isolation for different types of tasks and avoids the "head-of-queue blocking" problem in a single queue.

[0109] According to one embodiment, this application uses a work-stealing algorithm in a multi-level thread pool to achieve dynamic load balancing. The work-stealing algorithm allows idle threads to obtain tasks from the task queue of the busy thread pool, thereby effectively solving the problem of congestion in some task queues and achieving efficient utilization of resources.

[0110] Specifically, the number of queues in each thread pool is dynamically set based on the load. If one thread pool's queue is relatively idle while another thread pool's queue is busy, the work-stealing algorithm automatically redirects some tasks from the busy queue threads to the idle queue threads for execution, thus alleviating the pressure on the busy queue. This embodiment achieves dynamic load balancing through the work-stealing algorithm, improving overall throughput and ensuring efficient resource utilization.

[0111] According to one embodiment, the device employs an HTTP callback-driven event handling mechanism.

[0112] Specifically, after the screenshot file generation module 102 completes the screenshot, it sends a notification to the main service via an HTTP callback interface. Optionally, the notification carries necessary parameters for screenshot completion, such as the screenshot file path and timestamp. In response to receiving the HTTP callback indicating screenshot completion, the screenshot file upload module 103 executes its operations, constructs one or more types of screenshot upload tasks based on the generated screenshot file, and distributes each screenshot upload task to a multi-level thread pool for parallel processing.

[0113] Optionally, the device receives a callback notification upon screenshot completion by registering the ` / on_new_image` interface on an HTTP server. The screenshot process sends a notification to this interface via an HTTP POST request, ensuring that subsequent processing is triggered immediately upon screenshot completion. This callback-driven mechanism decouples screenshot generation from subsequent processing, allowing the screenshot module to focus on generating screenshots, while the main service focuses on task allocation and execution.

[0114] Compared with the traditional polling detection method, the HTTP callback-driven event handling mechanism of this application embodiment realizes the immediate response to events, significantly reduces processing latency, and improves the overall efficiency and response speed of the system.

[0115] According to one embodiment, the device employs a failover mechanism using multiple storage buckets to upload screenshot files. These storage buckets include a primary bucket and a backup bucket. Specifically, multiple storage buckets are used to upload screenshot files, and if uploading a screenshot file fails in the primary bucket, the system automatically switches to the backup bucket.

[0116] Optionally, a local proxy service can be used to connect to a distributed object storage service (such as S3 storage service), supporting multi-bucket configuration. When uploading screenshot files, the primary bucket is used first for file upload. Optionally, to manage the primary and standby buckets more efficiently, index numbers are created for the uploaders in the primary and standby buckets. Taking S3 storage service as an example, the failover mechanism for multiple buckets follows these steps: At startup, multiple uploader instances for screenshot files are created. Each instance corresponds to a bucket and contains independent authentication information, endpoint addresses, and proxy configurations. Under normal circumstances, screenshot upload tasks are executed using the uploader with index 0 (i.e., the primary bucket). When a screenshot upload fails and the task is marked for retry, an uploader is randomly selected from the list of backup buckets for retry.

[0117] The failover mechanism of multiple storage buckets in this embodiment eliminates the risk of single point of failure. By randomly selecting backup buckets, it achieves load distribution in failure scenarios, which significantly improves the upload success rate and the overall availability of the system.

[0118] According to one embodiment, the apparatus further includes an anomaly determination module and a process restart module.

[0119] The anomaly detection module monitors the screenshot upload status of each process to determine if any anomalies have occurred. If an anomaly is detected in a process, the process restart module triggers the restart of that process, thereby re-processing the screenshot.

[0120] Optionally, fault recovery can be achieved by periodically monitoring the timestamps of received screenshots based on Group of Pictures (GOPs). In video encoding and streaming transmission, a GOP represents a set of consecutive image frames in a video stream, and the GOP value reflects the interval period of keyframes. The size of a GOP is usually expressed in frames; for example, a GOP value of 12 means that each GOP contains 12 frames.

[0121] Specifically, the image group information (GOP) of the streaming media data is obtained. The anomaly determination module periodically monitors the timestamps of screenshot files uploaded by each process based on the image group information. If a process does not receive a new screenshot file within a time exceeding the sum of the GOP period and the fault tolerance threshold, it determines that an anomaly has occurred and triggers the restart of that process.

[0122] Optionally, each time a screenshot callback is received, the exception determination module updates the last screenshot timestamp (lastImageTimestamp) of the corresponding stream and obtains the GOP size (gopSize, in milliseconds) through the HLS stream's metadata interface. If a process does not receive a new screenshot within a time exceeding the sum of the GOP period and the fault tolerance threshold, it is determined that the process has encountered an exception.

[0123] Optionally, after restarting the process, wait for the initial screenshot callback to confirm successful recovery. If no callback is received within a specified time, trigger the restart process again.

[0124] The embodiments of this application are based on a dynamic threshold method of GOP period, which is more adaptable to live streams with different encoding configurations than a fixed threshold, reducing the possibility of false positives and false negatives. This automatic recovery mechanism realizes self-healing of faults and significantly improves the availability of the system.

[0125] According to one embodiment, the device stores screenshot files in a memory file system. The memory file system, such as / dev / shm, simulates a file system in memory, enabling high-speed file read and write operations.

[0126] Optionally, efficient uploading can be achieved through a local proxy mechanism, combined with a scheduled cleanup mechanism and fine-grained monitoring to ensure the system's efficient operation and high reliability.

[0127] Specifically, the storage and processing of screenshot files are as follows: First, the screenshot output directory is configured as a memory-based file system, allowing screenshot files to be written directly to memory, avoiding disk I / O overhead. Next, a local proxy service, such as a Unix Socket-based proxy, is used to connect to the upload service, replacing a direct network connection, reducing network overhead and facilitating unified management. Unix Socket is an inter-process communication mechanism that enables efficient data transfer between local processes through file system paths.

[0128] Optionally, the device initiates a timed cleanup task (e.g., every 3 minutes) to clean up screenshot files that have not been updated for a specified time, freeing up memory space and ensuring the efficient operation of the memory file system.

[0129] The apparatus according to the embodiments of this application significantly improves processing efficiency and reduces end-to-end processing latency by constructing an asynchronous parallel screenshot processing architecture based on a multi-level thread pool. This differs from the serial processing mode of existing technologies and effectively reduces processing time. The multi-bucket failover mechanism improves upload availability; when the primary bucket fails to upload, it automatically switches to a backup bucket, unlike the single storage solution of existing technologies, ensuring the continuity and stability of uploads. By adopting a memory file system, the disk I / O bottleneck is eliminated, achieving stable high throughput in high-concurrency scenarios and significantly improving the efficiency and reliability of live screenshots. The HTTP callback-driven event architecture achieves lower latency and higher efficiency event processing. The screenshot tool actively reports screenshot completion events via HTTP callbacks, triggering subsequent processing flows. Unlike the polling detection method of existing technologies, this achieves instant event response, significantly reducing processing latency and improving the overall efficiency and response speed of the system.

[0130] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0131] The electronic device can be a user device, or a device formed by integrating user devices and network devices through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0132] Figure 4The diagram illustrates the structure of an apparatus suitable for implementing the methods and / or technical solutions in the embodiments of this application. The apparatus 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes based on a program stored in a Read Only Memory (ROM) 1202 or a program loaded from a storage portion 1208 into a Random Access Memory (RAM) 1203. The RAM 1203 also stores various programs and data required for system operation. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An Input / Output (I / O) interface 1205 is also connected to the bus 1204.

[0133] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; a storage section 1208 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet.

[0134] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, it performs the functions defined in the methods of this application.

[0135] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0136] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0137] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0138] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0139] The program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or page components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0145] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0147] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in an apparatus may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for processing streaming media data, wherein, The method includes: Acquire monitoring and processing information of streaming media data to be processed. The monitoring and processing information includes one or more screenshot monitoring modes and their corresponding parameter information. The screenshot monitoring mode refers to a series of screenshot configuration schemes preset according to different business needs. A separate process is started for each screenshot monitoring mode, and screenshot processing and screenshot files are generated in the corresponding screenshot monitoring mode. Based on the generated screenshot files, one or more types of screenshot upload tasks are constructed, and each screenshot upload task is assigned to a multi-level thread pool for parallel processing. The multi-level thread pool includes multiple independently running thread pools, each thread pool being used to execute a certain type of screenshot upload task.

2. The method according to claim 1, wherein, The method further includes: Screenshot files are uploaded using multiple storage buckets. If the upload fails in the primary bucket, the system automatically switches to the backup bucket.

3. The method according to claim 1 or 2, wherein, The method further includes: Monitor the screenshot upload status of each process to determine if any anomalies occur; If an anomaly is detected in a process, the process is restarted, and the screenshot is reprocessed.

4. The method according to claim 3, wherein, The method further includes: Obtain the group of images (GOP) information of the streaming media data; The monitoring of screenshot uploads from each process to determine whether any anomalies have occurred includes: Based on the image group information, the timestamps of each process's uploaded screenshot files are monitored periodically. If a process does not receive a new screenshot file within a time exceeding the sum of the GOP period and the fault tolerance threshold, it is determined that an anomaly has occurred in that process.

5. The method according to claim 1, wherein, The monitoring and processing information for acquiring the streaming media data to be processed includes: Based on the streaming data publishing events received, the stream identifier and screenshot configuration parameters are parsed, and one or more screenshot monitoring modes matching the streaming data are determined through a bit flag mechanism; Obtain the parameter information corresponding to one or more screenshot monitoring modes to obtain the monitoring and processing information of streaming media data.

6. The method according to claim 5, wherein, The method of determining one or more screenshot monitoring modes that match streaming media data through a bit flag mechanism includes: Obtain the mapping relationship between predefined bit flags and screenshot monitoring modes; When a stream publishing event is received, the configuration identifier value is extracted from the event parameters. This value encodes the combination of screenshot monitoring modes that need to be activated. Based on the mapping relationship, a bitwise AND operation is performed on the bit flag and configuration identifier value of each mode. If the result of the operation is non-zero, it indicates that the screenshot monitoring mode needs to be started.

7. The method according to claim 1, wherein, Based on the generated screenshot files, construct one or more types of screenshot upload tasks, and allocate each screenshot upload task to a multi-level thread pool for parallel processing, including: Based on the generated screenshot files, the screenshot metadata is parsed and multiple upload tasks are constructed. Tasks are assigned to the corresponding thread pools based on their type, allowing different upload tasks to be executed in parallel across multiple thread pools.

8. The method according to claim 7, wherein, The method further includes: Dynamic load balancing is achieved by employing a work-stealing algorithm in a multi-level thread pool, which allows idle threads to obtain tasks from the task queue of the busy thread pool.

9. The method according to claim 1, wherein, The method employs an HTTP callback-driven event handling mechanism, and the method includes: After taking the screenshot, a notification is sent to the main service via an HTTP callback interface; In response to the HTTP callback indicating that the main service has received the screenshot, one or more types of screenshot upload tasks are constructed based on the generated screenshot file, and each screenshot upload task is distributed to a multi-level thread pool for parallel processing.

10. An apparatus for processing streaming media data, wherein, The device includes: The mode information acquisition module is used to acquire monitoring and processing information of streaming media data to be processed. The monitoring and processing information includes one or more screenshot monitoring modes and their corresponding parameter information. The screenshot monitoring mode refers to a series of screenshot configuration schemes preset according to different business needs. The screenshot file generation module is used to start an independent process for each screenshot monitoring mode, and in the process, screenshot processing is performed according to the corresponding screenshot monitoring mode and screenshot files are generated. The screenshot file upload module is used to construct one or more types of screenshot upload tasks based on the generated screenshot files, and to allocate each screenshot upload task to a multi-level thread pool for parallel processing. The multi-level thread pool includes multiple independently running thread pools, each thread pool is used to execute a certain type of screenshot upload task.

11. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

12. A computer-readable medium having stored thereon computer program instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 9.

Images