A web worker-based video playing method and device

Abstract

The present disclosure provides a Web Worker-based video playing method and device, which separates the video stream data downloading task and decoding task from the main thread by creating a downloading Worker thread and a decoding Worker thread, and acquires the buffer state information of the undecoded data packet queue in the decoding Worker thread, generates a feedback control message according to the buffer state information, and coordinates the downloading Worker thread to perform control operations such as pausing downloading, resuming downloading or breakpoint continuation. Thus, when the undecoded data packets accumulate, the data continues to be input, when the undecoded data is insufficient, the downloading is resumed in time, and the playing delay can be reduced in combination with the frame buffer frame chasing control in the real-time streaming scenario, thereby reducing the memory occupation risk, avoiding decoding starvation and playing lag, and improving the fluency, stability and real-time performance of the browser-side video playing.

Description

Technical Field

[0001] This disclosure relates to the field of video playback technology, and more specifically, to a video playback method and apparatus based on Web Worker. Background Technology

[0002] With the development of applications such as online video, real-time surveillance video, and live streaming, browser-based video playback demands increasingly higher levels of smoothness, low latency, and system responsiveness. In existing browser video playback solutions, the downloading, parsing, decoding, and rendering of video data, as well as user interface interaction, typically need to be completed collaboratively within the browser environment.

[0003] To reduce the load on the main thread, existing technologies have solutions that offload some video processing tasks to Web Worker threads. For example, by setting up a download worker thread to perform video stream data download tasks and a decoding worker thread to perform video data decoding tasks, the download and decoding processes can be executed in parallel on different threads. This type of solution improves the parallel processing capability of the video player to a certain extent and can reduce the impact of computationally intensive tasks such as downloading and decoding on the main thread's rendering and interactive operations.

[0004] However, existing multi-worker video playback solutions typically only implement unidirectional data transmission from the download worker to the decoding worker. The download worker continuously sends video stream data to the decoding worker according to the network download capacity, while the decoding worker mainly passively receives and decodes the data. Due to the lack of a control mechanism for feedback on the decoding load status from the decoder side to the downloader side, it is difficult to achieve dynamic matching between download speed and decoding capacity. When the network download speed exceeds the decoder's processing capacity, undecoded video data packets can easily accumulate upstream of the decoder, consuming a large amount of memory and potentially causing memory overflow. When the network fluctuates or the download speed is insufficient, the decoder may idle due to insufficient data to be decoded, leading to playback stuttering. Summary of the Invention

[0005] This disclosure provides at least one video playback method and apparatus based on Web Worker. By creating a download worker thread and a decoding worker thread, the video stream data download and decoding tasks are separated from the main thread. The decoding worker thread obtains the buffer status information of the undecoded data packet queue and generates feedback control messages based on this information. The main thread then coordinates the download worker thread to perform control operations such as pausing download, resuming download, or resuming interrupted downloads. This allows for the suppression of further data input when undecoded data packets accumulate, timely resumption of downloads when undecoded data is insufficient, and, combined with frame buffering and frame tracking control in real-time streaming scenarios, reduces playback latency. This reduces memory consumption risks, avoids decoding starvation and playback stuttering, and improves the smoothness, stability, and real-time performance of video playback on the browser side.

[0006] This disclosure provides a video playback method based on Web Worker, including: During the player initialization phase, a download worker thread is created to perform video stream data download tasks and a decoding worker thread is created to perform video data decoding tasks. The download worker thread acquires video stream data and sends the video stream data to the decoding worker thread. The received video stream data is decoded by the Decoding Worker thread, and the buffer status information of the undecoded data packet queue inside the decoder is obtained during the decoding process. Based on the buffer state information, a feedback control message corresponding to the decoder load state is generated, and the feedback control message is sent to the main thread; The main thread parses the feedback control message and sends a download control command to the download worker thread based on the parsing result, so as to adjust the video stream data acquisition status of the download worker thread. The decoded video frame data is returned to the main thread by the decoding worker thread, and the main thread renders and plays the video frame data.

[0007] In one optional implementation, obtaining the buffer state information of the queue of undecoded data packets inside the decoder specifically includes: The decoding worker thread calls the buffer status monitoring interface provided by the underlying decoding module. The cumulative playback duration is used as buffer state information to characterize the upstream load state of the decoder; The underlying decoding module is a WebAssembly decoding module, and the buffer status monitoring interface is used to call the queue query method implemented by the C / C++ decoder to obtain the cumulative duration of the queue of undecoded data packets. When the queue query method returns a null value, the cumulative duration is set to zero.

[0008] In one optional implementation, a feedback control message corresponding to the decoder load state is generated based on the buffer state information, specifically including: The cumulative playback duration is compared with a first preset threshold. When the cumulative playback duration exceeds the first preset threshold, it is determined that the queue of undecoded data packets is in a backlog state, and a first feedback control message is generated; The first feedback control message includes a message type identifier indicating that the decoding data is full and the cumulative playback duration.

[0009] In one optional implementation, the main thread parses the feedback control message and sends a download control command to the download worker thread based on the parsing result, specifically including: The main thread identifies the message type identifier in the first feedback control message; When the first feedback control message is detected, a pause download control command is generated through the main thread; The pause download control command is sent to the download worker thread to cause the download worker thread to pause the current video stream data acquisition task.

[0010] In an optional implementation, generating a feedback control message corresponding to the decoder load state based on the buffer state information further includes: During the decoding loop executed by the decoding Worker thread, undecoded data packets are decoded in preset batches. After completing a batch of decoding, the cumulative playback duration of the queue of undecoded data packets is retrieved again; When the cumulative playback time is less than the second preset threshold, it is determined that the queue of undecoded data packets is in a state of shortage, and a second feedback control message is generated; The second feedback control message includes a message type identifier indicating that more data is needed, the current decoding timestamp, and the cumulative playback duration.

[0011] In one optional implementation, the main thread parses the feedback control message and sends a download control command to the download worker thread based on the parsing result, further comprising: The main thread identifies the message type identifier in the second feedback control message; When the second feedback control message is detected, the main thread determines the download recovery position based on the current decoding timestamp. A resume download control command is generated based on the resume download location, and the resume download control command is sent to the download worker thread so that the download worker thread can resume the video stream data acquisition task or execute the breakpoint resume download.

[0012] In an optional implementation, before generating the first feedback control message or the second feedback control message, the method further includes: Get the next allowed sending time corresponding to the current feedback control message; Sending the current feedback control message is prohibited if the current time has not reached the next allowed sending time; When the current time reaches the next allowed transmission time, the current feedback control message is allowed to be sent, and the next allowed transmission time is updated according to the preset anti-jitter interval.

[0013] In one optional implementation, acquiring video stream data through the download worker thread specifically includes: The video stream data is obtained by calling the corresponding data loader according to the video stream protocol type. The video stream protocol type includes at least one of HLS protocol, FLV protocol, WebSocket protocol, HTTP protocol and private protocol. When transmitting video stream data to the decoding Worker thread, Transferable Objects are used to transfer ArrayBuffer data between threads.

[0014] In one optional implementation, the method further includes: The main thread maintains a queue of decoded video frames and calculates the cumulative playback time of the queue of decoded video frames. Determine whether the current playback source is a real-time streaming media type and whether the player is in real-time playback mode; When the current playback source is a real-time streaming media type, the player is in real-time playback mode, and the cumulative playback duration of the decoded video frame buffer queue is greater than the frame-tracking trigger threshold, the playback consumption speed is increased to accelerate the consumption of video frame data in the decoded video frame buffer queue. When the cumulative playback duration of the decoded video frame buffer queue falls below the standard buffer threshold, the playback consumption speed will be restored to the basic playback speed.

[0015] This disclosure also provides a video playback device based on Web Worker, including: The thread creation module is used to create a download worker thread for performing video stream data download tasks and a decoding worker thread for performing video data decoding tasks during the player initialization phase. The video stream acquisition module is used to acquire video stream data through the download worker thread and send the video stream data to the decoding worker thread; The decoding module is used to decode the received video stream data through the decoding Worker thread, and to obtain the buffer status information of the undecoded data packet queue inside the decoder during the decoding process; The feedback module is used to generate a feedback control message corresponding to the decoder load state based on the buffer state information, and send the feedback control message to the main thread; The download control module is used to parse the feedback control message through the main thread and send download control instructions to the download worker thread according to the parsing result, so as to adjust the video stream data acquisition status of the download worker thread; The rendering and playback module is used to return the decoded video frame data to the main thread through the decoding worker thread, and to render and play the video frame data through the main thread.

[0016] This disclosure also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the above-described Web Worker-based video playback method, or any possible implementation of the above-described Web Worker-based video playback method.

[0017] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described Web Worker-based video playback method, or any possible implementation thereof.

[0018] This disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the above-described Web Worker-based video playback method, or the steps in any possible implementation of the above-described Web Worker-based video playback method.

[0019] This disclosure provides a video playback method and apparatus based on Web Workers. By creating a download worker thread and a decoding worker thread, the video stream data download and decoding tasks are separated from the main thread. The decoding worker thread obtains the buffer status information of the undecoded data packet queue and generates feedback control messages based on this information. The main thread then coordinates the download worker thread to perform control operations such as pausing download, resuming download, or resuming interrupted downloads. This allows for the suppression of further data input when undecoded data packets accumulate, timely resumption of downloads when undecoded data is insufficient, and, combined with frame buffering and frame tracking control in real-time streaming scenarios, reduces playback latency. This reduces memory consumption risks, avoids decoding starvation and playback stuttering, and improves the smoothness, stability, and real-time performance of video playback on the browser side.

[0020] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a video playback method based on Web Worker provided in an embodiment of this disclosure is shown; Figure 2 A flowchart is shown below illustrating another Web Worker-based video playback method provided in this disclosure embodiment; Figure 3 A schematic diagram of a Web Worker-based video playback device provided in an embodiment of this disclosure is shown; Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] Research has revealed that existing multi-worker video playback solutions typically only achieve unidirectional data transmission from the download worker to the decoding worker. The download worker continuously sends video stream data to the decoding worker according to network download capacity, while the decoding worker primarily passively receives and decodes the data. Due to the lack of a control mechanism for feedback on the decoding load status from the decoder side to the downloader side, dynamic matching between download speed and decoding capacity is difficult to achieve. When the network download speed exceeds the decoder's processing capacity, undecoded video data packets can easily accumulate upstream of the decoder, consuming a large amount of memory and potentially causing memory overflow. Conversely, when network fluctuations or insufficient download speed occur, the decoder may idle due to insufficient data to decode, leading to playback stuttering.

[0027] Based on the above research, this disclosure provides a video playback method and apparatus based on Web Workers. By creating a download worker thread and a decoding worker thread, the video stream data download and decoding tasks are separated from the main thread. The decoding worker thread obtains the buffer status information of the undecoded data packet queue and generates feedback control messages based on this information. The main thread then coordinates the download worker thread to perform control operations such as pausing download, resuming download, or resuming interrupted downloads. This allows for the suppression of further data input when undecoded data packets accumulate, timely resumption of downloads when undecoded data is insufficient, and, combined with frame buffering frame tracking control in real-time streaming scenarios, reduces playback latency, thereby reducing memory consumption risks, avoiding decoding starvation and playback stuttering, and improving the smoothness, stability, and real-time performance of video playback on the browser side.

[0028] To facilitate understanding of this embodiment, a detailed description of the Web Worker-based video playback method disclosed in this disclosure is provided first. The execution entity of the Web Worker-based video playback method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this Web Worker-based video playback method can be implemented by a processor calling computer-readable instructions stored in memory.

[0029] See Figure 1 The diagram shows a flowchart of a video playback method based on Web Worker provided in this embodiment of the present disclosure. The method includes steps S101 to S106, wherein: S101. During the player initialization phase, a download worker thread is created to perform video stream data download tasks and a decoding worker thread is created to perform video data decoding tasks.

[0030] In practical implementation, when the player receives a video playback start command, the main thread first performs player initialization operations. These initialization operations may include reading playback parameters, parsing playback addresses, identifying video stream types, initializing the rendering canvas, and creating Worker threads. Based on preset Worker script addresses, the main thread creates download Worker threads and decoding Worker threads using a Worker constructor, separating the video stream data download and decoding tasks from the main thread so that they are executed independently by different background threads.

[0031] The Download Worker thread is used to perform network acquisition tasks for video stream data. After creating the Download Worker thread, the main thread can send a download initialization message to it. This message may contain at least one of the following: video stream address, video stream protocol type, whether it is a real-time streaming media, start playback time, request header information, authentication information, and download control parameters. Upon receiving the download initialization message, the Download Worker thread selects the corresponding data loading method based on the video stream protocol type and initiates a video stream data acquisition operation based on the video stream address.

[0032] In specific implementations, the video stream protocol type can include at least one of the following: HLS, FLV, WebSocket, HTTP, and proprietary protocols. For example, when the video stream protocol type is HLS, the download worker thread can obtain and parse the M3U8 playlist and download the corresponding TS segment data according to the M3U8 playlist; when the video stream protocol type is FLV or HTTP, the download worker thread can continuously obtain streaming media data through HTTP requests; when the video stream protocol type is WebSocket or proprietary, the download worker thread can establish a connection according to the corresponding protocol format and receive video stream data sent by the device or server.

[0033] Here, the Decoding Worker thread is used to perform video data decoding tasks. After creating the Decoding Worker thread, the main thread can send a decoding initialization message to it. This message may carry at least one of the following: video encoding format, decoder initialization parameters, screen size information, hardware or software decoding selection parameters, and WebAssembly decoding module loading parameters. Upon receiving the initialization message, the Decoding Worker thread loads the corresponding decoding module and completes decoder initialization to decode subsequently received video stream data.

[0034] Furthermore, after the download worker thread acquires the video stream data, it can send the video stream data to the main thread, which then forwards it to the decoding worker thread; alternatively, the download worker thread can transmit the video stream data to the decoding worker thread through a pre-defined thread communication link. The video stream data can be encapsulation format data, segmented data, video frame data, or data packets to be decoded. Upon receiving the video stream data, the decoding worker thread writes it into the data queue to be processed corresponding to the decoder and performs decoding processing on the video data in the data queue according to the decoding process.

[0035] It should be noted that, to reduce the memory copying overhead during inter-thread data transfer, when the download worker thread sends large amounts of video stream data to the main thread or decoding worker thread, TransferableObjects can be used to transfer data of type ArrayBuffer. By transferring ownership of the underlying data buffer between different threads, the repeated copying of video stream data during thread communication can be reduced, thereby reducing memory usage and CPU overhead, and improving the data processing efficiency of video playback on the browser side.

[0036] In this way, a playback processing structure is formed with division of labor among the download worker thread, the decoding worker thread, and the main thread. The download worker thread is mainly responsible for acquiring video stream data, the decoding worker thread is mainly responsible for decoding video data, and the main thread is mainly responsible for playback control, message coordination, image rendering, and user interaction. This avoids computationally intensive tasks such as downloading and decoding occupying the main thread for extended periods, improving the main thread's responsiveness, and providing a thread foundation for subsequent download pause, resume, and breakpoint restart control based on decoder buffer status feedback.

[0037] S102. Obtain video stream data through the download worker thread and send the video stream data to the decoding worker thread.

[0038] In practice, after the download worker thread and decoding worker thread are created, the main thread sends a download start message to the download worker thread based on the playback address and playback parameters of the video to be played. The download start message may include at least one of the following: video stream address, video stream protocol type, starting download position, current playback time, whether it is a real-time stream identifier, authentication parameters, and request configuration parameters. Upon receiving the download start message, the download worker thread selects the corresponding data acquisition method based on the video stream protocol type and acquires the video stream data based on the video stream address.

[0039] In its implementation, the download worker thread can support data acquisition from multiple video streaming protocols. For example, when the video streaming protocol is HLS, the download worker thread obtains the M3U8 playlist and downloads the corresponding TS segments sequentially according to the playlist; when the video streaming protocol is FLV or HTTP, the download worker thread continuously receives FLV stream data or other media stream data via HTTP requests; when the video streaming protocol is WebSocket, the download worker thread establishes a WebSocket connection with the server or device and continuously receives video streaming data through this connection; when the video streaming protocol is a proprietary protocol, the download worker thread receives and processes the video streaming data according to the corresponding proprietary protocol parsing rules. This allows the same player architecture to be compatible with multiple video streaming data sources.

[0040] Here, after the download worker thread acquires the video stream data, it can perform preliminary processing on the video stream data to obtain video data units to be transmitted. These video data units can be video segment data, encapsulation format data, raw bitstream data, data packets to be decoded, or binary data blocks used to compose the data packets to be decoded. For real-time streaming media data, the download worker thread can divide the continuously acquired data into multiple data blocks according to the receiving order, and configure corresponding time information, sequence number information, or stream status information for each data block so that the decoding worker thread can perform decoding processing in the correct order.

[0041] In one optional implementation, the download worker thread sends the video stream data to the main thread, which then acts as a coordinator to forward the data to the decoding worker thread. This method allows the main thread to centrally manage the download status, decoding status, playback control status, and thread communication status. In another optional implementation, the download worker thread can also directly or indirectly transmit the video stream data to the decoding worker thread via a pre-defined thread communication link, reducing the number of times the main thread participates in data forwarding. Regardless of the method used, after receiving the video stream data, the decoding worker thread writes it into the corresponding pending data queue of the decoder for use in subsequent decoding operations.

[0042] Furthermore, when the download worker thread sends video stream data to the main thread or the decoding worker thread, a message communication mechanism can be used to transmit data messages. The data message may include at least one of the following: message type identifier, video stream data body, data length, data sequence number, data timestamp, identifier of whether it is a keyframe, identifier of whether it is the last segment of data, and an abnormal state identifier. By carrying the above information in the data message, the receiving thread can accurately identify the type and processing method of the current data, avoiding confusion in data processing under different protocols, different segments, or different playback states.

[0043] Furthermore, when the video stream data is binary data of type ArrayBuffer, the download worker thread can use Transferable Objects technology to transfer the ArrayBuffer as a transferable object between threads. By transferring ownership of the underlying data buffer instead of copying the video stream data, memory copying caused when transferring large amounts of video stream data between threads can be reduced, lowering memory usage and CPU overhead, and improving the transmission efficiency of video stream data from the download end to the decoding end.

[0044] Furthermore, the download worker thread can also support download status control. When the main thread sends a pause download control command to the download worker thread, the download worker thread pauses its current video stream data acquisition task and stops inputting new video stream data to the decoding worker thread. When the main thread sends a resume download control command to the download worker thread, the download worker thread re-initiates the data acquisition operation based on the current playback time point, resume download position, or continuation address carried in the resume download control command, and continues to send the acquired video stream data to the decoding worker thread. Thus, the data input process of the download worker thread can be scheduled and controlled by the main thread, providing an execution basis for subsequent pause and resume downloads based on the decoder buffer status.

[0045] In this way, the download worker thread is responsible for continuously acquiring video stream data from the network side, while the decoding worker thread is responsible for receiving the video stream data and entering the decoding process. The main thread can then perform necessary message coordination and state scheduling between the two. This approach separates the network download task and the video decoding task into different worker threads, reducing the load on the main thread and improving the efficiency of video stream data processing.

[0046] S103. The received video stream data is decoded by the decoding worker thread, and the buffer status information of the undecoded data packet queue inside the decoder is obtained during the decoding process.

[0047] In practice, after initialization, the decoding worker thread receives video stream data forwarded from the download worker thread or the main thread. This video stream data can be video segment data, encapsulation format data, raw bitstream data, or data packets to be decoded. Upon receiving the video stream data, the decoding worker thread can first perform format parsing, data verification, or encapsulation layer splitting to obtain data packets that can be input into the decoder, and then write these data packets into the decoder's internal queue of undecoded data packets.

[0048] In its implementation, the decoding worker thread can call the underlying decoding module to decode the data packets to be decoded in the queue of undecoded data packets. This underlying decoding module can be a WebAssembly decoding module, which can be compiled from a decoder written in C / C++ to support decoding of H.264, H.265, or other video encoding formats. The decoding worker thread reads the data packets to be decoded one by one from the queue of undecoded data packets by calling the decoding interface of the WebAssembly decoding module, and outputs the corresponding decoded video frame data.

[0049] Furthermore, during the decoding process executed by the decoding worker thread, the buffer status information of the queue of undecoded data packets inside the decoder is obtained. This buffer status information is used to characterize the backlog or shortage of data to be processed upstream of the decoder. Unlike methods that only monitor the buffer queue of decoded video frames, this implementation obtains the status of the queue of undecoded data packets inside the decoder that have not yet been decoded. Therefore, it is possible to determine the decoder load status earlier, after data enters the decoder and before a decoded frame is formed.

[0050] In one optional implementation, the decoding worker thread encapsulates a buffer status monitoring interface, which is used to obtain the cumulative playback duration of the undecoded data packet queue. After receiving video stream data and writing it to the undecoded data packet queue, the decoding worker thread can call the buffer status monitoring interface to read the total playback duration corresponding to all undecoded data packets in the current undecoded data packet queue. The cumulative playback duration can be expressed in milliseconds and used as the buffer status information.

[0051] In another optional implementation, the buffer status monitoring interface calls a native method provided by the WebAssembly decoding module to directly query the queue of undecoded data packets within the decoder. That is, the decoding worker thread does not indirectly determine the buffer status by estimating the amount of downloaded data or the number of decoded frames, but instead directly obtains the cumulative playback duration of the undecoded data packet queue from the underlying decoding module. If the underlying decoding module returns a null value, an invalid value, or the queue is empty, the cumulative playback duration can be set to zero to avoid abnormal return values ​​affecting subsequent control decisions.

[0052] Furthermore, the decoding worker thread can obtain the buffer status information at multiple processing nodes. For example, after the decoding worker thread receives externally input video stream data and writes it to the undecoded data packet queue, it obtains the cumulative playback duration of the undecoded data packet queue to determine whether new input data has caused buffer backlog; after the decoding worker thread executes the decoding loop according to a preset batch, it obtains the cumulative playback duration of the undecoded data packet queue again to determine whether the current queue is approaching a data shortage state due to continuous decoding. By obtaining buffer status information separately during the data input and decoding consumption stages, the dynamic changes between data input and data consumption within the decoder can be reflected more accurately.

[0053] Furthermore, the buffer status information, in addition to including the cumulative playback duration of the undecoded data packet queue, may also include at least one of the following: the number of undecoded data packets, queue space occupied, current decoding timestamp, last write time, last decoding time, decoder running status, and an identifier indicating whether the data source has completed receiving data. By combining the above information, it is possible to more comprehensively determine whether the decoder is in a normal processing state, a data backlog state, a data shortage state, or a data completion state.

[0054] In this way, the decoding worker thread can not only complete the decoding of video stream data, but also obtain the buffer status information of the undecoded data packet queue inside the decoder in real time during the decoding process. Since this buffer status information directly reflects the load of the data to be processed upstream of the decoder, it can be used as the basis for generating feedback control messages, enabling the player to dynamically control the video stream data input process of the download worker thread according to the actual processing capacity of the decoder.

[0055] S104. Generate a feedback control message corresponding to the decoder load state based on the buffer state information, and send the feedback control message to the main thread.

[0056] In practical implementation, after obtaining the buffer status information of the queue of undecoded data packets inside the decoder, the decoding worker thread determines the current decoder load status based on the buffer status information. The decoder load status can include at least one of the following: normal state, backlog state, missing data state, and data completion state. The backlog state indicates that there is too much undecoded data inside the decoder, and the download input speed is greater than the decoding consumption speed; the missing data state indicates that there is insufficient data to be decoded inside the decoder, the decoding consumption speed is greater than the download input speed, or there are fluctuations in network input.

[0057] In one optional implementation, the buffer status information includes the cumulative playback duration of the undecoded data packet queue. The decoding worker thread compares the cumulative playback duration with a first preset threshold. When the cumulative playback duration exceeds the first preset threshold, it determines that the undecoded data packet queue is in a backlog state and generates a first feedback control message. The first feedback control message is used to indicate that the decoder-side data is full or nearly full, so that the main thread can subsequently control the download worker thread to pause or reduce the video stream data input.

[0058] Furthermore, the first feedback control message may include a first message type identifier and the current cumulative playback duration. The first message type identifier is used to indicate that the current feedback control message is a message indicating that the decoded data is full, and the current cumulative playback duration is used to indicate the total playback duration corresponding to the remaining undecoded data packets in the queue. By carrying the current cumulative playback duration in the first feedback control message, the main thread can know the data backlog level on the decoder side and perform download pause, download speed limit, or other traffic suppression processing accordingly.

[0059] In another optional implementation, the decoding worker thread decodes undecoded data packets in preset batches during the decoding loop, and reacquires the cumulative playback duration after completing one batch of decoding. The decoding worker thread compares the reacquired cumulative playback duration with a second preset threshold. When the cumulative playback duration is less than the second preset threshold, it determines that the undecoded data packet queue is in a shortage state and generates a second feedback control message. The second feedback control message is used to indicate that the decoder side needs more video stream data so that the main thread can subsequently control the download worker thread to resume downloading or execute breakpoint resume.

[0060] Furthermore, the second feedback control message may include a second message type identifier, a current decoding timestamp, and a current cumulative playback duration. The second message type identifier indicates that the current feedback control message requires more data; the current decoding timestamp indicates the playback time point currently processed by the decoder; and the current cumulative playback duration indicates the amount of data remaining to be decoded within the decoder. By carrying the current decoding timestamp in the second feedback control message, the main thread can determine the download resumption position based on the current decoding timestamp, thereby allowing the download worker thread to continue acquiring subsequent video stream data based on the current playback progress, reducing duplicate downloads or data gaps.

[0061] In one optional implementation, when the decoding worker thread determines that the queue of undecoded data packets is in a state of shortage, it can further determine whether the video stream data source has been completely received. If the video stream data source has been completely received and the cumulative playback time of the queue of undecoded data packets is zero, it can be determined that the current video data has been decoded, and the decoding worker thread can pause the decoding job or generate a data completion status message; if the video stream data source has not been completely received, the second feedback control message is generated to request the main thread to coordinate the download worker thread to continue inputting video stream data.

[0062] Furthermore, to avoid frequent triggering of feedback control messages near the threshold, which would cause the download worker thread to repeatedly pause and resume, the decoding worker thread can perform debouncing control when generating or sending feedback control messages. Specifically, the decoding worker thread can maintain a first allowed next transmission time corresponding to the first feedback control message and a second allowed next transmission time corresponding to the second feedback control message. When the current time has not reached the corresponding next allowed transmission time, the corresponding feedback control message will not be sent repeatedly, even if the buffer state information meets the triggering condition; when the current time reaches the corresponding next allowed transmission time, the corresponding feedback control message is allowed to be sent, and the next allowed transmission time is updated according to the preset debouncing interval.

[0063] For example, after sending the first feedback control message, the retransmission of new first feedback control messages can be prohibited within the first stabilization interval to avoid frequent triggering of download pause control due to the undecoded data packet queue remaining at a high level; similarly, after sending the second feedback control message, the retransmission of new second feedback control messages can be prohibited within the second stabilization interval to avoid frequent triggering of download resumption control due to the undecoded data packet queue remaining at a low level. The first and second stabilization intervals can be set according to the video stream type, network environment, decoding performance, or playback strategy.

[0064] During the actual transmission process, the decoding worker thread can send the feedback control message to the main thread through the thread message communication interface. The feedback control message can be transmitted in the form of an object message, a structured message, or a binary message, and must include at least a message type identifier so that the main thread can identify the control meaning corresponding to the feedback control message based on the message type identifier. After receiving the feedback control message, the main thread can route the feedback control message to the corresponding download control processing logic according to the message type identifier.

[0065] In this way, the decoding worker thread no longer simply acts as a passive decoding thread receiving video stream data. Instead, during the decoding process, it actively generates feedback control messages based on the buffer status of the undecoded data packet queue and sends these messages to the main thread. This allows the main thread to promptly detect data backlog or shortages on the decoder side, providing a basis for subsequently sending pause or resume download control commands to the download worker thread. This creates a download traffic adjustment mechanism driven by the decoder's load status.

[0066] S105. The main thread parses the feedback control message and sends a download control command to the download worker thread based on the parsing result, so as to adjust the video stream data acquisition status of the download worker thread.

[0067] In practice, the main thread acts as a coordinator between the download worker thread and the decoding worker thread, listening for feedback control messages sent by the decoding worker thread. Upon receiving the feedback control message, the main thread parses it, obtaining at least one of the following: message type identifier, cumulative playback duration of the undecoded data packet queue, current decoding timestamp, and other status parameters. Based on the message type identifier, the main thread determines the corresponding download control strategy.

[0068] In one optional implementation, when the main thread parses the feedback control message as a first feedback control message, it determines that the decoder's internal queue of undecoded data packets is in a backlog state. The first feedback control message can be a "decoding data full" message, indicating that the amount of data to be decoded by the decoding worker thread has exceeded the high-water mark control condition. At this time, the main thread generates a pause download control command and sends it to the download worker thread. Upon receiving the pause download control command, the download worker thread pauses its current video stream data acquisition task or stops sending new video stream data to the decoding worker thread, thereby reducing the continued backlog of data upstream of the decoder.

[0069] In a specific implementation, the pause download control command may include at least one of the following: a pause request identifier, a current playback address identifier, a current playback session identifier, and a pause reason parameter. The download worker thread enters a paused state based on the pause request identifier. In the paused state, the download worker thread can suspend network streaming tasks, pause fragment requests, stop reading WebSocket data, or stop sending data blocks that have been acquired but not yet distributed.

[0070] In another optional implementation, when the main thread parses the feedback control message as a second feedback control message, it determines that the decoder's internal queue of undecoded data packets is in a shortage state. The second feedback control message can be a "more data needed" message, indicating that the amount of data to be decoded by the decoding worker thread is below the low-water mark control condition. At this time, the main thread generates a resume download control command and sends it to the download worker thread. Upon receiving the resume download control command, the download worker thread resumes the video stream data acquisition task and continues to input video stream data to the decoding worker thread to avoid the decoder idling due to insufficient data to be decoded.

[0071] Furthermore, when generating the resume download control command, the main thread can prioritize reading the current decoding timestamp carried in the second feedback control message and use this timestamp as the basis for determining the resume download position. When the second feedback control message does not carry the current decoding timestamp, the main thread can read the last decoding time point recorded locally as the resume download position. Based on the resume download position, the identifier indicating whether the current playback stream is a real-time stream, and the original playback address, the main thread generates a resume address or resume download parameters and sends them to the download worker thread in the resume download control command.

[0072] In a specific implementation, the resume download control command may include at least one of the following: a resume request identifier, a resume download position, the current decoding timestamp, an identifier indicating whether it is a real-time stream, a resume address, a playback session identifier, and request configuration parameters. The download worker thread reactivates the network connection or re-initiates the download request based on the resume download control command, and continues to acquire data from the video stream data corresponding to the resume download position. For video streams that support time-point jumps or breakpoint resumes, the download worker thread can perform time offset processing on the playback address based on the resume download position, thereby achieving precise resume based on the current playback progress.

[0073] Furthermore, the main thread can also validate the validity of feedback control messages when parsing them. For example, the main thread can verify whether the feedback control message belongs to the current playback session, whether the message type identifier is valid, whether the current decoding timestamp falls within a valid time range, whether the cumulative playback duration is a valid value, and whether the feedback control message matches the current download worker thread state. If the feedback control message does not meet the validation conditions, the main thread can discard the feedback control message or record the abnormal state to prevent error messages from causing the download worker thread to pause or resume unintentionally.

[0074] Furthermore, the main thread can maintain the current download state of the download worker threads, which includes at least one of the following: downloading, paused, resumed, reconnected, and terminated. When the main thread receives the first feedback control message, if the download worker thread is already in a paused state, it does not need to resend the pause download control command; when the main thread receives the second feedback control message, if the download worker thread is already in a downloading state, it does not need to resend the resume download control command. By combining the current download state of the download worker threads for control, the issuance of duplicate commands can be reduced, and the stability of thread control can be improved.

[0075] In this way, the main thread can identify, route, and transform feedback control messages based on the decoder load status reported by the decoding worker thread, and issue corresponding pause or resume download control commands to the download worker thread. Consequently, the video stream data acquisition status of the download worker thread can be dynamically adjusted according to changes in the queue of undecoded data packets within the decoder, ensuring that the video stream data input speed matches the decoding processing capacity. This avoids data backlog due to excessively fast downloads and decoding starvation and playback stuttering due to insufficient downloads.

[0076] S106. The decoded video frame data is returned to the main thread through the decoding worker thread, and the main thread renders and plays the video frame data.

[0077] In practice, after decoding the received video stream data, the Decoding Worker thread generates decoded video frame data corresponding to the video stream data. The decoded video frame data may include at least one of the following: image frame data, frame timestamp, frame sequence number, frame width and height information, pixel format information, keyframe identifier, and playback duration information. The Decoding Worker thread encapsulates the decoded video frame data into a frame data message and sends it to the main thread through the thread message communication interface.

[0078] Here, after receiving the frame data message, the main thread parses the message to obtain the decoded video frame data and its corresponding frame time information. The main thread can write the decoded video frame data into a decoded video frame buffer queue, which is used to cache video frames waiting to be rendered. Based on the current playback state, playback timeline, and frame timestamps, the main thread sequentially retrieves the video frame data to be rendered from the decoded video frame buffer queue and renders it onto the Canvas or other video display areas, thereby achieving continuous playback of the video.

[0079] In the specific implementation, the main thread can determine the rendering timing of a video frame based on the frame timestamp of the video frame data and the current playback time of the player. When the frame timestamp of the video frame to be rendered matches the current playback time, or when the preset rendering time condition is met, the main thread submits the video frame to the rendering module for image drawing; when the frame timestamp of the video frame is significantly earlier than the current playback time, the main thread can discard the video frame according to the playback strategy to avoid image delay caused by rendering lag; when the frame timestamp of the video frame is later than the current playback time, the main thread can keep the video frame in the decoded video frame buffer queue for subsequent rendering.

[0080] See Figure 2 The diagram shows another video playback method based on Web Worker provided in this disclosure, the method including steps S201-S204, wherein: S201. Maintain the decoded video frame buffer queue through the main thread, and calculate the cumulative playback time of the decoded video frame buffer queue.

[0081] S202. Determine whether the current playback source is a real-time streaming media type and whether the player is in real-time playback mode.

[0082] S203. When the current playback source is a real-time streaming media type, the player is in real-time playback mode, and the cumulative playback duration of the decoded video frame buffer queue is greater than the frame-tracking trigger threshold, the playback consumption speed is increased to accelerate the consumption of video frame data in the decoded video frame buffer queue.

[0083] S204. When the cumulative playback duration of the decoded video frame buffer queue falls below the standard buffer threshold, the playback consumption speed is restored to the basic playback speed.

[0084] In practice, the main thread can continuously maintain the queue status of the decoded video frame buffer queue. The queue status may include at least one of the following: the number of decoded video frames, the cumulative playback duration of video frames in the queue, the earliest video frame timestamp, the latest video frame timestamp, and the current buffer latency.

[0085] In one optional implementation, the main thread is configured with a minimum buffer threshold, a standard buffer threshold, and a maximum buffer threshold. When the cumulative playback duration of the decoded video frame buffer queue is less than the minimum buffer threshold, the main thread can temporarily postpone rendering or remain in a waiting state to reduce playback stuttering caused by insufficient frame data. When the cumulative playback duration reaches the standard buffer threshold, the main thread performs normal rendering and playback at the basic playback speed. When the cumulative playback duration exceeds the maximum buffer threshold, the main thread can perform frame dropping, accelerated playback, or other buffer digestion processes based on the playback scenario to prevent playback latency from continuously increasing.

[0086] Furthermore, in real-time streaming playback scenarios, the main thread can perform frame-tracking control based on the decoded video frame buffer queue. Specifically, the main thread determines whether the current playback source is a real-time streaming media type and whether the player is in real-time playback mode. When the current playback source is a real-time streaming media type and the player is in real-time playback mode, the main thread calculates the cumulative playback duration of the decoded video frame buffer queue; when the cumulative playback duration exceeds the frame-tracking trigger threshold, the main thread increases the playback consumption speed to accelerate the consumption of video frame data in the decoded video frame buffer queue, thereby reducing the end-to-end latency of real-time streaming playback.

[0087] In one specific implementation, the main thread can increase the playback consumption speed to a preset multiple of the base playback speed when the cumulative playback duration of the decoded video frame buffer queue exceeds the frame-tracking trigger threshold; when the cumulative playback duration of the decoded video frame buffer queue falls back below the standard buffer threshold, the main thread restores the playback consumption speed to the base playback speed.

[0088] Furthermore, the main thread continues to listen for messages from the decoding worker thread and the download worker thread during rendering and playback. When the queue of decoded video frames continuously decreases, the main thread can determine whether to resume downloading based on the status of the queue of undecoded data packets reported by the decoding worker thread. When the queue of decoded video frames continuously increases and the queue of undecoded data packets is also at a high level, the main thread can maintain or trigger pause control on the download worker thread. Thus, the control of the decoded frame buffer and the feedback control of the undecoded data packets can work together.

[0089] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0090] Based on the same inventive concept, this disclosure also provides a Web Worker-based video playback device corresponding to the Web Worker-based video playback method. Since the principle of the device in this disclosure for solving the problem is similar to the Web Worker-based video playback method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0091] Please see Figure 3 , Figure 3 This is a schematic diagram of a video playback device based on Web Worker, provided as an embodiment of this disclosure. Figure 3 As shown in the illustration, the Web Worker-based video playback device 300 provided in this embodiment includes: The thread creation module 310 is used to create a download worker thread for performing video stream data download tasks and a decoding worker thread for performing video data decoding tasks during the player initialization phase.

[0092] The video stream acquisition module 320 is used to acquire video stream data through the download worker thread and send the video stream data to the decoding worker thread.

[0093] The decoding module 330 is used to decode the received video stream data through the decoding worker thread, and to obtain the buffer status information of the undecoded data packet queue inside the decoder during the decoding process.

[0094] Feedback module 340 is used to generate a feedback control message corresponding to the decoder load state based on the buffer state information, and send the feedback control message to the main thread.

[0095] The download control module 350 is used to parse the feedback control message through the main thread and send download control instructions to the download worker thread according to the parsing result, so as to adjust the video stream data acquisition status of the download worker thread.

[0096] The rendering and playback module 360 ​​is used to return the decoded video frame data to the main thread through the decoding Worker thread, and to render and play the video frame data through the main thread.

[0097] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0098] This disclosure provides a video playback method and apparatus based on Web Workers. By creating a download worker thread and a decoding worker thread, the video stream data download and decoding tasks are separated from the main thread. The decoding worker thread obtains the buffer status information of the undecoded data packet queue and generates feedback control messages based on this information. The main thread then coordinates the download worker thread to perform control operations such as pausing download, resuming download, or resuming interrupted downloads. This allows for the suppression of further data input when undecoded data packets accumulate, timely resumption of downloads when undecoded data is insufficient, and, combined with frame buffering and frame tracking control in real-time streaming scenarios, reduces playback latency. This reduces memory consumption risks, avoids decoding starvation and playback stuttering, and improves the smoothness, stability, and real-time performance of video playback on the browser side.

[0099] Corresponding to Figure 1 and Figure 2 In addition to the Web Worker-based video playback method, this disclosure also provides an electronic device 400, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 400 provided in an embodiment of this disclosure, including: Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including main memory 421 and external memory 422; main memory 421, also called internal memory, is used to temporarily store the computational data in processor 41, as well as the data exchanged with external memory 422 such as hard disk. Processor 41 exchanges data with external memory 422 through main memory 421. When the electronic device 400 is running, processor 41 and memory 42 communicate through bus 43, enabling processor 41 to execute... Figure 1 and Figure 2 The steps of the Web Worker-based video playback method.

[0100] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the Web Worker-based video playback method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0101] This disclosure also provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can perform the steps of the video playback method based on Web Worker described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0102] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0103] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A video playback method based on Web Worker, characterized in that, include: During the player initialization phase, a download worker thread is created to perform video stream data download tasks and a decoding worker thread is created to perform video data decoding tasks. The download worker thread acquires video stream data and sends the video stream data to the decoding worker thread. The received video stream data is decoded by the Decoding Worker thread, and the buffer status information of the undecoded data packet queue inside the decoder is obtained during the decoding process. Based on the buffer state information, a feedback control message corresponding to the decoder load state is generated, and the feedback control message is sent to the main thread; The main thread parses the feedback control message and sends a download control command to the download worker thread based on the parsing result, so as to adjust the video stream data acquisition status of the download worker thread. The decoded video frame data is returned to the main thread through the decoding worker thread, and the video frame data is rendered and played through the main thread. Obtain the buffer status information of the queue of undecoded data packets inside the decoder, specifically including: The decoding worker thread calls the buffer status monitoring interface provided by the underlying decoding module. The cumulative playback duration is used as buffer state information to characterize the upstream load state of the decoder; The underlying decoding module is a WebAssembly decoding module, and the buffer status monitoring interface is used to call the queue query method implemented by the C / C++ decoder to obtain the cumulative playback duration of the queue of undecoded data packets. When the queue query method returns an empty value, the cumulative playback duration is set to zero.

2. The method according to claim 1, characterized in that, Based on the buffer state information, a feedback control message corresponding to the decoder load state is generated, specifically including: The cumulative playback duration is compared with a first preset threshold. When the cumulative playback duration exceeds the first preset threshold, it is determined that the queue of undecoded data packets is in a backlog state, and a first feedback control message is generated; The first feedback control message includes a message type identifier indicating that the decoding data is full and the cumulative playback duration.

3. The method according to claim 2, characterized in that, The main thread parses the feedback control message and sends download control commands to the download worker thread based on the parsing result, specifically including: The main thread identifies the message type identifier in the first feedback control message; When the first feedback control message is detected, a pause download control command is generated through the main thread; The pause download control command is sent to the download worker thread to cause the download worker thread to pause the current video stream data acquisition task.

4. The method according to claim 2, characterized in that, The process of generating a feedback control message corresponding to the decoder load state based on the buffer state information also includes: During the decoding loop executed by the decoding Worker thread, undecoded data packets are decoded in preset batches. After completing a batch of decoding, the cumulative playback duration of the queue of undecoded data packets is retrieved again; When the cumulative playback time is less than the second preset threshold, it is determined that the queue of undecoded data packets is in a state of shortage, and a second feedback control message is generated; The second feedback control message includes a message type identifier indicating that more data is needed, the current decoding timestamp, and the cumulative playback duration.

5. The method according to claim 4, characterized in that, The main thread parses the feedback control message and sends download control instructions to the download worker thread based on the parsing result. The method also includes: The main thread identifies the message type identifier in the second feedback control message; When the second feedback control message is detected, the main thread determines the download recovery position based on the current decoding timestamp. A resume download control command is generated based on the resume download location, and the resume download control command is sent to the download worker thread so that the download worker thread can resume the video stream data acquisition task or execute the breakpoint resume download.

6. The method according to claim 4, characterized in that, Before generating the first feedback control message or the second feedback control message, the method further includes: Get the next allowed sending time corresponding to the current feedback control message; Sending the current feedback control message is prohibited if the current time has not reached the next allowed sending time; When the current time reaches the next allowed transmission time, the current feedback control message is allowed to be sent, and the next allowed transmission time is updated according to the preset anti-jitter interval.

7. The method according to claim 1, characterized in that, The acquisition of video stream data through the download worker thread specifically includes: The video stream data is obtained by calling the corresponding data loader according to the video stream protocol type. The video stream protocol type includes at least one of HLS protocol, FLV protocol, WebSocket protocol, HTTP protocol and private protocol. When transmitting video stream data to the decoding Worker thread, Transferable Objects are used to transfer ArrayBuffer data between threads.

8. The method according to claim 1, characterized in that, The method further includes: The main thread maintains a queue of decoded video frames and calculates the cumulative playback time of the queue of decoded video frames. Determine whether the current playback source is a real-time streaming media type and whether the player is in real-time playback mode; When the current playback source is a real-time streaming media type, the player is in real-time playback mode, and the cumulative playback duration of the decoded video frame buffer queue is greater than the frame-tracking trigger threshold, the playback consumption speed is increased to accelerate the consumption of video frame data in the decoded video frame buffer queue. When the cumulative playback duration of the decoded video frame buffer queue falls below the standard buffer threshold, the playback consumption speed will be restored to the basic playback speed.

9. A video playback device based on Web Worker, characterized in that, include: The thread creation module is used to create a download worker thread for performing video stream data download tasks and a decoding worker thread for performing video data decoding tasks during the player initialization phase. The video stream acquisition module is used to acquire video stream data through the download worker thread and send the video stream data to the decoding worker thread; The decoding module is used to decode the received video stream data through the decoding Worker thread, and to obtain the buffer status information of the undecoded data packet queue inside the decoder during the decoding process; Obtain the buffer status information of the queue of undecoded data packets inside the decoder, specifically including: The decoding worker thread calls the buffer status monitoring interface provided by the underlying decoding module. The cumulative playback duration is used as buffer state information to characterize the upstream load state of the decoder; The underlying decoding module is a WebAssembly decoding module, and the buffer status monitoring interface is used to call the queue query method implemented by the C / C++ decoder to obtain the cumulative playback duration of the queue of undecoded data packets. When the queue query method returns an empty value, the cumulative playback duration is set to zero; The feedback module is used to generate a feedback control message corresponding to the decoder load state based on the buffer state information, and send the feedback control message to the main thread; The download control module is used to parse the feedback control message through the main thread and send download control instructions to the download worker thread according to the parsing result, so as to adjust the video stream data acquisition status of the download worker thread; The rendering and playback module is used to return the decoded video frame data to the main thread through the decoding worker thread, and to render and play the video frame data through the main thread.

Images