A web multi-video concurrent processing method based on single thread multiplexing and off-screen rendering

Abstract

The application discloses a kind of based on single thread multiplexing and off-screen rendering Web multi-video concurrent processing method, comprising: creating globally unique shared decoding thread, for each video distribution virtual instance identifier and establish the mapping relationship with decoding context, the rendering control right of canvas element is handed over to the shared decoding thread, decoding is executed in the thread based on identifier routing and direct completion graphic rendering, form the closed loop that decoding and drawing are both closed in background;Measure first frame rendering time consumption, limit time trigger fuse degradation makes video instance switch to controlled step mode.The application will thread resource overhead fixation, eliminate pixel data cross thread copy, release main thread rendering load, and realize adaptive degradation alive under extreme load.

Description

Technical Field

[0001] This invention relates to the fields of multimedia stream processing and front-end web application technology, specifically to a web multi-video concurrent processing method based on single-threaded multiplexing and off-screen rendering. Background Technology

[0002] Currently, due to licensing restrictions or hardware ecosystem limitations, mainstream web browsers (such as Chrome and Edge) can only directly decode a few encoding formats, such as H.264, in their native video playback tags. For the H.265 (HEVC) format, which is widely used in the security monitoring field and has a higher compression rate, browsers do not natively support direct playback.

[0003] To enable plugin-free playback of H.265 videos in a browser, a known software decoding solution involves: loading a software decoding library (such as FFmpeg) using WebAssembly technology, and creating a separate background Web Worker thread for each video playback instance on the page; running the WebAssembly decoding module in this separate thread to complete the software decoding; serializing and copying the large-volume pixel frame data (e.g., YUV format data groups) generated by decoding through an inter-thread message passing mechanism, and then sending it back to the front-end main thread; finally, the main thread uses WebGL technology to map the pixel data and draw it onto the canvas object of the interface for display.

[0004] The above solution can run smoothly when playing single-channel video, but it shows obvious shortcomings in typical scenarios such as security monitoring screens and live streaming matrices where multiple video streams are played concurrently (e.g., playing 16, 32, or even 64 streams simultaneously on the same screen). First, the resource overhead of worker threads increases linearly with the number of concurrent streams. Because of the architecture of "one independent worker thread per video stream," as the number of concurrent streams increases, the browser needs to create and maintain an equal number of background threads in real time. This not only easily triggers the browser's or operating system's upper limit on the number of threads, causing thread creation failures or abnormal exits, but also significantly increases the context switching overhead due to frequent CPU scheduling contention among multiple threads. In severe cases, this can lead to memory overflow and cause the page to crash with a blank screen.

[0005] Secondly, the large amount of data copied across threads leads to significant performance overhead. Each independent decoding thread produces a large amount of single-frame pixel data. Under multi-threaded concurrency, this data needs to be serialized and transmitted repeatedly across the "background worker thread" and the "front-end main thread," consuming a high amount of memory bandwidth and constituting a significant performance bottleneck.

[0006] Third, the main thread becomes heavily loaded due to the concentrated redrawing tasks, which in turn slows down the interface interaction. All the final graphics drawing operations of the video screens are concentrated on the browser's single UI main thread. When there are a large number of high-frequency concurrent redrawing requests, the main thread is in a state of full load for a long time, which severely compresses the response time of normal interaction logic, and the user interface exhibits sluggish operation or even "freezing".

[0007] It is evident that existing software decoding solutions are insufficient to meet the actual needs of high-channel video concurrency processing on the Web platform in terms of ensuring system stability, memory bandwidth utilization efficiency, and interface interaction smoothness. The industry urgently needs a Web multi-channel video concurrency processing solution that can substantially reduce system resource consumption and alleviate the main thread rendering load. Summary of the Invention

[0008] The purpose of this invention is to solve the problems of system physical thread resource overload and memory overflow crashes caused by the linear expansion of background threads with the number of complex video streams during concurrent playback on the Web. It also addresses the issues of serialization performance loss and excessive memory bandwidth consumption caused by frequent cross-thread handling of large video pixel frame data, and eliminates the interface interaction lag and response failure caused by the main thread's concentrated handling of multiple video redrawing tasks. Therefore, it provides a Web multi-video concurrent processing method based on single-thread reuse and off-screen rendering. By constructing a fully background streaming rendering closed loop, it significantly reduces system resource consumption while ensuring the operating efficiency, interactive smoothness, and system stability under extreme loads for Web multi-video concurrent processing.

[0009] To achieve the above objectives, this invention provides a Web multi-video concurrent processing method based on single-threaded multiplexing and off-screen rendering, comprising: Obtain the identification information of the target video stream instance and detect whether a globally unique shared decoding thread that supports cross-thread communication has been mounted in the current runtime environment; if not, instantiate a shared decoding thread. Each video stream instance is assigned a unique virtual instance identifier. A logical routing mapping relationship between the virtual instance identifier and the corresponding decoding context is established within the shared decoding thread. The bitstream data with the virtual instance identifier attached is sent to the shared decoding thread through the cross-thread communication interface. Obtain the canvas element responsible for displaying each video stream in the main interface, call the rendering control of the canvas element to obtain the off-screen rendering proxy object through the decoupling interface, and transfer the ownership of the off-screen rendering proxy object to the shared decoding thread in one go; Within the shared decoding thread, based on the virtual instance identifier carried in the received bitstream data, the corresponding decoding context and the off-screen rendering proxy object are retrieved by searching the logical routing mapping relationship, and decoding is performed to obtain pixel data; The pixel data is directly rendered to the video memory associated with the off-screen rendering proxy object using the background graphics drawing context, so that decoding and rendering are completed in the shared decoding thread. Record the start time when the video stream data is sent to the shared decoding thread, and obtain the first frame rendering callback time of the background graphics drawing context; calculate the time difference between the start time and the first frame rendering callback time; When the time difference exceeds a preset health threshold, a circuit breaker command is sent to the corresponding video stream instance to switch it from autonomous loop decoding mode to controlled step mode, so that it only performs single-frame decoding and rendering in response to active step commands issued by the front end.

[0010] Furthermore, the step of transferring ownership of the off-screen rendering proxy object to the shared decoding thread in one go includes: The off-screen rendering proxy object is placed as a transferable object in the message payload of the initialization connection signaling, and the control handle of the off-screen rendering proxy object is transferred to the shared decoding thread through the one-time handshake protocol of the cross-thread communication interface.

[0011] Furthermore, it also includes: The decoded pixel data is retained in the shared decoding thread for background rendering, enabling cross-thread backhaul of zero-pixel data from the main interface. The decoded audio stream data is returned to the main thread through a message channel, and the browser's sound card is driven by the main thread's global audio manager to play the audio.

[0012] Furthermore, sending a circuit breaker command to the corresponding video stream instance to switch it from autonomous loop decoding mode to controlled step mode includes: In response to the circuit breaker command, the video stream instance interrupts the current full-speed decoding loop task sequence and enters a sleep suspension state, entering the controlled step mode.

[0013] Furthermore, in the controlled stepping mode, active pulse control logic is executed: When a step call pulse is received from the front-end main interface according to a preset cycle, the underlying decoding engine is driven to perform a single frame image parsing and video memory projection for this pulse, thereby achieving system survival under extreme load by actively limiting the decoding extraction cycle.

[0014] Furthermore, before sending the circuit breaker command to the corresponding video stream instance, the following steps are also included: Obtain the window size information or preset display priority labels for each video stream instance; Video stream instances with a window size smaller than a preset threshold or a display priority label of low are identified as secondary priority instances, and the circuit breaker command is sent to the secondary priority instances first.

[0015] Furthermore, the health threshold is dynamically determined based on the number of concurrent video stream instances currently carried by the shared decoding thread; when the number of concurrent streams exceeds the preset number, the first frame rendering callback time exhibits a non-linear delay increase relative to the start time.

[0016] Furthermore, in the controlled step mode, the decoding engine only performs intra-frame decoding on the current single-frame image, skipping the continuous parsing of bidirectional prediction frames, so as to control the decoding computational resource consumption within a preset rated range.

[0017] Furthermore, before the shared decoding thread performs decoding on the video stream data, a stream protocol adaptation step is also included: Identify the Private Business Extensions header carried in the received video stream data; The process involves performing a pre-stripping operation on the proprietary business extension identifier header and then re-encapsulating the stripped payload data according to the standard input specifications of the video decoder to convert heterogeneous, non-standard video streams into standard decodeable streams.

[0018] Furthermore, it also includes cache depth control steps based on stream attributes: Parse the live streaming status identifier carried in the initialization connection signaling; If the live streaming status identifier is true, then detect the frame stacking depth in the back buffer associated with the current video stream instance; When the frame accumulation depth exceeds the preset frame loss threshold, the old frame buffer is automatically cleared and the process jumps to the nearest key frame for parsing to compensate for the scheduling delay introduced by single physical thread reuse.

[0019] Compared with the prior art, the present invention has the following significant advantages: By using a routing architecture with a globally unique shared decoding thread and virtual instance identifier, this invention overcomes the drawback of traditional solutions where thread overhead increases linearly with the number of video streams. Regardless of how the total number of concurrent streams connected to the front end increases, the underlying layer maintains only a single physical worker thread, smoothing out the geometrically increasing computational thread load phenomenon into a normalized low resource consumption, effectively avoiding thread creation failures and memory overflow crashes caused by reaching the system thread limit.

[0020] Based on the one-time ownership transfer mechanism of off-screen rendering proxy objects, this invention completely transfers graphics rendering control off the main thread and completes the entire process of stream packet reception, software decoding, and video memory flushing within a background closed loop. By utilizing the background graphics drawing context to process pixel data directly in the GPU video memory, it completely eliminates the serialization and transportation of massive pixel matrices between worker threads and the main thread, significantly reducing the instantaneous load on memory bandwidth.

[0021] Since all video frame redraw operations are undone and processed on the background management bus, the front-end UI main thread is no longer burdened by heavy drawing tasks. This allows the UI thread to be fully dedicated to responding to user clicks, swipes, and other business logic, fundamentally solving the common interface hangs and responsiveness issues in high-concurrency monitoring scenarios.

[0022] By using a rendering latency monitoring model, this invention can detect performance bottlenecks in terminal hardware in real time. When the load exceeds limits, the decoding driver is switched from full-speed self-looping to active pulse triggering via a controlled stepping mode. This mechanism allows the system to sacrifice the continuous frame rate of secondary-priority videos to ensure the availability of core business modules, effectively preventing irreversible page crashes caused by data backlog, and playing a core protection and interception role.

[0023] By adapting the bitstream protocol and stripping away proprietary commercial extension identifiers, this invention can directly achieve web-based compatibility with older monitoring system streams that do not support standard transmission protocols and contain numerous proprietary prefixes. Compared to forcibly building an intermediate network adaptation platform or expensive server-side transcoding, this solution not only reduces the enterprise's hardware and software investment costs but also ensures the real-time performance and synchronization of live streams under a single-threaded architecture through "deep cache control." Attached Figure Description

[0024] Figure 1 This is a flowchart of the Web multi-video concurrent processing method of the present invention; Figure 2 This is a schematic diagram of the architecture of the Web multi-video concurrent processing system of the present invention. Detailed Implementation

[0025] Example 1

[0026] This embodiment provides a web-based multi-video concurrent processing method based on single-threaded multiplexing and off-screen rendering, applicable to multi-channel high-concurrency video scenarios such as security monitoring screens and live streaming matrices on a web browser. The method specifically includes the following steps: S1. Creation and reuse of globally shared decoding threads; Obtain the identification information of the target video stream instance and detect whether a globally unique shared decoding thread that supports cross-thread communication has been mounted in the current runtime environment; if not, instantiate a shared decoding thread.

[0027] In this step, a static global variable is used to detect whether the shared decoding thread has been mounted. This ensures that after the first video playback instance triggers thread creation, subsequent newly added video streams automatically reuse the existing thread without requiring re-initialization. If the shared decoding thread has not yet been established, a unique background shared decoding thread is instantiated and a high-performance video decoder compiled based on WebAssembly is loaded. This thread remains resident throughout its entire lifecycle. Therefore, regardless of the increase in the number of concurrent video streams accessed from the front end, the number of background decoding threads remains constant, fundamentally avoiding the risks of thread creation failure, increased context switching overhead, and memory overflow caused by the linear increase in the number of threads with the number of concurrent streams in existing solutions.

[0028] S2. Virtual instance identifier allocation and logical route mapping establishment; Each video stream instance is assigned a unique virtual instance identifier. A logical routing mapping relationship between the virtual instance identifier and the corresponding decoding context is established within the shared decoding thread. The bitstream data with the virtual instance identifier attached is sent to the shared decoding thread through the cross-thread communication interface.

[0029] In this step, whenever a new video playback instance is created, the system automatically assigns a globally unique virtual instance identifier to that instance and associates and encapsulates this identifier with the subsequently issued bitstream data and control signaling. Simultaneously, a mapping dictionary structure is built within the shared decoding thread, using the virtual instance identifier as the key to store the decoding context object, decoding buffer queue, and off-screen rendering proxy object for that video stream. Through this mapping dictionary, strong data-level isolation is achieved for multiple video decoding instances within a single physical thread, enabling the same thread to handle dozens of independent video decoding tasks in parallel, thus solving the core technical challenge of concurrent processing of multiple instances within a single thread.

[0030] S3. Acquiring and transferring ownership of off-screen rendering proxy objects; Obtain the canvas element responsible for displaying each video stream in the main interface, call the rendering control of the canvas element to obtain the off-screen rendering proxy object through the decoupling interface, and transfer the ownership of the off-screen rendering proxy object to the shared decoding thread in one go.

[0031] In this step, when the front-end main thread mounts the video player, it first checks the browser's operating environment's compatibility with the off-screen canvas interface. Upon confirmation of support, it calls the interface to detach the element's control, completely stripping the canvas element's rendering control from the visual tree and converting it into an off-screen rendering proxy object completely disconnected from the main thread's UI. This off-screen rendering proxy object is then sent to the shared decoding thread along with the initialization connection signaling via a one-time handshake protocol using the cross-thread communication interface, completing the transfer of rendering control ownership. This transfer is a one-time operation; thereafter, the shared decoding thread takes full control of the canvas element's graphics rendering, and the main thread no longer participates in any rendering operations, laying the foundation for a fully background rendering closed loop. In cases where some browser environments do not support the off-screen canvas interface, a data return mode can be switched. The shared decoding thread serializes the decoded pixel data and returns it to the main thread, which then performs the graphics rendering, ensuring overall compatibility across all environments.

[0032] S4. Decoding execution based on identifier routing; Within the shared decoding thread, based on the virtual instance identifier carried in the received bitstream data, the corresponding decoding context and the off-screen rendering proxy object are retrieved by searching the logical routing mapping relationship, and decoding is performed to obtain pixel data.

[0033] In this step, when the shared decoding thread receives the mixed multi-stream data packets, it first parses out the virtual instance identifier attached to the outer layer of each data packet. Using this identifier as an index, it searches the mapping dictionary structure for the independent decoding context object and off-screen rendering proxy object bound to it. Through this routing mechanism, the bitstream data of different video streams is accurately distributed to their respective decoding contexts for independent decoding, obtaining the pixel data of each stream. This routing lookup and distribution process is completed within the shared decoding thread, eliminating the need for inter-thread communication, resulting in high processing efficiency. Furthermore, the isolated management of the decoding contexts ensures that the decoding states of each video stream do not interfere with each other.

[0034] S5, background closed-loop graphic drawing; The pixel data is directly rendered to the video memory associated with the off-screen rendering proxy object using the background graphics drawing context, so that decoding and rendering are completed in the shared decoding thread.

[0035] In this step, after decoding the pixel data, the shared decoding thread does not perform any serialization and backhaul operations on the pixel data to the main thread. Instead, based on the off-screen rendering proxy object corresponding to the video stream, it obtains the graphics drawing context bound to its backend, directly calls the GPU drawing pipeline, and performs color space conversion and flushing / rearrangement of the decoded video frame data in the graphics processor's video memory to the layer video memory of the canvas elements. Thus, the entire process from receiving the bitstream data, software decoding to graphics rendering is completed within the shared decoding thread, enabling zero-byte video pixel frame data to cross the main thread boundary. This closed-loop architecture eliminates the memory bandwidth bottleneck caused by repeated cross-thread copying of large-volume pixel data under multi-channel concurrency, while completely freeing the main thread from the redrawing tasks of all video frames.

[0036] S6. Measurement of first frame rendering time; Record the start time when the video stream data is sent to the shared decoding thread, and obtain the first frame rendering callback time of the background graphics drawing context; calculate the time difference between the start time and the first frame rendering callback time.

[0037] In this step, the start time is defined as the moment the video stream data is transmitted to the shared decoding thread, and the end time is defined as the moment when the video stream first completes graphics rendering and triggers a callback. The difference between the two times is taken as the first frame rendering time. This first frame rendering time directly represents the peak of the overall decoding and rendering capabilities under the current terminal hardware environment. In scenarios where the number of concurrent streams continuously increases, this first frame rendering time exhibits a non-linear latency growth characteristic; that is, the more concurrent streams there are, the faster the rendering latency of each video stream increases. This characteristic can be used to accurately identify the current load state of the system, providing a quantitative basis for subsequent degradation control.

[0038] S7, Adaptive step-down control; When the time difference exceeds a preset health threshold, a circuit breaker command is sent to the corresponding video stream instance to switch it from autonomous loop decoding mode to controlled step mode, so that it only performs single-frame decoding and rendering in response to active step commands issued by the front end.

[0039] In this step, when the first frame rendering time exceeds a preset health threshold, the system determines that the current hardware and software capabilities are insufficient to handle the full-speed decoding and rendering of all video stream instances, and immediately initiates degradation control for the relevant video stream instances. Specifically, a circuit breaker command is sent to the corresponding video stream instance. In response, the instance interrupts its current full-speed decoding loop task sequence and enters a sleep / suspended state, switching from autonomous loop decoding mode to controlled step mode. In controlled step mode, the instance no longer autonomously schedules decoding tasks. Only when it receives a step call pulse initiated by the front-end main interface at a preset cycle does it drive the underlying decoding engine to perform a single-frame image parsing and memory projection for that pulse. By actively limiting the decoding extraction cycle, the system keeps computational resource consumption within a safe range, achieving system survival under extreme loads.

[0040] In a preferred implementation, before sending the circuit breaker command, the window size information or preset display priority tags of each video stream instance can be obtained. Video stream instances with window sizes smaller than a preset threshold or display priority tags of low level are determined as secondary priority instances, and circuit breaker commands are sent to these secondary priority instances first. This selective degradation strategy ensures smooth playback of large-window or high-priority video frames when system resources are scarce, making the degradation strategy more business-appropriate.

[0041] As a preferred implementation, the health threshold can be dynamically determined based on the number of concurrent video stream instances currently carried by the shared decoding thread. When the number of concurrent streams exceeds the preset number, the first frame rendering callback time exhibits a non-linear delay increase relative to the start time. Dynamically adjusting the health threshold can match the non-linear decay characteristics of system capabilities under different concurrency scales, making degradation triggering more accurate.

[0042] In a preferred implementation, under controlled step mode, the decoding engine can perform intra-frame decoding only on the current single-frame image, skipping the continuous parsing of bidirectional prediction frames. Since the decoding of bidirectional prediction frames requires reference to data from previous and subsequent frames, the computational load is far greater than that of intra-frame decoding frames. By skipping the continuous parsing of bidirectional prediction frames, the computational resource consumption of a single decoding can be controlled within a preset rated range, maintaining the basic effective display of the image with minimal computational overhead under extreme load.

[0043] As another implementation, this embodiment also includes asymmetric audio and video processing: the decoded pixel data is retained in the shared decoding thread for background rendering, enabling cross-thread backhaul of zero-pixel data from the main interface; the decoded audio stream data is returned to the main thread via a message channel, and the browser sound card is driven by the main thread's global audio manager for audio playback. Since the audio stream data is much smaller than the pixel data, the impact of separately returning it to the main thread on communication bandwidth and main thread performance is negligible. Simultaneously, audio playback can seamlessly integrate with the main interface's mute control, promiscuous monitoring, and other functions.

[0044] As another implementation, before the shared decoding thread decodes the video stream data, a stream protocol adaptation step is included: identifying the proprietary commercial extension identifier header carried in the received video stream data; performing a pre-stripping action on the proprietary commercial extension identifier header; and re-encapsulating the stripped payload data according to the standard input specification of the video decoder, so that the heterogeneous non-standard video stream is converted into a standard decodeable stream. This step enables this solution to be directly compatible with the large number of old monitoring system streams in existing networks that carry proprietary extension fields from equipment manufacturers, without the need to pre-deploy an intermediate transcoding server for stream standardization processing, thus reducing the system deployment cost and migration threshold.

[0045] As another implementation, a buffer depth control step based on stream attributes is also included: parsing the live broadcast status identifier carried in the initialization connection signaling; if the live broadcast status identifier is true, then detecting the frame accumulation depth in the background buffer associated with the current video stream instance; when the frame accumulation depth exceeds a preset frame dropping threshold, automatically clearing the old frame buffer and jumping to the nearest keyframe for parsing. Since multiple video streams share the same thread's scheduling cycle under a single physical thread multiplexing architecture, frame buffer data accumulation may occur in live broadcast scenarios due to scheduling delays. This step effectively compensates for the delay introduced by single-threaded scheduling by actively monitoring the frame accumulation depth and discarding old frames and jumping to keyframe parsing when the limit is exceeded, ensuring the timeliness of the live broadcast display.

[0046] This embodiment, through the coordinated operation of the above steps, achieves the technical effect of supporting concurrent decoding and rendering of multiple video streams with a fixed single-threaded resource on the web browser side. The thread overhead does not increase with the number of concurrent streams, there is zero cross-thread copying of pixel data, the main thread rendering load is completely released, and it has the ability to adaptively step downgrade and keep alive under extreme load. At the same time, it is compatible with heterogeneous bitstream environments, which significantly improves the system stability, operating efficiency and applicability of the web-side multi-channel video concurrent processing solution.

[0047] Example 2

[0048] This embodiment combines Figure 2The system architecture diagram shown above provides a further detailed explanation of the specific implementation architecture and core processes of the technical solution in Embodiment 1.

[0049] like Figure 2 As shown, the operational architecture of the method provided by this invention is generally divided into two collaborative logical layers: the main front-end thread (UI) and the shared decoding base (sharedWorker). The main front-end thread (UI) is responsible for the creation and management of video playback instances, the stripping and transfer of canvas rendering control, and degradation scheduling after overload. The shared decoding base (sharedWorker), as the globally unique background decoding thread, is responsible for the centralized decoding of all video channels, off-screen rendering, and performance status reporting. The two communicate with each other through the browser's cross-thread communication interface for signaling and data exchange.

[0050] This embodiment breaks away from the traditional design of web front-end players that create independent background threads for each video stream, and instead uses a single shared base engine for centralized demultiplexing and software decoding scheduling. Specifically, as shown... Figure 2 As shown, the main front-end thread (UI) creates / probes N concurrent player shells. When the browser side creates the first video playback instance, the main front-end thread first probes whether the global static instance `sharedWorker` has been mounted. If it has not yet been created, i.e. Figure 2 If the condition "[shared dock not running]" is met, then a single background decoding main thread (single physical thread) is created and initialized using `new Worker()` as the shared decoding dock, and a high-performance video decoder compiled based on WebAssembly is loaded. In all subsequent lifecycle stages, any additional video surveillance channels added to the business system will reuse this single underlying background thread, achieving single-threaded response to multiple video inputs.

[0051] To accurately distinguish and process dozens of unrelated real-time video streams in parallel within a single background physical thread, this embodiment employs a virtual instance identifier (instanceId) registration mechanism. Whenever a new player instance object is created on the webpage (e.g., an on-demand added split-screen monitor), the system automatically assigns it a globally unique ID. All signaling between the main front-end thread and the underlying layer (such as initialization, speed setting, stop / destroy, etc.) and the issued raw audio / video stream slices must have this identifier strictly sealed on their message payload. Internally, this shared decoding base maintains a dictionary mapping structure called instanceMap. When multi-channel mixed video network slice packets are pushed into the underlying layer, the base engine first extracts their instanceId for unpacking and routing, and then looks up the corresponding ID-bound independent decoding context object and cache queue. Through this software-layer logical arrangement, strong data isolation of multiple decoding instances is achieved within a single physical worker thread. Through the synergy of the above mechanisms, this embodiment smooths out the phenomenon of geometrically increasing workload computing threads caused by the addition of high-number video feeds, transforming it into a normalized, fixed, low-resource-consumption state. No matter how many concurrent feeds the system accesses, the underlying architecture retains only a single master scheduling pool, fundamentally solving the problems of creation failure, increased context switching overhead, and memory overflow caused by exceeding the thread limit.

[0052] Regarding off-screen rendering, this embodiment transfers canvas rendering control from the main front-end thread to the shared decoding dock in one go, and completes the fully enclosed processing from decoding to drawing within the dock. This completely avoids the problem of traditional playback architectures transmitting ultra-large video pixel data across threads, achieving true zero data copying. Specifically, as... Figure 2 As shown within the loop body, when the main front-end thread mounts the player's front-end interactive rendering node, its first step is to check the browser environment's support for off-screen graphics features. After confirming support, it retrieves the HTML responsible for rendering the interface. <canvas>Nodes, from <canvas>The rendering bindings are stripped from the element, and the system-provided `transferControlToOffscreen()` interface is actively called to completely strip the rendering control of the actual displayed element in the visual tree, generating an off-screen handle with a graphics context but completely disconnected from the main UI thread—that is, an off-screen rendering proxy object. Then, when the main front-end thread constructs the initial connection signaling for this child instance, it concatenates the extracted off-screen rendering proxy object with the instance's `instanceId`, such as... Figure 2 As shown by the horizontal arrow, the ownership of the offscreen rendering proxy object is transferred to the underlying shared decoding platform (sharedWorker) in one go via `postMessage(type:'init',instanceId,OffscreenCanvas)`. This handover is complete, and the front-end main thread completely loses direct layer modification rights over the visible area of ​​the video. Correspondingly, within the shared decoding platform, through... Figure 2 The instanceMap.set(instanceId,this) mapping operation shown below establishes a mapping association between the received off-screen rendering proxy object and the instanceId of the video stream, thus establishing a safety barrier for multiple concurrent streams within the same memory thread.

[0053] Subsequently, the heavy burden of concurrent drawing is entirely transferred to a single background process. After the shared decoding dock unpacks the video and uses the built-in WebAssembly decoder to decrypt the complete digital pixel matrix of a given instance, the system is configured not to perform serialization reverse cross-thread delivery to the main interface, i.e., zero-byte video frame return. The dock logic will, in place, find the pre-managed off-screen proxy handle based on the corresponding instanceId mapped in its own instanceMap, request its graphics drawing context, directly call the underlying GPU drawing pipeline, perform color conversion in the shader, and flush and rearrange the colors to the final interface's layer memory. This completely encloses the "receiving stream packets - software decoding - GPU drawing rendering" chain within the underlying computing pool. Taking 1080P resolution as an example, this mechanism can save the equivalent of megabytes of multi-dimensional large packet transmission and a series of processing actions on a single frame, achieving a leapfrog reduction in running memory. At the same time, the main front-end thread was completely freed from view constraints, and all video frame graphics redrawing operations were emptied and placed on the background management bus for processing. The main front-end UI event capture loop for the target terminal was no longer constrained by the large amount of data consumption of coloring and outlining the video layer.

[0054] In high-load scenarios where a single physical core handles a large number of security channels, this embodiment implements a callback strategy that keeps heavy-load images in the background and segments light-load images across boundaries, along with a performance over-limit detection defense. Specifically, the massive number of pixels in the image are locked in the background and rendered entirely in the background off-screen rendering loop as described above, without sending any pixel frame data back to the main thread. However, for the extremely small amount of environmental audio track streams (such as pulsed PCM signal points) obtained after decomposition, since the main interactive interface needs to be globally associated with application-layer functions such as mute control and one-click promiscuous monitoring, the system separately loads the only lightweight audio stream, returns it to the front end via a low-frequency, small-volume native cross-thread message channel, and connects to the built-in PCM audio manager. Playback is achieved using the browser's common sound card driver, which avoids redundant calculations and offloads the main UI without putting pressure on it.

[0055] In terms of performance monitoring, this embodiment constructs a statistical measurement model for the total time period from the initial raw material casting time to the moment the background graphics drawing context shading callback information is first obtained, and calculates the time difference between the two as the initial rendering time indicator. This initial rendering time directly represents the peak and limit pressure of the system's current terminal hardware environment and software decoding capabilities. When high concurrency is severely overloaded, this time will exhibit a non-linear time tail lengthening characteristic. When this time exceeds the preset health threshold boundary, it is determined that the user's business needs have exceeded the limit capacity safety valve of the hardware and software baseline, and the system will immediately trigger the subsequently configured anti-crash degradation control strategy.

[0056] Compared to previous implementations where peak concurrency would directly cause system crashes and memory overflows due to backlog, this embodiment incorporates a single-step controlled mitigation and degradation procedure. Specifically, when the scheduling and control code is triggered by an alarm from the underlying engine in the previous step, it sends a lower-level takeover command to the monitoring stream playback instance designated as having the lowest priority. This command invokes a dedicated control instruction from the engine, triggering the degradation process for that instance. The controlled video decoding entity hit by this command immediately shuts down, ceasing its autonomous attempt to consume system resources to accelerate parsing. It forcibly stops and cancels the full-speed loop task allocation sequence, switching from autonomous loop decoding mode to controlled step mode, forming a monitored, isolated, and suspended protected state. After entering controlled step mode, the instance exits the endless competition for decoding computing resources and is manually awakened by the calling end, which can periodically issue active step call commands by setting a specific low-frequency clock. This forced invocation method, which actively limits the extraction cycle, forces the underlying decoding engine to calculate and execute only one displayable frame for each valid invocation command, projecting it onto the rendering interface. This architectural logic achieves this by significantly sacrificing unrelated high-frame-rate updates of video due to overload, avoiding unnecessary overhead in non-critical, high-frequency jump calculations such as the time-consuming bidirectional prediction frame parsing, and forcibly locking and protecting the rated capacity of background memory and CPU allocation. Ultimately, even when testing on low-spec computers under excessive load, although the single-frame invocation results in discontinuous updates and a snapshot-like screen refresh, the overall multi-array image display modules remain stable and usable. This ensures that the system's anti-crash and anti-forced-quit programs play a crucial last-line defensive role, preventing serious business incidents such as browser crashes due to complete memory overflow causing a blank page and exit.

[0057] In the specific implementation of this embodiment, the following optional processing may also be included: Regarding the lifecycle management of the shared decoding base, a global reference counter can be used to count the total number of video stream instances currently carried by the shared decoding base. When the reference counter reaches zero, the shared decoding base is terminated and released, and the instanceMap dictionary mapping structure is cleared, releasing the system resources it occupies. Regarding off-screen rendering compatibility, if it is detected that the current browser runtime environment does not support the off-screen canvas interface, the full background rendering closed-loop mode can be automatically closed, switching to data return mode. The shared decoding base serializes the decoded pixel data and returns it to the main thread, which then performs graphics drawing on the interface canvas. Regarding heterogeneous bitstream adaptation, for non-standard video bitstreams carrying private commercial extension headers in existing monitoring systems, the shared decoding base can identify and pre-strip the private extension header before decoding, and repackage the stripped payload data according to the standard input specification of the video decoder, converting it into a standard decodeable stream. In terms of ensuring real-time performance in live streaming scenarios, the live streaming status identifier carried in the initial connection signaling can be parsed. If the current video stream is determined to be a live stream, the frame accumulation depth of the background decoding buffer associated with the video instance will be continuously probed. When the frame accumulation depth exceeds the preset frame dropping threshold, the old frame data in the buffer will be automatically cleared and the decoding will be performed on the most recent key frame.

[0058] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / canvas> < / canvas>

Claims

1. A Web multi-video concurrent processing method based on single-threaded multiplexing and off-screen rendering, characterized in that, include: Obtain the identification information of the target video stream instance and detect whether a globally unique shared decoding thread supporting cross-thread communication has been mounted in the current runtime environment; If it does not exist, instantiate a shared decoding thread; Each video stream instance is assigned a unique virtual instance identifier. A logical routing mapping relationship between the virtual instance identifier and the corresponding decoding context is established within the shared decoding thread. The bitstream data with the virtual instance identifier attached is sent to the shared decoding thread through the cross-thread communication interface. Obtain the canvas element responsible for displaying each video stream in the main interface, call the rendering control of the canvas element to obtain the off-screen rendering proxy object through the decoupling interface, and transfer the ownership of the off-screen rendering proxy object to the shared decoding thread in one go; Within the shared decoding thread, based on the virtual instance identifier carried in the received bitstream data, the corresponding decoding context and the off-screen rendering proxy object are retrieved by searching the logical routing mapping relationship, and decoding is performed to obtain pixel data; The pixel data is directly rendered to the video memory associated with the off-screen rendering proxy object using the background graphics drawing context, so that decoding and rendering are completed in the shared decoding thread. Record the start time when the video stream data is sent to the shared decoding thread, and obtain the first frame rendering callback time of the background graphics drawing context; calculate the time difference between the start time and the first frame rendering callback time; When the time difference exceeds a preset health threshold, a circuit breaker command is sent to the corresponding video stream instance to switch it from autonomous loop decoding mode to controlled step mode, so that it only performs single-frame decoding and rendering in response to active step commands issued by the front end.

2. The method according to claim 1, characterized in that, Transferring ownership of the off-screen rendering proxy object to the shared decoding thread in one go includes: The off-screen rendering proxy object is placed as a transferable object in the message payload of the initialization connection signaling, and the control handle of the off-screen rendering proxy object is transferred to the shared decoding thread through the one-time handshake protocol of the cross-thread communication interface.

3. The method according to claim 1, characterized in that, Also includes: The decoded pixel data is retained in the shared decoding thread for background rendering, enabling cross-thread backhaul of zero-pixel data from the main interface. The decoded audio stream data is returned to the main thread through a message channel, and the browser's sound card is driven by the main thread's global audio manager to play the audio.

4. The method according to claim 1, characterized in that, Send a circuit breaker command to the corresponding video stream instance to switch it from autonomous loop decoding mode to controlled step mode, including: In response to the circuit breaker command, the video stream instance interrupts the current full-speed decoding loop task sequence and enters a sleep suspension state, entering the controlled step mode.

5. The method according to claim 4, characterized in that, Execute active pulse control logic in controlled stepping mode: When a step call pulse is received from the front-end main interface according to a preset cycle, the underlying decoding engine is driven to perform a single frame image parsing and video memory projection for this pulse, thereby achieving system survival under extreme load by actively limiting the decoding extraction cycle.

6. The method according to claim 1, characterized in that, Before sending the circuit breaker command to the corresponding video stream instance, the following is also included: Obtain the window size information or preset display priority labels for each video stream instance; Video stream instances with a window size smaller than a preset threshold or a display priority label of low are identified as secondary priority instances, and the circuit breaker command is sent to the secondary priority instances first.

7. The method according to claim 1, characterized in that, The health threshold is dynamically determined based on the number of concurrent video stream instances currently carried by the shared decoding thread; when the number of concurrent streams exceeds the preset number, the first frame rendering callback time exhibits a non-linear delay increase relative to the start time.

8. The method according to claim 5, characterized in that, In controlled step mode, the decoding engine performs intra-frame decoding only on the current single-frame image, skipping the continuous parsing of bidirectional prediction frames, so as to keep the decoding computational resource consumption within a preset rated range.

9. The method according to claim 1, characterized in that, Before the shared decoding thread performs decoding on the video stream data, a stream protocol adaptation step is also included: Identify the Private Business Extensions header carried in the received video stream data; The process involves performing a pre-stripping operation on the proprietary business extension identifier header and then re-encapsulating the stripped payload data according to the standard input specifications of the video decoder to convert heterogeneous, non-standard video streams into standard decodeable streams.

10. The method according to any one of claims 2 to 9, characterized in that, It also includes a cache depth control step based on stream attributes: Parse the live streaming status identifier carried in the initialization connection signaling; If the live streaming status identifier is true, then detect the frame stacking depth in the back buffer associated with the current video stream instance; When the frame accumulation depth exceeds the preset frame loss threshold, the old frame buffer is automatically cleared and the process jumps to the nearest key frame for parsing to compensate for the scheduling delay introduced by single physical thread reuse.

Images