A seamless switching video stream forwarding method and device, electronic equipment and computer program product

Abstract

The application relates to the technical field of streaming media processing, in particular to a seamless switching video stream forwarding method and device, electronic equipment and computer program product. The video stream forwarding method comprises the following steps: acquiring a video stream forwarding task and a forwarding rule corresponding to the video stream forwarding task, wherein the forwarding rule at least comprises a current to-be-output video stream identifier, a candidate video stream identifier, a video source address, a streaming media server address and a video stream switching condition; determining at least two pull streamers and one push streamer according to the forwarding rule; determining a current effective pull streamer in the at least two pull streamers, and controlling the current effective pull streamer to write the decoded video frame into a forwarding queue, while controlling other pull streamers to keep a pre-pull stream state but not to write the video frame into the forwarding queue; reading the video frame from the forwarding queue through the push streamer, and pushing the read video frame to the streaming media server to form an externally output video stream.

Description

Technical Field

[0001] This application relates to the field of streaming media processing technology, and in particular to a seamless video stream forwarding method, apparatus, electronic device, and computer program product. Background Technology

[0002] With the development of video surveillance, network transmission, and streaming media technologies, various video surveillance platforms are widely used in scenarios such as public safety, urban governance, industrial production, transportation, park management, bank security, and airport / port monitoring. In these scenarios, video stream forwarding systems typically handle the task of acquiring video streams from front-end video sources and pushing them to streaming media servers or business clients. Video sources can be network cameras, video encoders, recording devices, video platforms, streaming media servers, or other devices that can provide video data. Video stream forwarding systems usually need to support one or more video transmission protocols such as RTSP, RTMP, GB / T 28181, HTTP-FLV, HLS, and WebRTC. During the forwarding process, the system generally needs to complete operations such as video source connection, video data acquisition, decapsulation, decoding, frame buffering, push stream encapsulation, streaming media server connection, and continuous push.

[0003] In real-world business scenarios, video streams are not always fixed. Many scenarios require dynamic switching of video sources based on business needs. For example: 1. In video carousel scenarios, the system needs to display video feeds from multiple cameras sequentially at preset time intervals; 2. In on-the-fly video scenarios, the system needs to automatically switch to the camera feed of the corresponding area based on changes in the location of vehicles, equipment, personnel, or target objects; 3. In alarm linkage scenarios, when an intrusion, fire, equipment malfunction, or other alarm event occurs in a certain area, the system needs to immediately switch the output video to the video stream of the alarm area; 4. In emergency command scenarios, dispatchers may manually select a video source as the current output video based on the situation on site; 5. In key target monitoring scenarios, the system may need to dynamically switch video streams based on the movement trajectory of the target object and the camera coverage area.

[0004] Existing video stream switching methods typically involve closing the currently playing or forwarding video stream and then opening the new target video stream when a switch is needed. The typical process includes: stopping the current video source's pull operation, stopping the push of the current video frames to the streaming media server, disconnecting the current video source connection, and releasing related decoding resources; then re-establishing a connection with the new video source, pulling video data from the new video source, decapsulating and decoding the new video data, resynchronizing the decoded video frames to the forwarding queue, and finally resuming the push operation.

[0005] While the existing methods described above can switch video sources, they have the following problems.

[0006] First, video stream interruptions are prone to occur during the switching process. Because the current stream needs to be closed before the new stream is opened, an unavoidable time interval exists between these two operations. During this interval, the streamer may be unable to retrieve consecutive video frames from the forwarding queue, resulting in the streaming media server not receiving continuous video data. This manifests as a black screen, stream interruption, stuttering, or frozen video on the client side.

[0007] Second, reopening a new video stream requires a certain amount of initialization time. Operations such as establishing video source connections, network handshakes, protocol negotiation, bitstream detection, decapsulation initialization, decoder initialization, keyframe waiting, and video frame synchronization all introduce latency. Especially for monitoring scenarios with high real-time requirements, even a delay of hundreds of milliseconds to several seconds can affect the user experience and security assessments.

[0008] Third, frequently closing and opening video streams wastes system resources. Each switch involves the creation and release of resources such as threads, connections, buffers, decoders, and network sockets. In scenarios involving video carousels or frequent switching between multiple streams, frequently performing these operations increases CPU usage, memory usage, network overhead, and thread scheduling pressure, thus reducing overall system performance.

[0009] Fourth, the existing control process is complex. Video switching involves multiple stages, including closing the current stream, opening a new stream, re-pulling the stream, re-decoding, resynchronizing, and resuming streaming. These stages require strict coordination. If any stage fails or times out, it may lead to switching failure, abnormal video, streaming interruption, or system instability.

[0010] Fifth, in security monitoring scenarios, video interruptions may lead to the loss of critical information. For example, in important scenarios such as airports, banks, traffic intersections, and industrial production lines, even short-term video interruptions may result in the omission of critical events, thereby affecting security assessments, incident tracking, and emergency response.

[0011] Therefore, there is an urgent need for a video stream forwarding method that can maintain streaming continuity, reduce switching latency, reduce system resource consumption, and improve system stability during video stream switching. Summary of the Invention

[0012] This application addresses the aforementioned technical problems by providing a seamless video stream forwarding method, apparatus, electronic device, and computer program product to solve the problems of video interruption, black screen, resynchronization delay, resource waste, and high system complexity in existing video stream switching methods.

[0013] A seamless switching video stream forwarding method according to one aspect of this application includes: S1, obtaining a video stream forwarding task and a forwarding rule corresponding to the video stream forwarding task, wherein the forwarding rule includes at least the identifier of the current video stream to be output, the identifier of the candidate video stream, the address of the video source, the address of the streaming media server, and the video stream switching conditions; S2, determining at least two pullers and one pusher according to the forwarding rule, wherein the at least two pullers establish connections with different video sources respectively, and pre-pull and decode the video streams of the corresponding video sources; S3, determining the currently effective puller among the at least two pullers, and controlling the currently effective puller to write its decoded video frames into the forwarding queue, while controlling the other pullers to maintain the pre-pull state but not to write video frames into the forwarding queue; S4, reading video frames from the forwarding queue through the pusher, and pushing the read video frames to the streaming media server to form an externally output video stream.

[0014] Furthermore, the seamless switching video stream forwarding method according to one aspect of this application further includes: S5, monitoring the business system status, forwarding status, and / or video stream switching conditions in real time through a forwarding monitor; when the video stream switching conditions are detected, dynamically updating the forwarding rules and switching the target stream puller to the new currently effective stream puller; S6, after updating the forwarding rules, controlling the original currently effective stream puller to stop writing video frames to the forwarding queue, and controlling the target stream puller to write its pre-pulled and decoded video frames to the forwarding queue, so that the pusher can continuously push the switched video frames without interrupting the connection with the streaming media server.

[0015] Furthermore, the seamless switching video stream forwarding method according to one aspect of this application also includes: S7, monitoring the operating status of the stream puller, the stream pusher, and the forwarding monitor, and when an abnormal state is detected, performing the corresponding abnormal recovery operation through the forwarding controller.

[0016] Furthermore, according to one aspect of the seamless switching video stream forwarding method of this application, the pre-pulling and decoding of the stream puller includes: S21, the stream puller establishes a network connection with the corresponding video source based on the video source address; S22, the stream puller continuously acquires raw video data from the video source; S23, the stream puller decapsulates and decodes the raw video data to generate video frames that can be forwarded by the pusher; S24, the stream puller caches the decoded video frames in a local buffer, or writes the decoded video frames into a forwarding queue when it is determined to be the current valid stream puller; wherein, the stream puller that is not determined to be the current valid stream puller maintains the connection with the corresponding video source and continues to perform pulling and decoding operations so as to provide video frames that can be forwarded immediately when a video stream switching occurs.

[0017] Furthermore, according to one aspect of the seamless switching video stream forwarding method of this application, the video stream switching conditions include at least one of the following: timed rotation conditions, target video source change conditions specified by the business system, on-the-fly video switching conditions, current video source abnormal conditions, current video source picture quality degradation conditions, user manual switching conditions, preset priority change conditions, or external event triggering conditions; the forwarding monitor generates or updates forwarding rules according to the video stream switching conditions, and sends the updated forwarding rules to the forwarding controller, which then schedules the puller and pusher streams according to the updated forwarding rules.

[0018] Furthermore, according to one aspect of the seamless switching video stream forwarding method of this application, in step S6, controlling the original currently active stream puller to stop writing video frames to the forwarding queue and controlling the target stream puller to write video frames to the forwarding queue includes: S61, setting forwarding status flags for the original currently active stream puller and the target stream puller respectively; S62, updating the forwarding status flag of the original currently active stream puller from active forwarding status to pre-pulling status; S63, updating the forwarding status flag of the target stream puller from pre-pulling status to active forwarding status; S64, controlling whether the corresponding stream puller writes video frames to the forwarding queue based on the updated forwarding status flag; wherein, the stream pusher remains running during the switching process and continuously reads video frames from the forwarding queue.

[0019] Furthermore, according to one aspect of the seamless switching video stream forwarding method of this application, the abnormal recovery operation in step S7 includes at least one of the following: when an abnormality is detected in the puller thread, the corresponding puller is restarted, and the video source connection, pull, and decoding status of the puller is restored according to the current forwarding rules; when an abnormality is detected in the pusher thread, the pusher is restarted, and the streaming media server is reconnected to restore the operation of reading video frames from the forwarding queue and pushing them to the streaming media server; when an abnormality is detected in the forwarding monitor, the forwarding monitor is restarted, and the current forwarding rules are reloaded or generated; when an abnormality is detected in the video source corresponding to the currently effective puller, the candidate puller in the pre-pulling state is switched to the new currently effective puller; when an abnormality is detected in the forwarding queue, the forwarding queue is cleaned up or rebuilt, and video frame writing is restored according to the currently effective puller.

[0020] A seamless switching video stream forwarding device according to another aspect of this application includes: a forwarding monitor configured to acquire video stream forwarding tasks, monitor the status of the business system, the forwarding status, and / or video stream switching conditions, and generate and dynamically update forwarding rules corresponding to the video stream forwarding tasks; a forwarding controller configured to determine and schedule at least two pullers and one pusher according to the forwarding rules; the at least two pullers configured to establish connections with different video sources respectively, pre-pull and decode the video streams of the corresponding video sources, wherein one puller is configured as the currently effective puller and is used to write the decoded video frames into the forwarding queue, and the other pullers are configured as pre-pull pullers and maintain the pulling and decoding status but do not write video frames into the forwarding queue; and a pusher configured to read video frames from the forwarding queue and push the read video frames to the streaming media server; wherein, when the forwarding monitor detects that the video stream switching conditions are met, the forwarding controller controls the target puller to switch to the new currently effective puller according to the updated forwarding rules, so that the pusher continuously pushes the switched video frames without interrupting the connection.

[0021] An electronic device according to another aspect of this application includes: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions, causing the electronic device to perform the video stream forwarding method described above.

[0022] According to another aspect of this application, an electronic device computer program product includes a computer program that, when executed by a processor, implements the video stream forwarding method described above.

[0023] This application uses multiple pullers to pre-pull and decode candidate video streams, ensuring the target video stream is already available before switching occurs. During switching, only the puller writing video frames to the forwarding queue changes; no re-establishment of connections or re-decoding is required, thus avoiding video interruption. The pusher continuously reads video frames from the forwarding queue and pushes them to the streaming media server during switching, without needing to disconnect from the server. Therefore, even if the current output video source changes, the streaming link remains continuous. Since the candidate video source has already completed pre-pulling and decoding, there is no need to wait for the new video stream's connection establishment and decoding initialization during switching, significantly reducing resynchronization time and improving video real-time performance. Although this application maintains the pre-pulling state of multiple pullers, it avoids resource jitter caused by frequent closing and reopening of video streams, reducing system overhead from frequent creation and destruction of threads, connections, and decoders. This application uses a forwarding controller to centrally schedule and monitor the pullers, pushers, and forwarding monitor. When an abnormal state is detected, recovery operations such as restarting, reconnecting, switching to a backup puller, and rebuilding the queue can be performed, improving system reliability. In scenarios such as security monitoring, emergency command, and on-board video surveillance, this application can reduce information loss caused by switching, ensure the continuity of monitoring footage, and help improve the accuracy of security judgments. Attached Figure Description

[0024] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 This is a flowchart illustrating a video stream forwarding method according to an embodiment of this application; Figure 2 This is a flowchart further illustrating a video stream forwarding method according to an embodiment of this application; Figure 3 This is a flowchart illustrating the pre-pulling and decoding processes of the stream puller in the video stream forwarding method according to an embodiment of this application; Figure 4 This is a flowchart illustrating the switching of the forwarding state of the stream puller in a video stream forwarding method according to an embodiment of this application; Figure 5 This is a structural block diagram illustrating a video stream forwarding device according to an embodiment of this application; Figure 6 This is a structural block diagram of an electronic device according to an embodiment of this application; Figure 7This is a schematic diagram illustrating a computer program product according to an embodiment of this application. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0027] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only used to illustrate this application and are not intended to limit the scope of protection of this application. Where there is no conflict, the technical features of the various embodiments can be combined with each other.

[0028] The core idea of ​​this application is as follows: Before switching video streams, pullers are pre-established for multiple candidate video sources, and each puller is in a pre-pulling and decoding state; at the current moment, only one currently valid puller is allowed to write video frames to the forwarding queue; the pusher always reads video frames from the forwarding queue and pushes them to the streaming media server; when it is necessary to switch video streams, the forwarding rules are dynamically updated, the target puller is set as the new currently valid puller, so that the target puller writes the pre-pulled and decoded video frames to the forwarding queue, and the pusher continuously pushes the switched video frames.

[0029] Figure 1 This is a flowchart illustrating a video stream forwarding method according to an embodiment of this application. Figure 1 As shown, the method includes steps S1 to S4.

[0030] Step S1: Obtain the video stream forwarding task and the forwarding rules corresponding to the video stream forwarding task.

[0031] In step S1, the system obtains the video stream forwarding task and the forwarding rules corresponding to the video stream forwarding task.

[0032] Video stream forwarding tasks can be issued by the business system, configured by users through the monitoring and management platform, or automatically generated by the system based on preset business policies. Video stream forwarding tasks instruct the system which video sources to acquire video streams from and push the specified video streams to the target streaming media server.

[0033] Video stream forwarding tasks can include information such as task identifier, task type, video source list, current output video source, candidate video sources, streaming media server address, push protocol, video encoding parameters, switching conditions, carousel cycle, priority rules, and exception handling strategy.

[0034] The task identifier is used to distinguish different video stream forwarding tasks. Task types can include fixed forwarding tasks, video carousel tasks, follow-up video tasks, alarm linkage tasks, manual switching tasks, etc. The video source list can include multiple video sources, each of which can correspond to a camera, encoder, video platform interface, or video file stream. The streaming media server address is used to indicate the target address to which the pusher will push video frames.

[0035] Forwarding rules are a crucial control mechanism in this application. Forwarding rules must include at least the identifier of the current video stream to be output, the identifier of the candidate video stream, the video source address, the streaming media server address, and the conditions for switching video streams.

[0036] Among them, the current video stream identifier is used to indicate the video stream that needs to be output to the outside world at present; the candidate video stream identifier is used to indicate the video stream that may be output in subsequent switching; the video source address is used to indicate the video source that each puller should connect to; the streaming media server address is used to indicate the target server that the pusher should push to; and the video stream switching condition is used to indicate under what circumstances the current video stream should be switched to the target video stream.

[0037] For example, in a video carousel task, the forwarding rule can be: the current output is the video stream of camera A, the candidate video streams include the video streams of camera B and camera C, and the next video source is switched every 10 seconds. The push address is the RTMP address of a certain streaming media server.

[0038] For example, in a live video task, the forwarding rule can be expressed as: determine the current video stream to be output based on the change in the location of the target device, and when the target device enters area B, switch to the video stream of the camera corresponding to area B.

[0039] Step S1 clarifies the business objectives and subsequent scheduling basis of the current forwarding task, providing a foundation for the initialization of the puller, pusher, and forwarding queue. Forwarding rules uniformly describe the video source, candidate video stream, push address, and switching conditions, giving subsequent scheduling a clear basis; the video stream switching process is rule-driven, rather than controlled independently by multiple scattered modules, reducing system complexity; it supports various business scenarios, such as timed rotation, on-device switching, alarm linkage, and manual user switching; and it provides a data foundation for dynamically updating forwarding rules, thereby achieving seamless subsequent switching.

[0040] Step S2: Determine at least two pullers and one pusher based on the forwarding rules.

[0041] In step S2, the system determines at least two pullers and one pusher based on the forwarding rules.

[0042] A streamer is a functional module used to acquire and decode a video stream from a video source. Each streamer can correspond to a video source, or in some implementations, to a video source address, a video channel, or a video stream identifier.

[0043] In this embodiment, at least two stream pullers establish connections with different video sources and pre-pull and decode the video streams of the corresponding video sources. Pre-pull refers to establishing a connection with a video source before it is officially used as the current output video stream, and continuously acquiring and decoding that video stream so that it can immediately provide forwardable video frames when switched to the current output video stream.

[0044] A pusher is a functional module used to push video frames to a streaming media server. Unlike existing technologies where the pusher may restart every time a video source is switched, in this embodiment, the pusher remains running during the switching process and reads video frames from a unified forwarding queue. The pusher does not directly depend on a specific video source, but rather on the continuous video frames provided by the forwarding queue.

[0045] Specifically, the forwarding controller can create or determine multiple streamers based on the list of video source addresses in the forwarding rules. For example, if the forwarding rules include video source A, video source B, and video source C, the system can determine streamers A, B, and C respectively. Streamer A establishes a connection with video source A, streamer B establishes a connection with video source B, and streamer C establishes a connection with video source C.

[0046] Once started, each stream puller can perform operations such as connection, streaming, decapsulation, decoding, and buffering. Different stream pullers can operate independently of each other, thus ensuring that a failure of one stream puller will not directly affect other stream pullers.

[0047] The pusher then establishes a connection with the target streaming media server based on the streaming media server address in the forwarding rules. The target streaming media server can be an RTMP server, RTSP server, GB / T 28181 platform, WebRTC media server, HTTP-FLV server, or other streaming media service node.

[0048] Step S2 establishes connections with different video sources in advance through multiple pullers, avoiding delays caused by establishing connections only during switching. By pre-pulling and decoding candidate video streams, candidate video frames can be used immediately during switching. The pusher is decoupled from the video source, and the pusher only reads video frames from the forwarding queue, so video source switching will not directly cause the pusher link to be interrupted. Multiple pullers work in parallel, improving the response speed of video stream switching. This provides a foundation for the rapid switching of the currently effective puller.

[0049] Step S3: Determine the currently active streamer and control its writing into the forwarding queue.

[0050] In step S3, the system determines the currently active streamer among at least two streamers and controls the currently active streamer to write its decoded video frames into the forwarding queue, while controlling the other streamers to maintain the pre-streaming state but not to write video frames into the forwarding queue.

[0051] The currently active streamer refers to the streamer that is currently allowed to write video frames to the forwarding queue. The video source corresponding to the currently active streamer is the video stream currently being output.

[0052] While other streamers do not write video frames to the forwarding queue, they still maintain a pre-streaming state. That is, other streamers maintain their connection to the corresponding video source, continuously acquiring and decoding the raw video data. Other streamers can either cache the decoded video frames in their local buffer or simply keep their decoder active and begin writing to the forwarding queue immediately after being switched to the current active streamer.

[0053] In one alternative implementation, the forwarding controller can set a forwarding status flag for each streamer. This forwarding status flag includes at least a valid forwarding status and a pre-streaming status. When a streamer's forwarding status flag is valid, the streamer writes the decoded video frames into the forwarding queue; when a streamer's forwarding status flag is pre-streaming, the streamer does not write video frames into the forwarding queue.

[0054] For example, the system contains stream pullers A, B, and C. If the current forwarding rule specifies video source A as the current video stream to be output, then stream puller A is determined to be the currently valid stream puller, and stream puller A writes its decoded video frames into the forwarding queue; stream pullers B and C remain in the pre-pull state and do not write video frames into the forwarding queue.

[0055] Forwarding queues can employ structures such as thread-safe queues, blocking queues, circular buffer queues, and shared memory queues. The forwarding queue buffers video frames written by the currently active puller for the pusher to read. Since typically only one active puller writes video frames to the forwarding queue at any given time, it avoids output anomalies caused by mixed writing of video frames from multiple video sources.

[0056] Step S3 ensures that only one video stream is written to the forwarding queue at any given time, thus forming a clear and stable current output video stream; it keeps the candidate pullers in a pre-pull state, ensuring that the candidate video streams have usable video frames when switching; through the forwarding state control mechanism, the video switching process is transformed into a write permission switching process, reducing the switching complexity; it avoids multiple pullers simultaneously writing video frames from different video sources to the forwarding queue, ensuring the orderliness of the push output; and it makes video source switching no longer dependent on reconnection and re-decoding, thereby improving the real-time performance of switching.

[0057] Step S4: The pusher reads video frames from the forwarding queue and pushes them to the streaming media server.

[0058] In step S4, the pusher reads video frames from the forwarding queue and pushes the read video frames to the streaming media server to form a video stream for external output.

[0059] The streaming device can run continuously and uses a producer-consumer model to read video frames from the forwarding queue. The currently active streaming device acts as a producer, writing the decoded video frames into the forwarding queue; the streaming device acts as a consumer, reading video frames from the forwarding queue and encapsulating and pushing them.

[0060] The streaming pusher can encapsulate the video frames it reads according to the protocols supported by the streaming media server. For example, when the target server supports the RTMP protocol, the streaming pusher can encapsulate the video frames into RTMP messages and push them; when the target server supports the HTTP-FLV protocol, the streaming pusher can encapsulate the video frames into FLV tags; when the target server supports GB / T28181, the streaming pusher can encapsulate and transmit them according to the corresponding protocol.

[0061] The streaming pusher can read video frames according to their timestamps, frame numbers, or queue writing order. In scenarios with high real-time requirements, the streaming pusher can also execute frame dropping strategies based on queue length. For example, when too many video frames accumulate in the forwarding queue, some older frames can be dropped to reduce playback latency.

[0062] In this embodiment, the pusher does not directly perceive which channel is currently the output video source. The pusher only focuses on whether there are video frames available for pushing in the forwarding queue. Therefore, when the currently active puller switches, as long as the new active puller can write video frames to the forwarding queue in a timely manner, the pusher can continuously push the video stream to the streaming media server.

[0063] Through step S4, the streamer is decoupled from the specific video source, so switching video sources will not require the streamer to be restarted; the streamer continuously reads video frames from a unified forwarding queue, keeping the external streaming link stable; it avoids disconnecting the streaming connection due to switching video sources, reducing client black screens and stream interruptions; it supports multiple streaming media push protocols, improving the applicability of the solution; and it improves the continuity and fault tolerance of the streaming process through queue caching and read control.

[0064] Figure 2 This is a flowchart further illustrating a video stream forwarding method according to an embodiment of this application. For example... Figure 2 As shown, in Figure 1 Based on steps S1 to S4 shown, the method also includes steps S5, S6 and S7.

[0065] Step S5: Monitor and dynamically update forwarding rules in real time through the forwarding monitor.

[0066] In step S5, the system monitors the business system status, forwarding status, and / or video stream switching conditions in real time through a forwarding monitor. When the video stream switching conditions are detected, the forwarding rules are dynamically updated, and the target stream puller is switched to the new currently valid stream puller.

[0067] The forwarding monitor can be a standalone software module or a submodule running within the forwarding controller. Its main functions are to monitor system status, determine whether video stream switching is necessary, and generate updated forwarding rules.

[0068] The business system status can include carousel timing status, target object location, alarm events, user operation commands, video source priority, business mode changes, etc. The forwarding status can include the current output video stream identifier, the current valid puller identifier, the connection status of each puller, the decoding status of each puller, the pusher running status, the forwarding queue length, the video frame writing frequency, the video frame timestamp continuity, etc.

[0069] Video stream switching conditions may include timed rotation conditions, changes in the target video source specified by the business system, on-the-fly video switching conditions, abnormal conditions of the current video source, deterioration of the current video source's picture quality, manual switching conditions by the user, changes in preset priority conditions, or external event triggering conditions.

[0070] When the forwarding monitor detects that the video stream switching conditions are met, it generates or updates the forwarding rules. For example, if the currently active streamer is streamer A and the target streamer is streamer B, then the identifier of the current output video stream in the forwarding rules can be updated from video source A to video source B, and the identifier of the currently active streamer can be updated from streamer A to streamer B.

[0071] The updated forwarding rules are sent to the forwarding controller. The forwarding controller then performs a puller state switch based on the updated forwarding rules.

[0072] Step S5 enables automatic triggering and rule-based control of video stream switching; it can adjust the current output video stream in real time according to the business status and forwarding status; it supports various business scenarios such as timed rotation, follow-up video, alarm linkage, and manual switching; it separates the switching decision from the pull and push stream execution, improving the clarity of the system structure; and it provides a control basis for the subsequent target puller to quickly take over the writing of the forwarding queue.

[0073] Step S6: Switch the currently active streamer according to the updated forwarding rules.

[0074] In step S6, after updating the forwarding rules, the system controls the original currently active streaming puller to stop writing video frames to the forwarding queue, and controls the target streaming puller to write its pre-pulled and decoded video frames to the forwarding queue, so that the pusher can continuously push the switched video frames without interrupting the connection with the streaming media server.

[0075] Specifically, after receiving the updated forwarding rules, the forwarding controller first identifies the original currently active stream puller and the target stream puller. Then, the forwarding controller updates the state of the original currently active stream puller from the active forwarding state to the pre-pull state, and updates the state of the target stream puller from the pre-pull state to the active forwarding state.

[0076] After the status update is complete, the previously active streamer no longer writes video frames to the forwarding queue, but can still maintain its connection with its video source and continue pre-streaming and decoding. The target streamer begins writing video frames to the forwarding queue. Because the target streamer has already completed pre-streaming and decoding before the switch, it can immediately provide video frames.

[0077] During the switching process, the streamer remains operational and continues to read video frames from the forwarding queue. The connection between the streamer and the streaming media server remains uninterrupted. In other words, video source switching in this application is not equivalent to stream connection switching. The stream link remains stable; only the source of video frames in the forwarding queue changes from the original currently active streamer to the target streamer.

[0078] To further ensure a smooth handover, in some optional implementations, the forwarding controller can perform the following processing during handover: 1. Before switching, check whether the target streamer has already captured usable video frames; 2. Clean up outdated video frames in the forwarding queue during switching; 3. Adjust the output timestamp of the target video frame based on the video frame timestamp; 4. Delay switching or select other candidate streamers if the target streamer is not yet ready; 5. Perform frame numbering continuation processing on the video frames before and after the switch.

[0079] The above-described processing methods can all be used as optional implementations of this application and are not intended to limit the scope of protection of this application.

[0080] Step S6 simplifies the video stream switching process to a switch of write permissions for the puller; the target puller has already pre-pulled and decoded the stream, so it can quickly take over the writing of the forwarding queue; the pusher does not interrupt the connection with the streaming media server, avoiding disconnection of external push; reducing black screen, stuttering and stream interruption; the original puller can still maintain the pre-pulled state after switching, making it easy to switch back later; improving the continuity in frequently switching scenarios such as video carousel and follow-up video.

[0081] Step S7: Monitor the running status and perform anomaly recovery operations.

[0082] In step S7, the system monitors the operating status of the puller, pusher, and forwarder. When an abnormal state is detected, the system executes the corresponding abnormal recovery operation through the forwarder controller.

[0083] The forwarding controller can periodically monitor the thread status, heartbeat status, connection status, data stream status, and queue status of each module. For example, the forwarding controller can detect whether the puller thread is alive, whether the puller is continuously receiving video data, whether the puller decoding is normal, whether the pusher is continuously pushing video frames, whether the forwarding monitor is updating rules normally, and whether the forwarding queue is blocked or growing abnormally.

[0084] Abnormal states may include, but are not limited to: 1. Streamer thread abnormality; 2. Streamer unable to connect to video source; 3. Streamer connection interruption; 4. Streamer decoding abnormality; 5. No video frame output from the currently active streamer; 6. Pusher thread abnormality; 7. Pusher unable to connect to streaming media server; 8. Pusher push failure; 9. Forwarding monitor abnormality; 10. Forwarding queue blocked, overflowing, or damaged.

[0085] When an abnormality is detected in the streamer thread, the forwarding controller can restart the corresponding streamer and restore the video source connection, streaming, and decoding status of the streamer according to the current forwarding rules.

[0086] When an abnormality is detected in the pusher thread, the forwarding controller can restart the pusher and reconnect to the streaming media server to resume the operation of reading video frames from the forwarding queue and pushing them to the streaming media server.

[0087] When an anomaly is detected in the forwarding monitor, the forwarding controller can restart the forwarding monitor and reload or generate the current forwarding rules.

[0088] When an anomaly is detected in the video source corresponding to the currently active streaming puller, the forwarding controller can switch the candidate streaming puller in the pre-streaming state to the new active streaming puller to reduce the impact of the current video source failure on the overall streaming.

[0089] When an anomaly is detected in the forwarding queue, the forwarding controller can clean up or rebuild the forwarding queue and resume video frame writing based on the currently active streamer.

[0090] Step S7 improves the system's fault detection capability; enables timely recovery when the stream puller, pusher, or forwarding monitor malfunctions; allows for rapid takeover of output via a candidate stream puller when the currently active stream puller malfunctions; reduces the probability of streaming interruption due to a single point of failure; and enhances the stability and reliability of the video stream forwarding system in long-term operating scenarios.

[0091] Figure 3 This is a flowchart illustrating the pre-pulling and decoding of a stream puller in a video stream forwarding method according to an embodiment of this application.

[0092] Step S21: The streamer establishes a network connection with the corresponding video source based on the video source address.

[0093] In step S21, the stream puller establishes a network connection with the corresponding video source based on the video source address configured in the forwarding rules. The video source address can be an RTSP address, RTMP address, HTTP-FLV address, GB / T 28181 device address, WebRTC signaling address, video platform interface address, or other addresses that can access the video stream.

[0094] When establishing a connection, the streaming adapter can perform corresponding handshakes, authentication, channel establishment, and media negotiation according to the video source protocol. For example, for RTSP video sources, the streaming adapter can sequentially execute interactive processes such as OPTIONS, DESCRIBE, SETUP, and PLAY; for RTMP video sources, the streaming adapter can perform handshakes and stream playback requests; for GB / T 28181 video sources, the streaming adapter can request the device to push the bitstream through the signaling platform.

[0095] In some implementations, the puller can also support configuration of authentication parameters, username and password, token, device channel number, and stream type.

[0096] Step S21 establishes a connection with the candidate video source in advance, avoiding the need to rebuild the connection during handover; ensures that the candidate video source is accessible before the formal handover; reduces the impact of video source network connection latency on the handover process; and supports multiple video source protocols, improving system applicability.

[0097] Step S22: The streamer continuously acquires raw video data from the video source.

[0098] In step S22, the streamer continuously acquires raw video data from the video source. The raw video data can be an encoded and encapsulated video stream, such as data in encoding formats like H.264, H.265, MPEG-4, and MJPEG. The raw video data can be transmitted in the form of RTP packets, RTMP messages, FLV tags, TS fragments, or other formats.

[0099] While continuously acquiring raw video data, the streaming adapter can detect network jitter, packet loss, timeouts, and other issues. When short-term network fluctuations occur, the streaming adapter can perform retry, buffering, or reconnection to maintain the pre-streaming state.

[0100] When the candidate stream puller is in the pre-stream pull state, although it does not write video frames to the forwarding queue, it continues to acquire raw video data, thereby ensuring that its internal data link is active.

[0101] Step S21 ensures that the candidate video stream remains available, avoids waiting for video data to arrive during switching, detects connection or data anomalies of the candidate video source in advance, and guarantees that video frames can be output immediately when the target streamer is switched to the current effective streamer.

[0102] Step S23: The streamer decapsulates and decodes the raw video data.

[0103] In step S23, the puller decapsulates and decodes the original video data to generate video frames that can be forwarded by the pusher.

[0104] Decapsulation refers to extracting encoded video data from a network transmission format or container format. For example, reconstructing H.264 / H.265 streams from RTP packets, or extracting video data from FLV tags.

[0105] Decoding refers to converting encoded video data into video frames that can be processed and pushed later. Depending on the system design, a video frame can be a raw image frame or an encoded frame that has been parsed and timestamped. In this application, "video frame" is a broad concept, which can include decoded image frames, as well as encoded frames or frame data units that can be directly encapsulated and forwarded by the streaming device.

[0106] During the decoding process, the streaming adapter can maintain information such as video frame timestamps, frame sequence numbers, keyframe identifiers, and frame types. This information can be used for continuous streaming by the pusher, or for timestamp adjustment during switching.

[0107] Step S23 completes the decapsulation and decoding of candidate video streams in advance, reducing processing delays after switching; ensures that the target streamer can directly provide forwardable video frames after switching to the current effective streamer; improves the continuity of video frame push through timestamp and frame sequence number management; and reduces the probability of waiting for key frames and decoding initialization during the switching process.

[0108] Step S24: The streamer buffers video frames or writes them to the forwarding queue when it becomes the currently active streamer.

[0109] In step S24, the streamer buffers the decoded video frames in a local buffer, or writes the decoded video frames into a forwarding queue when it is determined to be the current valid streamer.

[0110] For a streamer not currently identified as the active streamer, it can buffer the decoded video frames in a local buffer. The local buffer can either store only the most recent few frames to avoid excessive memory usage, or it can store only the most recent keyframe and a few subsequent frames for quick output from a decodeable frame during switching.

[0111] For a streamer that is identified as the current valid streamer, its decoded video frames are written into the forwarding queue, and then read and pushed by the pusher.

[0112] Pullers that are not identified as currently active pullers maintain their connection to the corresponding video source and continue to perform pull and decode operations in order to provide video frames that can be forwarded immediately when a video stream switch occurs.

[0113] In step S24, available video frames are retained in the candidate streamers to improve the switching response speed; the currently effective streamer writes video frames to the unified forwarding queue to ensure a stable data source for the streamer; streamers that are not selected are not written to the forwarding queue to avoid video frame mixing; and the balance between memory usage and switching availability is controlled through the local buffer.

[0114] Figure 4 This is a flowchart illustrating the switching state of the stream forwarding device in a video stream forwarding method according to an embodiment of this application.

[0115] Step S61: Set forwarding status flags for the current active streamer and the target streamer respectively.

[0116] In step S61, the forwarding controller sets forwarding status flags for both the current active streamer and the target streamer. The forwarding status flag indicates whether the streamer currently has the permission to write video frames to the forwarding queue. The forwarding status flag can be represented by a Boolean value, an enumerated value, a status field, a state machine node, or other equivalent forms.

[0117] For example, forwarding status identifiers may include: valid forwarding status; pre-pull stream status; abnormal status; initialization status; and stopped status.

[0118] Among them, the effective forwarding state indicates that the corresponding puller can write video frames to the forwarding queue; the pre-pull state indicates that the corresponding puller keeps pulling and decoding, but does not write video frames to the forwarding queue.

[0119] Step S61 clarifies the role of each stream puller through status identifiers; provides a control basis for subsequent write permission switching; avoids multiple stream pullers writing to the forwarding queue simultaneously; and makes the forwarding control logic clearer and more maintainable.

[0120] Step S62: Update the forwarding status flag of the current active stream puller from active forwarding status to pre-pull status.

[0121] In step S62, the forwarding controller updates the forwarding status flag of the currently active stream puller from active forwarding status to pre-pull status.

[0122] After the status update, the previously active streaming puller stops writing video frames to the forwarding queue, but can maintain the connection with the original video source and continue streaming and decoding operations. This way, when it's necessary to switch back to the original video source later, the original streaming puller remains available and can quickly become the active streaming puller again.

[0123] In some implementations, the forwarding controller can wait for the currently active puller to complete writing the current frame before updating its write permission, thus avoiding half-frame writes or incomplete data. Thread safety during state updates can also be ensured through mutexes, atomic variables, message queues, etc.

[0124] Step S62 removes the original active streamer from the forwarding queue writing role; prevents the original video source from continuing to write video frames to the forwarding queue, thus affecting the output after switching; maintains the original streamer's pre-streaming state, facilitating quick switching back later; and reduces the impact of video source switching on the streamer's connection status.

[0125] Step S63: Update the forwarding status flag of the target stream puller from the pre-pull status to the valid forwarding status.

[0126] In step S63, the forwarding controller updates the forwarding status flag of the target streamer from the pre-streaming status to the valid forwarding status.

[0127] Before switching, the target streamer has already established a connection with the target video source and continuously pulls and decodes the video stream. Therefore, once its status is updated to a valid forwarding status, the pre-pulled and decoded video frames can be immediately written into the forwarding queue.

[0128] In some implementations, the forwarding controller can determine whether the target streamer meets the switching readiness conditions before performing a state update. For example, the target streamer may have a normal connection, normal decoding, available video frames in its local buffer, and a normal video frame output frequency recently. Only when the target streamer meets the readiness conditions is it set to a valid forwarding state.

[0129] If the target streamer does not meet the preparation conditions, the forwarding controller can select other candidate streamers, continue to maintain the output of the currently valid streamer, or execute the abnormal recovery process.

[0130] Step S63 enables the target stream puller to quickly take over the writing of the forwarding queue; utilizes the results of pre-pulling and pre-decoding to avoid waiting for video frames after switching; improves the real-time performance of video stream switching; and enhances switching reliability through preparation condition judgment.

[0131] Step S64: Control whether the corresponding puller writes video frames to the forwarding queue based on the updated forwarding status flag.

[0132] In step S64, each streamer determines whether to write a video frame to the forwarding queue based on the updated forwarding status flag.

[0133] If the streamer's forwarding status is marked as valid forwarding, the streamer will write the decoded video frames into the forwarding queue. If the streamer's forwarding status is marked as pre-streaming, the streamer will continue streaming and decoding, but will not write video frames into the forwarding queue.

[0134] The streamer remains operational during the switching process, continuously reading video frames from the forwarding queue. Because the streamer does not need to reconnect to the streaming media server, the switching process appears to the outside world as a change in the video source, rather than an interruption of the streaming link.

[0135] Step S64 enables dynamic control of video frame write permissions through status flags; ensures that video frames in the forwarding queue after switching originate from the target streaming device; the streaming device continuously reads the forwarding queue to maintain continuous external streaming; decouples video source switching from the streaming link; and reduces black screen, streaming interruption, and resynchronization delays during switching.

[0136] Figure 5 This is a structural block diagram illustrating a video stream forwarding device according to an embodiment of this application.

[0137] like Figure 5 As shown, the video stream forwarding device 50 according to an embodiment of this application includes a forwarding monitor 51, a forwarding controller 52, at least two stream pullers 53, and a stream pusher 54.

[0138] The forwarding monitor 51 is configured to acquire video stream forwarding tasks, monitor the status of the business system, forwarding status and / or video stream switching conditions, and generate and dynamically update forwarding rules corresponding to the video stream forwarding tasks.

[0139] The forwarding monitor 51 may include a service status monitoring unit, a forwarding status monitoring unit, a switching condition judgment unit, a rule generation unit, and a rule update unit.

[0140] The business status monitoring unit is used to obtain the status of the business system. For example, in a video carousel scenario, the business status can be the status of the carousel timer; in a live video scenario, the business status can be the location of the target object; in an alarm linkage scenario, the business status can be the alarm event; and in a manual operation scenario, the business status can be the video source selected by the user.

[0141] The forwarding status monitoring unit is used to monitor the running status of the current forwarding task, including the current output video stream, candidate video stream, puller status, pusher status, forwarding queue status, etc.

[0142] The switching condition judgment unit is used to determine whether the video stream switching conditions are met. Video stream switching conditions may include timed rotation conditions, changes in the target video source specified by the business system, on-the-fly video switching conditions, abnormal conditions of the current video source, conditions of degraded picture quality of the current video source, manual switching conditions by the user, preset priority change conditions, or external event triggering conditions.

[0143] The rule generation unit generates initial forwarding rules during system startup or task initialization. The rule update unit dynamically updates the forwarding rules when switching conditions are met.

[0144] The forwarding monitor 51 enables rule-based management of video stream forwarding tasks; dynamically determines whether to switch based on service status and forwarding status; decouples switching decisions from pull and push stream execution; improves the system's adaptability to multiple service scenarios; and provides clear scheduling basis for the forwarding controller.

[0145] The forwarding controller 52 is configured to determine and schedule at least two pullers 53 and one pusher 54 according to forwarding rules.

[0146] The forwarding controller 52 may include a task initialization unit, a streamer management unit, a streamer management unit, a status management unit, a thread monitoring unit, and an exception recovery unit.

[0147] The task initialization unit is used to initialize the system state when the video stream forwarding task starts.

[0148] The streamer management unit is used to determine or create multiple streamers according to the forwarding rules, and to control the streamers to start, stop, connect to video sources, enter pre-streaming state or effective forwarding state.

[0149] The pusher management unit is used to identify or create pushers and control the pushers to connect to the streaming media server, read video frames from the forwarding queue, and push them.

[0150] The status management unit is used to maintain the forwarding status identifiers of each stream puller, such as the valid forwarding status and the pre-pull status.

[0151] The thread monitoring unit is used to monitor the running status of the puller, pusher, and forwarder monitors.

[0152] The anomaly recovery unit is used to perform recovery operations when an abnormal state is detected, such as restarting the puller, restarting the pusher, reloading forwarding rules, switching to a backup puller, and cleaning or rebuilding the forwarding queue.

[0153] The forwarding controller 52 enables centralized scheduling of the puller and pusher; controls video stream output according to forwarding rules; switches write permissions for the puller through state management; improves system reliability through thread monitoring and exception recovery; and simplifies the multi-step coordination process in traditional video switching.

[0154] At least two pullers 53 are configured to establish connections with different video sources, respectively, to pre-pull and decode the video streams of the corresponding video sources. One puller is configured as the currently active puller and is used to write the decoded video frames to the forwarding queue, while the other pullers are configured as pre-pull pullers and maintain the pulling and decoding state but do not write video frames to the forwarding queue.

[0155] Each streamer may include a connection unit, a streamer unit, a decapsulation unit, a decoding unit, a buffer unit, and a write control unit.

[0156] The connection unit establishes a network connection with the video source based on the video source address. The streaming unit continuously acquires raw video data. The decapsulation unit extracts the video bitstream from the transport protocol or encapsulation format. The decoding unit generates video frames that can be forwarded. The buffering unit buffers the decoded video frames. The write control unit determines whether to write video frames to the forwarding queue based on the forwarding status flag.

[0157] At least two pullers 53 establish candidate video source connections in advance; pull and decode candidate video streams in advance; enable the target video stream to be forwarded immediately during switching; avoid the mixing of frames from different video sources through write control; and improve the switching speed and continuity of video streams.

[0158] Forwarding queues can be thread-safe queues, blocking queues, circular buffer queues, shared memory queues, or other data structures suitable for the producer-consumer model.

[0159] The forwarding queue can be configured with a queue length threshold. When the queue length exceeds the threshold, the system can discard older video frames to ensure real-time performance. The forwarding queue can also record video frame timestamps and frame sequence numbers so that the streamer can read them in order.

[0160] The pusher 54 is configured to read video frames from the forwarding queue and push the read video frames to the streaming media server.

[0161] The streaming unit 54 may include a queue reading unit, a frame processing unit, an encapsulation unit, a pushing unit, and a connection maintenance unit.

[0162] The queue reading unit reads video frames from the forwarding queue. The frame processing unit processes video frame timestamps, frame numbers, or encoding parameters as needed. The encapsulation unit encapsulates video frames according to the target streaming protocol. The push unit sends the encapsulated video data to the streaming media server. The connection maintenance unit maintains the connection with the streaming media server.

[0163] During video stream switching, the pusher 54 does not interrupt the connection and continues to read video frames from the forwarding queue, thus enabling it to continuously push the switched video frames.

[0164] The Streamer 54 maintains a stable external streaming link; is decoupled from specific video sources; does not require re-establishing a streaming connection when switching video sources; reduces client black screens, stream interruptions, and stuttering; and supports the push requirements of different streaming media protocols.

[0165] Figure 6 This is a structural block diagram of an electronic device according to an embodiment of this application. For example... Figure 6 As shown, the electronic device 60 includes a memory 61 and a processor 62.

[0166] The memory 61 is used to store computer-readable instructions. When executed by the processor 62, the computer-readable instructions cause the electronic device 60 to perform the video stream forwarding method described in any embodiment of this application.

[0167] Memory 61 may include volatile memory and / or non-volatile memory. For example, memory 61 may include random access memory, read-only memory, flash memory, hard disk, solid-state drive, storage array, or other media capable of storing computer-readable instructions.

[0168] The memory 61 can store forwarding rules, video source configuration, streaming media server configuration, puller status information, pusher status information, exception recovery strategy, log data and program code.

[0169] The processor 62 is used to execute computer-readable instructions stored in the memory 61, causing the electronic device 60 to perform the video stream forwarding method described in any embodiment of this application.

[0170] The processor 62 may be a central processing unit, a graphics processing unit, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, a microcontroller, or other computing device capable of executing program instructions.

[0171] When the processor 62 executes computer-readable instructions, it can perform the following operations: obtain video stream forwarding tasks and forwarding rules; determine the puller and pusher based on the forwarding rules; control multiple pullers to perform pre-pulling and decoding; determine the currently effective puller; control the currently effective puller to write video frames to the forwarding queue; control the pusher to read video frames from the forwarding queue and push them; monitor the status of the business system and the forwarding status; dynamically update the forwarding rules; control the target puller to take over the writing to the forwarding queue; monitor anomalies and perform recovery operations.

[0172] Figure 7 This is a schematic diagram illustrating a computer program product according to an embodiment of this application. For example... Figure 7 As shown, the computer program product 70 includes a computer program 71. When executed by a processor, the computer program 71 implements the video stream forwarding method described in any embodiment of this application.

[0173] The computer program product 70 may exist as a software installer, container image, script file, executable file, plugin, service program, firmware program, or other form. The computer program product 70 may be stored on a computer-readable storage medium or provided via network download, cloud deployment, or platform distribution.

[0174] When a computer program is executed by a processor, it can enable a computing device to perform the following functions: load video stream forwarding tasks; parse forwarding rules; initialize multiple pullers and one pusher; control the pullers to connect to video sources; control the pullers to perform pre-pulling and decoding; select the currently active puller; control the currently active puller to write video frames to the forwarding queue; control the pusher to continuously push video frames to the streaming media server; monitor video stream switching conditions; dynamically update forwarding rules; switch the target puller to the new currently active puller; and perform anomaly detection and recovery.

[0175] The video stream forwarding method described in any embodiment of this application can be used in a variety of application scenarios.

[0176] In video carousel scenarios, the system needs to switch between multiple cameras at preset time intervals. For example, the system needs to output video streams from camera A, camera B, and camera C in sequence, with each video stream output for 10 seconds.

[0177] In traditional solutions, each switch requires closing the current video stream and opening the next one, which can easily cause black screens or stuttering on the user's end.

[0178] When using this application, the system determines streamer A, streamer B, and streamer C for camera A, camera B, and camera C, respectively. All three streamers establish connections with their corresponding cameras in advance and continuously stream and decode. Initially, streamer A is the currently active streamer, writing video frames into the forwarding queue; streamers B and C remain in a pre-streaming state.

[0179] When the 10-second loop cycle is reached, the forwarding monitor detects that the timed loop conditions are met and dynamically updates the forwarding rules, switching the currently active puller from puller A to puller B. Based on the updated forwarding rules, the forwarding controller sets puller A to pre-pulling state and puller B to active forwarding state. Puller B immediately writes the pre-pulled and decoded video frames into the forwarding queue. The pusher continuously reads video frames from the forwarding queue and pushes them to the streaming media server.

[0180] In this scenario, this application can significantly reduce black screens and interruptions caused by carousel switching, and improve the continuity of video carousels.

[0181] In on-camera video scenarios, the system needs to switch video sources based on changes in the target object's location. For example, when the vehicle is in area A, the video from camera A is output; when the vehicle enters area B, the video from camera B is output; and when the vehicle enters area C, the video from camera C is output.

[0182] When using this application, the streaming devices corresponding to the cameras in each area are pre-connected and pre-streamed. When the business system detects a change in vehicle location, the forwarding monitor updates the forwarding rules based on the target location. The forwarding controller then switches the currently active streaming device according to the updated forwarding rules.

[0183] Since the streamer corresponding to the target area is already in pre-streaming mode, it can quickly take over the output. This allows the follow-up video to switch continuously as the target object moves, reducing latency caused by reopening the video stream.

[0184] In alarm linkage scenarios, when an abnormal event occurs in a certain area, the system needs to immediately switch to the video stream of the corresponding camera in that area.

[0185] For example, the current output video is from camera A. When an intrusion alarm is triggered in the area where camera C is located, the forwarding monitor detects the external event trigger condition, generates a new forwarding rule, and sets the stream puller C corresponding to camera C as the target stream puller. The forwarding controller switches stream puller C to the currently valid stream puller, causing it to write video frames to the forwarding queue.

[0186] Because the stream puller C pre-pulls and decodes the stream, it can quickly output the alarm area image. This method can reduce information loss caused by video switching delays when a security incident occurs.

[0187] In some alternative implementations, the forwarding queue can be a circular buffer queue. A circular buffer queue can be configured with a maximum capacity, allowing the system to overwrite the oldest video frames when the write speed exceeds the read speed, thus preventing latency from accumulating.

[0188] In some alternative implementations, the puller can pre-pull the candidate video stream without fully decoding it, performing fast decoding before or at the moment of switching. This approach reduces CPU usage, but the switching speed may be slower than a fully pre-decoded solution.

[0189] In some optional implementations, the system can set priorities for different video sources. When the currently valid video source is abnormal, the forwarding controller prioritizes a candidate streamer with higher priority that is in the pre-streaming state as the new current valid streamer.

[0190] In some optional implementations, the forwarding monitor can trigger a switch based on video quality metrics. For example, when the current video source experiences screen tearing, freezing, low frame rate, abnormal bitrate, or degraded image quality, the forwarding monitor can update the forwarding rules and switch the current output to an alternative video source.

[0191] In some alternative implementations, the system can process the video frame timestamps continuously during switching to reduce playback anomalies caused by sudden changes in timestamps on the streaming media server or client.

[0192] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0193] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0194] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0195] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0196] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0197] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0198] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A seamless video stream forwarding method, characterized in that, include: S1. Obtain the video stream forwarding task and the forwarding rules corresponding to the video stream forwarding task. The forwarding rules include at least the current video stream identifier to be output, the candidate video stream identifier, the video source address, the streaming media server address, and the video stream switching conditions. S2. According to the forwarding rules, determine at least two streaming pullers and one streaming pusher, wherein the at least two streaming pullers establish connections with different video sources respectively, and pre-pull and decode the video streams of the corresponding video sources; S3. Determine the currently effective streamer among the at least two streamers, and control the currently effective streamer to write its decoded video frames into the forwarding queue, while controlling the other streamers to maintain the pre-streaming state but not to write video frames into the forwarding queue. S4. Read video frames from the forwarding queue through the pusher and push the read video frames to the streaming media server to form a video stream for external output.

2. The seamless switching video stream forwarding method according to claim 1, characterized in that, Also includes: S5. Monitor the status of the business system, forwarding status and / or video stream switching conditions in real time through the forwarding monitor. When the video stream switching conditions are detected, dynamically update the forwarding rules and switch the target streamer to the new currently effective streamer. S6. After updating the forwarding rules, control the original currently active streaming device to stop writing video frames to the forwarding queue, and control the target streaming device to write its pre-pulled and decoded video frames to the forwarding queue, so that the streaming device can continuously push the switched video frames without interrupting the connection with the streaming media server.

3. The seamless switching video stream forwarding method according to claim 2, characterized in that, Also includes: S7. Monitor the operating status of the puller, pusher, and forwarding monitor. When an abnormal state is detected, execute the corresponding abnormal recovery operation through the forwarding controller.

4. The seamless switching video stream forwarding method according to any one of claims 1 to 3, characterized in that, The stream puller performs pre-stream pulling and decoding, including: S21. The streaming device establishes a network connection with the corresponding video source based on the video source address; S22. The streamer continuously acquires raw video data from the video source; S23. The streaming device decapsulates and decodes the original video data to generate video frames that can be forwarded by the streaming device. S24. The streamer buffers the decoded video frames in a local buffer, or writes the decoded video frames into a forwarding queue when it is determined to be the current valid streamer. Among them, the pullers that are not identified as the current valid pullers maintain the connection with the corresponding video source and continue to perform pull and decoding operations so as to provide video frames that can be forwarded immediately when a video stream switch occurs.

5. The seamless switching video stream forwarding method according to claim 2 or 3, characterized in that, The video stream switching conditions include at least one of the following: timed rotation conditions, target video source change conditions specified by the business system, on-the-fly video switching conditions, current video source abnormal conditions, current video source picture quality degradation conditions, user manual switching conditions, preset priority change conditions, or external event triggering conditions. The forwarding monitor generates or updates forwarding rules based on the video stream switching conditions, and sends the updated forwarding rules to the forwarding controller, which then schedules the puller and the pusher according to the updated forwarding rules.

6. The seamless switching video stream forwarding method according to claim 2 or 3, characterized in that, In step S6, controlling the currently active streaming device to stop writing video frames to the forwarding queue and controlling the target streaming device to write video frames to the forwarding queue includes: S61. Set forwarding status flags for the current active streamer and the target streamer respectively; S62. Update the forwarding status flag of the current active stream puller from active forwarding status to pre-pull status; S63. Update the forwarding status flag of the target streamer from the pre-streaming status to the valid forwarding status; S64. Control whether the corresponding streamer writes video frames to the forwarding queue based on the updated forwarding status identifier; The streamer remains operational during the switching process and continuously reads video frames from the forwarding queue.

7. The seamless switching video stream forwarding method according to claim 3, characterized in that, The anomaly recovery operation in S7 includes at least one of the following: When an abnormality is detected in the streaming thread, the corresponding streaming thread is restarted, and the video source connection, streaming, and decoding status of the streaming thread are restored according to the current forwarding rules. When an abnormality is detected in the pusher thread, the pusher is restarted and the streaming media server is reconnected to resume the operation of reading video frames from the forwarding queue and pushing them to the streaming media server. When an anomaly is detected in the forwarding monitor, the forwarding monitor is restarted, and the current forwarding rules are reloaded or generated. When an anomaly is detected in the video source corresponding to the currently valid streamer, the candidate streamer in the pre-streaming state will be switched to the new currently valid streamer. When an anomaly is detected in the forwarding queue, the forwarding queue is cleaned up or rebuilt, and video frame writing is resumed based on the currently active streamer.

8. A seamless video stream forwarding device, characterized in that, include: The forwarding monitor is configured to acquire video stream forwarding tasks, monitor the status of the business system, forwarding status and / or video stream switching conditions, and generate and dynamically update forwarding rules corresponding to the video stream forwarding tasks. A forwarding controller is configured to determine and schedule at least two pullers and one pusher based on the forwarding rules. At least two streamers are configured to establish connections with different video sources respectively, and to pre-pull and decode the video streams of the corresponding video sources. One streamer is configured as the currently active streamer and is used to write the decoded video frames into the forwarding queue. The other streamers are configured as pre-pull streamers and maintain the pulling and decoding state but do not write video frames into the forwarding queue. A pusher is configured to read video frames from the forwarding queue and push the read video frames to the streaming media server; When the forwarding monitor detects that the video stream switching conditions are met, the forwarding controller controls the target stream puller to switch to the new currently effective stream puller according to the updated forwarding rules, so that the pusher can continuously push the switched video frames without interrupting the connection.

9. An electronic device, characterized in that, include: Memory, used to store computer-readable instructions; as well as A processor for executing the computer-readable instructions, causing the electronic device to perform the video stream forwarding method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the video stream forwarding method as described in any one of claims 1 to 7.

Images