Method for realizing seamless switching by playing live stream in split screen mode

Abstract

The invention discloses a method for realizing seamless switching by playing live streaming in a split-screen manner, belongs to the technical field of live streaming, and solves the technical problems of overhigh pressure of a mixed streaming server, jamming, black screen and the like caused by no network load planning in the existing live streaming switching. The method comprises a stream pushing terminal and a playing terminal. The method comprises the following steps of: respectively arranging a non-stream mixing server, a front stream mixing server and a rear stream mixing server; the priority is preset at the flow pushing terminal; the stream pushing terminal is responsible for pushing the live stream to the non-stream mixing server or the front stream mixing server; the non-mixed-flow server directly pushes the live broadcast flow to a rear mixed-flow server; the front stream mixing server mixes the live streams according to a set layout and then pushes the mixed streams to the rear stream mixing server; the rear flow mixing server always superposes the live stream pulled from the non-flow mixing server to the live stream pulled from the front flow mixing server according to the set layout, and finally pushes the live stream to the playing terminal, thereby ensuring that the seamless switching effect can be achieved when the flow pushing terminal cuts the flow.

Description

Technical Field

[0001] The present invention relates to the field of live broadcast technology, and more specifically, to a method for achieving seamless switching of split-screen live broadcast streams. Background Art

[0002] With the rapid development of online live broadcast technology, users have higher and higher requirements for live broadcast experience. Live stream switching technology allows users to seamlessly switch between different live broadcast contents, providing a richer and more personalized viewing experience. Among them, multi-screen playback technology allows users to watch multiple video images on the same screen, which is very useful in application scenarios such as video surveillance and multi-angle live broadcast.

[0003] Patent application document CN116684655A designs a method for switching from a single live stream to at least two live streams playing simultaneously without disconnecting from the client. However, since the same server will simultaneously create new streaming media components for mixed streaming, the process of switching to mixed streaming takes a long time, resulting in freezes and black screens in the live stream, leading to a poor user experience. Patent application document CN115997384A designs a method that uses only a single-pull and push-back stream cloud server for processing when playing on a single screen, and uses a dedicated mixed stream server for processing when playing on split screens, and then passes it to the cloud server mentioned in the former for pushing the stream. This method has a better live stream switching experience, but every time a new live stream is added, it will all be pushed to the mixed stream server by default, without network load planning, causing excessive pressure on the mixed stream server.

[0004] Therefore, there is an urgent need for a method based on dynamic live stream switching to solve the above problems. Summary of the invention

[0005] The technical problem to be solved by the present invention is aimed at the above-mentioned shortcomings of the prior art. The purpose of the present invention is to provide a method for achieving seamless switching of live streams during split-screen playback, which can improve the efficiency of live stream switching during split-screen playback and user experience, while optimizing the allocation and utilization of server resources, reducing resource waste, and maintaining stable operation of the system under high load conditions. It can be widely used in the fields of online anchors, online education, production monitoring, etc.

[0006] The technical solution of the present invention is: a method for realizing seamless switching of split-screen live streaming, comprising a streaming terminal and a playing terminal, and respectively setting a non-mixed streaming server, a front-end mixed streaming server, and a rear-end mixed streaming server;

[0007] Preset the priority level on the streaming terminal;

[0008] The streaming terminal is responsible for pushing the live stream to the non-mixed stream server or the front-end mixed stream server;

[0009] The non-mixed stream server directly pushes the live stream to the post-mixed stream server, and the number of pushed streams is consistent with the number of pulled streams;

[0010] The front-end mixing server mixes the live streams according to the set layout, and pushes the mixed live streams to the back-end mixing server, triggering only one back-pushing stream;

[0011] The post-mixing server always superimposes the live stream pulled from the non-mixing server on the live stream pulled from the pre-mixing server according to the set layout, and finally pushes it to the playback terminal, thereby ensuring that a seamless switching effect can be achieved when the streaming terminal switches the stream.

[0012] As a further improvement, when only one streaming terminal is performing live streaming, it is pushed to the non-mixed streaming server by default. The non-mixed streaming server directly pushes the received live streaming to the post-mixed streaming server, and the post-mixed streaming server finally pushes it to the playback terminal.

[0013] Furthermore, when a new streaming terminal is pushing live stream, if the new streaming terminal has a higher priority, the live stream of the new streaming terminal will be pushed to the non-mixed stream server, and the non-mixed stream server will independently push the received live stream to the post-mixed stream server, and the post-mixed stream server will mix the live stream according to the set layout and push it to the playback terminal.

[0014] Furthermore, when a new streaming terminal is pushing live streams, if the new streaming terminal has a lower priority, the live stream of the new streaming terminal is pushed to the front-end mixing server, the front-end mixing server mixes all received live streams and pushes them to the rear-end mixing server, the rear-end mixing server mixes the live streams of the non-mixed server with the front-end mixing server, and pushes them to the playback terminal.

[0015] Furthermore, when the CPU load of the non-mixed stream server is too high or the network fluctuates, the streaming terminal with a low priority is notified to push the live stream to the front-end streaming server for mixing at the same time, and the rear-end streaming server superimposes the live stream of the front-end streaming server on the live stream of the rear-end streaming server according to the set layout for mixing. After the GOP frames of the streams to be switched are aligned, the streaming terminal with a low priority disconnects the live stream pushed to the non-mixed stream server.

[0016] Furthermore, when the CPU load of the front-end mixing server is too high or the network fluctuates, the streaming terminal with a high priority is notified to push the live stream to the non-mixed stream server at the same time. The non-mixed stream server pushes the newly received live stream to the post-mixing server. The post-mixing server superimposes the live stream of the front-end mixing server on the live stream of the post-mixing server according to the set layout for mixing. After the GOP frames of the streams to be switched are aligned, the streaming terminal with a high priority disconnects the live stream pushed to the front-end mixing server.

[0017] Beneficial Effects

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] The present invention performs dynamic live stream switching according to the preset priority of the push terminal through the service architecture of a non-mixed stream server, a front-end mixed stream server, and a rear-end mixed stream server. The rear-end mixed stream server always superimposes the live stream pulled from the non-mixed stream server on the live stream pulled from the front-end mixed stream server according to a predetermined layout, thereby achieving the effect of seamless switching of dynamic live streams. The effect is more significant when the number of push terminals is large and the number and priority are clear.

[0020] It can improve the efficiency of live stream switching and user experience during split-screen playback, while optimizing the allocation and utilization of server resources, reducing resource waste, and maintaining stable system operation under high load conditions. It can be widely used in network anchors, online education, production monitoring and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of streaming by a single streaming terminal;

[0022] Figure 2 This is a schematic diagram of allocating the streaming of newly added streaming terminals to non-mixed streaming servers;

[0023] Figure 3 This is a schematic diagram of allocating the push stream of the newly added push stream terminal to the front-end mixing stream player;

[0024] Figure 4 Schematic diagram of a streaming terminal simultaneously streaming to a front-end streaming mixing server;

[0025] Figure 5 Schematic diagram of the streaming terminal completing the stream switching to the front-end mixing server;

[0026] Figure 6 This is a schematic diagram of a streaming terminal simultaneously streaming to a non-mixed streaming server;

[0027] Figure 7 Schematic diagram of the streaming terminal completing the stream switching to the non-mixed stream server. DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the specific embodiments in the accompanying drawings.

[0029] See also Figure 1 to Figure 7 , a method for realizing seamless switching of split-screen live streaming playback, comprising a streaming terminal and a playback terminal, and respectively setting a non-mixed streaming server, a front-end mixed streaming server, and a rear-end mixed streaming server;

[0030] Preset the priority on the streaming terminal;

[0031] The streaming terminal is responsible for pushing the live stream to the non-mixed stream server or the front-end mixed stream server;

[0032] The non-mixed stream server pushes the live stream directly to the post-mixed stream server, and the number of pushed streams is the same as the number of pulled streams;

[0033] After the front-end mixing server mixes the live streams according to the set layout, it pushes the mixed live stream to the back-end mixing server, triggering only one back-end push stream, that is, the front-end mixing server only pushes one mixed live stream;

[0034] The post-mixing server always superimposes the live stream pulled from the non-mixing server on the live stream pulled from the front-end mixing server according to the set layout, and finally pushes it to the playback terminal, thereby ensuring that seamless switching can be achieved when the streaming terminal switches the stream.

[0035] like Figure 1 As shown, when only one streaming terminal is pushing live stream, it is pushed to the non-mixed stream server by default. The non-mixed stream server directly pushes the received live stream to the post-mixed stream server, and the post-mixed stream server finally pushes it to the playback terminal.

[0036] like Figure 2 As shown, when a new streaming terminal is pushing a live stream, if the new streaming terminal has a higher priority (i.e., a higher priority than the old streaming terminal), the live stream of the new streaming terminal will be pushed to the non-mixed stream server, and the non-mixed stream server will independently push the received live stream to the post-mixed stream server, and the post-mixed stream server will mix the live stream according to the set layout and push it to the playback terminal.

[0037] like Figure 3 As shown, when a new streaming terminal is pushing a live stream, if the priority of the new streaming terminal is lower (i.e., lower than the priority of the old streaming terminal), the live stream of the new streaming terminal is pushed to the front-end mixing server, the front-end mixing server mixes all received live streams and pushes them to the rear-end mixing server, the rear-end mixing server mixes the live streams of the non-mixed server and the front-end mixing server, and pushes them to the playback terminal.

[0038] like Figure 4 , Figure 5 As shown, when the CPU load of the non-mixed stream server is too high or the network fluctuates, the low-priority streaming terminal is notified to push the live stream to the front-end streaming server for mixing at the same time, and the rear-end streaming server superimposes the live stream of the front-end streaming server on the live stream of the rear-end streaming server according to the set layout for mixing. After the GOP frames of the streams to be switched are aligned, the low-priority streaming terminal disconnects the live stream pushed to the non-mixed stream server.

[0039] On the contrary, if Figure 6 , Figure 7 As shown, when the CPU load of the front-end mixing server is too high or the network fluctuates, the push terminal with a high priority is notified to push the live stream to the non-mixed stream server at the same time. The non-mixed stream server pushes the newly received live stream to the rear-end mixing server. The rear-end mixing server superimposes the live stream of the front-end mixing server on the live stream of the rear-end mixing server according to the set layout for mixing. After the GOP frames of the streams to be switched are aligned, the push terminal with a high priority disconnects the live stream pushed to the front-end mixing server.

[0040] That is, the non-mixed stream server and the front-end mixed stream server notify the push stream terminal according to the push stream terminal priority, network bandwidth, CPU load, etc., and push the live stream to each other at the same time. After the post-mixed stream player aligns the GOP frame, the push terminal disconnects the old live stream.

[0041] The above are only preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A method for achieving seamless switching of split-screen live streaming, comprising a streaming terminal and a playback terminal, characterized in that: Set up a non-mixed flow server, a front-end mixed flow server, and a rear-end mixed flow server respectively; Preset the priority level on the streaming terminal; The streaming terminal is responsible for pushing the live stream to the non-mixed stream server or the front-end mixed stream server; The non-mixed stream server directly pushes the live stream to the post-mixed stream server, and the number of pushed streams is consistent with the number of pulled streams; The front-end mixing server mixes the live streams according to the set layout, and pushes the mixed live streams to the back-end mixing server, triggering only one back-pushing stream; The post-mixing server always superimposes the live stream pulled from the non-mixing server on the live stream pulled from the pre-mixing server according to the set layout, and finally pushes it to the playback terminal, thereby ensuring that a seamless switching effect can be achieved when the streaming terminal switches the stream.

2. According to claim 1, a method for realizing seamless switching of split-screen live streaming is characterized in that: When only one streaming terminal is performing live streaming, it is pushed to the non-mixed streaming server by default. The non-mixed streaming server directly pushes the received live streaming to the post-mixed streaming server, and the post-mixed streaming server finally pushes it to the playback terminal.

3. The method for realizing seamless switching of split-screen live streaming according to claim 1, characterized in that: When a new streaming terminal is pushing live stream, if the new streaming terminal has a higher priority, the live stream of the new streaming terminal will be pushed to the non-mixed stream server, and the non-mixed stream server will independently push the received live stream to the post-mixed stream server, and the post-mixed stream server will mix the live stream according to the set layout and push it to the playback terminal.

4. The method for realizing seamless switching of split-screen live streaming according to claim 1, characterized in that: When a new streaming terminal is pushing live stream, if the priority of the new streaming terminal is lower, the live stream of the new streaming terminal is pushed to the front-end mixing server, and the front-end mixing server mixes all received live streams and pushes them to the rear-end mixing server, and the rear-end mixing server mixes the live streams of the non-mixed server and the front-end mixing server and pushes them to the playback terminal.

5. The method for realizing seamless switching of split-screen live streaming according to claim 1, characterized in that: When the CPU load of the non-mixed stream server is too high or the network fluctuates, the low-priority streaming terminals are notified to push the live streams to the front-end streaming server for mixing at the same time. The rear-end streaming server superimposes the live stream of the front-end streaming server on the live stream of the rear-end streaming server according to the set layout for mixing. After the GOP frames of the streams to be switched are aligned, the low-priority streaming terminals disconnect the live streams pushed to the non-mixed stream server.

6. The method for realizing seamless switching of split-screen live streaming according to claim 1, characterized in that: When the CPU load of the front-end mixing server is too high or the network fluctuates, the streaming terminal with a high priority is notified to push the live stream to the non-mixed stream server at the same time. The non-mixed stream server pushes the newly received live stream to the post-mixing server. The post-mixing server superimposes the live stream of the front-end mixing server on the live stream of the post-mixing server according to the set layout for mixing. After the GOP frames of the streams to be switched are aligned, the streaming terminal with a high priority disconnects the live stream pushed to the front-end mixing server.

Images