Live stream switching method and system, computing device, and computer storage medium

By switching the stream processing components during the live stream switching process, the problem of long-term switching of live streams is solved, and smooth switching of live streams and user experience is improved.

WO2024244838A1PCT designated stage expired Publication Date: 2024-12-05ALIBABA CLOUD COMPUTING CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/089663
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-04-24
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

During the live stream switching process, the existing technology causes the switching of live stream push tasks to take a long time, resulting in the live stream switching stuttering, black screen, etc., which makes it impossible to achieve smooth switching and poor user experience.

Method used

By receiving the live stream switching request sent by the client, and then switching the stream processing component in the current push task after response, it will generate a mixed stream, avoid terminating and restarting the push task, and shortening the live stream switching time.

Benefits of technology

It realizes smooth switching of live streams, improves user experience, avoids lag and black screen phenomena, and shortens the time for live streams to switch.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024089663_05122024_PF_FP_ABST
    Figure CN2024089663_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a live stream switching method and system, a computing device, and a computer storage medium. The live stream switching method comprises: receiving a first live stream switching request sent by a first client; in response to the first live stream switching request, switching from a second stream processing component, which is used for processing a second live stream sent by a second client, to a first stream processing component in a current push task; receiving, by using the first stream processing component, a first live stream sent by the first client; processing the first live stream sent by the first client and the second live stream by using the first stream processing component to generate a third live stream; and pushing the third live stream to a third client. The technical solution provided by the embodiments of the present application realizes smooth switching of live streams.
Need to check novelty before this filing date? Find Prior Art

Description

Live stream switching method, system computing device and computer storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 31, 2023, with application number 202310641682.3 and invention name “Live stream switching method, system computing device and computer storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of live broadcast technology, and in particular to a live broadcast stream switching method, system, computing device, and computer storage medium. Background Art

[0003] A live broadcast is a mode of interaction between at least two hosts, each broadcasting live simultaneously in the same live broadcast room. During the live broadcast, the host can initiate a live broadcast request to the audience or other hosts, achieving a live broadcast interaction effect.

[0004] When switching from a standard live broadcast to a live broadcast with microphones connected, or ending a live broadcast with microphones connected, the live stream needs to be switched. For example, in a standard live broadcast, the server sends the live stream of the host to the viewer's client, which is the live stream generated by the host's client. In a live broadcast with microphones connected, the server needs to switch the live stream sent to the viewer's client from the host's live stream to a mixed live stream of the host's live stream and the live stream sent by the live broadcast client.

[0005] In some cases, when performing a live stream switching process, it usually involves switching of the live stream push task. For example, when live stream switching is required, the first push task currently used to push the anchor's live stream is terminated to close the second stream processing component in the first push task. The second stream processing component is used to receive the anchor's live stream and forward the anchor's live stream to the audience client; then a new second push task is created to generate the first stream processing component, and the first stream processing component is used to mix the anchor's live stream and the microphone live stream to generate a new live stream, and then the new live stream is pushed to the audience client.

[0006] In the process of realizing the concept of this application, the inventors found that the process of switching push tasks takes a long time, resulting in freezes, black screens, etc. when switching live streams, and smooth switching of live streams cannot be achieved, resulting in a poor user experience.

[0007] Summary of the Invention

[0008] Embodiments of the present application provide a live stream switching method, system, apparatus, computing device, and computer storage medium.

[0009] In a first aspect, an embodiment of the present application provides a live stream switching method, including: receiving a first live stream switching request sent by a first client; in response to the first live stream switching request, switching the second stream processing component that processes the second live stream sent by the second client to the first stream processing component in the current push task; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to a third client.

[0010] In a second aspect, an embodiment of the present application provides a live stream switching method, which is applied to a first client, wherein a live screen is displayed in the first client, and the live screen includes a microphone connection control; the method includes: generating a first live stream switching request in response to a microphone connection operation triggered by a user on the microphone connection control; sending the first live stream switching request to a stream mixing server, so that the stream mixing server switches the second stream processing component for processing the second live stream sent by the second client to the first stream processing component in the current push task in response to the first live stream switching request; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to a third client.

[0011] In a third aspect, an embodiment of the present application provides a live stream switching method, including: receiving a first live stream switching request sent by a first client; generating a stream processing component switching instruction in response to the first live stream switching request; sending the stream processing component switching instruction to a stream mixing server, so that the stream mixing server responds to the stream processing component switching instruction and switches the second stream processing component that processes the second live stream sent by the second client to the first stream processing component in the current push task; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to a third client.

[0012] In a fourth aspect, an embodiment of the present application provides a live stream switching system, including: a first client, used to send a first live stream generated by the first client and a first live stream switching request to a mixing server; a mixing server, used to receive the first live stream switching request sent by the first client; in response to the first live stream switching request, switching the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to a third client; a third client, used to receive the third live stream sent by the mixing server, and parse the third live stream to display a mixed stream picture.

[0013] In a fifth aspect, an embodiment of the present application provides a live stream switching device, including: a first request receiving module for receiving a first live stream switching request sent by a first client; a first switching module for responding to the first live stream switching request, switching the second stream processing component for processing the second live stream sent by the second client to the first stream processing component in the current pushed task; a live stream receiving module for using the first stream processing component to receive the first live stream sent by the first client; a mixing module for using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; a first live stream pushing module for pushing the third live stream to a third client.

[0014] In a sixth aspect, an embodiment of the present application provides a live stream switching device, which is applied to a first client, wherein a live screen is displayed in the first client, and the live screen includes a microphone connection control; the method includes: a first request generating module, which is used to generate a first live stream switching request in response to a microphone connection operation triggered by a user on the microphone connection control; a first request sending module, which is used to send the first live stream switching request to a mixing server, so that the mixing server switches the second stream processing component for processing the second live stream sent by the second client in the current push task to the first stream processing component in response to the first live stream switching request; uses the first stream processing component to receive the first live stream sent by the first client; uses the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushes the third live stream to a third client.

[0015] In the seventh aspect, an embodiment of the present application provides a live stream switching device, including: a second request receiving module for receiving a first live stream switching request sent by a first client; a switching instruction generating module for generating a stream processing component switching instruction in response to the first live stream switching request; a switching instruction sending module for sending the stream processing component switching instruction to a mixed stream server, so that the mixed stream server switches the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component in response to the stream processing component switching instruction; uses the first stream processing component to receive the first live stream sent by the first client; uses the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushes the third live stream to a third client.

[0016] In an eighth aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the live stream switching method provided in an embodiment of the present application.

[0017] In a ninth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, the live stream switching method provided in the embodiment of the present application is implemented.

[0018] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in some situations, the following is a brief introduction to the drawings required for use in the description of the embodiments or some situations. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] FIG1 shows a system architecture diagram of a technical solution of an embodiment of the present application to which the live stream switching method provided by the embodiment of the present application can be applied.

[0022] FIG2 schematically shows a flow chart of a live stream switching method provided by an embodiment of the present application.

[0023] FIG3 schematically shows a schematic diagram of a live stream switching method provided in an embodiment of the present application.

[0024] FIG4 schematically shows a schematic diagram of a live stream switching method provided in an embodiment of the present application.

[0025] FIG5 schematically shows a flow chart of a live stream switching method provided by another embodiment of the present application.

[0026] FIG6 schematically shows a flow chart of a live stream switching method provided by another embodiment of the present application.

[0027] FIG7 schematically shows a block diagram of a live stream switching system provided by an embodiment of the present application.

[0028] FIG8 schematically shows a block diagram of a live stream switching device provided by an embodiment of the present application.

[0029] FIG9 schematically shows a block diagram of a live stream switching device provided by another embodiment of the present application.

[0030] FIG10 schematically shows a block diagram of a live stream switching device provided by another embodiment of the present application.

[0031] FIG11 schematically shows a block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0033] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0034] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0035] A live broadcast is a mode of interaction between at least two hosts, each broadcasting live simultaneously in the same live broadcast room. During the live broadcast, the host can initiate a live broadcast request to the audience or other hosts, achieving a live broadcast interaction effect.

[0036] When switching from a standard live broadcast to a live broadcast with microphones connected, or ending a live broadcast with microphones connected, the live stream needs to be switched. For example, in a standard live broadcast, the server sends the live stream generated by the live broadcast client to the viewer's client; in a live broadcast with microphones connected, the server needs to switch the live stream sent to the viewer's client from the host's live stream to a mixed live stream of the host's live stream and the live stream sent by the live broadcast client.

[0037] In some cases, when performing a live stream switching process, it usually involves switching of the live stream push task. For example, when live stream switching is required, the first push task currently used to push the anchor's live stream is terminated first, and then a new second push task is created, and the second push task is used to receive and push a mixed live stream of the anchor's live stream and the live stream with the microphone.

[0038] In the process of realizing the concept of this application, the inventors found that in the process of switching push tasks, multiple push components on the push link will be involved, such as the shutdown and restart of codecs, transcoders, and synthesizers. It may also involve resetting audio and video parameters, changing video layout, switching picture sources, etc., which takes a long time and causes the live stream switching to be stuck, black screen, etc., and it is impossible to achieve smooth switching of the live stream, resulting in a poor user experience.

[0039] In order to solve technical problems existing in some situations, an embodiment of the present application provides a live stream switching method, which adopts the following methods: receiving a first live stream switching request sent by a first client; in response to the first live stream switching request, switching the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to a third client. The technical solution converges the switching of the live stream into one push task, without terminating the currently ongoing push task, and only needs to switch the second stream processing component running in the push task to the first stream processing component, thereby shortening the time consumption of the live stream switching, avoiding phenomena such as freezes and black screens, achieving smooth switching of the live stream, and improving user experience.

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0041] FIG1 shows a system architecture diagram of a technical solution of an embodiment of the present application to which the live stream switching method provided by the embodiment of the present application can be applied. The system architecture may include a client 101 and a server 102 .

[0042] The client 101 and the server 102 are connected via a network. The network provides a medium for the communication link between the client 101 and the server 102. The network can include various connection types, such as wired or wireless communication links or fiber optic cables.

[0043] The client 101 can interact with the server 102 through the network to receive or send messages, etc.

[0044] Among them, the client 101 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5, Hypertext Markup Language 5th Edition) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The client 101 can be deployed in an electronic device and needs to rely on the device to run or certain apps in the device to run, etc. The electronic device can, for example, have a display screen and support information browsing, etc., such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. For ease of understanding, Figure 1 mainly uses the image of a device to represent the user end. Various other types of applications can usually be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0045] The server 102 may include servers that provide various services, such as a server for background training that provides support for the model used on the user terminal 101, or a server that processes interactive information sent by the user terminal.

[0046] It should be noted that the server 102 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0047] It should be noted that the live stream switching method provided in the embodiments of the present application is generally executed by the server 102, and the corresponding live stream switching device is generally installed in the server 102. However, in other embodiments of the present application, the client 101 may also have similar functions to the server 102, thereby executing the live stream switching method provided in the embodiments of the present application. In other embodiments, the live stream switching method provided in the embodiments of the present application may also be jointly executed by the user terminal 101 and the server 102.

[0048] It should be understood that the number of user terminals and server terminals in FIG1 is merely illustrative and any number of user terminals and server terminals may be provided according to implementation requirements.

[0049] The implementation details of the technical solution of the embodiment of the present application are described in detail below.

[0050] Figure 2 schematically shows a flowchart of a live stream switching method provided by an embodiment of the present application. As shown in Figure 2, the live stream switching method may include the following steps: 201, receiving a first live stream switching request sent by a first client; 202, in response to the first live stream switching request, switching the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component; 203, using the first stream processing component to receive the first live stream sent by the first client; 204, using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; 205, pushing the third live stream to the third client.

[0051] FIG3 schematically shows a schematic diagram of a live stream switching method provided in an embodiment of the present application.

[0052] Figure 301 schematically shows a display screen of a third client in a normal live broadcast mode, where the normal live broadcast can be a single-stream live broadcast of a second client. As shown in Figure 301 , the third client only displays a live screen 3011 generated by rendering the second live stream collected and generated by the second client.

[0053] Figure 302 schematically illustrates the display screen of a third client in the live broadcast mode of the microphone connection. The live broadcast of the microphone connection can be a mixed stream of the second client and the first client. As shown in Figure 302, the third client not only displays the live broadcast screen 3011 collected and generated by the second client, but also displays the live broadcast screen 3021 collected and generated by the first client.

[0054] According to an embodiment of the present application, the first client, the second client, and the third client may be applications with a live broadcast function.

[0055] According to an embodiment of the present application, the first client, the second client, and the third client may be the same application, or the first client, the second client, and the third client may be applications of the same type on different platforms.

[0056] According to an embodiment of the present application, the first client and the second client may be live streaming terminals, which generate and push live streaming to the mixing server based on the collected live streaming data, such as audio data, video data, etc.

[0057] According to an embodiment of the present application, the third client may be a live stream pulling terminal, which sends a live stream pulling request to the stream mixing server based on the received live broadcast viewing operation.

[0058] According to an embodiment of the present application, the stream mixing server can be used to receive the live streams sent by each live stream push terminal and push the received live streams to the live stream pull terminal.

[0059] According to an embodiment of the present application, in the initial state, as shown in Figure 301, the second client can broadcast live to at least one third client, that is, the second live stream collected by the second client is sent to at least one third client via the mixed stream server, so that at least one answer client parses the received second live stream and displays the live screen corresponding to the second live stream.

[0060] According to an embodiment of the present application, for the second live stream sent by the second client, since the second live stream is a single stream, that is, the second live stream is a live stream collected and generated by a single client, the second live stream processing component deployed in the mixing server can be directly used to forward the second live stream to the third client.

[0061] According to an embodiment of the present application, the function of the second stream processing component can be to transparently transmit live stream data, that is, to directly forward the received live stream without performing additional processing.

[0062] According to an embodiment of the present application, the first client may be a viewer of the first live broadcast room where the second client is located, or the first client may be another live broadcast client of a second live broadcast room different from the first live broadcast room where the second client is located. When the user requests to connect with the second client by operating the first client, a first live stream switching request may be generated and sent.

[0063] According to an embodiment of the present application, when a first client and a second client are connected to each other, as shown in FIG302 , it is necessary to mix the first live stream collected by the first client and the live stream collected by the second client, that is, to mix the first live stream and the second live stream into one live stream, so that a third client can simultaneously display the live images collected by the first client and the second client.

[0064] According to an embodiment of the present application, when the mixing server receives a first live stream switching request, it means that the mixing server needs to switch the live stream, that is, switch the live stream from the single stream originally sent by the second client to the mixed stream sent by the first client and the second client.

[0065] According to an embodiment of the present application, the mixing server can execute the switching of the stream processing component in response to the first live stream switching request, that is, switch the second stream processing component that retweets the second live stream in the current push task to the first stream processing component.

[0066] According to an embodiment of the present application, since the mixed stream involves live streams collected by two different clients respectively, in order to enable a third client to display the mixed stream, a mixing stream server can be used to mix the first live stream and the second live stream according to a specified layout to generate a third live stream.

[0067] According to an embodiment of the present application, according to an embodiment of the present application, pushing the third live stream to the second client can be specifically implemented as: sending the third live stream to a content distribution network, so that the second client can pull the third live stream from the content distribution network.

[0068] According to an embodiment of the present application, the stream mixing server can push the third live stream through a content delivery network (CDN).

[0069] According to an embodiment of the present application, the stream mixing server can send the third live stream to a preset network node of the CDN, and then the third client can pull the third live stream from the preset network node.

[0070] According to the embodiments of the present application, since CDN has a powerful load balancing and fault tolerance mechanism, it can cache the live stream at an edge node closer to the third client according to the needs of the third client and the network environment, thereby achieving local access, improving the speed at which the third client pulls the third live stream, and ensuring the stability, continuity and clarity of the live broadcast.

[0071] According to an embodiment of the present application, the first client and the second client can be connected to the mixing server through real-time communication technology (Web Real-Time Communication, RTC) based on Web technology, and the mixing server communicates with the CDN's RTMP (Real-Time Messaging Protocol), thereby ensuring the real-time nature of live stream pulling and pushing.

[0072] According to an embodiment of the present application, when the mixing server switches the live stream, by adopting a switching method of only switching the stream processing component instead of terminating the current push task and starting a new push task, the RTC connection between the mixing server and the client, and the RTMP connection between the mixing server and the CDN can be ensured not to be disconnected, thereby shortening the live stream switching time and avoiding abnormal phenomena such as freezes and black screens.

[0073] According to an embodiment of the present application, in response to the first live stream switching request, switching the forwarding module that forwards the second live stream sent by the second client in the current push task to the mixing module can be specifically implemented as follows: in response to the first live stream switching request, destroying the second stream processing component; generating the first stream processing component.

[0074] According to an embodiment of the present application, in response to the first live stream switching request, after the second stream processing component that processes the second live stream sent by the second client in the current push task is switched to the first stream processing component, the live stream switching method may further include: generating a first decoder corresponding to the first client and a second decoder corresponding to the second client.

[0075] According to an embodiment of the present application, when a first live stream switching request arrives at the mixing server, the mixing server may destroy the currently deployed second stream processing component. Destroying the second stream processing component may mean that the mixing server stops the currently running second stream processing component in order to generate a new first stream processing component. In one embodiment, destroying the second stream processing component may also be implemented by stopping the process of the second stream processing component via a command line or an API (Application Programming Interface).

[0076] According to an embodiment of the present application, after destroying the second stream processing component, the stream mixing server may generate a first decoder corresponding to the first client and a second decoder corresponding to the second client.

[0077] According to an embodiment of the present application, the live stream switching method may further include: using a first decoder to decode the first live stream to generate a fourth live stream in a predetermined format; using a second decoder to decode the second live stream to generate a fifth live stream in a predetermined format.

[0078] According to an embodiment of the present application, using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate the third live stream can be specifically implemented as follows: using the first stream processing component to mix the fourth live stream and the fifth live stream to obtain the third live stream.

[0079] According to the embodiments of the present application, since the first and second clients independently capture the first and second live streams, it is generally necessary to configure and optimize the first and second live streams accordingly, such as with respect to resolution, frame rate, and audio encoding format, to ensure the effectiveness of the stream mixing. Therefore, it is necessary to generate a first decoder and a second decoder for the first and second clients, respectively, so that the first and second decoders can process the first and second live streams, respectively.

[0080] According to an embodiment of the present application, the first live stream and the second live stream can be converted into live stream data in YUV format (a coding method usually used to represent the brightness (Y) and chroma (U, V) information of a color image. Y, U, V represent brightness, blue chroma and red chroma respectively).

[0081] According to an embodiment of the present application, after generating the fourth live stream and the fifth live stream, the first stream processing component can mix and combine the fourth live stream and the fifth live stream to generate a new third live stream containing multiple channels of data.

[0082] According to an embodiment of the present application, before receiving a first live stream switching request sent by a first client, the live stream switching method also includes: receiving a second live stream sent by a second client; using a second stream processing component to forward the second live stream to a third client, so that the third client can decode the second live stream and display the live screen corresponding to the second live stream.

[0083] According to an embodiment of the present application, in the initial state, the second client can broadcast live to at least one third client, that is, the second live stream collected by the second client is sent to at least one third client via the mixing server, so that at least one answer client parses the received second live stream and displays the live screen corresponding to the second live stream.

[0084] According to an embodiment of the present application, for the second live stream sent by the second client, since the second live stream is a single stream, that is, the second live stream is a live stream collected and generated by a single client, the second live stream processing component deployed in the mixing server can be directly used to forward the second live stream to the third client.

[0085] According to an embodiment of the present application, the function of the second stream processing component can be to transparently transmit live stream data, that is, to directly forward the received live stream without performing additional processing.

[0086] According to an embodiment of the present application, receiving the first live stream switching request sent by the first client can be specifically implemented as: receiving the stream processing component switching instruction sent by the management server, and the stream processing component switching instruction is generated after the management server receives the first live stream switching request sent by the first client.

[0087] According to an embodiment of the present application, the management server can be used for the management of the live broadcast room. The management of the live broadcast room can, for example, include the creation and deletion of the live broadcast room, the management of the live broadcast room users, etc., and can also subscribe to the live broadcast users.

[0088] According to an embodiment of the present application, after the first client generates a first live stream switching request, it can first send the first live stream switching request to the management server, so that the management server notifies the mixing server to subscribe to the first live stream generated by the first client, and generates and sends a stream processing component switching instruction to the mixing server, so as to instruct the mixing server to switch the second stream processing component to the first stream processing component through the stream processing component switching instruction.

[0089] According to an embodiment of the present application, the live stream switching method also includes: receiving a second live stream switching request sent by the first client; in response to the second live stream switching request, switching the first stream processing component that mixes the first live stream and the second live stream to the second stream processing component to stop receiving the first live stream; and using the second stream processing component to forward the second live stream to a third client.

[0090] According to an embodiment of the present application, after the first client and the second client end the microphone connection, the first client can send a second live stream switching request to the mixing server or the management server, so that the mixing server can switch the deployed stream processing component from the first stream processing component to the second stream processing component, thereby switching the live stream from a mixed stream to a single stream.

[0091] According to an embodiment of the present application, in response to a second live stream switching request, switching the first stream processing component that mixes the first live stream and the second live stream to the second stream processing component can be specifically implemented as follows: in response to the second live stream switching request, destroying the first stream processing component; generating a second stream processing component; and using the second stream processing component to receive the second live stream.

[0092] According to an embodiment of the present application, when a second live stream switching request arrives at the mixing server, the mixing server may destroy the currently deployed first stream processing component. Destroying the first stream processing component may mean that the mixing server stops the currently running first stream processing component in order to generate a new second stream processing component. In one embodiment, destroying the first stream processing component may also be implemented by stopping the process of the first stream processing component via a command line or an API (Application Programming Interface).

[0093] According to an embodiment of the present application, after destroying the first stream processing component, the stream mixing server may generate a second stream processing component corresponding to the second client.

[0094] FIG4 schematically shows a schematic diagram of a live stream switching method provided in an embodiment of the present application.

[0095] As shown in Figure 401, a schematic diagram of a normal live broadcast can be a single-stream live broadcast of a second client 4011. Specifically, the second client 4011 collects and generates a second live stream, and sends the second live stream to a stream mixing server 4012. The stream mixing server 4012 uses the deployed second stream processing component 4013 to push the second live stream to a third client 4014.

[0096] As shown in Figure 402, it can be a schematic diagram of a live broadcast with microphones connected, which can be a mixed stream live broadcast of a first client 4021 and a second client 4011. When the live broadcast scene switches from a normal live broadcast to a live broadcast with microphones connected, it is necessary to involve switching of the live stream.

[0097] When switching live streams, the first client may first send a first live stream switching instruction to the management server 4022, causing the management server to instruct the stream mixing server 4012 to subscribe to the first live stream generated by the first client, and then send a stream processing component switching instruction to the first client, instructing the stream mixing server 4012 to switch the currently deployed second stream processing component 4013 to the first stream processing component 4023. The first stream processing component 4023 may include a first decoder 40231 and a second decoder 40232, which are respectively used to decode the first live stream and the second live stream.

[0098] After generating the first stream processing component 4023, the first and second live streams can be received using the first decoder 40231 and the second decoder 40232, respectively, so that the first and second live streams can be decoded to generate the fourth and fifth live streams. In one embodiment, the first stream processing component 4023 can mix the fourth and fifth live streams to generate the third live stream.

[0099] After the third live stream is generated, the third live stream can be sent to the third client 4014. For example, after the third live stream is generated, the third live stream can be first sent to the content distribution network so that the third client 4014 pulls the third live stream from the content distribution network.

[0100] Figure 5 schematically shows a flowchart of a live stream switching method provided by another embodiment of the present application. The live stream switching method can be applied to a first client, and a live broadcast screen is displayed in the first client, and the live broadcast screen includes a microphone connection control; the live stream switching method may specifically include the following steps: 501, in response to the microphone connection operation triggered by the user on the microphone connection control, generating a first live stream switching request; 502, sending the first live stream switching request to the mixing server, so that the mixing server responds to the first live stream switching request and switches the second stream processing component for retweeting the second live stream sent by the second client in the current push task to the first stream processing component; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to the third client.

[0101] Figure 6 schematically shows a flowchart of a live stream switching method provided by another embodiment of the present application. The live stream switching method may specifically include the following steps: 601, receiving a first live stream switching request sent by a first client; 602, generating a stream processing component switching instruction in response to the first live stream switching request; 603, sending the stream processing component switching instruction to the stream mixing server, so that the stream mixing server switches the second stream processing component that retweets the second live stream sent by the second client to the first stream processing component in response to the stream processing component switching instruction; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to the third client.

[0102] According to an embodiment of the present application, in response to the first live stream switching request, generating a stream processing component switching instruction can be specifically implemented as follows: obtaining identification information of the first client; generating a stream processing component switching instruction based on the identification information, so that after receiving the stream processing component switching instruction, the mixed stream server receives the first live stream of the first client according to the identification information.

[0103] Figure 7 schematically shows a block diagram of a live stream switching system provided by an embodiment of the present application. As shown in Figure 7, the live stream switching system may specifically include: a first client 701, used to send the first live stream generated by the first client 701 and the first live stream switching request to the mixing server 702; the mixing server 702 is used to receive the first live stream switching request sent by the first client 701; in response to the first live stream switching request, switch the second stream processing component that retweets the second live stream sent by the second client in the current push task to the first stream processing component; use the first stream processing component to receive the first live stream sent by the first client; use the first stream processing component to mix the second live stream and the first live stream sent by the first client 701 to generate a third live stream; and push the third live stream to the third client 703; the third client 703 is used to receive the third live stream sent by the mixing server 702, and parse the third live stream to display the mixed stream picture.

[0104] An embodiment of the present application provides a live stream switching method, which adopts the following methods: receiving a first live stream switching request sent by a first client; in response to the first live stream switching request, switching the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component; using the first stream processing component to receive the first live stream sent by the first client; using the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushing the third live stream to a third client. The technical solution converges the switching of the live stream into one push task, without terminating the currently ongoing push task, and only needs to switch the second stream processing component running in the push task to the first stream processing component, thereby shortening the time consumption of the live stream switching, avoiding phenomena such as freezes and black screens, achieving smooth switching of the live stream, and improving user experience.

[0105] Figure 8 schematically shows a block diagram of a live stream switching device provided by an embodiment of the present application. As shown in Figure 8, the live stream switching device may specifically include: a first request receiving module 801, used to receive a first live stream switching request sent by a first client; a first switching module 802, used to switch the second stream processing component that processes the second live stream sent by the second client to the first stream processing component in the current push task in response to the first live stream switching request; a live stream receiving module 803, used to use the first stream processing component to receive the first live stream sent by the first client; a mixing module 804, used to use the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; a first live stream pushing module 805, used to push the third live stream to a third client.

[0106] According to an embodiment of the present application, the live stream switching device 800 further includes: a generation module, configured to generate a first decoder corresponding to the first client and a second decoder corresponding to the second client.

[0107] According to an embodiment of the present application, the live stream switching device 800 also includes: a first decoding module, used to decode the first live stream using a first decoder to generate a fourth live stream in a predetermined format; a second decoding module, used to decode the second live stream using a second decoder to generate a fifth live stream in a predetermined format.

[0108] According to an embodiment of the present application, the stream mixing module 804 includes: a stream mixing submodule, configured to use the first stream processing component to mix the fourth live stream and the fifth live stream to obtain a third live stream.

[0109] According to an embodiment of the present application, before receiving the first live stream switching request sent by the first client, the live stream switching device 800 also includes: a second live stream receiving module, used to receive the second live stream sent by the second client; a forwarding module, used to use the second stream processing component to forward the second live stream to the third client, so that the third client can decode the second live stream and display the live screen corresponding to the second live stream.

[0110] According to an embodiment of the present application, the first live stream pushing module 804 includes: a first live stream pushing submodule, configured to send the third live stream to a content distribution network, so that the second client can pull the third live stream from the content distribution network.

[0111] According to an embodiment of the present application, the first request receiving module 801 includes: a switching instruction receiving module, which is used to receive a stream processing component switching instruction sent by the management server, and the stream processing component switching instruction is generated after the management server receives the first live stream switching request sent by the first client.

[0112] According to an embodiment of the present application, the live stream switching device 800 also includes: a second request receiving module, used to receive a second live stream switching request sent by the first client; a second stream processing component switching module, used to respond to the second live stream switching request, and switch the first stream processing component that mixes the first live stream and the second live stream to the second stream processing component to stop receiving the first live stream; a second live stream pushing module, used to use the second stream processing component to push the second live stream to the third client.

[0113] According to an embodiment of the present application, the second stream processing component switching module includes: a component destruction submodule, which is used to destroy the first stream processing component in response to the second live stream switching request; a component generation submodule, which is used to generate a second stream processing component; and a second live stream receiving submodule, which is used to use the second stream processing component to receive the second live stream.

[0114] The live stream switching device of FIG8 can execute the live stream switching method of the embodiment shown in FIG2, and its implementation principle and technical effects are not described in detail here. The specific manner in which each module and unit performs operations in the live stream switching device of the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0115] Figure 9 schematically shows a block diagram of a live stream switching device provided by another embodiment of the present application. As shown in Figure 9, the live stream switching device may specifically include: a first request generating module 901, used to generate a first live stream switching request in response to a microphone connection operation triggered by a user on a microphone connection control; a first request sending module 902, used to send the first live stream switching request to a stream mixing server, so that the stream mixing server switches the second stream processing component for retweeting the second live stream sent to the second client in the current push task to the first stream processing component in response to the first live stream switching request; uses the first stream processing component to receive the first live stream sent by the first client; uses the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushes the third live stream to a third client.

[0116] The live stream switching device of FIG9 can execute the live stream switching method of the embodiment shown in FIG5 , and its implementation principle and technical effects are not described in detail here. The specific manner in which each module and unit performs operations in the live stream switching device of the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0117] Figure 10 schematically shows a block diagram of a live stream switching device provided by another embodiment of the present application. As shown in Figure 10, the live stream switching device may specifically include: a second request receiving module 1001, used to receive a first live stream switching request sent by a first client; a switching instruction generating module 1002, used to generate a stream processing component switching instruction in response to the first live stream switching request; a switching instruction sending module 1003, used to send the stream processing component switching instruction to the stream mixing server, so that the stream mixing server switches the second stream processing component that retweets the second live stream sent by the second client in the current push task to the first stream processing component in response to the stream processing component switching instruction; uses the first stream processing component to receive the first live stream sent by the first client; uses the first stream processing component to mix the second live stream and the first live stream sent by the first client to generate a third live stream; and pushes the third live stream to the third client.

[0118] According to an embodiment of the present application, the switching instruction generation module includes: an identification acquisition module for acquiring identification information of the first client; an instruction generation module for generating a stream processing component switching instruction based on the identification information, so that the mixed stream server receives the first live stream of the first client according to the identification information after receiving the stream processing component switching instruction.

[0119] The live stream switching device shown in FIG10 can execute the live stream switching method described in the embodiment shown in FIG6 , and its implementation principle and technical effects are not described in detail here. The specific manner in which each module and unit performs operations in the live stream switching device in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0120] In one possible design, the live stream switching apparatus provided in the embodiment of the present application can be implemented as a computing device. As shown in FIG11 , the computing device may include a storage component 1101 and a processing component 1102; the storage component 1101 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 1102 to implement the live stream switching method provided in the embodiment of the present application.

[0121] Of course, a computing device may also include other components, such as input / output interfaces and communication components. The input / output interface provides an interface between the processing component and peripheral interface modules, such as output devices and input devices. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices.

[0122] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0123] When the computing device is a physical device, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device.

[0124] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a computer, the live stream switching method provided in the embodiment of the present application can be implemented.

[0125] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a computer, can implement the live stream switching method provided in the embodiment of the present application.

[0126] The processing components in the above embodiments may include one or more processors to execute computer instructions to complete all or part of the steps in the above methods. Of course, the processing components may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0127] The memory component is configured to store various types of data to support operations in the device. The memory component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0128] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A live stream switching method, comprising: Receiving a first live stream switching request sent by a first client; In response to the first live stream switching request, switching the second stream processing component for processing the second live stream sent by the second client to the first stream processing component in the current push task; Using the first stream processing component to receive a first live stream sent by the first client; Using the first stream processing component to process the second live stream and the first live stream sent by the first client to generate a third live stream; The third live stream is pushed to a third client.

2. The method according to claim 1, wherein: After the second stream processing component for processing the second live stream sent by the second client in the current push task is switched to the first stream processing component in response to the first live stream switching request, the method includes: A first decoder corresponding to the first client and a second decoder corresponding to the second client are generated.

3. The method according to claim 2, wherein: The method further comprises: Decoding the first live stream using the first decoder to generate a fourth live stream in a predetermined format; Decoding the second live stream using the second decoder to generate a fifth live stream in a predetermined format; The using the first stream processing component to process the second live stream and the first live stream sent by the first client to generate a third live stream includes: The fourth live stream and the fifth live stream are mixed by using the first stream processing component to obtain the third live stream.

4. The method according to claim 1, wherein: Before receiving the first live stream switching request sent by the first client, the method further includes: receiving the second live stream sent by the second client; The second live stream is forwarded to the third client by using the second stream processing component, so that the third client can decode the second live stream and display the live screen corresponding to the second live stream.

5. The method according to claim 1, wherein: The pushing the third live stream to the second client comprises: The third live stream is sent to a content distribution network, so that the second client can pull the third live stream from the content distribution network.

6. The method according to claim 1, wherein: The receiving a first live stream switching request sent by a first client comprises: A stream processing component switching instruction sent by a management server is received, where the stream processing component switching instruction is generated after the management server receives a first live stream switching request sent by the first client.

7. The method according to claim 1, wherein: The method further comprises: Receive a second live stream switching request sent by the first client; In response to the second live stream switching request, switching the first stream processing component that processes the first live stream and the second live stream to the second stream processing component to stop receiving the first live stream; The second live stream is forwarded to the third client by using the second stream processing component.

8. The method according to claim 7, wherein: In response to the second live stream switching request, switching the first stream processing component that processes the first live stream and the second live stream to the second stream processing component includes: In response to the second live stream switching request, destroying the first stream processing component; generating the second stream processing component; The second live stream is received by the second stream processing component.

9. A live stream switching method, applied to a first client, wherein a live screen is displayed in the first client, and the live screen includes a microphone connection control; the method comprises: In response to a microphone connection operation triggered by a user on the microphone connection control, generating a first live stream switching request; Sending the first live stream switching request to the mixing stream server, so that the mixing stream server switches the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component in response to the first live stream switching request; using the first stream processing component to receive the first live stream sent by the first client; Using the first stream processing component to process the second live stream and the first live stream sent by the first client to generate a third live stream; The third live stream is pushed to a third client.

10. A live stream switching method, comprising: Receiving a first live stream switching request sent by a first client; In response to the first live stream switching request, generating a stream processing component switching instruction; The stream processing component switching instruction is sent to the mixed stream server, so that the mixed stream server switches the second stream processing component that processes the second live stream sent by the second client in the current push task to the first stream processing component in response to the stream processing component switching instruction; and uses the first stream processing component to receive the first live stream sent by the first client; Using the first stream processing component to process the second live stream and the first live stream sent by the first client to generate a third live stream; The third live stream is pushed to a third client.

11. The method according to claim 10, wherein: The generating, in response to the first live stream switching request, a stream processing component switching instruction comprises: Obtaining identification information of the first client; The stream processing component switching instruction is generated based on the identification information, so that after receiving the stream processing component switching instruction, the mixed stream server receives the first live stream of the first client according to the identification information.

12. A live stream switching system, comprising: A first client is used to send a first live stream generated by the first client and a first live stream switching request to a stream mixing server; The stream mixing server is configured to receive a first live stream switching request sent by a first client; in response to the first live stream switching request, switch a second stream processing component that processes a second live stream sent by a second client in a current push task to a first stream processing component; and receive the first live stream sent by the first client using the first stream processing component; Using the first stream processing component to process the second live stream and the first live stream sent by the first client to generate a third live stream; And pushing the third live stream to a third client; The third client is used to receive the third live stream sent by the mixed stream server, and parse the third live stream to display the mixed stream picture.

13. A computing device comprising a processing component and a storage component; The storage component stores one or more computer instructions; the one or more computer instructions are used by the processor The processing component calls and executes to implement the live stream switching method as described in any one of claims 1 to 8, or implements the live stream switching method as described in any one of claims 9 to 10.

14. A computer storage medium storing a computer program, wherein when the computer program is executed by a computer, the live stream switching method according to any one of claims 1 to 8 is implemented, or the live stream switching method according to any one of claims 9 to 10 is implemented.

Citation Information

Patent Citations

  • Multimedia stream switching method, device and system

    CN115767128A

  • Live broadcast stream switching method and device, server, terminal and program product

    CN115997384A

  • Live broadcast stream switching method and system, computing device and computer storage medium

    CN116684655A

  • Method and apparatus for playing video signal of multi-user interaction, and device

    US20230034686A1