Live-streaming content presentation method and apparatus, and device and storage medium

Abstract

Provided in the embodiments of the present disclosure are a live-streaming content presentation method and apparatus, and a device and a storage medium. The method comprises: during the live streaming of a live-streaming object, in response to having detected the selection of a first display mode or a second display mode which corresponds to the live-streaming object, acquiring a target media stream corresponding to the selected display mode, wherein the target media stream corresponding to the first display mode comprises a media stream of the live-streaming object, and the target media stream corresponding to the second display mode comprises a media stream obtained by means of merging the media stream of the live-streaming object with a media stream of at least one participant in the live streaming; on the basis of the selected display mode, executing picture rendering on the target media stream, so as to obtain a target picture; and on a live-streaming interface of the live streaming, presenting the target picture.

Description

Methods, devices, equipment, and storage media for presenting live content

[0001] This application claims priority to Chinese Patent Application No. 202411514238.6, filed on October 28, 2024, entitled "Method, Apparatus, Device and Storage Medium for Presenting Live Content", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for presenting live content. Background Technology

[0003] With the development of computer technology, the internet has become an important platform for information exchange. For example, people can use live streaming to transmit video content in real time, allowing users to watch the video content instantly and interact with other users in real time. During live streaming, people not only expect high-quality and smooth video content, but also expect reduced system resource consumption. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for presenting live content is provided. The method includes: during a live stream of a live object, in response to detecting the selection of a first display mode or a second display mode corresponding to the live object, acquiring a target media stream corresponding to the selected display mode, wherein the target media stream corresponding to the first display mode includes the media stream of the live object, and the target media stream corresponding to the second display mode includes a media stream obtained by merging the media stream of the live object with the media streams of at least one participant in the live stream; performing screen rendering on the target media stream based on the selected display mode to obtain a target screen; and presenting the target screen in the live stream interface.

[0005] In a second aspect of this disclosure, an apparatus for presenting live content is provided. The apparatus includes: an acquisition module configured to, during a live stream of a live object, acquire a target media stream corresponding to the selected display mode in response to detecting the selection of a first display mode or a second display mode corresponding to the live object, wherein the target media stream corresponding to the first display mode includes the media stream of the live object, and the target media stream corresponding to the second display mode includes a media stream obtained by merging the media stream of the live object with the media streams of at least one participant in the live stream; a rendering module configured to perform screen rendering on the target media stream based on the selected display mode to obtain a target image; and a presentation module configured to present the target image in a live stream interface.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the electronic device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions, wherein when executed by a processor, the computer-executable instructions implement the method according to a first aspect of this disclosure.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;

[0012] Figure 2 illustrates a flowchart of the live content presentation process according to some embodiments of the present disclosure;

[0013] Figure 3 illustrates a schematic diagram of an example architecture for presenting live content according to some embodiments of the present disclosure;

[0014] Figures 4A and 4B respectively illustrate flowcharts of the signaling flow for presenting live content according to some embodiments of the present disclosure;

[0015] Figure 5 illustrates a schematic diagram of an example architecture for presenting live content according to some embodiments of the present disclosure;

[0016] Figure 6 illustrates a schematic diagram of an example architecture for presenting live content according to some embodiments of the present disclosure;

[0017] Figure 7 shows a schematic structural block diagram of an example device for presenting live content according to some embodiments of the present disclosure; and

[0018] Figure 8 shows a block diagram of an electronic device capable of implementing several embodiments of the present disclosure. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0021] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0022] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0024] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0025] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0026] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0027] As mentioned above, during live streaming, people can transmit video content in real time, allowing users to watch the video content instantly and interact with it in real time. Among the many live streaming methods, cloud gaming live streaming has become a very popular one. The streamer can run games that require high processing power on a cloud device and then share the game footage with the audience in real time, interacting with them on the screen. In cloud gaming live streaming, the streamer's terminal device typically has two main display modes, which can be called clear screen mode and self-preview mode. While the streamer is playing the game, they can choose clear screen mode, where their terminal device only displays the game footage. During interaction between the streamer and guests / viewers, they can switch to self-preview mode. In self-preview mode, the streamer's terminal device simultaneously displays the game footage, as well as previews of the streamer's and guests' screens.

[0028] Traditionally, to meet the requirements of real-time audio and video streaming and display in two modes, the broadcaster's terminal device typically needs to establish two isolated Real-Time Communication (RTC) channels. A container receives the game's audio and video streams from the cloud device via one RTC channel, performs rendering on the game's audio and video streams within the container, and creates a game screen that can be displayed in clear screen mode. Then, the rendered game screen's audio and video stream is merged with the broadcaster's and guests' audio and video streams to form an audio and video stream containing the game screen, broadcaster, and guests. The terminal device can then share this audio and video stream containing the game screen, broadcaster, and live stream with the audience in real time via the other RTC channel. Simultaneously, the terminal device also renders the audio and video stream containing the game screen, broadcaster, and live stream, creating a self-preview screen that can be displayed in self-preview mode. This results in the broadcaster's terminal device repeatedly rendering the game screen, consuming significant system resources.

[0029] Embodiments of this disclosure propose an improved scheme for presenting live content. In this scheme, during the live streaming of a live object, in response to detecting the selection of a first display mode or a second display mode for the live object, a target media stream corresponding to the selected display mode is acquired. The target media stream corresponding to the first display mode includes the media stream of the live object, and the target media stream corresponding to the second display mode includes a media stream obtained by merging the media stream of the live object with the media streams of at least one participant in the live stream. Based on the selected display mode, image rendering is performed on the target media stream to obtain a target image. The target image is then presented in the live streaming interface.

[0030] In this way, the embodiments of this disclosure render the media stream corresponding to the selected display mode according to the display mode selected by the broadcaster, which can avoid repeatedly rendering the screen of the live broadcast object (such as a game), reduce the system resource consumption of the terminal device, improve the smoothness and picture quality of the live broadcast, and reduce the power consumption and extend the standby time of the terminal device.

[0031] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0032] Figure 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In environment 100, one or more users 110-1, 110-2, 110-3, ..., 110-N can watch live streams and participate in interactive live stream sessions through their respective associated terminal devices 120-1, 120-2, ..., 120-N. User 110 may also be referred to as a participant, for example. For ease of discussion, users 110-1, 110-2, ..., 110-N may be collectively referred to as user 110 or individually, and terminal devices 120-1, 120-2, ..., 120-N may be collectively referred to as terminal device 120 or individually. Among all users 110, one or more users, such as user 110-1, can establish a live stream session and initiate a live stream through their associated terminal device 120-1.

[0033] In some scenarios, the user 110-1 who initiates the live stream can also be referred to as the host of the live stream session, a host participant, the live streamer, or the live stream administrator. One or more other users 110 may include guest participants 110-N to indicate users participating in the live stream interaction initiated by the host. In addition to guest participants in the live stream interaction, other users 110 may also include viewers, listeners, or viewers of the live stream session.

[0034] In some embodiments, the terminal device 120 may have an application installed that can provide live streaming services, or may have access to a website that can provide live streaming services. The user 110 can operate the terminal device 120 to access the corresponding application or website. Accordingly, the terminal device 120 may present a corresponding live streaming interface, which may provide live streaming content, such as audio live streaming content or video live streaming content.

[0035] In embodiments of this disclosure, environment 100 may include one or more server devices 130-1, 130-2, ..., 130-N. In some embodiments, terminal device 120 may also communicate with server devices 130-1, 130-2, ..., 130-N via network 132 to provide live streaming services. The one or more server devices 130-1, 130-2, ..., 130-N may individually or collaboratively provide live streaming services to terminal device 120. In one example, server device 130-1 may leverage its computing power to remotely run applications with high processing power requirements, such as games. Media streams used to display application screens are provided to terminal device 120 as live video content. In another example, server device 130-2 may also provide functions such as application or website management, configuration, and maintenance. In some embodiments, server devices 130-1, 130-2, ..., 130-N may store data generated during the live streaming session, including live interaction data generated during live interaction.

[0036] Terminal device 120 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 120 may also support any type of user-facing interface (such as "wearable" circuitry). Server device 130 can be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0037] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0038] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0039] Figure 2 illustrates a flowchart of a live content presentation process 200 according to some embodiments of the present disclosure. For ease of discussion, some embodiments of the present disclosure will be described below in conjunction with the environment 100 in Figure 1 and from the perspective of a terminal device 120, but this is merely exemplary. In some embodiments, the actions described relative to the terminal device may be performed by the terminal device in coordination with the server device.

[0040] In block 210 of process 200, during the live broadcast of the live broadcast object, the terminal device 120, in response to detecting the selection of a first display mode or a second display mode corresponding to the live broadcast object, acquires the target media stream corresponding to the selected display mode.

[0041] The live streaming object can be understood as the object targeted by the live stream, used to form the content of the live stream. The live streaming object can include, but is not limited to, content presented by people, products, activities, landscapes, events, applications (such as game applications, video applications), etc. The media stream can include video streams, audio and video streams, etc., and of course, the media stream can also include text streams (such as subtitles), media stream metadata, etc. The embodiments of this disclosure do not limit the type of live streaming object or the type of media stream. In practical applications, any appropriate live streaming object and media stream can be selected according to actual needs. In some live streaming application scenarios, the broadcaster may broadcast the content presented on the screen of the broadcaster's terminal device through screen recording. An example of screen recording live streaming includes game live streaming, that is, broadcasting game content. Of course, in addition to game live streaming, content presented by other types of applications can also be broadcast live.

[0042] During a live stream featuring a specific subject, the streamer may wish to switch between different display modes. For example, in a game live stream, in some situations the streamer may need to control the game or clearly present the game content to guests or viewers. In this case, the streamer might want to switch to a display mode that only displays the game content. In other situations, the streamer may want to interact with guests or viewers. In this case, the streamer might want to switch to a display mode that simultaneously displays the game content, the streamer's view, and the guest's view.

[0043] Here, a first display mode and a second display mode can be used to distinguish between two different display modes for the live streaming object. If the selection of the first display mode for the live streaming object is detected, the terminal device 120 acquires the target media stream corresponding to the first display mode. The target media stream corresponding to the first display mode may include the media stream of the live streaming object. If the selection of the second display mode for the live streaming object is detected, the terminal device 120 acquires the target media stream corresponding to the second display mode. The target media stream corresponding to the second display mode may include the media stream obtained by merging the media stream of the live streaming object with the media streams of at least one participant in the live stream. It can be understood that merging multiple media streams can also be called merging.

[0044] It should be noted that the display mode for a live stream object is not limited to the first and second display modes; it can also include, for example, a third, fourth, and so on. In practical applications, these two different display modes can be indicated in any way, not just by using terms such as "first display mode" and "second display mode." As an example, these two different display modes can be indicated by text, icons, or a combination of text and icons. For instance, the first display mode can be called "clear screen mode," and the second display mode can be called "non-clear screen mode."

[0045] In some embodiments, if the terminal device 120 detects the selection of a first display mode, the terminal device 120 can receive the media stream of the live stream object from the server device 130-1 associated with the live stream object through a first functional component to form a target media stream corresponding to the first display mode. Regarding the server device 130-1 associated with the live stream object, the server device 130-1 may differ depending on the live stream object. For example, if the live stream object is the display screen of a target application (e.g., game software), the server device 130-1 may be the server device running the target application. As another example, if the live stream object is the screen of another participant, the server device 130-1 may also be the server device acquiring the media stream of that other participant. The first functional component is configured to process the object content of the live stream object, which may include, but is not limited to, the media stream of the live stream object. As an example, the first functional component may be configured to establish a communication channel between the terminal device 120 and the server device 130-1. The first functional component can send control commands to the server device 130-1 via the established communication channel, thereby controlling the live streaming object through the server device 130-1. The first functional component can also receive the media stream of the live streaming object from the server device 130-1 via the established communication channel. Furthermore, the first functional component can be configured to process the media stream of the live streaming object, such as encoding, decoding, rendering, forwarding, etc. As an example, the first functional component may include a container corresponding to the live streaming object.

[0046] To illustrate the process of acquiring the target media stream, the following description, in conjunction with Figure 3, will provide an exemplary explanation of the acquisition process. Figure 3 shows a schematic diagram of an example architecture 300 for presenting live content according to some embodiments of this disclosure. The example architecture 300 illustrates a first functional component 310 of a terminal device 120. The terminal device 120 can establish a real-time communication (RTC) channel between the terminal device 120 and the server device 130 through the first functional component 310, and receive the media stream 301 (e.g., audio and video stream) of the live object from the server device 130-1 based on the RTC channel.

[0047] In some embodiments, the live streaming object may include the display screen of the target application. Based on this, the terminal device 120 can send a request to the server device 130-1 via the first functional component 310 to request the server device 130-1 to run the target application. Subsequently, the terminal device 120 can receive a media stream corresponding to the user interface of the target application from the server device 130-1 via the first functional component 310, as the media stream of the live streaming object.

[0048] As an example, Figure 4A illustrates a flowchart of a signaling stream 400A for live content presentation according to some embodiments of the present disclosure. As shown in Figure 4A, the signaling stream 400A involves a broadcaster component 410, a first functional component 310, and a server device 130-1. A terminal device 120 can establish an RTC channel with the server device 130-1 via the first functional component 310. A broadcaster participant can trigger a launch operation of a target application via the terminal device 120 or an attached device of the terminal device 120. In response to this launch operation, the broadcaster component 410 can send a (401) launch command to the first functional component 310 to request the launch of the target application. The first functional component 310 can send a (402) launch command to the server device 130-1. Upon receiving the launch command, the server device 130-1 can run (403) the target application (e.g., a game). Server device 130-1 can use a media stream corresponding to the user interface of the target application (e.g., audio and video streams of game screens) as the media stream of the live streaming object, and feed back (405) the media stream of the live streaming object to the first functional component 310 based on the RTC channel. It is understood that the target application can include various applications suitable for running on server device 130-1, such as game applications, video applications, or other types of applications. The embodiments of this disclosure do not limit the type of target application.

[0049] As another example, Figure 5 illustrates a schematic diagram of an example architecture 500 for presenting live content according to some embodiments of the present disclosure. The example architecture 500 shows a first functional component 310, which may include a container corresponding to a live object. This container may contain an interface 510 and a real-time communication (RTC) engine 520, which can establish an RTC channel between the terminal device 120 and the server device 130-1. The broadcast component 410 can send a start command to the container's interface 510, which can then send the start command to the RTC engine 520. The RTC engine 520 then sends the start command to the server device 130-1.

[0050] In some embodiments, if terminal device 120 detects the selection of a second display mode, terminal device 120 can receive the media stream of the live stream object from server device 130-1 associated with the live stream object through a first functional component. Terminal device 120 can merge the media stream of the live stream object with the media stream of at least one participant through a second functional component to form a target media stream corresponding to the second display mode. The second functional component is configured to process the live stream content. As an example, the second functional component can be configured to perform merge, trim, effects, and other processing on the media stream used for live streaming to form the live stream content. The second functional component can also be configured to establish a communication channel with server device 130-2, so that terminal device 120 can interact with server device 130-2 through the second functional component. For example, it can push the live stream content to server device 130-2 through the second functional component, and then push the live stream content to the terminal devices of the guests or the audience through server device 130-2.

[0051] As an example, example structure 300 illustrates a second functional component 320, an image acquisition component 330, and terminal devices 120-1, 120-2, ..., 120-M for guest participants. Terminal device 120 can acquire the media stream 302 of the broadcast participant acquired by the image acquisition component 330. The second functional component 320 merges the media stream 301 of the live broadcast object and the media stream 302 of the broadcast participant to form a first media stream 303. Terminal device 120 can also receive media streams 304-1, 304-2, ..., 304-M from terminal devices 120-1, 120-2, ..., 120-M via the second functional component 320, and merge the first media stream 303 and the media streams of the guest participants 304-1, 304-2, ..., 304-M to form the target media stream corresponding to the second display mode.

[0052] Alternatively, terminal device 120 can also send the first media stream 303 to server device 130-2 via the second functional component 320. Server device 130-2 obtains the media streams 304-1, 304-2, ..., 304-M from terminal devices 120-1, 120-2, ..., 120-M, and merges these media streams with the first media stream 303 to form the target media stream corresponding to the second display mode. Afterwards, terminal device 120 can receive the target media stream corresponding to the second display mode from server device 130-2 via the second functional component 303. It should be noted that server device 130-2, used to receive the media stream from the guest participants, and server device 130-1, used to provide the media stream to the live broadcast audience, can be implemented as the same server device or as different server devices.

[0053] As another example, Figure 4B illustrates a flowchart of a signaling stream 400B for presenting live content according to some embodiments of the present disclosure. The signaling stream 400B involves a broadcast component 410, a first functional component 310, a second functional component 320, and a server device 130-1. The second functional component 320 includes a first functional node 420 and a second functional node 430. The first functional node 420 may include a live streaming kernel for performing processes such as image rendering and streaming, and the second functional node 430 may include a merging engine for performing processes such as merging.

[0054] Server device 130-1 can feed back (415) the media stream of the live object to first functional component 310 via the RTC channel. First functional component 310 can send (416) the media stream of the live object to anchor component 410. Anchor component 410 sends (417) the media stream of the live object to second functional node 430. Terminal device 120 can use, for example, an image acquisition component to acquire the media stream of the anchor participant. Anchor component 410 can receive the media stream of the anchor participant from the image acquisition component. Anchor component 410 can send (418) the media stream of the anchor participant to first functional node 420. The media stream of the anchor participant is processed by first functional node 420, for example, by cropping or applying special effects to the media stream of the anchor participant. First functional node 420 can send (418) the processed media stream of the anchor participant to second functional node 430. The second functional node 430 can perform (419) merging (also known as pre-merging) on ​​the media streams of the broadcaster participants and the media streams of the live broadcast object to form the first media stream.

[0055] The second functional node 430 can also perform a merge (421) of the media streams of at least one guest participant and the first media stream (also known as a live stream merge) to form a target media stream corresponding to the second display mode. Additionally, the second functional node 430 can send the first media stream to the server device 130-2 or other remote device, and the server device 130-2 can perform a merge of the media streams of the at least one guest participant and the first media stream. The second functional node 430 can receive the target media stream corresponding to the second display mode from the server device 130-2. In another example, the second functional node 430 can receive the media stream of the at least one guest participant and perform a merge of the media stream of the at least one guest participant and the first media stream locally on the terminal device 120 to form the target media stream corresponding to the second display mode.

[0056] Referring back to Figure 2, in block 220 of process 200, terminal device 120 performs screen rendering on the target media stream based on the selected display mode to obtain the target screen. Specifically, if a selection of a first display mode is detected, terminal device 120 can perform screen rendering on the target media stream corresponding to the first display mode to obtain the target screen corresponding to the first display mode. If a selection of a second display mode is detected, terminal device 120 can perform screen rendering on the target media stream corresponding to the second display mode to obtain the target screen corresponding to the second display mode.

[0057] In some embodiments, the terminal device 120 determines a rendering area corresponding to the selected display mode based on the selected display mode and the configuration information of the live streaming interface. Then, the terminal device 120 can perform image rendering on the target media stream within the rendering area. The configuration information of the live streaming interface can include information related to the construction of the rendering area. For example, the configuration information of the live streaming interface can include the size, resolution, color mode, etc., of the live streaming interface. The terminal device 120 can set the size, resolution, and color mode of the rendering area based on the size, resolution, and color mode of the live streaming interface.

[0058] In some embodiments, if a selection of a first display mode is detected, the terminal device 120 determines the rendering area corresponding to the first display mode based on configuration information using a first functional component. As an example, FIG6 shows an example diagram of an example architecture 600 for live content presentation according to some embodiments of the present disclosure. Example architecture 600 shows a first functional component 310 and a second functional component 320. In block 610, the terminal device 120 can determine the selected display mode, specifically whether it is the first display mode or the second display mode. If it is determined in block 610 that the first display mode is selected, the terminal device 120 can set the rendering area 621 corresponding to the first functional component 310 based on the configuration information of the display interface, so that the first functional component 310 can render the image in the rendering area 621.

[0059] As another example, in signaling flow 400A of Figure 4A, the broadcast component 410 can send a (401) selection instruction to the first functional component 310 in response to the broadcast participant's selection of a first display mode, indicating the selection of the first display mode. This selection instruction and the launch instruction can be sent synchronously or separately. For example, the broadcast component 410 can respond to a launch operation on the target application, defaulting to the first display mode, and simultaneously send the launch instruction and the selection instruction to the container. The first functional component 310 can respond to this selection instruction by constructing a rendering area within itself, such as a canvas or view, etc.

[0060] As another example, referring to the example architecture 500 in Figure 5, the container can receive selection instructions from the broadcaster component 410 through interface 510. The container can send a request to the RTC engine 520 based on the configuration information of the live broadcast interface, requesting the RTC engine 520 to set the canvas, such as the remote video canvas, based on the configuration information of the live broadcast interface.

[0061] In some embodiments, if a selection of a second display mode is detected, the terminal device 120 can determine the rendering area corresponding to the second display mode based on configuration information using a second functional component. As an example, as shown in FIG6, if it is determined in box 610 that the selected display mode is the second display mode, the terminal device 120 can set the rendering area 622 corresponding to the second functional component 320 based on the configuration information of the live streaming interface, so that the second functional component 320 can render the image in the rendering area 622.

[0062] As another example, in the signaling stream 400B shown in Figure 4B, the broadcast component 410 may send a selection instruction (411) to the first functional node 420 in response to the selection of a second display mode, indicating the selection of the second selection mode. The first functional node 420 may set a rendering area (e.g., a view) based on the configuration information of the live broadcast interface.

[0063] In some embodiments, if a second display mode is in effect and a selection of a first display mode is detected, the rendering area corresponding to the second display mode can be removed via a second functional component. For example, the broadcast component 410 can send a removal command to the first functional node 420 in response to a selection of the first display mode. The first functional node 420 can then remove the rendering area at the first working base point 420 in response to the removal command. This completely avoids duplicate rendering.

[0064] In some embodiments, if a first display mode is in effect and a selection of a second display mode is detected, the rendering area corresponding to the first display mode can be removed via a first pass component. As an example, in the signaling stream 400B shown in FIG4B, the broadcast component 410, in response to the selection of the second display mode, sends a removal command (413) to the first functional component 310. Upon receiving the removal command, the first functional component 310 removes (414) the rendering area at its location. As another example, as shown in FIG5, the RTC engine 520 can be requested to remove the canvas via interface 510 to completely avoid rendering the image by the RTC engine 520.

[0065] In some embodiments, as shown in FIG3, if a selection of a first display mode is detected, the first functional component 310 can perform screen rendering on the target media stream (i.e., the media stream of the live broadcast object) corresponding to the first display mode in the rendering area corresponding to the first display mode to form a target screen 305 containing the live broadcast object. If a selection of a second display mode is detected, the second functional component 320 can perform screen rendering on the target media stream (i.e., the media stream formed by merging the media stream of the live broadcast object with the media stream of at least one participant) corresponding to the second display mode in the rendering area corresponding to the second display mode to form a target screen containing the live broadcast object and at least one participant.

[0066] As an example, as shown in Figure 5, a rendering component 521 capable of performing screen rendering can be set within the RTC engine 520. An RTC channel is established between the terminal device 120 and the server device 130-1 via the RTC engine 520, and audio / video streams, such as those from a game, are received from the server device 130-1 through this RTC channel. The rendering component 521 parses the audio / video stream to obtain the parsed audio / video stream. In box 530, the rendering component 521 determines whether a canvas is set. If a canvas is set in box 530, the process proceeds to box 540. In box 540, the rendering component 521 performs screen rendering on the parsed audio / video stream to obtain the target screen (e.g., the game screen). Then, the target screen is fed back to the broadcast component 410 via interface 510. If it is determined in box 530 that no canvas is set within the RTC engine 520, the parsed audio / video stream is fed back to the broadcast component 410 via interface 510.

[0067] In frame 230 of process 200, terminal device 120 presents the target image in the live streaming interface. Here, the live streaming interface may include the live streaming interface of the host participant's terminal device. As an example, as shown in Figure 6, in the first display mode, the first functional component 310 can render and form the target image 631 in the rendering area 621, and present the target image 631 through the host participant's live streaming interface. The target image 631 may include the image 641 of the live streaming object. In the second display mode, the second functional component 320 can render and form the target image 632 in the rendering area 622, and present the target image 632 through the host participant's live streaming interface. The target image 632 may include the image 641 of the live streaming object, the image 642 of the host participant, and the images 643-1, 643-2, ..., 643-N of at least one guest participant.

[0068] In some embodiments, the live streaming interface may further include a live streaming interface for the guest participants and a live streaming interface for the audience. As an example, terminal device 120 can display the target image on the local live streaming interface of the host participant. Simultaneously, terminal device 120 can also encode the target media stream and push the encoded target media stream to the terminal devices of the guest participants and the audience. Thus, in both the first and second display modes, the live streaming images seen by the guest participants and the audience are identical to those seen by the host participant.

[0069] In some embodiments, the live streaming interface may not include the live streaming interfaces of the guest participants and the audience. As an example, terminal device 120 can encode the first media stream 303 formed by pre-streaming merging and push the encoded first media stream 303 to the guest participant's terminal device. Terminal device 120 can also encode the target media stream corresponding to the second display mode formed by live streaming merging and push the encoded target media stream corresponding to the second display mode to the audience's terminal device. Thus, the live streaming images seen by the guest participants and the live streaming images seen by the audience are unrelated to the display mode of the live streaming object displayed by the host participant.

[0070] In this way, the embodiments of this disclosure render the media stream corresponding to the selected display mode according to the display mode selected by the broadcaster, which can avoid repeatedly rendering the screen of the live broadcast object (such as a game), reduce the system resource consumption of the broadcaster's terminal device, improve the smoothness and picture quality of the live broadcast, and reduce the power consumption and extend the standby time of the terminal device.

[0071] Embodiments of this disclosure also provide corresponding apparatus for implementing the methods or processes described above. Figure 7 shows a schematic structural block diagram of an example apparatus 700 for presenting live content according to certain embodiments of this disclosure. Apparatus 700 may be implemented as or included in terminal device 120. The various modules / components in apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0072] As shown in Figure 7, the device 700 includes an acquisition module 710, a rendering module 720, and a presentation module 730. The acquisition module 710 is configured to, in the live streaming of a live streaming object, in response to detecting the selection of a first display mode or a second display mode corresponding to the live streaming object, acquire a target media stream corresponding to the selected display mode. The target media stream corresponding to the first display mode includes the media stream of the live streaming object, and the target media stream corresponding to the second display mode includes a media stream obtained by merging the media stream of the live streaming object with the media streams of at least one participant in the live stream. The rendering module 720 is configured to perform screen rendering on the target media stream based on the selected display mode to obtain a target image. The presentation module 730 is configured to present the target image in the live streaming interface.

[0073] In some embodiments, the rendering module 720 is further configured to: determine a rendering area corresponding to the selected display mode based on the selected display mode and the configuration information of the live streaming interface; and perform screen rendering on the target media stream in the rendering area.

[0074] In some embodiments, the rendering module 720 is further configured to: in response to the selection of a first display mode, determine a rendering area corresponding to the first display mode based on configuration information by a first functional component, wherein the first functional component is configured to process the object content of the live object; or in response to the selection of a second display mode, determine a rendering area corresponding to the second display mode based on configuration information by a second functional component, wherein the second functional component is configured to process the live content of the live stream.

[0075] In some embodiments, the rendering module 720 is further configured to: in response to the selection of a first display mode, remove the rendering area corresponding to the second display mode via a second functional component; or in response to the selection of a second display mode, remove the rendering area corresponding to the first display mode via a first functional component.

[0076] In some embodiments, the acquisition module 710 is further configured to: in response to detecting the selection of a first display mode, receive the media stream of the live object from the server device associated with the live object through a first functional component to form a target media stream corresponding to the first display mode, wherein the first functional component is configured to process the object content of the live object.

[0077] In some embodiments, the acquisition module 710 is further configured to: in response to detecting the selection of a second display mode, receive the media stream of the live object from a server device associated with the live object through a first functional component, the first functional component being configured to process the object content of the live object; and merge the media stream of the live object and the media stream of at least one participant through a second functional component to form a target media stream corresponding to the second display mode, the second functional component being configured to process the live content of the live stream.

[0078] In some embodiments, the at least one participant includes a broadcaster participant and at least one guest participant. The acquisition module 710 is further configured to: merge the media stream of the live object and the media stream of the broadcaster participant through a second functional component to form a first media stream; and merge the first media stream and the media stream of at least one guest participant through a second functional component to form a target media stream corresponding to the second display mode.

[0079] In some embodiments, the acquisition module 710 is further configured to: send a request to the server device through the first functional component to request the server device to run the target application; and receive a media stream corresponding to the user interface of the target application from the server device through the first functional component as a media stream for live streaming.

[0080] In some embodiments, the presentation module is further configured to present the target image in the live streaming interface related to the host participant in the live stream.

[0081] The units and / or modules included in device 700 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 700 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0082] Figure 8 shows a block diagram of an electronic device 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 800 shown in Figure 8 is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. The electronic device 800 shown in Figure 8 may include or be implemented as the terminal device 120 of Figure 1, or the device 700 of Figure 7.

[0083] As shown in Figure 8, the electronic device 800 is in the form of a general-purpose electronic device. Components of the electronic device 800 may include, but are not limited to, one or more processing units or processors 810, memory 820, storage devices 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processor 810 may be a physical or virtual processor and is capable of performing various processes according to programs stored in the memory 820. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device 800.

[0084] Electronic device 800 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 830 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 800.

[0085] Electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG8, disk drives for reading or writing from removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading or writing from removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 820 may include computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0086] The communication unit 840 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 800 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 800 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0087] Input device 850 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 860 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 800 can also communicate with one or more external devices (not shown) via communication unit 840 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 800, or with any device that enables electronic device 800 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0088] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0089] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0090] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0091] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0093] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for presenting live streaming content, comprising: In the live streaming of a live streaming object, in response to detecting the selection of a first display mode or a second display mode corresponding to the live streaming object, a target media stream corresponding to the selected display mode is obtained, wherein the target media stream corresponding to the first display mode includes the media stream of the live streaming object, and the target media stream corresponding to the second display mode includes a media stream obtained by merging the media stream of the live streaming object with the media streams of at least one participant in the live streaming; Based on the selected display mode, image rendering is performed on the target media stream to obtain the target image. The target image is displayed on the live streaming interface.

2. The method of claim 1, wherein performing screen rendering on the target media stream based on the selected display mode comprises: Based on the selected display mode and the configuration information of the live streaming interface, determine the rendering area corresponding to the selected display mode; as well as Rendering is performed on the target media stream in the rendering area.

3. The method according to claim 2, wherein determining the rendering area corresponding to the selected display mode based on the selected display mode and the configuration information of the live streaming interface includes: In response to the selection of the first display mode, a first functional component determines the rendering area corresponding to the first display mode based on the configuration information, and the first functional component is configured to process the object content of the live object. or In response to the selection of the second display mode, a second functional component determines the rendering area corresponding to the second display mode based on the configuration information, and the second functional component is configured to process the live content of the live broadcast.

4. The method according to claim 3, wherein determining the rendering area corresponding to the selected display mode further includes: In response to the selection of the first display mode, the rendering area corresponding to the second display mode is removed by the second functional component; or In response to the selection of the second display mode, the rendering area corresponding to the first display mode is removed by the first functional component.

5. The method of claim 1, wherein obtaining the target media stream corresponding to the selected display mode comprises: In response to detecting the selection of the first display mode, a first functional component receives the media stream of the live object from a server device associated with the live object to form the target media stream corresponding to the first display mode, wherein the first functional component is configured to process the object content of the live object.

6. The method of claim 1, wherein obtaining the target media stream corresponding to the selected display mode comprises: In response to detecting a selection of the second display mode, the media stream of the live object is received from a server device associated with the live object via a first functional component, the first functional component being configured to process the object content of the live object; as well as The media stream of the live streaming object and the media stream of the at least one participant are merged by the second functional component to form the target media stream corresponding to the second display mode. The second functional component is configured to process the live streaming content.

7. The method of claim 6, wherein the at least one participant includes a broadcaster participant and at least one guest participant, and wherein merging the media stream of the live stream object and the acquired media stream of the at least one participant via the second functional component includes: The second functional component merges the media stream of the live stream object and the media stream of the anchor participant to form a first media stream; as well as The second functional component merges the first media stream and the media stream of the at least one guest participant to form the target media stream corresponding to the second display mode.

8. The method of claim 5 or 6, wherein receiving the media stream of the live stream object from a server device associated with the live stream object via a first functional component comprises: The first functional component sends a request to the server device to request the server device to run the target application; as well as The first functional component receives a media stream corresponding to the user interface of the target application from the server device as the media stream of the live streaming object.

9. The method according to claim 1, wherein presenting the target image comprises: The target image is displayed on the live streaming interface related to the host and other participants in the live stream.

10. An apparatus for presenting live streaming content, comprising: The acquisition module is configured to, in the live broadcast of a live broadcast object, in response to detecting the selection of a first display mode or a second display mode corresponding to the live broadcast object, acquire a target media stream corresponding to the selected display mode, wherein the target media stream corresponding to the first display mode includes the media stream of the live broadcast object, and the target media stream corresponding to the second display mode includes a media stream obtained by merging the media stream of the live broadcast object with the media streams of at least one participant in the live broadcast; The rendering module is configured to perform image rendering on the target media stream based on the selected display mode to obtain the target image. The presentation module is configured to present the target image in the live streaming interface of the live stream.

11. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 9 when executed by the at least one processor.

12. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 9.

Images