Live-streaming method and apparatus, and device and medium

Abstract

Provided in the embodiments of the present disclosure are a method and apparatus for managing a live-streaming room, and a device, and a medium. The method comprises: in response to an access request for accessing a virtual environment at a server device having been received, controlling a first terminal device of a first user of a live-streaming room, so as to start a container at the first terminal device; controlling the first terminal device to acquire state data of the virtual environment via the container; and obtaining live-streaming data on the basis of the state data, and presenting the live-streaming data in the live-streaming room. In this way, live-streaming modes are enriched, and the convenience of live streaming is improved.

Description

Live streaming method, apparatus, device and medium

[0001] The present application claims priority to the Chinese patent application No. 202410842285.7, filed on June 26, 2024, entitled “Live streaming method, apparatus, device and medium”, the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] Exemplary implementations of the present disclosure generally relate to the field of computers, and in particular, to a live streaming method, apparatus, device and computer readable storage medium. BACKGROUND

[0003] With the development of computer technology, more and more applications can provide live streaming function. During the live streaming process, for example, a host can run a virtual environment (e.g., a game environment, a virtual reality environment, etc.) at a terminal device of the host, and provide a live streaming room with a picture of the host user’s activities in the virtual environment. However, the virtual environment can involve a complex scene and require the use of more resources, which can result in that the live streaming can only be started at a terminal device with high performance, thereby reducing the richness and convenience of the live streaming. SUMMARY

[0004] In a first aspect of the present disclosure, a live streaming method is provided. The method comprises: in response to receiving an access request for accessing a virtual environment at a server device, controlling a first terminal device of a first user of a live streaming room to start a container at the first terminal device; controlling the first terminal device to obtain state data of the virtual environment via the container; and obtaining live streaming data based on the state data, the live streaming data being presented in the live streaming room.

[0005] In a second aspect of the present disclosure, a live streaming apparatus is provided. The apparatus comprises: a container starting module configured to, in response to receiving an access request for accessing a virtual environment at a server device, control a first terminal device of a first user of a live streaming room to start a container at the first terminal device; a data obtaining module configured to control the first terminal device to obtain state data of the virtual environment via the container; and a data generating module configured to obtain live streaming data based on the state data, the live streaming data being presented in the live streaming room.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The electronic device comprises: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to the first aspect of the present disclosure.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, causes the processor to implement the method according to the first aspect of the present disclosure.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to the first aspect of the present disclosure.

[0009] It is to be understood that the particulars shown herein are by way of example and for purposes of illustrative discussion of the various embodiments of the present disclosure only and are not intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above-mentioned and other features and advantages of various embodiments of the present disclosure will become more apparent from the following description, taken in conjunction with the accompanying drawings, wherein:

[0011] FIG. 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG. 2 shows a schematic diagram of an example for managing a live room, according to some embodiments of the present disclosure;

[0013] FIG. 3 shows a schematic diagram of an example for managing a live room, according to some other embodiments of the present disclosure;

[0014] FIG. 4A shows a schematic diagram of a signaling flow for starting a game, according to some embodiments of the present disclosure;

[0015] FIG. 4B shows a schematic diagram of a signaling flow for ending a game, according to some embodiments of the present disclosure;

[0016] FIG. 5 shows a schematic diagram of an example of generating live data, according to some embodiments of the present disclosure;

[0017] FIG. 6 shows a schematic diagram of an example of processing data by a first service, according to some embodiments of the present disclosure;

[0018] FIG. 7 shows a flowchart of a process for managing a live room, according to some embodiments of the present disclosure;

[0019] FIG. 8 shows a schematic structural block diagram of an apparatus for managing a live room, according to some embodiments of the present disclosure; and

[0020] FIG. 9 shows a block diagram of an electronic device in which one or more embodiments of the present disclosure can be implemented. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided so that the present disclosure can be understood more thoroughly and completely. It is understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] In the description of embodiments of the present disclosure, the term "comprising" and its conjugations should be understood to encompass the meaning of "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "an embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". The terms "first", "second" and the like can refer to different or identical objects. Other explicit and implicit definitions can also be included below.

[0023] In this document, unless explicitly stated, performing a step "in response to" A does not mean that the step is performed immediately after A, but can include one or more intermediate steps.

[0024] In this document, the term "virtual environment" can include but is not limited to: game environment, simulation environment, simulation environment, virtual reality environment, augmented reality environment, etc., and embodiments of the present disclosure are not limited in this respect. For ease of description, only game environment will be described as an example of virtual environment in the following.

[0025] It can be understood that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0026] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the scope of use, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0027] For example, in response to receiving the user's active request, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can voluntarily choose whether to provide personal information to the software or hardware such as electronic device, application program, server or storage medium, etc. which performs the operation of the technical solutions of the present disclosure.

[0028] As an optional but non-limiting implementation, in response to receiving the active request of the user, the manner of sending the prompt information to the user can be, for example, a pop-up window manner, in which the prompt information can be presented in a text manner. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0029] It can be understood that the above notification and user authorization obtaining process is only illustrative and does not limit the implementation of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0030] Example Environment

[0031] FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In the example environment 100, an electronic device 110 has an application 120 installed therein. A user 140 can interact with the application 120 via the electronic device 110 and / or an attached device of the electronic device 110.

[0032] In some embodiments, the application 120 can be a content sharing application (e.g., a video application mainly for video sharing), which is capable of providing various services related to media items (also referred to as media content, media content items, content items, etc.) to the user 140, including browsing, commenting, forwarding, creating (e.g., shooting and / or editing), publishing, etc. of the media items. In some embodiments, the application 120 can also be any other appropriate application in which media items can be presented. In this document, each "media item" includes one or more types of content, such as video, image, moving image, image set, audio, text, etc.

[0033] In the environment 100 of FIG. 1, if the application 120 is in an active state, the electronic device 110 can present a page 150 of the application 120. The page 150 can include various pages that can be provided by the application 120, such as a media item presentation page, a media item creation page, a media item publishing page, a message page, a personal homepage, etc. The application 120 can provide a media item viewing function to view various media items published in the application 120.

[0034] In some embodiments, the electronic device 110 communicates with the server 130 to enable provisioning of services for the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media player, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a game device, or any combinations of these and the like, including accessories and peripherals related thereto, or any combinations thereof. In some embodiments, the electronic device 110 can also support any type of interface to the user (such as “wearable” circuitry, etc.). The server 130 can be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, and the like.

[0035] It should be appreciated that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and without implying any limitation on the scope of the present disclosure.

[0036] Example Environment

[0037] FIG. 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In the environment 100, a user 110 can establish a live room (e.g., a live room in a live application can correspond to a live session) and provide live content, etc. via an associated terminal device 120. In some scenarios, the user 110 is also referred to as a host, a broadcaster, or a manager of the live room, for example. The terminal device 120 can also be referred to as a host end of the live room.

[0038] One or more users 130-1, 130-2, …, 130-N can watch the live and participate in the interaction of the live room, etc. via respective associated terminal devices 140-1, 140-2, …, 140-N. For ease of discussion, the users 130-1, 130-2, …, 130-N can be collectively or individually referred to as the users 130, and the terminal devices 140-1, 140-2, …, 140-N can be collectively or individually referred to as the terminal devices 140. In some scenarios, the users 130 can also be referred to as viewers, listeners, watching parties, or participating parties of the live room. The terminal devices 140 can also be referred to as viewer ends of the live room. Alternatively and / or additionally, the users 130 can be invited to join the live, at which time the invited users can be converted to guest users and go on the mic to talk with the host.

[0039] It should be appreciated that although only a single anchor user is shown in FIG. 1, in some embodiments, there can be multiple anchor users participating in a live broadcast in a certain live broadcast room. In some embodiments, the terminal device 120 and the terminal device 140 can each have an application installed thereon that is capable of providing a live broadcast service, or can access a website that is capable of providing a live broadcast service. The user 110 and the user 130 can operate the terminal device 120 and the terminal device 140 to access the corresponding application or website.

[0040] Accordingly, the terminal device 120 and the terminal device 140 can present a corresponding live broadcast interface, which can provide, for example, live broadcast content of the live broadcast room, such as audio live broadcast content or video live broadcast content, etc.

[0041] In some embodiments, the terminal device 120 and the terminal device 140 can also communicate with the server 150 through the network 152 to implement the provision of the live broadcast service. The server 150 can provide functions such as management, configuration and maintenance of the application or website. The server 150 can include multiple servers, for example, a server for providing support for the live broadcast service, a server for providing support for a game service, etc.

[0042] The terminal device 120 and the terminal device 140 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a subnotebook computer, a netbook computer, a tablet computer, a media player, a navigation device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a television receiver, a radio broadcast receiver, an e-book device, a game device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, the terminal device 120, 140 can also be capable of supporting any type of interface to user (such as "wearable" circuitry, etc.).

[0043] The server 150 can be various types of computing systems / servers capable of providing computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, etc. The server 150 can provide, for example, background services for the applications in the terminal device 120 and the terminal device 140 that provide the live broadcast service. In some embodiments, if an anchor user initiates an activity in a live broadcast room, the server 150 can include a server for providing the live broadcast service and a server for providing the activity service. Alternatively and / or additionally, the live broadcast service and the activity service can be provided by a single server.

[0044] It should be appreciated that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and without implying any limitation on the scope of the present disclosure. Some example embodiments of the present disclosure will be described hereinafter with continuous reference to the drawings.

[0045] For ease of description, the process of managing a live room is described below only as an example of a user accessing a virtual environment (e.g., a game environment, a virtual reality environment, an augmented reality environment, etc.) as an activity. Traditionally, a host can run a small virtual environment at a terminal device, however, since a large virtual environment can involve a complex scene and require the use of more resources (e.g., computing resources and storage resources). This results in that the host can only load a large virtual environment by means of a computing device with rich resources (e.g., a desktop computer, etc.) and live in a live room. This results in that the live environment is limited, reducing the richness and convenience of the live. At this time, it is desirable to be able to load a large virtual environment at a portable terminal device and provide live.

[0046] Summary of managing a live room

[0047] In view of this, embodiments of the present disclosure propose a live improvement scheme. According to the scheme, in response to receiving an access request for accessing a virtual environment at a server device, a first terminal device (e.g., terminal device 120) of a first user (e.g., a host user of a live application, e.g., user 110) of a live room is controlled to start a container at the first terminal device. The first terminal device can be controlled to obtain state data of the virtual environment via the container. Live data is obtained based on the state data, and the live data is rendered in the live room. The terminal device 120 can be a portable device with limited resources. Here, the container can be a container located locally at the first terminal device, and the virtual environment can be located at a remote server device different from the first terminal device. In this way, the virtual environment runs at the server device, and the terminal device 120 can access the virtual environment with the container, thereby enabling the terminal device 120 to provide a live room.

[0048] In the context of the present disclosure, the state data of the virtual environment can represent data from the virtual environment, for example, including but not limited to sound data and image data in the virtual environment. Specifically, under a game environment, the state data of the virtual environment can include an audio and video stream from the game environment. The live data can include data transmitted during live, for example, voice (e.g., game commentary, etc.), video (e.g., video during participation in the game, etc.), and / or other required data from the host and / or guests can be added on the basis of the audio and video stream of the game environment.

[0049] Taking a virtual environment including a game environment, a container of the terminal device 120 as a cloud game container as an example, referring to FIG. 2 describing a summary according to one example embodiment of the present disclosure, FIG. 2 shows a schematic diagram of an example 200 for managing a live room according to some embodiments of the present disclosure. The example 200 involves the terminal device 120, a cloud game container 210, a cloud game server 220, a game client 240 running on a cloud instance, and an RTC (real-time communication) room 230 (i.e., a live room).

[0050] In the live state, the terminal device 120 can start the cloud game container 210 in response to receiving an access request for accessing a game environment at a server device (in the game scenario, the server device can be a cloud game server) (e.g., a start operation of a game by a host user). The cloud game container 210 can start and run the game client 240 on a cloud instance by means of the cloud game server 220. After the game client 240 is running, the cloud game SDK (software development kit) 250 can create the RTC room 230.

[0051] The RTC room 230 can transmit state data of the game environment at the game client 240 (e.g., which can include an audio and video stream of the game environment) to the cloud game container 210. The RTC room 230 can also receive game instructions sent by the cloud game container 210 (i.e., implement game instruction transmission). Thus, the host user can operate the game running at the game client 240 (e.g., operate a virtual character to perform an action in the game environment corresponding to the game client 240) at the terminal device 120.

[0052] The terminal device 120 obtains state data of the game environment (i.e., an audio and video stream of the game environment) via the cloud game container 210, and obtains live data based on the state data. The live data in the live room can be presented, i.e., presented to each user who joins the live room. The method of the present disclosure can be executed at any computing device with computing capability, e.g., the above method can be executed at a server (e.g., the server 150 and / or other servers in the network) managing the live room, so as to control the terminal device 120 to start a local container, thereby supporting game live streaming. In the context of the present disclosure, the server managing the live room can control the first terminal device by a control instruction. For example, in response to the access request described above, the server managing the live room can send a control instruction to the first terminal device. At this time, the first terminal device can start the container in response to receiving the control instruction. For another example, the above method can be executed at the terminal device. At this time, the first terminal device can control the first terminal device to start the container in the case where the access request is detected.

[0053] In this way, in embodiments of the present disclosure, the anchor user can obtain the state data of the virtual environment by means of the container (e.g., the cloud game container 210), and obtain the live broadcast data based on the state data. Since the virtual environment is actually run at the cloud game server with powerful resources, the terminal device of the anchor does not require powerful resources, but can utilize the container to transfer data between the cloud game server and the terminal device. This enables the anchor user to perform live broadcast associated with the virtual environment at any appropriate terminal device, which helps to enrich the live broadcast mode and improve the convenience of live broadcast. Some example embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0054] Detailed process of managing a live broadcast room

[0055] In a live broadcast scenario, the live broadcast platform often uses RTC and CDN services. When the anchor user and the guest user are in a live microphone, the live microphone data will use the RTC service to ensure low-latency real-time interaction (the data stream corresponding to the RTC service can be referred to as an RTC stream). The RTC service will then push the live microphone content to the CDN service, and the audience users who are not in a live microphone will still obtain the live broadcast content from the CDN service (the data stream corresponding to the CDN service can be referred to as a CDN stream).

[0056] FIG. 3 shows a schematic diagram of an example 300 for managing a live broadcast room according to some embodiments of the present disclosure. As shown in FIG. 3, when the terminal device 120 is in a live microphone 330 state, the terminal device 120 can receive an access request of the anchor user for accessing a virtual environment (e.g., a game environment) at a server device (e.g., a server providing a game). For example, the terminal device 120 can determine that the access request of the anchor user for accessing the virtual environment is received in response to receiving a user operation of the anchor user for initiating a game.

[0057] Further, the terminal device 120 can be controlled to start a container (e.g., the cloud game container 210) in response to receiving the access request. The cloud game container 210 can also send the game instructions received by the terminal device 120 from the anchor user to the game client 240, and the game client 240 can also send the generated game instructions to the cloud game container 210. The terminal device 120 can also obtain the state data of the game environment (e.g., which can include an audio and video stream of the game environment) from the game client 240 running at the cloud instance 360 by means of the cloud game container 210.

[0058] Under the control of the method of the present disclosure, the terminal device 120 can directly acquire the state data by means of the cloud game container 210. In some embodiments, the terminal device 120 can receive the state data from the first service (for example, the RTC service 320 provided by the first server of the live room) for managing the live room via the cloud game container 210. Specifically, the terminal device 120 can forward the interaction request to the server device to update the state data of the virtual environment in response to receiving the interaction request (for example, the interaction request for the game environment), and receive the updated state data. For example, the terminal device 120 can forward the interaction request to the server device in response to receiving the interaction request of the host user for operating the game. The server device can operate the game environment based on the interaction request to update the game environment. After the game environment is updated, the terminal device 120 can receive the updated state data corresponding to the updated game environment via the cloud game container 210. In this way, the cloud game container 210 can assist in forwarding data between the terminal device 120 and the cloud instance 360, thereby achieving the goal of controlling the virtual environment by the host at his own terminal device 120.

[0059] Regarding the specific way of forwarding the interaction request, in some embodiments, the terminal device 120 can forward the state data to the first service via the container (for example, the cloud game container 210) so that the forwarded state data is transmitted to the server device. In this way, the terminal device 120 can forward data between itself and the server device by means of the RTC service 320 and the cloud game container 210, thereby operating the virtual character in the game and / or performing other actions in real time.

[0060] The terminal device 120 can generate live data based on the acquired state data of the virtual environment, and this live data can be presented in the live room. In some embodiments, in addition to the state data, under the control of the method of the present disclosure, the terminal device 120 can also acquire first collection data from the first collection device. The first collection device may, for example, include an image collection device (for example, a camera) of the terminal device 120, an audio collection device (for example, a loudspeaker), and the like. The first collection data may, for example, include image data, audio data, and the like associated with the host user. For example, the terminal device 120 can acquire image data and audio data of the host user from the camera, thereby generating video data.

[0061] Further, the first collection data can be added in the live data, so as to generate anchor data (which can also be referred to as anchor audio / video stream). At this time, the anchor data can include video data of the virtual environment itself, and video data from the anchor. Specifically, the video data of the virtual environment can be presented in a predetermined area (for example, a large window in the display area), and the live video data from the anchor can be presented in another predetermined area (for example, a small window in the middle position at the top of the display area). In this way, it is possible to support providing more rich information in the live room.

[0062] In some embodiments, under the control of the method of the present disclosure, this anchor data can also be generated by the terminal device 120. For example, the terminal device 120 can obtain the state data, and generate the anchor data based on the first collection data collected by the terminal device 120 via the first collection device and the obtained state data. In this case, the terminal device 120 can directly provide the anchor data to the RTC service 320 for further processing by the RTC service 320.

[0063] In some embodiments, if there are guest users in the live room, the terminal device 340 corresponding to the guest users is also in the live mic-in 350 state. The terminal device 340 can obtain second collection data of a second collection device. The second collection device can include, for example, an image collection device (such as a camera) of the terminal device 340, an audio collection device (such as a speaker), and the like. The second collection data can include, for example, image data, audio data, and the like associated with the guest user. For example, the terminal device 340 can obtain image data and audio data of the guest user from the camera, thereby generating video data. The second collection data can also be referred to as guest audio / video stream. Although FIG. 3 only shows one guest user, alternatively and / or additionally, the anchor user can invite multiple guest users, and the terminal device of each guest user can collect the corresponding guest audio / video stream in a similar manner.

[0064] In some embodiments, the terminal device 340 can also provide the second collection data to the RTC service 320, respectively. Thus, the RTC service 320 can provide the guest audio / video stream to the terminal device 120, so as to add data about the guest in the live data.

[0065] The RTC service 320 can process the data. For example, the RTC service 320 can receive the anchor audio / video stream sent by the terminal device 120 and the guest audio / video stream sent by the terminal device 340. The RTC service 320 can also provide, for example, the guest audio / video stream of the guest user participating in the live room to the terminal device 120 and the anchor data of the anchor user to the terminal device 340. If there are multiple guest users, for each guest user, the RTC service 320 can merge the anchor data and the second collection data corresponding to other guests (i.e., the second collection data corresponding to the other guest users other than the guest user) to merge into one RTC stream, and provide the RTC stream to the terminal device 340 corresponding to the guest user.

[0066] The RTC service 320 can also merge the multiple second collection data corresponding to multiple guests into one RTC stream, and provide the RTC stream to the terminal device 120. Under the control of the method of the present disclosure, the second collection data of the guest user can also be obtained and the live data can be updated based on the second collection data, for example, in response to determining that the live room includes a guest user. The updated live data can include, for example, the anchor data (i.e., the state data and the first collection data) and the second collection data. The terminal device 120 can then present the updated live data. In this way, the live data can include more rich content.

[0067] It should be noted that the live data can be generated by the terminal device 120 or by the RTC service 320. In the mode in which the terminal device 120 obtains the live data, the terminal device 120 can directly obtain the live data, thereby avoiding the time delay of transmitting data to the RCT service to obtain the live data. In the mode in which the RTC service 320 obtains the live data, the RTC service 320 can obtain the live data based on the anchor data and the second collection data, for example. Although data transmission involves a certain time delay, this mode can greatly reduce the workload of the terminal device 120, and thus is particularly suitable for terminal devices 120 with limited performance.

[0068] In some embodiments, the terminal device 120 / RTC service 320 can also transmit the live data to a second service (e.g., a CDN service 310 provided by a second server of the live room) of the live room to generate content distribution data. The CDN service 310 can push the content distribution data to the terminal device of the audience user 301 in the form of a CDN stream, so that the content distribution data is presented at the terminal device of the audience user 301 of the live room. It can be understood that similarly, the terminal device 120 / RTC service 320 can also transmit the updated live data to the CDN service 310 to generate updated content distribution data.

[0069] The specific example of the anchor user accessing the game environment is described in detail below in combination with FIG. 4A and FIG. 4B. Referring to FIG. 4A, FIG. 4A shows a schematic diagram of a signaling flow 400 of starting a game according to some embodiments of the present disclosure. The terminal device 120 of the anchor user can be referred to as an anchor end for short, the terminal device of the guest user can be referred to as a guest end 402 for short, and the terminal device of the spectator user can be referred to as a spectator end 401 for short. The terminal device 120 can send (411) an access request for accessing the game environment at the server device to the live server 403 in response to receiving the access request from the anchor user. The live server 403 can start the game environment on the cloud instance in response to the access request, and return (412) a starting result to the terminal device 120.

[0070] The terminal device 120 can switch (413) the layout of the live page for presenting the live room in response to receiving the starting result. For example, the terminal device 120 can present a game page corresponding to the game environment in the background layer of the live page, and superimposedly present an anchor window for presenting the image of the anchor user and a guest window for presenting the image of the guest user at any appropriate position on the game page. In some embodiments, the live server 403 can also send (i.e., 414 and 416) the starting result to the guest end 402 and the spectator end 401, respectively. The guest end 402 and the spectator end 401 can switch (i.e., 415 and 417) the layout of the live page for presenting the live room based on the starting result, respectively.

[0071] Further, the terminal device 120 can initialize (418) a cloud game container 405. After the cloud game container 405 is initialized, the cloud game container 405 can call (419) a cloud game server 406 to pull up (420) a cloud game 407 (i.e., a game environment running on a cloud instance). The cloud game 407 can instruct (421) a cloud game RTC 408 to create an RTC room, and send (422) a cloud game starting result to the cloud game server 406. The cloud game server 406 can forward (423) the received cloud game starting result to the cloud game container 405. The live server 403 can acquire the cloud game starting result via the cloud game container 405 in response to the cloud game container 405 acquiring the cloud game starting result, and send (424) the cloud game starting result to the terminal device 120.

[0072] After the cloud game is started, the cloud game container 405 can acquire (425) state data of the game environment (e.g., which can include an audio and video stream of the game environment) from the cloud game RTC 408. The terminal device 120 can in turn acquire (426) the state data of the game environment via the cloud game container 405. The terminal device 120 can also acquire first capture data (e.g., which can include video, images, and audio associated with the host user) from a first capture device (e.g., which can include a camera and a speaker). In some embodiments, the terminal device 120 can generate (427) host data based on the first capture data and the state data.

[0073] The terminal device 120 can send (428) the generated host data to the co-host RTC 404 (which can also be referred to as a first service, a RTC service, a co-host RTC service, etc.). In some embodiments, in the case that the live room also includes a guest user, a terminal device corresponding to the guest user (i.e., the guest end 402) can also send second capture data (e.g., which can include video, images, and audio associated with the guest user) from a second capture device (e.g., which can also include a camera and a speaker) to the co-host RTC 404. The co-host RTC 404 can perform (429) a merge operation based on the acquired host data and the second capture data to obtain live data, for example.

[0074] In some embodiments, the co-host RTC 404 can generate content distribution data based on the live data by means of a second service (e.g., a CDN service) for each audience user participating in the live room. The co-host RTC 404 can provide (430) the content distribution data to the audience end 401, so that the audience user participating in the live room can browse to the content distribution data.

[0075] In some embodiments, the co-host RTC 404 can also send (431) the live data to the guest end 402 corresponding to the guest user in the live room. Illustratively, if the live room includes only one guest user, the co-host RTC 404 can obtain the live data based on the host data and send the live data to the guest end 402. If the live room includes multiple guest users, the co-host RTC 404 can obtain the live data based on the host data and second capture data of the other guest users other than the guest user, and send the live data to the guest end 402. In this way, the co-host RTC 404 can distribute the live data including the state data of the game environment to the audience end 401, the guest end 402, and the host end 120.

[0076] Further, referring to FIG. 4B, FIG. 4B shows a schematic diagram of a signaling flow 400 of ending a game according to some embodiments of the present disclosure. The terminal device 120 can send (441) a quit request to the live streaming server 403 in response to receiving the quit request (e.g., a game quit operation from the host user) for quitting the game environment at the server device. The live streaming server 403 can quit the game environment on the cloud instance and return (442) a quit result to the terminal device 120 in response to the quit request.

[0077] The terminal device 120 can switch (443) the layout of the live streaming page for presenting the live room and cancel (444) generating the host data based on the state data and the first collection data in response to receiving the quit result. For example, the terminal device 120 can cancel presenting the game page corresponding to the game environment in the background layer of the live streaming page, the terminal device 120 can directly determine the first collection data as the host data, and the terminal device 120 can directly present the host page corresponding to the host data in the background layer of the live streaming page.

[0078] In some embodiments, the terminal device 120 can further instruct (445) the cloud game container 405 to quit the virtual environment. The cloud game container 405 can close (447) the cloud game 407 via (446) the cloud game server 406. The cloud game 407 can instruct the cloud game RTC 408 to destroy (448) the previously created RTC room and return (449) a cloud game close result to the cloud game server 406. The cloud game server 406 can forward (450) the received cloud game close result to the cloud game container 405.

[0079] The live streaming server 403 can acquire the cloud game close result via the cloud game container 405 in response to the cloud game container 405 acquiring the cloud game close result. In some embodiments, the terminal device 120 can further release the cloud game container 405 in response to determining that the game environment is quit. The live streaming server 403 can send (i.e., 451 and 452) the cloud game close result to the guest end 402 and the audience end 401, respectively. The guest end 402 and the audience end 401 can switch (i.e., 453 and 454) the layout of the live streaming page based on the cloud game close result, respectively.

[0080] Referring to FIG. 5, FIG. 5 shows a schematic diagram of an example of obtaining live streaming data according to some embodiments of the present disclosure. As shown in FIG. 5, the terminal device 120 can acquire first collection data 501 (which can include video, audio, image, etc. associated with the host user, which can also be referred to as a host audio / video stream) from a first collection device.

[0081] The terminal device 120 can perform special effect processing 502 on the first collected data 501 based on a user setting. For example, the terminal device 120 can perform whitening on the anchor image in the anchor audio and video stream, perform voice changing on the anchor audio, and the like based on the user setting. The terminal device 120 can generate an anchor preview small window 503 based on the first collected data 501 on which the special effect processing 502 is performed. The anchor preview small window 503 can be, for example, a window for presenting an anchor picture 504 (i.e., an anchor image, a video).

[0082] The terminal device 120 can also acquire state data of the game environment from the game client 370 in the cloud instance 360 by means of the cloud game container 210. The terminal device 120 can determine a game picture based on the state data of the game environment. The game picture can be, for example, for presenting the state data. The terminal device 120 can also acquire second collected data corresponding to the guest users in the live room from the RTC service 320 (which can also be referred to as guest audio and video streams), and based on the second collected data, determine a plurality of guest windows corresponding to the plurality of guests participating in the live room. The plurality of guest windows can be used to present the respective guest pictures (e.g., pictures 507-1, 507-2, and 507-3) of the plurality of guests.

[0083] The terminal device 120 can generate a live page 506 based on, for example, the game picture, the anchor picture 504, the pictures 507-1, 507-2, and 507-3. The live page 506 is used to present live data, and the live data includes the first collected data, the state data, and the second collected data. Therefore, the live page 506 can include the game picture, the anchor picture 504, the pictures 507-1, 507-2, and 507-3. The game picture can be presented, for example, in a background layer of the live page 506. The anchor picture 504, the pictures 507-1, 507-2, and 507-3 can be superimposed and presented on the game picture. The terminal device 120 can send the live data to the RTC service for further processing, for example.

[0084] Referring to FIG. 6, FIG. 6 shows a schematic diagram of an example 600 of processing data of a first service according to some embodiments of the present disclosure. As shown in FIG. 6, the terminal device 120 can obtain state data 602 of a virtual environment and first capture data 601. The state data 602 may, for example, include audio and video data of a game environment. The first capture data may, for example, include anchor audio and video data. The terminal device 120 can perform special effect processing 604 on the first capture data 601. The terminal device 120 can generate an anchor preview window 603 based on the first capture data 601 on which the special effect processing 604 is performed. The anchor preview window 603 may, for example, be a window for presenting an anchor picture. The terminal device 120 can input the first capture data 601 on which the special effect processing 604 is performed and the state data 602 to a pre-merge point 605. The pre-merge point 605 may, for example, generate anchor data based on the first capture data 601 and the state data 602. The terminal device 120 can provide the anchor data to an RTC service 606.

[0085] The RTC service 606 can push the anchor data in the form of an RTC stream. The anchor data may, for example, be as shown in a live page 609, in which a game picture corresponding to the game environment is in a background layer, and an anchor picture is presented on the game picture. The RTC service 606 can further obtain second capture data 608 corresponding to a guest user. The RTC service 606 can perform RTC merging based on the anchor data and the second capture data 608. Specifically, the RTC service 606 can obtain live data based on the anchor data and the second capture data 608. The live data may, for example, be as shown in a live page 610, in which a game picture is in a background layer, and an anchor picture and guest pictures corresponding to a plurality of guest users are presented on the game picture. The RTC service 606 may, for example, send the live data to a CDN service, so that the CDN service pushes the live data to a viewer user in the form of a CDN stream.

[0086] In summary, according to embodiments of the present disclosure, the state data of a virtual environment can be obtained by means of a container, and live data to be presented in a live room can be generated based on the state data. This enables an anchor user to access a virtual environment at a cloud end at any terminal device, and the anchor user can perform a live show associated with the virtual environment at any appropriate terminal device, which helps to enrich live show modes and improve the convenience of live shows.

[0087] Example process

[0088] The specific details of each step of the query have been described above, and a method of managing a live room is provided. FIG. 7 shows a flowchart of a live process 700 according to some embodiments of the present disclosure. The process 700 can be implemented at any device having computing capability. The process 700 is described below with reference to FIG. 1.

[0089] At block 710, in response to receiving the access request for accessing the virtual environment at the server device, the first terminal device of the first user of the live room is controlled to start a container at the first terminal device.

[0090] At block 720, the first terminal device is controlled to obtain state data of the virtual environment via the container.

[0091] At block 730, live data is obtained based on the state data, the live data being rendered in the live room.

[0092] In some embodiments, obtaining the state data comprises receiving, via the container, the state data from a first service for managing the live room, the first service being started by the server device.

[0093] In some embodiments, the process 700 further comprises: in response to receiving the interaction request at the first terminal device, controlling the first terminal device to forward the interaction request to the server device to update the state data of the virtual environment; and receiving the updated state data.

[0094] In some embodiments, controlling the first terminal device to forward the interaction request comprises: controlling the first terminal device to forward, via the container, the state data to the first service, such that the forwarded state data is transmitted to the server device.

[0095] In some embodiments, the first user is a host user of the live application, and obtaining the live data comprises: obtaining first capture data from a first capture device; obtaining host data based on the first capture data and the state data; and obtaining the live data based on the host data.

[0096] In some embodiments, obtaining the live data further comprises: in response to determining that the live room comprises a guest user, obtaining second capture data from a second capture device of the guest user; and updating the live data based on the second capture data.

[0097] In some embodiments, the process 700 further comprises: the updated live data is rendered at the first terminal device.

[0098] In some embodiments, obtaining the second capture data comprises: obtaining the second capture data via the first service.

[0099] In some embodiments, the process 700 further comprises: transmitting the updated live data to a second service of the live room to generate content distribution data, the content distribution data being rendered at a terminal device of a viewer user of the live room.

[0100] In some embodiments, the process 700 further includes: in response to determining that the live room includes a guest user, obtaining, by the first service, second capture data from a second capture device of the guest user; updating, by the first service, the live data based on the second capture data; and transmitting, by the first service, the updated live data to a second service of the live room to generate content distribution data, the content distribution data being rendered at a terminal device of a spectator user of the live room.

[0101] In some embodiments, the process 700 further includes: in response to receiving a quit request for quitting the virtual environment, controlling the first terminal device to request, by the container, a server device to quit the virtual environment; and in response to receiving a message that the virtual environment is quit, controlling the first terminal device to release the container.

[0102] Example apparatuses and devices

[0103] Embodiments of the present disclosure also provide corresponding apparatuses for implementing the above-described methods or processes. FIG. 8 shows a schematic structural block diagram of a live apparatus 800 according to some embodiments of the present disclosure. Various modules / components in the apparatus 800 can be implemented by hardware, software, firmware, or any combination thereof.

[0104] As shown, the apparatus 800 includes a container starting module 810 configured to, in response to receiving an access request for accessing a virtual environment at a server device, control a first terminal device of a first user of a live room to start a container at the first terminal device. The apparatus 800 also includes a data obtaining module 820 configured to control the first terminal device to obtain, via the container, state data of the virtual environment. The apparatus 800 further includes a data deriving module 830 configured to derive live data based on the state data, the live data being rendered in the live room.

[0105] In some embodiments, the data obtaining module 820 includes a data receiving module configured to receive, via the container, the state data from a first service for managing the live room, the first service being started by the server device.

[0106] In some embodiments, the apparatus 800 further includes a request forwarding module configured to, in response to receiving an interaction request at the first terminal device, control the first terminal device to forward the interaction request to the server device to update the state data of the virtual environment; and an updated data receiving module configured to receive the updated state data.

[0107] In some embodiments, the request forwarding module includes a first forwarding module configured to control the first terminal device to forward, via the container, the state data to the first service, such that the forwarded state data is transmitted to the server device.

[0108] In some embodiments, the first user is a host user of the live application, and the data obtaining module 830 includes: a first data obtaining module configured to obtain first collection data from a first collection device; a host data obtaining module configured to obtain host data based on the first collection data and the state data; and a first data obtaining module configured to obtain live data based on the host data.

[0109] In some embodiments, the data generating module 830 further includes: a second data obtaining module configured to, in response to determining that the live room includes a guest user, obtain second collection data from a second collection device of the guest user; and a data updating module configured to update the live data based on the second collection data.

[0110] In some embodiments, the apparatus 800 further includes: an updated data presenting module configured to cause the updated live data to be presented at the first terminal device.

[0111] In some embodiments, the second data obtaining module is specifically configured to: obtain the second collection data via the first service.

[0112] In some embodiments, the apparatus 800 further includes: a distribution data generating module configured to transmit the updated live data to a second service of the live room to generate content distribution data, the content distribution data being presented at a terminal device of a spectator user of the live room.

[0113] In some embodiments, the apparatus 800 further includes: a third data obtaining module configured to, in response to determining that the live room includes a guest user, obtain, by the first service, second collection data from a second collection device of the guest user; a live data updating module configured to update, by the first service, the live data based on the second collection data; and an updated data transmitting module configured to transmit, by the first service, the updated live data to a second service of the live room to generate content distribution data, the content distribution data being presented at a terminal device of a spectator user of the live room.

[0114] In some embodiments, the apparatus 800 further includes: an environment exiting module configured to, in response to receiving an exit request for exiting the virtual environment, control the first terminal device to exit the virtual environment by the container request server device; and a container releasing module configured to, in response to receiving a message that the virtual environment is exited, control the first terminal device to release the container.

[0115] The units and / or modules included in device 800 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 800 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0116] It should be understood that one or more steps in the above methods can be performed by appropriate electronic devices or combinations of electronic devices. Such electronic devices or combinations of electronic devices may, for example, include server 150, terminal device 120, and / or terminal device 140 in FIG. 1.

[0117] Figure 9 shows a block diagram of an electronic device 900 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 900 shown in Figure 9 is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. The electronic device 900 shown in Figure 9 can be used to implement the server 150, terminal device 120, and / or terminal device 140 of Figure 1.

[0118] As shown in Figure 9, the electronic device 900 is in the form of a general-purpose electronic device. Components of the electronic device 900 may include, but are not limited to, one or more processors or processing units 910, a memory 920, a storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. The processing unit 910 may be a physical or virtual processor and is capable of performing various processes according to the program stored in the memory 920. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device 900.

[0119] Electronic device 900 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 900, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 920 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 930 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 900.

[0120] Electronic device 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 9, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 920 may include computer program product 925 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0121] The communication unit 940 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 900 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 900 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0122] Input device 950 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 960 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 900 can also communicate with one or more external devices (not shown) via communication unit 940 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 900, or with any device that enables electronic device 900 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0123] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0124] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0125] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0126] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0128] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for live streaming, comprising: In response to receiving an access request for accessing the virtual environment at the server device, control the first terminal device of the first user in the live broadcast room to start a container at the first terminal device; The first terminal device is controlled to obtain the status data of the virtual environment via the container; as well as Live data is obtained based on the status data and is then presented in the live streaming room.

2. The method according to claim 1, wherein obtaining the status data includes: The status data is received via the container from a first service for managing the live stream, the first service being started by the server device.

3. The method according to claim 2, further comprising: In response to receiving an interaction request at the first terminal device, the first terminal device is controlled to forward the interaction request to the server device to update the state data of the virtual environment; as well as Receive the updated status data.

4. The method of claim 3, wherein controlling the first terminal device to forward the interaction request comprises: The first terminal device is controlled to forward the status data to the first service via the container, so that the forwarded status data is transmitted to the server device.

5. The method according to claim 1, wherein the first user is a broadcaster user of the live streaming application, and obtaining the live streaming data includes: Acquire first data from the first acquisition device; Based on the first collected data and the status data, the broadcaster data is generated; as well as The live streaming data is obtained based on the streamer data.

6. The method according to claim 5, wherein obtaining the live data further comprises: In response to determining that the live broadcast room includes guest users, second acquisition data is obtained from the second acquisition device of the guest users; as well as The live streaming data is updated based on the second collected data.

7. The method of claim 6, wherein the updated live data is presented at the first terminal device.

8. The method according to claim 6, wherein acquiring the second collected data includes: The second collection data is obtained through the first service.

9. The method of claim 6, further comprising: The updated live streaming data is transmitted to a second service in the live streaming room to generate content distribution data, which is then presented on the terminal devices of the viewers in the live streaming room.

10. The method of claim 5, further comprising: In response to determining that the live broadcast room includes guest users, the first service obtains second acquisition data from the second acquisition device of the guest users; The first service updates the live stream data based on the second collected data; as well as The first service transmits updated live streaming data to the second service of the live streaming room to generate content distribution data, which is then presented on the terminal devices of the viewers in the live streaming room.

11. The method of claim 1, further comprising: In response to receiving an exit request for exiting the virtual environment, the first terminal device is controlled to request the server device to exit the virtual environment via the container; as well as In response to receiving a message that the virtual environment has been exited, the first terminal device is controlled to release the container.

12. A live streaming device, comprising: The container startup module is configured to, in response to receiving an access request for accessing a virtual environment at a server device, control the first terminal device of the first user in the live broadcast room to start a container at the first terminal device. The data acquisition module is configured to control the first terminal device to acquire the state data of the virtual environment via the container; as well as The data generation module is configured to obtain live streaming data based on the status data, and the live streaming data is presented in the live streaming room.

13. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processor.

14. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 11.

Images