A video live streaming method, device, system, apparatus and storage medium

Abstract

Embodiments of the present disclosure provide a video live streaming method, device, system, equipment and storage medium. The method comprises: in response to a virtual video live streaming operation triggered by a target user, sending a virtual video live streaming request to a server, so that the server assigns a target collection device to the target user based on the virtual video live streaming request, and returns a virtual video live streaming start instruction; in response to the virtual video live streaming start instruction, obtaining a current viewing position and a current viewing angle of the target user; sending the current viewing position and the current viewing angle to the server through a real-time communication network, so that the server synchronously adjusts a current shooting position and a current shooting angle of the target collection device, and performs real-time shooting on a three-dimensional virtual scene, and returns a current live video stream through the real-time communication network; and playing the current live video stream. Through the technical solution of the embodiments of the present disclosure, the same video live streaming effect as the virtual video is achieved without constructing the virtual video.

Description

Technical Field

[0001] This disclosure relates to Internet technology, and more particularly to a video live streaming method, apparatus, system, device, and storage medium. Background Technology

[0002] With the rapid development of internet technology, live video streaming often requires virtual video to allow users to freely change their viewing angle and move freely within a virtual scene, thus achieving an immersive experience. Currently, virtual video live streaming typically requires the real-time construction of the virtual video and its distribution to the playback device, enabling users to watch from any position and perspective. However, since virtual video is essentially a continuous sequence of 3D models, the construction process is cumbersome and complex, requiring high-performance equipment, which hinders the widespread adoption of virtual video live streaming. Summary of the Invention

[0003] This disclosure provides a video live streaming method, apparatus, system, device, and storage medium to reduce the complexity of live streaming virtual video, eliminate the need to construct virtual video, and achieve the same video live streaming effect as virtual video, making it easy to popularize and apply.

[0004] In a first aspect, embodiments of this disclosure provide a video live streaming method, including:

[0005] In response to a virtual video live streaming operation triggered by a target user, a virtual video live streaming request corresponding to the target user is sent to the server, so that the server allocates an idle target acquisition device to the target user based on the virtual video live streaming request, and returns a virtual video live streaming start instruction;

[0006] In response to the virtual video live stream start command, the current viewing position and current viewing angle of the target user are obtained;

[0007] The current viewing position and viewing angle are sent to the server via a real-time communication network, so that the server can synchronously adjust the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle, and shoot the three-dimensional virtual scene in real time based on the adjusted target acquisition device, and return the current live video stream through the real-time communication network.

[0008] Play the current live video stream captured in real time by the target acquisition device.

[0009] Secondly, embodiments of this disclosure provide a video live streaming method, including:

[0010] Receive a virtual video live streaming request corresponding to the target user, wherein the virtual video live streaming request is sent in response to a virtual video live streaming operation triggered by the target user;

[0011] Based on the virtual video live streaming request, an idle target acquisition device is assigned to the target user;

[0012] A virtual video live stream start command is sent to the video playback terminal, so that the video playback terminal can obtain the target user's current viewing position and current viewing angle based on the virtual video live stream start command, and return the current viewing position and current viewing angle through the real-time communication network;

[0013] Based on the current viewing position and viewing angle, the current shooting position and shooting angle of the target acquisition device are simultaneously adjusted, and the three-dimensional virtual scene is captured in real time based on the adjusted target acquisition device. The captured live video stream is sent to the video playback terminal through a real-time communication network so that the video playback terminal can play the live video stream captured in real time by the target acquisition device.

[0014] Thirdly, this disclosure also provides a video live streaming device, including:

[0015] The virtual video live streaming request sending module is used to respond to the virtual video live streaming operation triggered by the target user, send the virtual video live streaming request corresponding to the target user to the server, so that the server allocates an idle target acquisition device to the target user based on the virtual video live streaming request, and returns a virtual video live streaming start instruction;

[0016] The current viewing information acquisition module is used to acquire the target user's current viewing position and current viewing angle in response to the virtual video live broadcast start command;

[0017] The current viewing information sending module is used to send the current viewing position and current viewing angle to the server through a real-time communication network, so that the server can synchronously adjust the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and perform real-time shooting of the three-dimensional virtual scene based on the adjusted target acquisition device, and return the captured current live video stream through the real-time communication network;

[0018] The current live video stream playback module is used to play the current live video stream captured in real time by the target acquisition device.

[0019] Fourthly, embodiments of this disclosure also provide a video live streaming device, including:

[0020] The virtual video live streaming request receiving module is used to receive virtual video live streaming requests corresponding to the target user. The virtual video live streaming request is sent in response to the virtual video live streaming operation triggered by the target user.

[0021] The acquisition device allocation module is used to allocate idle target acquisition devices to the target user based on the virtual video live streaming request.

[0022] The live streaming start command sending module is used to send a virtual video live streaming start command to the video playback terminal, so that the video playback terminal can obtain the current viewing position and current viewing angle of the target user based on the virtual video live streaming start command, and return the current viewing position and current viewing angle through the real-time communication network;

[0023] The current live video stream sending module is used to synchronously adjust the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and to shoot the three-dimensional virtual scene in real time based on the adjusted target acquisition device. The captured current live video stream is then sent to the video playback terminal through a real-time communication network so that the video playback terminal can play the current live video stream captured in real time by the target acquisition device.

[0024] Fifthly, this disclosure also provides a video live streaming system, the system including a video playback terminal and a server;

[0025] The video playback terminal is used to implement the live video streaming method provided in the first aspect;

[0026] The server is used to implement the live video streaming method provided in the second aspect.

[0027] Sixthly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0028] One or more processors;

[0029] Storage device for storing one or more programs.

[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement the video live streaming method as described in any of the embodiments of this disclosure.

[0031] In a seventh aspect, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the video live streaming method as described in any of the embodiments of this disclosure.

[0032] In this embodiment, the video playback terminal responds to a virtual video live streaming operation triggered by a target user by sending a virtual video live streaming request corresponding to the target user to the server. Based on the virtual video live streaming request, the server allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command. In response to the virtual video live streaming start command, the video playback terminal acquires the target user's current viewing position and viewing angle in real time and quickly sends these information to the server via a real-time communication network, thereby ensuring the real-time transmission of user viewing information. The server synchronously adjusts the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle of each transmission. Based on the adjusted target acquisition device, it performs real-time shooting of the 3D virtual scene. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time, and return the captured live video stream through a real-time communication network. This allows the target user to quickly see the live video stream from their current viewing position and viewing angle, avoiding live streaming delays. Therefore, the same video live streaming effect as virtual video can be achieved simply by synchronously adjusting the target acquisition device allocated to the target user, without the need to construct virtual video, reducing the complexity of virtual video live streaming, and facilitating widespread application. Attached Figure Description

[0033] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0034] Figure 1 This is a schematic flowchart of a video live streaming method provided in an embodiment of this disclosure;

[0035] Figure 2 This is a schematic flowchart of a video live streaming method provided in an embodiment of this disclosure;

[0036] Figure 3 This is a schematic diagram of the structure of a video live streaming device provided in an embodiment of this disclosure;

[0037] Figure 4 This is a schematic diagram of the structure of a video live streaming device provided in an embodiment of this disclosure;

[0038] Figure 5 This is a schematic diagram of the structure of a video live streaming system provided in an embodiment of this disclosure;

[0039] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0040] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0041] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0042] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0043] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0044] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0045] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0046] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0047] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0048] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0049] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0050] Figure 1 This is a flowchart illustrating a video live streaming method provided in an embodiment of this disclosure. This embodiment is applicable to situations where video live streaming is performed in a virtual video format, allowing users to watch from any position and perspective. The method can be executed by a video live streaming device, which can be implemented in software and / or hardware and integrated into a video playback terminal. The video playback terminal can be any terminal device used to play virtual videos. For example, the video playback terminal can be a wearable device, such as a VR (Virtual Reality) headset or VR glasses. The video playback terminal can also be a mobile terminal or a PC, such as a smartphone or desktop computer.

[0051] like Figure 1 As shown, the video live streaming method specifically includes the following steps:

[0052] S110. In response to the virtual video live streaming operation triggered by the target user, a virtual video live streaming request corresponding to the target user is sent to the server, so that the server allocates an idle target acquisition device to the target user based on the virtual video live streaming request, and returns a virtual video live streaming start instruction.

[0053] In this context, "target user" refers to any user who enters the live stream room to watch the live video broadcast, or any user currently watching the live video broadcast in the live stream room. Target users can be logged-in users on the video playback platform. A live stream room can be a virtual room used to play virtual videos. Different live stream rooms can play different virtual videos. Users can choose to enter different live stream rooms to watch virtual videos based on their needs. Virtual video, also known as volumetric video, is essentially a format that captures information in three-dimensional space (such as depth and color information) and generates a sequence of three-dimensional models. These models, when connected, form a new video format that can be viewed from any perspective. Virtual video allows users to adjust to any position and perspective for viewing, thus experiencing six degrees of freedom: yaw, pitch, roll, forward / backward, up / down, and left / right free movement. The server can be a device used to produce and transmit the live video stream. For example, the server can be a single server or a service cluster consisting of several servers, or a virtualization platform or cloud server. The target acquisition device can refer to the acquisition device assigned to the target user. The acquisition device can be used to capture two-dimensional video from a specified position and perspective. For example, the acquisition device can be a 2D camera. The idle state refers to the state where the acquisition device has not yet been assigned. It should be noted that the target acquisition device can be considered as the eyes of the target user, used to capture the scene content that the target user wants to see in the 3D virtual scene in real time.

[0054] Specifically, if a target user wants to watch live video from different positions and perspectives, they can trigger a virtual live video operation on the video playback device. This can be done by clicking a button to experience the virtual live video effect, or by selecting a virtual live video mode if the video playback device supports at least two live video modes. In response to the target user's virtual live video operation, the video playback device generates a corresponding virtual live video request and sends it to the server. Upon receiving the request, the server can allocate an idle target acquisition device to the target user, allowing them to experience the virtual live video effect. If the target user is successfully allocated an idle target acquisition device, it indicates that the user is allowed to experience the virtual live video effect, and a virtual live video start command can be sent to the target user's video playback device. If all acquisition devices are currently occupied and it is impossible to allocate an idle acquisition device to the target user, a virtual video live stream pause command can be sent to the target user's video playback terminal to inform the target user that the virtual video live stream effect cannot be experienced at present, and that they can wait for a period of time to retry or watch in another video live stream mode.

[0055] S120, In response to the virtual video live streaming start command, obtain the target user's current viewing position and current viewing angle.

[0056] The current viewing position refers to the target user's current viewing position in the 3D virtual scene. The current viewing position can be changed by moving forward / backward, up / down, or left / right, allowing the user to view scene content from different positions within the three degrees of freedom (forward / backward, left / right, and up / down). For example, a preset viewing position in the 3D virtual scene can be set as the initial value for the current viewing position. The target user can then change their current viewing position by moving forward / backward, up / down, or left / right from the preset position. The current viewing angle refers to the target user's current viewing angle in the 3D virtual scene. The current viewing angle can be changed by yaw, pitch, or roll, allowing the user to view scene content from different perspectives within the three degrees of freedom (yaw, pitch, and roll). For example, a preset viewing angle in the 3D virtual scene can be set as the initial value for the current viewing angle. The target user can then change their current viewing angle by yaw, pitch, and roll from the preset viewing angle.

[0057] Specifically, after receiving the virtual video live stream start command, the video playback client indicates that the server is ready for the virtual video live stream, and at this time, it can obtain the target user's current viewing position and current viewing angle in real time.

[0058] For example, the target user's current viewing position and current viewing angle can be obtained in at least two of the following ways:

[0059] As one implementation method, S120 may include: determining the current viewing position and current viewing angle of the target user based on the current pose information collected by the sensors in the video playback terminal.

[0060] Specifically, if the pose of the video playback device can change synchronously with the pose of the target user—for example, if the video playback device is a wearable or handheld device—the target user can change their viewing position and viewing angle by altering the pose of the video playback device. The video playback device can collect its current pose information using built-in sensors such as gyroscopes, and determine the target user's current viewing position and viewing angle based on this information. For example, the initial viewing pose corresponding to the initial viewing position and initial viewing angle can be determined, along with the change in current position and current viewing angle between the current pose and the initial viewing pose. Based on the initial viewing position and the change in current position, the current viewing position can be determined; based on the initial viewing angle and the change in current time, the current viewing angle can be determined. Thus, the sensors in the video playback device enable real-time acquisition and processing of the current viewing position and current viewing angle. The initial viewing position refers to the pre-set viewing position upon first entering the 3D virtual scene. The initial viewing angle refers to the pre-set viewing angle upon first entering the 3D virtual scene.

[0061] As another implementation, S120 may include: determining the target user's current viewing position and current viewing angle based on the control button trigger information in the video playback terminal.

[0062] Specifically, if the pose of the video playback device cannot change synchronously with the pose of the target user, such as when the video playback device is a desktop computer, control buttons for adjusting position and viewing angle information can be set on the video playback device, for example, using the keys on the keyboard connected to the desktop computer. The target user can adjust the current viewing position and viewing angle by triggering the control buttons. For example, the change in current position and the change in current viewing angle can be determined based on the control button trigger information on the video playback device, and the current viewing position can be determined based on the initial viewing position and the change in current position, and the current viewing angle can be determined based on the initial viewing angle and the change in current time. Thus, the current viewing position and the current viewing angle can be adjusted in real time using the control buttons on the video playback device.

[0063] S130. The current viewing position and current viewing angle are sent to the server through the real-time communication network, so that the server can synchronously adjust the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and shoot the three-dimensional virtual scene in real time based on the adjusted target acquisition device, and return the current live video stream through the real-time communication network.

[0064] In this context, "real-time communication network" refers to a network where the video playback end and the server communicate via the Real-Time Communication (RTC) protocol. "3D virtual scene" refers to the VR scene that the live streaming room needs to broadcast, which is pre-built on the server. A 3D virtual scene can be a 3D virtual model of a real-world location. A real-world location refers to a physically existing entity, such as a tourist attraction. A 3D virtual scene can be viewed as a 3D virtual model of a real-world location scaled down to a certain proportion. The current live video stream can refer to a 2D video stream captured from the current viewing position and perspective.

[0065] Specifically, the video playback terminal can send the obtained current viewing position and viewing angle to the server in real time through the command transmission channel in the real-time communication network. The server synchronously adjusts the current shooting position of the target acquisition device to the current viewing position of the target user, and synchronously adjusts the current shooting angle of the target acquisition device to the current viewing angle of the target user, thereby placing the target acquisition device in the target user's current position and viewing angle in the 3D virtual scene, and then using the target acquisition device to capture the current live video stream seen by the target user from the current viewing position and viewing angle. The captured current live video stream can be quickly sent to the video playback terminal through the audio and video transmission channel in the real-time communication network to meet the requirements of low latency in live streaming.

[0066] It should be noted that if the target user's current viewing position and viewing angle change, the target acquisition device's current shooting position and viewing angle will also change synchronously in real time. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time.

[0067] For example, a 3D virtual scene can be constructed based on a 3D virtual model of a real-world location and the configuration information of at least one model asset. The model assets can include, but are not limited to, at least one of: video, audio, images, text, and 3D asset models. Video, audio, images, and text can all be considered 2D asset models. The 3D asset model can be any type of 3D model that can be added to the 3D virtual model. For example, the 3D asset model can be a 3D building model, a 3D animation model, or a 3D virtual character model, etc. The asset configuration information can be pre-configured based on business requirements, including the display position and size information of the model assets within the 3D virtual model.

[0068] Specifically, before live video streaming, a 3D virtual model can be constructed by scaling and rendering the real-world location according to a certain proportion. Based on the material configuration information of each model's materials, these materials are added to the 3D virtual model, adhering to basic physical collision principles. This results in a richer 3D virtual scene, enabling configurability and meeting the personalized needs of live streaming. For example, if the 3D virtual model is a model of a tourist attraction, and introductory videos are added to the attractions within this model, a virtual scene of the tourist attraction containing these videos is created. This allows users to watch the introductory videos upon arriving at the attraction, thus enhancing the richness of the live stream content.

[0069] S140: Play the current live video stream captured in real time by the target acquisition device.

[0070] Specifically, the current live video stream is captured by a target acquisition device, showing the scene as seen by the target user from their current viewing position and perspective. The video playback terminal then plays this live video stream, allowing the target user to view the video from any position and angle. The entire process achieves the same live video streaming effect as virtual video without requiring the construction of virtual video, thus realizing a simple and efficient virtual video live streaming method that is easy to widely adopt.

[0071] The technical solution of this disclosure embodiment involves the video playback terminal responding to a virtual video live streaming operation triggered by a target user by sending a virtual video live streaming request corresponding to the target user to the server. Based on the virtual video live streaming request, the server allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command. In response to the virtual video live streaming start command, the video playback terminal acquires the target user's current viewing position and current viewing angle in real time and quickly sends these to the server via a real-time communication network, thereby ensuring the real-time transmission of user viewing information. The server synchronously adjusts the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle of each transmission. Based on the adjusted target acquisition device, it performs real-time shooting of the 3D virtual scene. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time, and return the captured live video stream through a real-time communication network. This allows the target user to quickly see the live video stream from their current viewing position and viewing angle, avoiding live streaming delays. Therefore, the same video live streaming effect as virtual video can be achieved simply by synchronously adjusting the target acquisition device allocated to the target user, without the need to construct virtual video, reducing the complexity of virtual video live streaming, and facilitating widespread application.

[0072] Based on the above technical solution, the method may further include: responding to a live sharing operation triggered by a target user, sending a live sharing request corresponding to the target user to the server, so that the server determines the sharing user corresponding to the target user based on the live sharing request, and sending the current live video stream captured in real time by the target acquisition device to the video playback terminal of the sharing user through a content delivery network.

[0073] Live streaming sharing requests refer to requests to send the current live video stream viewed by a target user to other users for viewing, enabling the target user to lead more users to watch the live stream. A Content Delivery Network (CDN) is an internet transmission protocol technology that accelerates user access to website or application content by storing data across multiple service nodes. A CDN can be a network that distributes content using CDN protocols. CDN protocols can include RTMP (Real Time Message Protocol), HTTP-FLV, or SRT (Secure Reliable Transport). The HTTP-FLV protocol is a streaming media transmission method that encapsulates audio and video data in FLV (Flash Video) file format, and then encapsulates the FLV format data in HTTP (Hypertext Transfer Protocol) for transmission.

[0074] Specifically, if a target user wants to share their virtual live video experience with other users, they can trigger a live sharing operation on the video playback client, such as by clicking the share button in the live stream. In response to the triggered live sharing operation, the video playback client generates a live sharing request corresponding to the target user and sends it to the server. Upon receiving the live sharing request, the server can determine the sharing user corresponding to the target user and send the current live video stream being watched by the target user to the sharing user's video playback client, allowing the sharing users to watch the live content watched by the target user on their respective video playback clients. There are two ways to determine the sharing user. The first is that the target user can specify the sharing user; for example, the target user can input sharing user information in the video playback client and generate a live sharing request based on that information. The server can then determine the sharing user the target user wants to share the video with based on the sharing user information in the received live sharing request. The second method is for users to actively apply to become the target user's sharing user. For example, after the server receives the live stream sharing request, it waits for other users to send sharing application information. The user who receives the sharing application information is designated as the target user's sharing user, so that the current live video stream being watched by the target user can be sent to the user's video playback terminal, allowing the user to see the live content being watched by the target user in real time.

[0075] It should be noted that shared users cannot control or change the viewing position or perspective of the video; they are only controlled by the target user. Shared users experience the live streaming effect of the virtual video by following the target user.

[0076] Based on the above technical solution, the method may further include: responding to a switching operation from virtual video live streaming mode to panoramic video live streaming mode triggered by the target user, sending a panoramic video live streaming request to the server, so that the server controls the panoramic acquisition device to shoot the three-dimensional virtual scene in real time based on the panoramic video live streaming request, and returns the current full video stream through the content delivery network; determining the current perspective video stream based on the received current panoramic video stream and the target user's current viewing angle, and playing the current perspective video stream.

[0077] Panoramic video can be a type of video captured from all 360 degrees using panoramic acquisition equipment. Panoramic video allows users to view it from any angle. The viewing position in panoramic video is fixed, thus only supporting changes in the viewing angle; that is, users can only experience changes in three degrees of freedom: yaw, pitch, and roll. Virtual video live streaming mode can refer to a mode that supports users watching the live stream from different positions and angles. Panoramic video live streaming mode can refer to a mode that only supports users watching the live stream from different angles. The shooting position of the panoramic acquisition equipment can be pre-set based on business needs. There can be one or more panoramic acquisition devices. Different panoramic acquisition devices correspond to different shooting positions to obtain panoramic video from different locations in the 3D virtual scene. For example, a panoramic acquisition device can refer to a panoramic camera.

[0078] Specifically, the video playback client can include both virtual and panoramic live streaming modes, allowing users to switch between them based on their needs. When a target user switches from virtual to panoramic live streaming (e.g., currently streaming in virtual video mode), they can trigger the switch by selecting panoramic live streaming mode. In response, the video playback client sends a panoramic live streaming request to the server. Upon receiving the request, the server controls a panoramic capture device to capture the 3D virtual scene in real-time at a preset shooting position, obtaining the full video stream. Since the panoramic video stream includes video information from all 360-degree perspectives, and all user-sent panoramic video streams are identical, the full video stream can be quickly distributed to the target user's video playback client via a content delivery network. The video playback client determines the target user's current viewing angle and identifies the current perspective video stream (i.e., a single perspective video stream) from the received panoramic video stream, then plays the current perspective video stream. This allows users to watch videos from any perspective, achieving a panoramic live streaming effect and providing users with a selectable live streaming experience within the same live streaming room.

[0079] Figure 2 This is a flowchart illustrating a video live streaming method provided in an embodiment of this disclosure. This embodiment is applicable to situations where video live streaming is performed in a virtual video format, allowing users to watch from any position and perspective. The method can be executed by a video live streaming device, which can be implemented in software and / or hardware and integrated into a server. The server can be a device used to produce and transmit live video streams. For example, the server can be a single server or a service cluster consisting of several servers, or it can be a virtualization platform or a cloud server, etc.

[0080] like Figure 2As shown, the video live streaming method specifically includes the following steps:

[0081] S210. Receive the virtual video live streaming request corresponding to the target user. The virtual video live streaming request is sent in response to the virtual video live streaming operation triggered by the target user.

[0082] The target user can refer to any user who enters the live stream room to watch the live video broadcast, or any user who is currently watching the live video broadcast in the live stream room. Virtual video, also known as volumetric video, allows users to adjust to any position and perspective for viewing, thus experiencing six degrees of freedom: yaw, pitch, roll, forward / backward, up / down, and left / right free movement.

[0083] Specifically, if a target user wants to watch a live video from different positions and perspectives, they can trigger a virtual live video operation on the video playback device. In response to this operation, the video playback device generates a virtual live video request corresponding to the target user and sends it to the server, allowing the server to receive the request.

[0084] S220: Based on the virtual video live streaming request, allocate the target acquisition device that is in an idle state to the target user.

[0085] The target acquisition device can refer to the acquisition device assigned to the target user. The acquisition device can be used to capture 2D video from a specified location and viewpoint. For example, the acquisition device can refer to a 2D camera. The idle state refers to the state where the acquisition device has not yet been assigned.

[0086] Specifically, after receiving a virtual video live stream request, the server can allocate a currently idle target acquisition device to the target user, allowing the user to experience the live virtual video effect. It should be noted that the target acquisition device can be considered the target user's eyes, used to capture in real-time the scene content the user wants to see in the 3D virtual environment.

[0087] For example, S220 may include: detecting whether the target user meets the virtual video live streaming conditions based on the target user information in the virtual video live streaming request; if the target user meets the virtual video live streaming conditions, determining the target acquisition device that is idle from a preset number of acquisition devices, and establishing an allocation relationship between the target user and the target acquisition device.

[0088] The virtual video live streaming conditions can be pre-set, representing the conditions that users must meet to experience the virtual video live streaming effect. For example, a virtual video live streaming condition could be: users allowed to experience the virtual video live streaming effect must be members. The preset quantity refers to the total number of acquisition devices that can be placed in the 3D virtual scene, which is the number of users who can experience the virtual video live streaming effect.

[0089] Specifically, based on the target user information in the received virtual video live streaming request, the server checks whether the target user meets the conditions for virtual video live streaming. If so, it indicates that the target user has the right to experience virtual video. At this time, based on the current state of each acquisition device, the server determines the target acquisition device that is currently idle from a preset number of acquisition devices. If only one acquisition device is idle at that time, that acquisition device is determined as the target acquisition device. If at least two acquisition devices are idle, one acquisition device can be randomly selected as the target acquisition device. By establishing an allocation relationship between the target user and the target acquisition device, the server can utilize the target acquisition device to provide the target user with a virtual video viewing experience and update the current state of the target acquisition device to an occupied state to avoid allocating the target acquisition device to other users.

[0090] S230. Send the virtual video live streaming start command to the video playback terminal so that the video playback terminal can obtain the target user's current viewing position and current viewing angle based on the virtual video live streaming start command, and return the current viewing position and current viewing angle through the real-time communication network.

[0091] In this context, the real-time communication network refers to the network through which the video playback terminal and the server communicate using the Real-Time Communication (RTC) protocol. The 3D virtual scene refers to the VR scene that the live streaming room needs to broadcast, which is pre-built on the server. The construction process of the 3D virtual scene can be referred to the relevant descriptions in the above embodiments, and will not be repeated here.

[0092] Specifically, if a target user is successfully assigned to an idle target acquisition device, it indicates that the target user is allowed to experience the virtual video live streaming effect. At this time, the server can send a virtual video live streaming start command to the target user's video playback terminal. After receiving the virtual video live streaming start command, the video playback terminal indicates that the server is ready for virtual video live streaming. At this time, it can obtain the target user's current viewing position and current viewing angle in real time, and send the obtained current viewing position and current viewing angle to the server in real time through the command transmission channel in the real-time communication network.

[0093] If all acquisition devices are currently occupied and an idle acquisition device cannot be allocated to the target user, the server can send a virtual video live stream pause command to the target user's video playback terminal to inform the target user that the virtual video live stream effect cannot be experienced at present, and that they can wait for a period of time to retry or watch in another video live stream mode.

[0094] S240. Based on the current viewing position and viewing angle, synchronously adjust the current shooting position and shooting angle of the target acquisition device, and shoot the three-dimensional virtual scene in real time based on the adjusted target acquisition device. Send the current live video stream to the video playback terminal through the real-time communication network so that the video playback terminal can play the current live video stream shot in real time by the target acquisition device.

[0095] Specifically, the server synchronously adjusts the target acquisition device's current shooting position to match the target user's current viewing position and viewing angle. This places the target acquisition device within the 3D virtual scene, placing it at the target user's current position and perspective. The server then captures the live video stream as seen from the user's current viewing position and angle. This captured live video stream can be quickly transmitted to the video playback terminal via the audio / video transmission channel of the real-time communication network, allowing for real-time playback and meeting the low-latency requirements of live streaming. The entire process achieves the same live video streaming effect as virtual video without requiring the construction of a virtual video, thus realizing a simple and efficient virtual video live streaming method that is easy to widely adopt.

[0096] For example, if a target user needs to end the virtual video live stream, they can trigger a virtual video live stream end operation. In response to the virtual video live stream end operation triggered by the target user, the video playback terminal stops playing the current two-dimensional video stream captured by the target acquisition device and sends a virtual video live stream end request corresponding to the target user to the server. Based on the virtual video live stream end request, the server releases the allocation relationship between the target user and the target acquisition device and changes the current state of the target acquisition device from occupied to idle, so that the target acquisition device can be allocated to other users, thereby realizing the reuse of the acquisition device.

[0097] The technical solution of this disclosure embodiment involves the video playback terminal responding to a virtual video live streaming operation triggered by a target user by sending a virtual video live streaming request corresponding to the target user to the server. Based on the virtual video live streaming request, the server allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command. In response to the virtual video live streaming start command, the video playback terminal acquires the target user's current viewing position and current viewing angle in real time and quickly sends these to the server via a real-time communication network, thereby ensuring the real-time transmission of user viewing information. The server synchronously adjusts the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle of each transmission. Based on the adjusted target acquisition device, it performs real-time shooting of the 3D virtual scene. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time, and return the captured live video stream through a real-time communication network. This allows the target user to quickly see the live video stream from their current viewing position and viewing angle, avoiding live streaming delays. Therefore, the same video live streaming effect as virtual video can be achieved simply by synchronously adjusting the target acquisition device allocated to the target user, without the need to construct virtual video, reducing the complexity of virtual video live streaming, and facilitating widespread application.

[0098] Based on the above technical solution, S210 may include: receiving a virtual video live streaming request corresponding to the target user, wherein the virtual video live streaming request is sent in response to a switching operation triggered by the target user from panoramic video live streaming mode to virtual video live streaming mode; adjusting the current shooting position and current shooting angle of the target acquisition device for the first time based on the shooting position of the panoramic acquisition device and the current viewing angle of the target user in the virtual video live streaming request, and shooting the three-dimensional virtual scene based on the adjusted target acquisition device to obtain a switched video stream; and sending the switched video stream to the video playback terminal through a real-time communication network so that the video playback terminal plays the switched video stream.

[0099] Here, "switching video streams" refers to the video stream played when switching from panoramic video live streaming mode to virtual video live streaming mode. Virtual video live streaming mode can refer to a mode that allows users to watch the live stream from different positions and perspectives. Panoramic video live streaming mode can also refer to a mode that allows users to watch the live stream from different perspectives.

[0100] Specifically, the video playback client can include both virtual video live streaming and panoramic video live streaming modes, allowing users to switch between different modes based on their needs. When a target user switches from panoramic video live streaming to virtual video live streaming, for example, if they are currently live streaming in panoramic mode, they can trigger the switch by selecting virtual video live streaming mode. In response to this switch, the video playback client generates a virtual video live streaming request based on the target user's current viewing angle and sends it to the server. The server obtains the target user's current viewing angle from the received request and adjusts the target acquisition device's current shooting position to that of the panoramic acquisition device, as well as its current shooting angle to match the target user's current viewing angle. Based on this adjustment, the server captures a 3D virtual scene, obtaining a switched video stream. This switched video stream is then transmitted to the target user's video playback client via the audio and video transmission channel in the real-time communication network. The switched video stream is then played on the video playback client, enabling a seamless transition between panoramic and virtual video live streaming modes, ensuring the continuity of the live streaming content and further enhancing the user experience.

[0101] It should be noted that when the virtual video live streaming operation is triggered by switching methods, the server also needs to synchronously execute the operations of steps S220-S240 based on the received virtual video live streaming request, so that after switching, the current shooting position and current shooting angle of the target acquisition device can be synchronously adjusted based on the current viewing position and current viewing angle of the target user, thereby realizing the live streaming effect of virtual video.

[0102] Based on the above technical solution, the method may further include: obtaining the virtual character model corresponding to the user currently watching the virtual video live stream; and adding the virtual character model corresponding to the user to the corresponding three-dimensional virtual scene based on the user's current viewing position and current viewing perspective, so as to update the three-dimensional virtual scene.

[0103] In this context, "viewer" refers to a user who has entered the same live stream room as the target user and is currently experiencing the virtual video live stream effect. There can be one or more viewers. Each viewer corresponds to a virtual character model. The virtual character model can be a 3D virtual model representing the viewer. Different viewers can correspond to the same virtual character model or different virtual character models, allowing viewers to choose from multiple virtual character models.

[0104] Specifically, for each new viewer joining the live stream in virtual video mode, a virtual character model corresponding to that viewer is added to the corresponding 3D virtual scene at their current viewing position. The virtual character model's viewing angle is the viewer's current perspective. This synchronously adds the virtual character models of all viewers entering the same live stream to the corresponding 3D virtual scene. This allows any viewer's capture device to photograph the updated 3D virtual scene, capturing images of other viewers' virtual character models and thus determining the number and location of other viewers. All viewers in the same live stream can meet and interact within the 3D virtual scene, following basic physical collision principles, further enhancing the live streaming effect of the virtual video.

[0105] Figure 3 This is a schematic diagram of the structure of a video live streaming device provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the device specifically includes: a virtual video live streaming request sending module 310, a current viewing information acquisition module 320, a current viewing information sending module 330, and a current live video stream playback module 340.

[0106] The virtual video live streaming request sending module 310 is used to send a virtual video live streaming request corresponding to the target user to the server in response to the virtual video live streaming operation triggered by the target user, so that the server allocates an idle target acquisition device to the target user based on the virtual video live streaming request and returns a virtual video live streaming start command; the current viewing information acquisition module 320 is used to acquire the current viewing position and current viewing angle of the target user in response to the virtual video live streaming start command; the current viewing information sending module 330 is used to send the current viewing position and current viewing angle to the server through a real-time communication network, so that the server synchronously adjusts the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and performs real-time shooting of the three-dimensional virtual scene based on the adjusted target acquisition device, and returns the captured current live video stream through the real-time communication network; the current live video stream playback module 340 is used to play the current live video stream captured in real time by the target acquisition device.

[0107] The technical solution provided in this disclosure involves the video playback terminal responding to a virtual video live streaming operation triggered by a target user by sending a virtual video live streaming request corresponding to the target user to the server. Based on the virtual video live streaming request, the server allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command. In response to the virtual video live streaming start command, the video playback terminal acquires the target user's current viewing position and viewing angle in real time and quickly sends these information to the server via a real-time communication network, thereby ensuring the real-time transmission of user viewing information. The server synchronously adjusts the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle of each transmission. Based on the adjusted target acquisition device, it performs real-time shooting of the 3D virtual scene. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time, and return the captured live video stream through a real-time communication network. This allows the target user to quickly see the live video stream from their current viewing position and viewing angle, avoiding live streaming delays. Therefore, the same video live streaming effect as virtual video can be achieved simply by synchronously adjusting the target acquisition device allocated to the target user, without the need to construct virtual video, reducing the complexity of virtual video live streaming, and facilitating widespread application.

[0108] Based on the above technical solution, the device is integrated into the video playback terminal. The current viewing information acquisition module 320 is specifically used to: determine the current viewing position and current viewing angle of the target user based on the current pose information collected by the sensors in the video playback terminal; or, determine the current viewing position and current viewing angle of the target user based on the control button trigger information in the video playback terminal.

[0109] Based on the above technical solutions, the three-dimensional virtual scene is constructed based on a three-dimensional virtual model of the real scene and the material configuration information of at least one model material;

[0110] The model materials include at least one of the following: video, audio, images, text, and 3D material models.

[0111] Based on the above technical solutions, the device also includes:

[0112] The live streaming sharing request sending module is used to respond to the live streaming sharing operation triggered by the target user, send a live streaming sharing request corresponding to the target user to the server, so that the server determines the sharing user corresponding to the target user based on the live streaming sharing request, and sends the current live video stream captured in real time by the target acquisition device to the video playback terminal of the sharing user through the content delivery network.

[0113] Based on the above technical solutions, the device also includes:

[0114] The panoramic video live streaming request sending module is used to send a panoramic video live streaming request to the server in response to the switching operation from virtual video live streaming mode to panoramic video live streaming mode triggered by the target user. This allows the server to control the panoramic acquisition device to capture the three-dimensional virtual scene in real time based on the panoramic video live streaming request, and return the captured full video stream through the content delivery network.

[0115] The current viewpoint video stream playback module is used to determine the current viewpoint video stream based on the received current panoramic video stream and the target user's current viewing angle, and to play the current viewpoint video stream.

[0116] The video live streaming device provided in this disclosure can execute the video live streaming method applied to the video playback terminal provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the video live streaming method.

[0117] Figure 4 This is a schematic diagram of the structure of a video live streaming device provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, the device specifically includes: a virtual video live streaming request receiving module 410, a data acquisition device allocation module 420, a live streaming start command sending module 430, and a current live streaming video stream sending module 440.

[0118] The system includes a virtual video live streaming request receiving module 410, which receives a virtual video live streaming request corresponding to a target user. The virtual video live streaming request is sent in response to a virtual video live streaming operation triggered by the target user. A data acquisition device allocation module 420 is used to allocate an idle target data acquisition device to the target user based on the virtual video live streaming request. A live streaming start command sending module 430 is used to send a virtual video live streaming start command to a video playback terminal, so that the video playback terminal can obtain the target user's current viewing position and current viewing angle based on the virtual video live streaming start command, and return the current viewing position and current viewing angle through a real-time communication network. A current live video stream sending module 440 is used to synchronously adjust the current shooting position and current shooting angle of the target data acquisition device based on the current viewing position and current viewing angle, and perform real-time shooting of the three-dimensional virtual scene based on the adjusted target data acquisition device, and send the captured current live video stream to the video playback terminal through a real-time communication network, so that the video playback terminal can play the current live video stream captured in real-time by the target data acquisition device.

[0119] The technical solution provided in this disclosure involves the video playback terminal responding to a virtual video live streaming operation triggered by a target user by sending a virtual video live streaming request corresponding to the target user to the server. Based on the virtual video live streaming request, the server allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command. In response to the virtual video live streaming start command, the video playback terminal acquires the target user's current viewing position and viewing angle in real time and quickly sends these information to the server via a real-time communication network, thereby ensuring the real-time transmission of user viewing information. The server synchronously adjusts the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle of each transmission. Based on the adjusted target acquisition device, it performs real-time shooting of the 3D virtual scene. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time, and return the captured live video stream through a real-time communication network. This allows the target user to quickly see the live video stream from their current viewing position and viewing angle, avoiding live streaming delays. Therefore, the same video live streaming effect as virtual video can be achieved simply by synchronously adjusting the target acquisition device allocated to the target user, without the need to construct virtual video, reducing the complexity of virtual video live streaming, and facilitating widespread application.

[0120] Based on the above technical solution, the data acquisition device allocation module 420 is specifically used for:

[0121] Based on the target user information in the virtual video live streaming request, it is detected whether the target user meets the virtual video live streaming conditions; if the target user meets the virtual video live streaming conditions, an idle target acquisition device is determined from a preset number of acquisition devices, and an allocation relationship is established between the target user and the target acquisition device.

[0122] Based on the above technical solutions, the virtual video live streaming request receiving module 410 is specifically used to: receive a virtual video live streaming request corresponding to a target user, wherein the virtual video live streaming request is sent in response to a switching operation from panoramic video live streaming mode to virtual video live streaming mode triggered by the target user.

[0123] The device also includes:

[0124] The video stream switching acquisition module is used to adjust the current shooting position and current shooting angle of the target acquisition device based on the shooting position of the panoramic acquisition device and the current viewing angle of the target user in the virtual video live broadcast request, and to shoot the three-dimensional virtual scene based on the adjusted target acquisition device to obtain the switched video stream;

[0125] The video stream switching sending module is used to send the switched video stream to the video playback terminal via a real-time communication network, so that the video playback terminal plays the switched video stream.

[0126] Based on the above technical solutions, the device also includes:

[0127] The 3D virtual scene update module is used to obtain the virtual character model corresponding to the currently watching virtual video live stream user; based on the current viewing position and current viewing angle of the user, the virtual character model corresponding to the user is added to the 3D virtual scene to update the 3D virtual scene.

[0128] The video live streaming device provided in this disclosure can execute the video live streaming method applied to the server provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the video live streaming method.

[0129] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0130] Figure 5 This is a schematic diagram of the structure of a video live streaming system provided in an embodiment of this disclosure. This embodiment is applicable to situations where video live streaming is conducted in a virtual video manner, allowing users to watch from any position and perspective. Figure 5 As shown, the system specifically includes: a video playback terminal 510 and a server terminal 520.

[0131] The video playback terminal 510 is used to implement the video stuttering prediction method provided in the above-described embodiment applied to the video playback terminal; the server terminal 520 is used to implement the video stuttering prediction method provided in the above-described embodiment applied to the server terminal.

[0132] The video live streaming system in this embodiment responds to a virtual video live streaming operation triggered by a target user by sending a virtual video live streaming request corresponding to the target user to the server. Based on the virtual video live streaming request, the server allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command. In response to the virtual video live streaming start command, the video playback terminal acquires the target user's current viewing position and viewing angle in real time and quickly sends these information to the server via a real-time communication network, thereby ensuring the real-time transmission of user viewing information. The server synchronously adjusts the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle of each transmission. Based on the adjusted target acquisition device, it performs real-time shooting of the 3D virtual scene. Thus, the target acquisition device can capture the current live video stream from the target user's current viewing position and viewing angle in real time, and return the captured live video stream through a real-time communication network. This allows the target user to quickly see the live video stream from their current viewing position and viewing angle, avoiding live streaming delays. Therefore, the same video live streaming effect as virtual video can be achieved simply by synchronously adjusting the target acquisition device allocated to the target user, without the need to construct virtual video, reducing the complexity of virtual video live streaming, and facilitating widespread application.

[0133] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 6 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0134] like Figure 6As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0135] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0136] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0137] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0138] The electronic device provided in this embodiment and the video live streaming method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0139] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the live video streaming method provided in the above embodiments.

[0140] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0141] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0142] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0143] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: respond to a virtual video live streaming operation triggered by a target user, send a virtual video live streaming request corresponding to the target user to a server, so that the server, based on the virtual video live streaming request, allocates an idle target acquisition device to the target user and returns a virtual video live streaming start command; respond to the virtual video live streaming start command, obtain the target user's current viewing position and current viewing angle; send the current viewing position and current viewing angle to the server via a real-time communication network, so that the server, based on the current viewing position and current viewing angle, synchronously adjusts the current shooting position and current shooting angle of the target acquisition device, and performs real-time shooting of the three-dimensional virtual scene based on the adjusted target acquisition device, and returns the captured current live video stream via a real-time communication network; and play the current live video stream captured in real time by the target acquisition device.

[0144] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a virtual video live streaming request corresponding to a target user, the virtual video live streaming request being sent in response to a virtual video live streaming operation triggered by the target user; allocate an idle target acquisition device to the target user based on the virtual video live streaming request; send a virtual video live streaming start command to a video playback terminal, so that the video playback terminal obtains the target user's current viewing position and current viewing angle based on the virtual video live streaming start command, and returns the current viewing position and current viewing angle through a real-time communication network; synchronously adjust the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and perform real-time shooting of a three-dimensional virtual scene based on the adjusted target acquisition device, and send the captured current live video stream to the video playback terminal through a real-time communication network, so that the video playback terminal plays the current live video stream captured in real-time by the target acquisition device.

[0145] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0147] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0148] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0149] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0150] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0151] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0152] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A video live streaming method, characterized in that, include: In response to a virtual video live streaming operation triggered by a target user, a virtual video live streaming request corresponding to the target user is sent to the server. If the server detects that the target user meets the virtual video live streaming conditions based on the target user information in the virtual video live streaming request, it determines an idle target acquisition device from a preset number of acquisition devices, establishes an allocation relationship between the target user and the target acquisition device, and returns a virtual video live streaming start command. The virtual video live streaming conditions are the conditions that users are required to meet to experience the virtual video live streaming effect. In response to the virtual video live stream start command, the current viewing position and current viewing angle of the target user are obtained; The current viewing position and viewing angle are sent to the server via a real-time communication network, so that the server can synchronously adjust the current shooting position and shooting angle of the target acquisition device based on the current viewing position and viewing angle, and shoot the three-dimensional virtual scene in real time based on the adjusted target acquisition device, and return the current live video stream through the real-time communication network. Play the current live video stream captured in real time by the target acquisition device.

2. The video live streaming method according to claim 1, characterized in that, The method is applied to a video playback terminal, and obtaining the target user's current viewing position and current viewing angle includes: Based on the current pose information collected by the sensors in the video playback device, the current viewing position and current viewing angle of the target user are determined; or... Based on the control button trigger information in the video playback terminal, the current viewing position and current viewing angle of the target user are determined.

3. The video live streaming method according to claim 1, characterized in that, The three-dimensional virtual scene is constructed based on a three-dimensional virtual model of the real-world location and the material configuration information of at least one model material; The model materials include at least one of the following: video, audio, image, text, and 3D material models.

4. The video live streaming method according to claim 1, characterized in that, The method further includes: In response to the live streaming sharing operation triggered by the target user, a live streaming sharing request corresponding to the target user is sent to the server, so that the server determines the sharing user corresponding to the target user based on the live streaming sharing request, and sends the current live video stream captured in real time by the target acquisition device to the video playback terminal of the sharing user through the content delivery network.

5. The video live streaming method according to any one of claims 1-4, characterized in that, The method further includes: In response to the switching operation from virtual video live streaming mode to panoramic video live streaming mode triggered by the target user, a panoramic video live streaming request is sent to the server, so that the server controls the panoramic acquisition device to shoot the three-dimensional virtual scene in real time based on the panoramic video live streaming request, and returns the current full video stream through the content delivery network. Based on the received current panoramic video stream and the target user's current viewing angle, determine the current viewpoint video stream and play the current viewpoint video stream.

6. A video live streaming method, characterized in that, include: Receive a virtual video live streaming request corresponding to the target user, wherein the virtual video live streaming request is sent in response to a virtual video live streaming operation triggered by the target user; If, based on the target user information in the virtual video live streaming request, it is detected that the target user meets the virtual video live streaming conditions, then an idle target acquisition device is determined from a preset number of acquisition devices, and an allocation relationship is established between the target user and the target acquisition device. The virtual video live streaming conditions are the conditions that users are required to meet to experience the virtual video live streaming effect. A virtual video live stream start command is sent to the video playback terminal, so that the video playback terminal can obtain the target user's current viewing position and current viewing angle based on the virtual video live stream start command, and return the current viewing position and current viewing angle through the real-time communication network; Based on the current viewing position and viewing angle, the current shooting position and shooting angle of the target acquisition device are simultaneously adjusted, and the three-dimensional virtual scene is captured in real time based on the adjusted target acquisition device. The captured live video stream is sent to the video playback terminal through a real-time communication network so that the video playback terminal can play the live video stream captured in real time by the target acquisition device.

7. The video live streaming method according to claim 6, characterized in that, The step of receiving a virtual video live streaming request corresponding to the target user, wherein the virtual video live streaming request is sent in response to a virtual video live streaming operation triggered by the target user, includes: Receive a virtual video live streaming request corresponding to the target user, wherein the virtual video live streaming request is sent in response to the switching operation from panoramic video live streaming mode to virtual video live streaming mode triggered by the target user; Based on the shooting position of the panoramic acquisition device and the current viewing angle of the target user in the virtual video live streaming request, the current shooting position and current shooting angle of the target acquisition device are adjusted for the first time, and the three-dimensional virtual scene is captured based on the adjusted target acquisition device to obtain the switched video stream; The switched video stream is sent to the video playback terminal via a real-time communication network, so that the video playback terminal plays the switched video stream.

8. The video live streaming method according to any one of claims 6-7, characterized in that, The method further includes: Get the virtual character model corresponding to the user currently watching the virtual video live stream; Based on the current viewing position and viewing angle of the user, the virtual character model corresponding to the user is added to the three-dimensional virtual scene to update the three-dimensional virtual scene.

9. A video live streaming device, characterized in that, include: The virtual video live streaming request sending module is used to respond to a virtual video live streaming operation triggered by a target user, and send a virtual video live streaming request corresponding to the target user to the server. If the server detects that the target user meets the virtual video live streaming conditions based on the target user information in the virtual video live streaming request, it will determine the target acquisition device that is idle from a preset number of acquisition devices, establish an allocation relationship between the target user and the target acquisition device, and return a virtual video live streaming start command. The virtual video live streaming conditions are the conditions that users are required to meet to experience the virtual video live streaming effect. The current viewing information acquisition module is used to acquire the target user's current viewing position and current viewing angle in response to the virtual video live broadcast start command; The current viewing information sending module is used to send the current viewing position and current viewing angle to the server through a real-time communication network, so that the server can synchronously adjust the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and perform real-time shooting of the three-dimensional virtual scene based on the adjusted target acquisition device, and return the captured current live video stream through the real-time communication network; The current live video stream playback module is used to play the current live video stream captured in real time by the target acquisition device.

10. A video live streaming device, characterized in that, include: The virtual video live streaming request receiving module is used to receive virtual video live streaming requests corresponding to the target user. The virtual video live streaming request is sent in response to the virtual video live streaming operation triggered by the target user. The acquisition device allocation module is used to determine the target acquisition device that is idle from a preset number of acquisition devices if the target user meets the virtual video live streaming conditions based on the target user information in the virtual video live streaming request, and to establish an allocation relationship between the target user and the target acquisition device. The virtual video live streaming conditions are the conditions that users are required to meet to experience the virtual video live streaming effect. The live streaming start command sending module is used to send a virtual video live streaming start command to the video playback terminal, so that the video playback terminal can obtain the current viewing position and current viewing angle of the target user based on the virtual video live streaming start command, and return the current viewing position and current viewing angle through the real-time communication network; The current live video stream sending module is used to synchronously adjust the current shooting position and current shooting angle of the target acquisition device based on the current viewing position and current viewing angle, and to shoot the three-dimensional virtual scene in real time based on the adjusted target acquisition device. The captured current live video stream is then sent to the video playback terminal through a real-time communication network so that the video playback terminal can play the current live video stream captured in real time by the target acquisition device.

11. A video live streaming system, characterized in that, The system includes a video playback terminal and a server. The video playback terminal is used to implement the video live streaming method as described in any one of claims 1-5; The server is used to implement the video live streaming method as described in any one of claims 6-8.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the video live streaming method as described in any one of claims 1-8.

13. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the live video streaming method as described in any one of claims 1-8.

Images