Transmission method, device and storage medium of VR live video

Abstract

The present disclosure provides a transmission method, device and storage medium for VR live video, relating to the technical field of communication, wherein the method comprises: performing encoding processing on video data transmitted by a VR live source to generate fragmented encoding data; performing encoding processing on the fragmented encoding data to generate panoramic encoding data and main perspective encoding data; performing encapsulation processing on the panoramic encoding data, the main perspective encoding data, unicast fragmented data index information and audio to generate a multicast stream and add a PMT table; transmitting the multicast stream in a multicast group, and when receiving a unicast switching request transmitted by a terminal in the multicast group based on the PMT table, transmitting unicast fragmented data corresponding to the unicast switching request to the terminal. The method, device and medium of the present disclosure can be combined for transmission by using multicast and unicast, save network bandwidth, take advantage of operator multicast bearing, reduce CDN bearing cost, be compatible with multiple types of terminals, and improve user experience.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a method, apparatus, and storage medium for transmitting VR live video. Background Technology

[0002] Existing VR live video solutions typically encapsulate live streams using DASH or HLS files and deliver services to users via CDN (Content Delivery Network) in a unicast manner. To reduce bandwidth and terminal decoding requirements, existing VR live video solutions utilize VR FOV technology. However, current VR FOV technology can only use unicast, failing to fully utilize the advantages of operator multicast networks, and resulting in high CDN costs. Summary of the Invention

[0003] In view of this, one technical problem to be solved by this disclosure is to provide a method, apparatus and storage medium for transmitting VR live video.

[0004] According to a first aspect of this disclosure, a method for transmitting VR live video is provided, comprising: encoding video data sent from a virtual reality (VR) live streaming source to generate segmented encoded data; encoding the segmented encoded data to generate panoramic encoded data and first-person perspective encoded data; encapsulating the panoramic encoded data, the first-person perspective encoded data, and corresponding unicast segmented data index information and audio to generate a multicast stream; wherein a PMT table corresponding to the multicast stream is added to the multicast stream; sending the multicast stream to all terminals within the multicast group, so that all terminals play the panoramic encoded data or the first-person perspective encoded data; receiving a unicast switching request sent by a terminal within the multicast group based on the PMT table, and sending unicast segmented data corresponding to the unicast switching request to the terminal, so that the terminal simultaneously receives the multicast stream and the unicast segmented data and performs corresponding display processing.

[0005] Optionally, the step of encoding the video data sent by the virtual reality (VR) live stream source to generate segmented encoded data includes: encoding the video data based on a preset encoding method to generate the segmented encoded data; wherein the encoding method includes: Motion Constraint Block Set (MCTS) encoding method.

[0006] Optionally, the unicast fragment data includes HLS fragment data; the method further includes: encapsulating the fragment encoded data using the HLS protocol to generate the HLS fragment data.

[0007] Optionally, the unicast fragment data index information includes: the URL link and offset of the HLS fragment data; the encapsulation processing of the panoramic encoded data, the main view encoded data, the corresponding unicast fragment data index information, and the audio to generate a multicast stream includes: performing TS encapsulation processing on the panoramic encoded data, the main view encoded data, the URL link and offset of the HLS fragment data, and the audio to generate a TS stream, which is used as the multicast stream.

[0008] Optionally, the information carried by the PMT table includes: a first PID of the panoramic encoded data, a second PID of the main view encoded data, a third PID of the HLS slice data index, a URL link of the HLS slice data index, and an offset and a fourth PID of the audio.

[0009] Optionally, the terminal extracts and parses the PMT table from the TS stream, and obtains the corresponding encoded data and audio based on the parsing result for display processing; wherein, after receiving the TS stream, the terminal first displays the first-person perspective encoded data.

[0010] Optionally, the unicast switching request includes: the URL link and offset of the HLS fragment data; sending the unicast fragment data corresponding to the unicast switching request to the terminal includes: obtaining the corresponding HLS fragment data according to the URL link and offset of the HLS fragment data index, and sending it to the terminal via unicast.

[0011] According to a second aspect of this disclosure, a VR live video transmission device is provided, comprising: a first encoding module for encoding video data sent from a virtual reality (VR) live streaming source to generate segmented encoded data; a second encoding module for encoding the segmented encoded data to generate panoramic encoded data and first-person view encoded data; a multicast stream encapsulation module for encapsulating the panoramic encoded data, the first-person view encoded data, and corresponding unicast segmented data index information and audio to generate a multicast stream; wherein a PMT table corresponding to the multicast stream is added to the multicast stream; a multicast stream sending module for sending the multicast stream to all terminals within the multicast group, so that all terminals play the panoramic encoded data or the first-person view encoded data; and a unicast stream sending module for receiving unicast switching requests sent by terminals within the multicast group based on the PMT table, and sending unicast segmented data corresponding to the unicast switching request to the terminals, so that the terminals simultaneously receive the multicast stream and the unicast segmented data and perform corresponding display processing.

[0012] Optionally, the first encoding module is specifically used to encode the video data based on a preset encoding method to generate the segmented encoded data; wherein, the encoding method includes: Motion Constraint Block Set (MCTS) encoding method.

[0013] Optionally, the unicast fragmented data includes HLS fragmented data; the apparatus further includes a unicast encapsulation module, used to encapsulate the fragmented encoded data using the HLS protocol to generate the HLS fragmented data.

[0014] Optionally, the unicast fragment data index information includes: the URL link and offset of the HLS fragment data; the multicast stream encapsulation module is specifically used to perform TS encapsulation processing on the panoramic encoded data, the main view encoded data, the URL link and offset of the HLS fragment data, and the audio to generate a TS stream, which is used as the multicast stream.

[0015] Optionally, the information carried by the PMT table includes: a first PID of the panoramic encoded data, a second PID of the main view encoded data, a third PID of the HLS slice data index, a URL link of the HLS slice data index, and an offset and a fourth PID of the audio.

[0016] Optionally, the unicast switching request includes: the URL link and offset of the HLS fragment data; the unicast stream sending module is specifically used to obtain the corresponding HLS fragment data according to the URL link and offset of the HLS fragment data index, and send it to the terminal via unicast. According to a third aspect of this disclosure, a VR live video transmission device is provided, including: a memory; and a processor coupled to the memory, the processor being configured to execute the method described above based on instructions stored in the memory.

[0017] According to a third aspect of this disclosure, a VR live video transmission apparatus is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the method described above based on instructions stored in the memory.

[0018] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions which are executed by a processor as described above.

[0019] The disclosed VR live video transmission method, apparatus, and storage medium encode video data sent from the VR live source to generate segmented encoded data; encode the segmented encoded data to generate panoramic encoded data and first-person view encoded data; encapsulate the panoramic encoded data, first-person view encoded data, unicast segmented data index information, and audio to generate a multicast stream and add a PMT table; send the multicast stream to all terminals in the multicast group; and when receiving a unicast switching request from a terminal in the multicast group based on the PMT table, send the unicast segmented data corresponding to the unicast switching request to the terminal. This method and apparatus utilize a combination of multicast and unicast for transmission, saving network bandwidth, leveraging the multicast carrying advantages of operators, reducing CDN carrying costs, and ensuring compatibility with various types of terminals, thus improving user experience. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of existing VR live streaming technology solutions;

[0022] Figure 2 This is a flowchart illustrating an embodiment of the VR live video transmission method according to the present disclosure;

[0023] Figure 3 This is an application architecture diagram of an embodiment of the VR live video transmission method according to the present disclosure;

[0024] Figure 4 This is a data processing diagram of one embodiment of the VR live video transmission method according to the present disclosure;

[0025] Figure 5A This is a schematic diagram of the initial multicast reception and decoding on the terminal side; Figure 5B This is a schematic diagram illustrating the switching from multicast to VR FOV on the terminal side. Figure 5C This is a schematic diagram illustrating stable playback after the field of view is switched on the terminal side;

[0026] Figure 6 This is a schematic diagram illustrating a practical application of one embodiment of the VR live video transmission method disclosed herein.

[0027] Figure 7 This is a schematic diagram of a module of an embodiment of a VR live video transmission device according to the present disclosure;

[0028] Figure 8 This is a schematic diagram of a module of another embodiment of the VR live video transmission device according to the present disclosure;

[0029] Figure 9 This is a schematic diagram of a module of yet another embodiment of the VR live video transmission device according to the present disclosure. Detailed Implementation

[0030] The present disclosure will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments of the present disclosure. The technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present disclosure. The technical solutions of the present disclosure will be described in various aspects below with reference to the various figures and embodiments.

[0031] VR (Virtual Reality) live streaming refers to a real-time virtual reality audio and video content service provided to users by a live stream source that captures 360-degree panoramic video in real time, projects and encodes it. VR live streaming videos provide end users with an immersive experience by displaying 360-degree video, supporting panoramic video content, allowing users to interactively switch viewing angles, and the terminal can dynamically render images, videos and their associated audio according to the user's viewing angle.

[0032] Existing VR live streaming technology solutions such as Figure 1 As shown, VR live video typically encapsulates the live stream using DASH (Dynamic Adaptive Streaming over HTTP) or HLS (HTTP Live Streaming) files and delivers the service to users via CDN in a unicast manner. Existing VR live video solutions have the following problems:

[0033] 1. To achieve the same high-definition live video quality as current VR live streaming, the required bandwidth is extremely large (e.g., 120-180Mb / s for 8K 60P), which is insufficient for the access bandwidth of many existing IPTV (Internet Protocol Television) users; 2. Existing VR live streaming solutions use VR FOV (Field of View) technology to reduce bandwidth and terminal decoding requirements, but current VR FOV technology can only use unicast, failing to fully utilize the advantages of telecom operators' multicast networks, resulting in very high CDN costs; 3. Existing IPTV terminals do not support VR projection, therefore they cannot play VR live streaming content.

[0034] Figure 2 This is a flowchart illustrating an embodiment of the VR live video transmission method according to the present disclosure, as shown below. Figure 2 As shown:

[0035] Step 201: Encode the video data sent by the virtual reality (VR) live stream source to generate segmented encoded data.

[0036] Step 202: Encode the segmented encoded data to generate panoramic encoded data and main view encoded data.

[0037] Step 203: Encapsulate the panoramic encoded data, the main view encoded data, the corresponding unicast segment data index information, and the audio to generate a multicast stream; wherein, a PMT (Program Map Table) table corresponding to the multicast stream is added to the multicast stream.

[0038] Step 204: Send a multicast stream to all terminals in the multicast group so that all terminals can play the panoramic encoded data or the first-view encoded data.

[0039] Step 205: Receive unicast switching requests sent by terminals within the multicast group based on the PMT table, and send the unicast fragment data corresponding to the unicast switching request to the terminals so that the terminals can simultaneously receive multicast streams and unicast fragment data and perform corresponding display processing. Various methods can be used to play panoramic encoded data or first-view encoded data, and to perform display processing on multicast streams and unicast fragment data.

[0040] The VR live video transmission method disclosed herein is compatible with existing IPTV multicast, can leverage the multicast carrying advantages of operators, reduces CDN carrying costs, and is compatible with various types of terminals, which is conducive to increasing the audience reach of VR live streaming.

[0041] In one embodiment, encoding video data sent from a virtual reality (VR) live stream source can be performed using various methods. For example, the video data can be encoded using a preset encoding method to generate segmented encoded data. Various encoding methods can be used, such as MCTS (motion-constrained tile set) encoding. Unicast segmented data includes HLS segmented data, which is encapsulated using the HLS protocol to generate HLS segmented data. Existing methods can also be used to encapsulate the segmented encoded data to generate HLS segmented data.

[0042] Unicast fragment data index information includes URL links and offsets of HLS fragment data. Multiple methods can be used to generate multicast streams. For example, panoramic encoded data, first-person view encoded data, URL links and offsets of HLS fragment data, and audio can be encapsulated in TS (Transport Stream) to generate a TS stream for use as a multicast stream. Various existing methods can be used for TS encapsulation to generate TS streams.

[0043] The PMT table can use a variety of existing formats. For example, the data carried by the PMT table includes the first PID of the panoramic encoded data, the second PID of the main view encoded data, the third PID of the HLS fragment data index, the URL link of the HLS fragment data index, and the fourth PID of the offset and audio.

[0044] The VR live video transmission method in the above embodiments addresses the problems of existing VR FOV technology, which can only use unicast to carry out the transmission, does not fully utilize the advantages of telecom operators' multicast networks, and has high CDN carrying costs. It adopts a new hybrid encapsulation format that uses the TS protocol to encapsulate the VR live background stream, the main view, and the FOV (field of view), and uses a hybrid transmission method of multicast and unicast.

[0045] In one embodiment, such as Figure 3 As shown, the VR live video transmission method of this disclosure is applied in a VR live video transmission device. For VR stream fusion and encapsulation processing, such as... Figure 4 As shown, the data sent by the VR live stream source is decapsulated and decoded. High-definition segmented TILE encoding: TILE encoding is performed based on MCTS to facilitate subsequent main-view encoding.

[0046] Panoramic low-definition encoding: Transcode the segmented encoded data output by TILE high-definition encoding to reduce resolution and achieve a low bit rate; First-view high-definition projection encoding: Project TILE into a first-view plane based on the first-view perspective and then perform high-definition encoding; TS coupling encapsulation: Based on the TS format standard, fuse and encapsulate VR full-view low-definition video, first-view high-definition video, and HLS index information (including URL links and offsets of HLS segmented data), which can adopt various existing fusion and encapsulation methods.

[0047] After TS coupling encapsulation, a TS-coupled encapsulated stream is output, enabling synchronization among the video streams. For example, any format of a VR live stream source is first decoded and then re-encoded using TILE high-definition encoding via MCTS. When encapsulating panoramic low-definition, first-person perspective, or DASH formats, the frame rate, timing, and other information of the GOP (Group of Pictures) must not be altered. TS coupling encapsulation is based on time-synchronized fusion encapsulation using the same audio, requiring PTS for rendering and display synchronization. TS coupling encapsulation must conform to the TS standard, and the information for each video stream is carried by the PMT table. The PMT filling rules are shown in Table 1 below.

[0048] Panoramic Low Definition PID1 Panoramic video stream First-person perspective HD PID2 Projection-encoded main view bitstream HLS Index PID3 The URL link of the HLS fragment carries the offset. audio PID4 Audio shared by all video streams is synchronized using audio time.

[0049] Table 1 - PMT Table for TS Coupled Package

[0050] The terminal extracts and parses the PMT table from the TS stream, and obtains the corresponding encoded data and audio for display processing based on the parsing results. After receiving the TS stream, the terminal first displays the first-person perspective encoded data. The unicast switching request includes the URL link and offset of the HLS fragment data, etc. The corresponding HLS fragment data is obtained according to the URL link and offset of the HLS fragment data index, and sent to the terminal via unicast.

[0051] In one embodiment, such as Figure 5A As shown, the initial multicast reception and decoding on the terminal side: different types of terminals receive the same multicast stream, first obtain and parse the PMT table; according to their respective decoding and VR processing capabilities, they obtain the PID of the audio and video stream from the PMT table; initially, all terminals decode and display the main view video.

[0052] like Figure 5BAs shown, the multicast to VR FOV switching on the terminal side is as follows: When the user switches the field of view, a multicast to VRFOV unicast process is triggered (traditional IPTV terminals do not support this switching); the terminal parses the FOV unicast URL from the FOV index (the FOV index is an HLS index, containing URL links of HLS segment data and offsets) and receives the panoramic low-definition. The FOV unicast URL (URL link of HLS segment data) is the URL of the specific live segment (HLS segment data) and carries an offset, without needing to point to the overall index URL.

[0053] like Figure 5C As shown, the terminal side plays stably after the field of view is switched: After stabilization, the terminal simultaneously receives multicast panoramic low-definition and FOV unicast high-definition and realizes superimposed display; when playing FOV unicast, the terminal should render and superimpose the background stream and FOV stream according to the audio clock; various existing methods can be used to achieve superimposed display and rendering superimposed, etc.

[0054] like Figure 6 As shown, for IPTV users, they can receive VR-fused and encapsulated video streams and, based on the PID indication in the PMT table, separate the main-view high-definition video stream. Since the main-view high-definition video stream is projected and encoded as a normal planar video stream, it can be decoded by ordinary IPTV terminals. However, due to the functional limitations of ordinary IPTV terminals, these users cannot use VR live streaming's panoramic playback, perspective switching, and other similar functions.

[0055] For VR IPTV terminals and VR headsets (head displays), these terminals initially play a normal planar video stream. When the user switches views, the terminal separates the VR background stream and FOV index according to the instructions of the PMT table, and requests FOV video live streaming segment files from the CDN according to the FOV index, and finally realizes FOV playback.

[0056] In one embodiment, such as Figure 7 As shown, this disclosure provides a VR live video transmission device 70, including: a first encoding module 71, a second encoding module 72, a multicast stream encapsulation module 73, a multicast stream sending module 74, and a unicast stream sending module 75.

[0057] The first encoding module 71 encodes the video data sent by the virtual reality (VR) live stream source to generate segmented encoded data. The second encoding module 72 encodes the segmented encoded data to generate panoramic encoded data and first-person view encoded data. The multicast stream encapsulation module 73 encapsulates the panoramic encoded data, first-person view encoded data, and corresponding unicast segmented data index information and audio to generate a multicast stream; wherein, a PMT table corresponding to the multicast stream is added to the multicast stream.

[0058] The multicast stream sending module 74 sends the multicast stream to all terminals within the multicast group, enabling all terminals to play panoramic encoded data or first-view encoded data. The unicast stream sending module 75 receives unicast switching requests sent by terminals within the multicast group based on the PMT table, and sends the unicast fragment data corresponding to the unicast switching request to the terminal, so that the terminal can simultaneously receive the multicast stream and unicast fragment data and perform corresponding display processing.

[0059] In one embodiment, the first encoding module 71 encodes the video data based on a preset encoding method to generate segmented encoded data; wherein the encoding method includes MCTS encoding method, etc. Figure 8 As shown, the unicast fragmented data includes HLS fragmented data; the VR live video transmission device 70 also includes a unicast encapsulation module 76, which uses the HLS protocol to encapsulate the fragmented encoded data to generate HLS fragmented data.

[0060] In one embodiment, the unicast fragment data index information includes URL links and offsets of the HLS fragment data. The multicast stream encapsulation module 73 performs TS encapsulation processing on the panoramic encoded data, the main view encoded data, the URL links and offsets of the HLS fragment data, and the audio to generate a TS stream as a multicast stream. The PMT table includes a first PID for the panoramic encoded data, a second PID for the main view encoded data, a third PID for the HLS fragment data index, and a fourth PID for the URL links and offsets of the HLS fragment data index and the audio.

[0061] The unicast switching request includes the URL link and offset of the HLS fragment data. The unicast stream sending module 75 obtains the corresponding HLS fragment data according to the URL link and offset of the HLS fragment data index, and sends it to the terminal via unicast.

[0062] Figure 9 This is a schematic diagram of a module of yet another embodiment of the VR live video transmission apparatus according to the present disclosure. Figure 9 As shown, the device may include a memory 91, a processor 92, a communication interface 93, and a bus 94. The memory 91 is used to store instructions, and the processor 92 is coupled to the memory 91. The processor 92 is configured to execute the VR live video transmission method described above based on the instructions stored in the memory 91.

[0063] The memory 91 can be a high-speed RAM, non-volatile memory, or a memory array. The memory 91 may also be divided into blocks, and these blocks can be combined into virtual volumes according to certain rules. The processor 92 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the VR live video transmission method of this disclosure.

[0064] In one embodiment, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the VR live video transmission method as described in any of the preceding embodiments.

[0065] The VR live video transmission method, apparatus, and storage medium in the above embodiments encode the video data sent by the VR live source to generate segmented encoded data; encode the segmented encoded data to generate panoramic encoded data and first-view encoded data; encapsulate the panoramic encoded data, first-view encoded data, unicast segmented data index information, and audio to generate a multicast stream and add a PMT table; send the multicast stream to all terminals in the multicast group; and when receiving a unicast switching request sent by a terminal in the multicast group based on the PMT table, send the unicast segmented data corresponding to the unicast switching request to the terminal. This allows for combined transmission of multicast and unicast, saving network bandwidth, enabling multicast and unicast switching, compatibility with existing IPTV multicast, leveraging the multicast carrying advantages of operators, reducing CDN carrying costs, and compatibility with various types of terminals, which helps to broaden the audience of VR live streaming and improve the user experience.

[0066] The methods and systems of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0067] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for transmitting VR live video, comprising: The video data sent by the virtual reality (VR) live stream source is encoded to generate segmented encoded data. The segmented encoded data is encoded to generate panoramic encoded data and main view encoded data; The panoramic encoded data, the main view encoded data, the corresponding unicast fragment data index information, and the audio are encapsulated and processed to generate a multicast stream; The unicast fragment data includes HLS fragment data, and the unicast fragment data index information includes the URL link and offset of the HLS fragment data; a PMT table corresponding to the multicast stream is added to the multicast stream, and the information carried by the PMT table includes: the first PID of the panoramic encoded data, the second PID of the main view encoded data, the third PID of the HLS fragment data index, the URL link and offset of the HLS fragment data index, and the fourth PID of the audio; The multicast stream is sent to all terminals in the multicast group so that all terminals can play the panoramic encoded data or the first-view encoded data. The system receives a unicast switching request sent by a terminal within the multicast group based on the PMT table, and sends the unicast fragment data corresponding to the unicast switching request to the terminal, so that the terminal can simultaneously receive the multicast stream and the unicast fragment data and perform corresponding display processing.

2. The method as described in claim 1, wherein encoding the video data sent by the virtual reality (VR) live stream source to generate segmented encoded data includes: The video data is encoded based on a preset encoding method to generate the segmented encoded data; wherein, the encoding method includes: Motion Constraint Block Set (MCTS) encoding method.

3. The method of claim 1 or 2, further comprising: The fragmented encoded data is encapsulated using the HLS protocol to generate the HLS fragmented data.

4. The method as described in claim 3, wherein encapsulating the panoramic encoded data, the main view encoded data, the corresponding unicast fragment data index information, and the audio to generate a multicast stream includes: The panoramic encoded data, the main view encoded data, the URL links and offsets of the HLS segmented data, and the audio are encapsulated in TS to generate a TS stream, which is used as the multicast stream.

5. The method of claim 4, further comprising: The terminal extracts and parses the PMT table from the TS stream, and obtains the corresponding encoded data and audio based on the parsing results for display processing; wherein, after receiving the TS stream, the terminal first displays the first-person perspective encoded data.

6. The method of claim 4, wherein the unicast switching request includes: The URL links and offsets of the HLS sharded data; Sending the unicast fragment data corresponding to the unicast switching request to the terminal includes: The corresponding HLS fragment data is obtained based on the URL link and offset of the HLS fragment data index, and then sent to the terminal via unicast.

7. A VR live video transmission device, comprising: The first encoding module is used to encode the video data sent by the virtual reality (VR) live streaming source and generate segmented encoded data. The second encoding module is used to encode the segmented encoded data to generate panoramic encoded data and main view encoded data; A multicast stream encapsulation module is used to encapsulate the panoramic encoded data, the main-view encoded data, the corresponding unicast fragment data index information, and the audio to generate a multicast stream. The unicast fragment data includes HLS fragment data. The unicast fragment data index information includes the URL link and offset of the HLS fragment data. A PMT table corresponding to the multicast stream is added to the multicast stream. The PMT table carries information including: a first PID of the panoramic encoded data, a second PID of the main-view encoded data, a third PID of the HLS fragment data index, the URL link and offset of the HLS fragment data index, and a fourth PID of the audio. The multicast stream sending module is used to send the multicast stream to all terminals in the multicast group so that all terminals can play the panoramic encoded data or the main view encoded data. The unicast stream sending module is used to receive unicast switching requests sent by terminals in the multicast group based on the PMT table, and send unicast fragment data corresponding to the unicast switching request to the terminals, so that the terminals can simultaneously receive the multicast stream and the unicast fragment data and perform corresponding display processing.

8. The apparatus of claim 7, wherein, The first encoding module is specifically used to encode the video data based on a preset encoding method to generate the segmented encoded data; wherein, the encoding method includes: Motion Constraint Block Set (MCTS) encoding method.

9. The apparatus of claim 7 or 8, further comprising: The unicast encapsulation module is used to encapsulate the fragmented encoded data using the HLS protocol to generate the HLS fragmented data.

10. The apparatus of claim 9, wherein, The multicast stream encapsulation module is specifically used to perform TS encapsulation processing on the panoramic encoded data, the main view encoded data, the URL links and offsets of the HLS segmented data, and the audio to generate a TS stream, which is used as the multicast stream.

11. The apparatus of claim 10, wherein the unicast switching request includes: The URL links and offsets of the HLS sharded data; The unicast stream sending module is specifically used to obtain the corresponding HLS fragment data according to the URL link and offset of the HLS fragment data index, and send it to the terminal via unicast.

12. A VR live video transmission device, comprising: Memory; And a processor coupled to the memory, the processor being configured to perform the method as described in any one of claims 1 to 6 based on instructions stored in the memory.

13. A computer-readable storage medium storing computer instructions that are executed by a processor according to any one of claims 1 to 6.

Images