Video playback method, virtual reality device, server and computer storage medium

Abstract

A video playback method, device and system. The method includes: a VR device receives a VR video stream of a first perspective with a first resolution sent by a server and a full-perspective VR video stream of a second resolution continuously sent by the server, wherein the first resolution is greater than the second resolution; the first perspective VR video stream is used to play the picture of the first perspective; when it is determined that the perspective has changed, the first or both of the full-perspective VR video stream and the first perspective VR video stream are used to play the picture of the current perspective. The solution provided by the embodiment of the present application overcomes the problems of large bandwidth occupation and high latency of network VR services, and can reduce the storage overhead of the server and avoid traffic bursts.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of virtual reality (VR) technology, and in particular to a video playback method, device, and system. Background Art

[0002] VR technology is a computer simulation technology that allows users to create and experience virtual worlds. It provides users with an immersive experience across multiple senses, including vision, hearing, and touch. Network VR technology provides VR services to users over the network. Service servers deployed on the network transmit VR video streams to VR devices over the network, which then play the received VR video streams.

[0003] At present, the following two schemes are usually used to transmit and play VR video streams. In the first scheme, the server sends a 360-degree panoramic VR video stream to the VR device, and the VR device uses the 360-degree panoramic VR video stream to play the picture corresponding to the user's viewing perspective. Since the resolution and data volume of the 360-degree panoramic VR video stream are relatively high, the transmission of the 360-degree panoramic VR video stream requires a large bandwidth, and the bandwidth of ordinary home users cannot meet the transmission requirements of the 360-degree panoramic VR video stream. In the second scheme, the server extracts the VR video stream corresponding to the user's viewing perspective from the 360-degree panoramic VR video stream, and the server only transmits the VR video stream corresponding to the user's viewing perspective to the VR device, and the VR device directly plays it after receiving the above VR video stream. The above-mentioned first scheme can be called a 360-degree transmission scheme, and the second scheme can be called a field of view (FOV) transmission scheme. Compared to the first solution, the second solution can save bandwidth, but it also has another problem. When the user's perspective switches, the VR device needs to request the VR video stream corresponding to the switched perspective from the server. Due to the time consumed by transmission and caching, the Motion to Photon (MTP) latency is high, which cannot meet the VR technology requirement of less than 20 milliseconds. MTP latency refers to the delay from the user's perspective switching to the VR device displaying the image corresponding to the switched perspective.

[0004] Therefore, the above-mentioned existing technologies cannot simultaneously overcome the problems of large bandwidth occupation and high latency of network VR services. Summary of the Invention

[0005] The embodiments of the present application provide a video playback method, device, and system to solve the problem in the prior art that network VR services cannot simultaneously overcome the large bandwidth occupation and high latency.

[0006] On the one hand, an embodiment of the present application provides a video playback method, which includes: a VR device receives a first-perspective VR video stream of a first resolution sent by a server and a full-perspective VR video stream of a second resolution continuously sent by the server, wherein the first resolution is greater than the second resolution; the VR device uses the first-perspective VR video stream of the first resolution to play the picture of the first perspective; when the VR device determines that the perspective has changed, the VR device uses the first or both of the full-perspective VR video stream of the second resolution and the first-perspective VR video stream of the first resolution to play the picture of the current perspective; the VR device sends perspective information to the server, and the perspective information carries parameters for indicating the change to the second perspective; the server sends the second-perspective VR video stream of the first resolution to the VR device according to the above parameters; the VR device uses the second-perspective VR video stream of the first resolution to play the picture of the second perspective.

[0007] In the solution provided by the embodiments of this application, the server sends two service data streams to the VR device: one is a high-resolution VR video stream with a viewing angle, which is used to meet display requirements when the viewing angle is fixed, and the other is a low-resolution full-view VR video stream, which is used to meet display requirements when the viewing angle switches. Compared with the 360-degree transmission solution, bandwidth usage is reduced, and because the low-resolution full-view VR video stream covers the 360-degree viewing angle, the image data required for viewing angle switching does not need to be requested from the server, thereby reducing display latency when viewing angle switching.

[0008] In one possible design, the VR device uses a full-view VR video stream of a second resolution to play the picture of the current view, including: the VR device decodes the full-view VR video stream of the second resolution to obtain a full-view picture; the VR device extracts the picture of the current view from the full-view picture; and the VR device plays the picture of the current view.

[0009] In the solution provided by the embodiment of the present application, during the process of changing the viewing angle, the VR device reads the full-view VR video stream of the second resolution from the video buffer, and uses the full-view VR video stream of the second resolution to play the picture of the current viewing angle. Since the full-view VR video stream of the second resolution is stored in the local video buffer, the VR device does not need to request it from the server, so it can reduce the MTP delay when the viewing angle is switched, and meet the requirement of MTP delay of less than 20 milliseconds. In addition, during the process of changing the viewing angle (for example, when the user turns his head), although a low-resolution picture is displayed, since the picture perceived during the movement of the human eye is itself blurred, presenting a low-resolution picture to the user does not affect the visual experience.

[0010] In one possible design, a VR device uses a full-view VR video stream with a second resolution and a first-view VR video stream with a first resolution to play the picture of the current view, including: the VR device decodes the first-view VR video stream with a first resolution to obtain the picture of the first view; the VR device decodes the full-view VR video stream with a second resolution to obtain the full-view picture; the VR device extracts overlapping pictures from the picture of the first view, and extracts non-overlapping pictures from the full-view picture, wherein the overlapping picture refers to the picture of the overlapping area between the picture of the first view and the picture of the current view, and the non-overlapping picture refers to the picture of the current view excluding the overlapping area; the VR device stitches the overlapping pictures and the non-overlapping pictures to obtain the picture of the current view; and the VR device plays the picture of the current view.

[0011] In the solution provided in the embodiment of the present application, during the process of perspective change, there may still be a certain overlap between the current perspective and the first perspective. For the overlapping area, high-resolution images can still be used for playback to minimize clarity loss.

[0012] In one possible design, after the VR device receives the VR video stream of the second perspective with the first resolution sent by the server, it also includes: the VR device detects whether the received VR video stream of the second perspective with the first resolution is synchronized with the timestamp of the currently played picture, and the above synchronization means that the timestamp of the received VR video stream of the second perspective with the first resolution includes the timestamp of the currently played picture; if synchronized, the VR device executes the step of using the VR video stream of the second perspective with the first resolution to play the picture of the second perspective; if not synchronized, the VR device uses the first or both of the VR video stream of the full perspective with the second resolution and the VR video stream of the first perspective with the first resolution to play the picture of the second perspective.

[0013] In the solution provided in the embodiment of the present application, the VR device is also used to detect whether the received VR video stream of the first resolution and the second perspective is synchronized with the timestamp of the current playback screen, ensuring smooth and accurate playback of the VR video to the user.

[0014] In one possible design, after the VR device receives the first-perspective VR video stream of the first resolution sent by the server and the full-perspective VR video stream of the second resolution continuously sent by the server, it also includes: the VR device stores the first-perspective VR video stream of the first resolution to a first video buffer, and stores the full-perspective VR video stream of the second resolution to a second video buffer.

[0015] In the solution provided in the embodiment of the present application, the VR device independently stores VR video streams of different resolutions through two video buffers, which is conducive to quickly selecting different VR video streams for playback and improving the smoothness of selection and switching.

[0016] In one possible design, the method also includes: when the VR device determines that the perspective has changed, or when the VR device uses a VR video stream with a first resolution and a second perspective to play a picture with a second perspective, the VR device performs data aging processing on the VR video stream with a first resolution and a first perspective cached in the first video buffer.

[0017] In the solution provided in the embodiment of the present application, when the VR device no longer needs to use the VR video stream of the first resolution and the first perspective cached in the first video buffer, data aging processing is performed on the above data, so that the cache resources of the first video buffer can be released as soon as possible.

[0018] In one possible design, a VR device uses a first-perspective VR video stream with a first resolution to play a first-perspective picture, including: the VR device decodes the first-perspective VR video stream with the first resolution to obtain a first-perspective picture, and the VR device plays the first-perspective picture. The VR device uses a second-perspective VR video stream with a first resolution to play a second-perspective picture, including: the VR device decodes the second-perspective VR video stream with the first resolution to obtain a second-perspective picture, and the VR device plays the second-perspective picture.

[0019] In the technical solution provided in the embodiment of the present application, since the VR device decodes different VR video streams separately, different VR video streams can be distinguished.

[0020] On the other hand, embodiments of the present application provide a VR device that has the functionality to implement the VR device-side behavior described in the above method examples. The functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functionality.

[0021] In one possible design, the VR device includes a processor and a communication interface. The processor is configured to support the VR device in performing the corresponding functions of the above method. The communication interface is used to support communication between the VR device and other devices. Furthermore, the VR device may also include a memory, which is coupled to the processor and stores program instructions and data necessary for the VR device.

[0022] In another aspect, embodiments of the present application provide a server that implements the server-side behavior described in the aforementioned method examples. The functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the aforementioned functionality.

[0023] In one possible design, the server structure includes a processor and a communication interface. The processor is configured to support the server in performing the corresponding functions of the above method. The communication interface is used to support communication between the server and other devices. Furthermore, the server may also include a memory, which is coupled to the processor and stores program instructions and data necessary for the server.

[0024] On the other hand, an embodiment of the present application provides a video playback system, which includes the VR device and server described in the above aspects.

[0025] On the other hand, an embodiment of the present application provides a computer storage medium for storing computer software instructions used for the above-mentioned VR device, which includes a program designed for executing the above-mentioned aspects.

[0026] On the other hand, an embodiment of the present application provides a computer storage medium for storing computer software instructions used by the above-mentioned server, which includes a program designed for executing the above-mentioned aspects.

[0027] On the other hand, an embodiment of the present application provides a computer program product, which, when executed, is used to execute the method on the VR device side described in the above aspects.

[0028] On the other hand, an embodiment of the present application provides a computer program product, which, when executed, is used to execute the server-side method described in the above aspects.

[0029] Compared to the existing technology, the solution provided by the embodiments of this application uses a server to send two service data streams to the VR device: one is a high-resolution VR video stream with a viewing angle, which is used to meet display requirements when the viewing angle is fixed, and the other is a low-resolution full-view VR video stream, which is used to meet display requirements when the viewing angle switches. Compared to the 360-degree transmission solution, bandwidth usage is reduced, and because the low-resolution full-view VR video stream covers the 360-degree viewing angle, the image data required for viewing angle switching does not need to be requested from the server, thereby reducing display latency when viewing angle switching. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1A This is a schematic diagram of an example of generating a single-view VR image;

[0031] Figure 1B This is a schematic diagram of an example of generating a VR screen with multiple perspectives;

[0032] Figure 2A This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0033] Figure 2B is a schematic diagram of an application scenario provided by another embodiment of the present application;

[0034] Figure 3 This is a flowchart of a video playback method provided by an embodiment of the present application;

[0035] Figure 4 This is a schematic diagram of stitching a picture of the current viewing angle provided by an embodiment of the present application;

[0036] Figure 5A is a flowchart of a video playback method provided by another embodiment of the present application;

[0037] Figure 5B is with Figure 5A Interaction diagram corresponding to the embodiment;

[0038] Figure 6A is a schematic block diagram of a device provided by an embodiment of the present application;

[0039] Figure 6B It is a structural diagram of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings.

[0041] The system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. A person skilled in the art will appreciate that, with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0042] Please refer to Figure 2A , which shows a schematic diagram of a possible application scenario provided by an embodiment of the present application. The application scenario may be a network VR system, which refers to a system that provides VR services to users over a network. The application scenario may include: VR device 21 and server 22.

[0043] The VR device 21 is used to receive the VR video stream sent by the server 22 and play images based on the VR video stream to provide users with the function of watching VR videos. The VR device 21 can be a head-mounted VR display device, such as a VR helmet, VR glasses, etc. Of course, the VR device can also be any other device capable of playing VR videos.

[0044] Server 22 stores VR video resources and is used to provide VR video streams to VR device 21. Server 22 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Optionally, server 22 is a backend server for providing network VR services, such as a backend server for a website or application that provides network VR services.

[0045] There is a communication connection between the VR device 21 and the server 22. The communication connection can be a wireless network connection or a wired network connection.

[0046] exist Figure 2A Based on the application scenarios provided in the embodiment, combined with reference Figure 2B The present application embodiment also provides a schematic diagram of another possible application scenario. In addition to the VR device 21 and the server 22, the application scenario also includes at least one camera 23.

[0047] The camera 23 is used to collect VR video streams and send the collected VR video streams to the server 22.

[0048] There is a communication connection between the camera 23 and the server 22. The communication connection can be a wireless network connection or a wired network connection.

[0049] Figure 2B The application scenario provided in the embodiment is applicable to live broadcast scenarios.

[0050] In one example, a method for transmitting and playing VR videos is provided. The server divides the 360-degree viewing angle into 30 viewing angles in advance, and the generation process of the VR image of each viewing angle is as follows: Figure 1A As shown. Taking the target perspective as an example, the target perspective is any one of the above 30 perspectives. The server extracts the target perspective area picture 12 from the 360-degree panoramic picture 11. The remaining picture after removing the target perspective area picture 12 from the 360-degree panoramic picture 11 can be called the background area picture 13. The server reduces the resolution of the background area picture 13 to obtain a low-resolution background area picture 14. The VR picture 15 of the target perspective includes the target perspective area picture 12 and the low-resolution background area picture 14. In this way, the VR pictures of the 30 perspectives can be as follows Figure 1B shown.

[0051] In one example, assuming the user is currently viewing perspective 2, the VR device receives the VR video stream of perspective 2 from the server and stores it in the video buffer. The VR device then uses the VR video stream of perspective 2 to play the image of perspective 2. When the user switches from perspective 2 to perspective 3, the VR device requests the VR video stream of perspective 3 from the server. Because network transmission and video buffering both take a certain amount of time, before the VR video stream of perspective 3 reaches the VR device and can be used for playback, the VR device first extracts the image of the current perspective from the low-resolution background area of ​​the locally cached VR video stream of perspective 2 for playback. During the perspective switching process, although a low-resolution image is displayed, the image perceived by the human eye during movement is inherently blurry. Therefore, displaying a low-resolution image during the perspective switching process does not significantly reduce the visual experience.

[0052] While the solution provided in the preceding example overcomes the issues of high bandwidth usage and high latency for network VR services, it still presents the following technical issues:

[0053] First, the server-side storage overhead is high. The server needs to pre-store VR video streams from 30 perspectives, and each VR video stream includes a low-resolution background area. The content of these low-resolution background areas overlaps significantly, increasing the server-side storage overhead.

[0054] Second, switching perspectives will cause traffic bursts. During the perspective switching process, it takes a certain amount of time (including the time required for data transmission and decoding) for the VR device to request the VR video stream of the switched perspective and then receive and play the VR video stream of the switched perspective. If the VR video stream of the perspective before the switch cached by the VR device is played out within the above-mentioned time, the VR device will have a display interruption. Therefore, in order to avoid display interruption, during the perspective switching process, the server needs to send the VR video stream of the perspective before the switch and the VR video stream of the perspective after the switch to the VR device at the same time. This means that double bandwidth is required when switching perspectives, resulting in traffic bursts.

[0055] In order to overcome the problem of large bandwidth occupation and high latency of network VR services, and to avoid the problems existing in the above examples, the embodiment of the present application provides a video playback method, as well as a device and system based on this method. In the embodiment of the present application, the server sends two service data streams to the VR device, one of which is a high-resolution viewing perspective VR video stream for meeting the display requirements when the viewing angle is fixed, and the other is a low-resolution full-view VR video stream for meeting the display requirements when the viewing angle is switched. Compared with the 360-degree transmission solution, the bandwidth occupation is reduced, and because the low-resolution full-view VR video stream covers the 360-degree viewing angle, the image data required when the viewing angle is switched does not need to be requested from the server, thereby reducing the display delay when the viewing angle is switched. Compared with the above example, the server does not need to repeatedly store multiple low-resolution full-view VR video streams, the storage overhead is reduced, and when the server determines that the viewing angle of the VR device has changed based on the viewing angle information, it can stop sending the first-view VR video stream to the VR device and instead send the second-view VR video stream to which it has changed, so there will be no traffic burst problem.

[0056] The embodiments of the present application will be further described in detail below based on the common aspects involved in the embodiments of the present application described above.

[0057] Please refer to Figure 3 , which shows a flow chart of a video playback method provided by an embodiment of the present application. This method can be applied to Figure 2A or Figure 2B In the application scenario shown in FIG. , the method may include the following steps.

[0058] In step 301, the server sends a first-perspective VR video stream of a first resolution to the VR device, and continuously sends a full-perspective VR video stream of a second resolution to the VR device, where the first resolution is greater than the second resolution.

[0059] In an embodiment of the present application, the server sends two business data streams to the VR device, one of which is a high-resolution viewing perspective VR video stream (that is, a first-resolution viewing perspective (such as a first perspective, a second perspective) VR video stream), which is used to meet the display requirements when the viewing angle is fixed, and the other is a low-resolution full-view VR video stream (that is, a second-resolution full-view VR video stream), which is used to meet the display requirements when the viewing angle is switched. Among them, the second-resolution full-view VR video stream is not affected by the change of viewing angle, and is continuously sent by the server to the VR device during the service provision process. In an embodiment of the present application, the resolution determines the clarity of the VR video. The higher the resolution, the higher the clarity of the VR video; the lower the resolution, the lower the clarity of the VR video.

[0060] In one example, a server stores a full-view VR video stream of a first resolution and a full-view VR video stream of a second resolution. When the server detects that a VR device requests to obtain a VR video stream of a certain perspective, the server extracts the VR video stream of the perspective from the full-view VR video stream of the first resolution. For example, when the server detects that a VR device requests to obtain a VR video stream of the first perspective, the server extracts the VR video stream of the first perspective from the full-view VR video stream of the first resolution. In the above manner, the server only needs to store the full-view VR video stream of the first resolution and the full-view VR video stream of the second resolution. Since the storage overhead occupied by the full-view VR video stream of the second resolution is approximately 0.1 times that of the full-view VR video stream of the first resolution, the total storage overhead occupied is approximately 1.1 times that of the full-view VR video stream of the first resolution. Compared with the above example, the storage overhead of the server is substantially reduced.

[0061] In another example, the server stores VR video streams of n perspectives at a first resolution and a full-perspective VR video stream at a second resolution, where n is an integer greater than 1. In practical applications, the value of n can be obtained based on actual experience, and the combination of the n perspectives can cover the full perspective, for example, n is 25, 30, 35, and so on. When the server detects that the VR device requests to obtain a VR video stream of a certain perspective, it selects the VR video stream of the perspective from the VR video streams of the n perspectives at the first resolution. For example, when the server detects that the VR device requests to obtain a VR video stream of the first perspective, it selects the VR video stream of the first perspective from the VR video streams of the n perspectives at the first resolution, where the first perspective is one of the n perspectives. Through the above method, the server does not need to repeatedly store multiple copies of the full-perspective VR video stream of the second resolution, but only needs to store one copy. Compared with the above example, the storage overhead of the server is also reduced to a certain extent. In addition, in this example, after receiving the request from the VR device, the server does not need to extract the VR video stream of the perspective required by the VR device from the full-view VR video stream. It only needs to directly select the VR video stream of the perspective required by the VR device from the stored n perspectives of VR video streams, which helps to shorten the server's response delay to the VR device's request.

[0062] In addition, the n perspective VR video streams of the first resolution can be extracted separately by the server from the full-perspective VR video stream of the first resolution; or they can be n independent video streams, and each perspective VR video stream is independently captured by one or more cameras.

[0063] Optionally, the full-view VR video stream of the second resolution is converted from the full-view VR video stream of the first resolution. The above-mentioned resolution conversion operation can be performed by the server, or by other devices connected to the server (such as a source server for providing VR video resources to the server). Of course, in other possible implementations, if the VR video streams of n perspectives of the first resolution are n independent video streams, and the VR video streams of each perspective are independently captured by one or more cameras, then in one embodiment, the server can first integrate the VR video streams of n perspectives of the first resolution to obtain a full-view VR video stream of the first resolution, and then convert the full-view VR video stream of the first resolution into a full-view VR video stream of the second resolution; in another embodiment, the server can also convert the VR video streams of n perspectives of the first resolution into VR video streams of n perspectives of the second resolution, and then integrate the VR video streams of n perspectives of the second resolution to obtain a full-view VR video stream of the second resolution.

[0064] Optionally, resolution conversion is achieved by changing the image transmission format of the VR video stream. For example, the VR video stream of the first resolution can adopt an image transmission format such as Double Common Intermediate Format (DCIF) or Quad Common Intermediate Format (4CIF); the VR video stream of the second resolution can adopt an image transmission format such as Common Intermediate Format (CIF) or Quarter Common Intermediate Format (QCIF). For example, taking the full-view VR video stream of the first resolution adopting 4CIF and the full-view VR video stream of the second resolution adopting QCIF as an example, the first resolution is 704×576 and the second resolution is 176×144.

[0065] VR video streams can be transmitted using a streaming protocol. In streaming protocols, different identifiers are used to distinguish between multiple different media streams. For example, when the streaming protocol is the Real-time Transport Protocol (RTP), the Synchronization Source (SSRC) identifier and the Contributing Source (CSRC) identifier are used in RTP to distinguish between multiple different media streams. For another example, when the streaming protocol is the Real-time Messaging Protocol (RTMP), a message stream ID is used to distinguish between multiple different media streams. For another example, when the streaming protocol is the HTTP Live Streaming (HLS) protocol, a Uniform Resource Identifier (URI) is used to distinguish between multiple different media streams. Therefore, when the server sends two VR video streams to the VR device, the two VR video streams can be distinguished by the identifiers defined by the streaming protocol. In addition, depending on the provisions of different streaming protocols, the identifier used to distinguish media streams can be carried in each data packet or only in the header packet (i.e., the first data packet). In other possible examples, since the VR device decodes different media streams (in the embodiment of this application, the media stream is also the VR video stream) separately, it can distinguish different media streams.

[0066] Correspondingly, the VR device receives the VR video stream of the first perspective at the first resolution sent by the server and the full-perspective VR video stream of the second resolution continuously sent by the server. The VR device stores the VR video stream of the first perspective at the first resolution in the first video buffer, and stores the full-perspective VR video stream at the second resolution in the second video buffer. The first video buffer is used to store the VR video stream of a certain perspective at the first resolution, and the second video buffer is used to store the full-perspective VR video stream at the second resolution. In an embodiment of the present application, the VR device independently stores VR video streams of different resolutions through two video buffers, which is conducive to quickly selecting different VR video streams for playback and improving the fluency when selecting and switching.

[0067] In step 302, the VR device uses a first-viewpoint VR video stream with a first resolution to play the first-viewpoint image.

[0068] The VR device decodes the VR video stream of the first perspective at the first resolution to obtain the picture of the first perspective, and then plays the picture of the first perspective. In the embodiment of the present application, the encoding and decoding algorithm of the VR video stream is not limited. For example, the encoding and decoding algorithm provided by the Moving Picture Experts Group (MPEG) standard, the related algorithms involved in the Video Codec (VC), the H.264 encoding and decoding algorithm, etc.

[0069] Optionally, when the VR device detects that the amount of data of the VR video stream of the first perspective cached in the first video buffer exceeds a preset threshold, it starts reading the VR video stream of the first perspective from the first video buffer and decoding and playing it. The preset threshold can be pre-set based on actual experience.

[0070] In step 303, when the VR device determines that the viewing angle has changed, the VR device uses the first one or both of the full-view VR video stream with the second resolution and the first-view VR video stream with the first resolution to play the picture of the current viewing angle.

[0071] When playing a VR video, the VR device detects in real time whether the viewing angle has changed, that is, whether it has changed from a first viewing angle to another viewing angle. When the VR device determines that the viewing angle has changed, the VR device uses the first or both of the second-resolution full-view VR video stream and the first-resolution first-view VR video stream to play the image from the current viewing angle. The current viewing angle refers to the viewing angle determined in real time by the VR device. When the VR device determines that the viewing angle has not changed, the VR device continues to use the first-resolution first-view VR video stream to play the image from the first viewing angle.

[0072] In one example, the VR device uses a full-view VR video stream of the second resolution to play the image of the current view. This step may include the following sub-steps:

[0073] 1. The VR device decodes the full-view VR video stream of the second resolution to obtain a full-view image;

[0074] 2. The VR device extracts the current perspective from the full perspective image;

[0075] 3. The VR device plays the image from the current perspective.

[0076] During the process of changing the viewing angle, the VR device reads the full-view VR video stream of the second resolution from the second video buffer, and uses the full-view VR video stream of the second resolution to play the picture of the current viewing angle. Since the full-view VR video stream of the second resolution is stored in the local second video buffer, the VR device does not need to request it from the server, so it can reduce the MTP delay when the viewing angle is switched, and meet the requirement of MTP delay of less than 20 milliseconds. In addition, during the process of changing the viewing angle (for example, when the user turns his head), although a low-resolution picture is displayed, the picture perceived by the human eye during movement is itself blurred, so presenting a low-resolution picture to the user does not affect the visual experience.

[0077] In another example, the VR device uses a full-view VR video stream of a second resolution and a first-view VR video stream of a first resolution to play the image of the current view. This step may include the following sub-steps:

[0078] 1. The VR device decodes the first-perspective VR video stream of the first resolution to obtain a first-perspective image;

[0079] 2. The VR device decodes the full-view VR video stream of the second resolution to obtain a full-view image;

[0080] 3. The VR device extracts overlapping images from the first-person perspective and non-overlapping images from the full-perspective perspective.

[0081] 4. The VR device combines the overlapping and non-overlapping images to obtain the image of the current perspective;

[0082] 5. The VR device plays the image from the current perspective.

[0083] The overlapping image refers to the area where the first-view and current-view images overlap, while the non-overlapping image refers to the area of ​​the current view except for the overlapping area. During the perspective change process, there may still be some overlap between the current and first-view images. For this overlapping area, high-resolution images can still be used for playback to minimize loss of clarity.

[0084] Combined with reference Figure 4, which shows a schematic diagram of a picture of the current perspective provided by an embodiment of the present application. There is an overlapping area 43 (shown in a rectangular box filled with diagonal lines in the figure) between the first perspective 41 (shown in a solid square box in the figure) and the current perspective 42 (shown in a dotted square box in the figure). The VR device extracts an overlapping picture 45 from the picture 44 of the first perspective of the first resolution, and extracts a non-overlapping picture 47 from the full-perspective picture 46 of the second resolution, and stitches the above-mentioned overlapping pictures 45 and non-overlapping pictures 47 to obtain a picture 48 of the current perspective. Optionally, the VR device scales the overlapping pictures 45 and the non-overlapping pictures 47 to the same resolution and stitches them together to obtain a picture 48 of the current perspective.

[0085] In the embodiment of the present application, there is no limitation on the order of executing sub-steps 1 and 2 in the above another example. The above sub-step 1 can be executed before sub-step 2, or after sub-step 2, or simultaneously with sub-step 2.

[0086] In addition, in the embodiment of the present application, a change in viewing angle may mean that the current viewing angle is different from the original viewing angle, or it may mean that the change between the current viewing angle and the original viewing angle is less than a preset threshold value, which may be pre-set according to actual needs.

[0087] In step 304, the VR device sends viewing angle information to the server, where the viewing angle information carries parameters for indicating the second viewing angle to be changed to.

[0088] In one example, the VR device proactively reports viewing angle information to the server. For example, the VR device sends viewing angle information to the server once every first preset time interval, and each reported viewing angle information carries a parameter for indicating the current viewing angle (i.e., the viewing angle at the time of reporting). The above-mentioned first preset time interval can be preset according to actual needs. For another example, when the VR device detects a change in viewing angle, it sends viewing angle information to the server, and the viewing angle information carries a parameter for indicating the second viewing angle to which it has changed. When it detects that the viewing angle has not changed, the VR device does not send viewing angle information to the server.

[0089] In another example, after receiving a polling request from the server, the VR device sends viewing angle information to the server. For example, the server sends a polling request to the VR device every second preset time interval. After receiving the polling request, the VR device sends viewing angle information to the server. Each reported viewing angle information carries a parameter indicating the current viewing angle (i.e., the viewing angle at the time of reporting). The second preset time interval can be pre-set according to actual needs.

[0090] Accordingly, the server receives the viewing angle information sent by the VR device. Through the method provided by the embodiment of the present application, it is possible to ensure that the server is informed of changes in the viewing angle of the VR device in a timely manner.

[0091] In step 305 , the server sends a VR video stream of the second perspective at the first resolution to the VR device according to the above parameters.

[0092] The server obtains a VR video stream of the second perspective with a first resolution according to the parameters for indicating the second perspective carried in the perspective information, and then sends the VR video stream of the second perspective with the first resolution to the VR device.

[0093] In one example, if a full-view VR video stream with a first resolution is stored in the server, the server extracts a VR video stream with a second view from the full-view VR video stream with the first resolution.

[0094] In another example, if the server stores VR video streams of n perspectives with a first resolution, the server selects a VR video stream of a second perspective from the VR video streams of n perspectives with the first resolution.

[0095] Correspondingly, the VR device receives the VR video stream of the second perspective at the first resolution sent by the server and stores the VR video stream of the second perspective at the first resolution in the first video buffer.

[0096] In the embodiment of the present application, when the server determines that the perspective of the VR device has changed based on the perspective information, it can stop sending the VR video stream of the first perspective to the VR device and instead send the VR video stream of the second perspective to which it has changed, thereby avoiding the problem of traffic bursts. In addition, because the low-resolution full-perspective VR video stream is sent separately and continuously, before the second-perspective VR video stream reaches the VR device, the VR device can use the low-resolution full-perspective VR video stream to play the image of the current perspective, without causing display interruption.

[0097] In step 306, the VR device uses the VR video stream of the second perspective at the first resolution to play the image of the second perspective.

[0098] The VR device decodes the VR video stream of the second perspective at the first resolution to obtain a picture of the second perspective, and then plays the picture of the second perspective.

[0099] Optionally, when the VR device no longer needs to use the VR video stream of the first perspective at the first resolution cached in the first video buffer, the VR device performs data aging processing on the VR video stream of the first perspective cached in the first video buffer. Data aging processing refers to clearing cached data, and performing data aging processing on the VR video stream of the first perspective refers to clearing the VR video stream of the first perspective in the first video buffer. In this way, the cache resources of the first video buffer can be released as soon as possible.

[0100] When the VR device determines that the viewing angle has changed, if the VR device uses a full-view VR video stream with a second resolution to play the picture of the current viewing angle, then when the VR device determines that the viewing angle has changed, data aging processing can be performed on the VR video stream with a first resolution and a first-view VR video stream with a first resolution cached in the first video buffer. When the VR device determines that the viewing angle has changed, if the VR device uses the first or both of the full-view VR video stream with a second resolution and the VR video stream with a first resolution and a first-view VR video stream to play the picture of the current viewing angle, then when the VR device uses the VR video stream with a second perspective with a first resolution to play the picture of the second perspective, data aging processing can be performed on the VR video stream with a first resolution and a first-view VR video stream with a first resolution cached in the first video buffer.

[0101] In the solution provided in the embodiment of the present application, the server sends two business data streams to the VR device, one of which is a high-resolution viewing perspective VR video stream, which is used to meet the display requirements when the viewing angle is fixed, and the other is a low-resolution full-view VR video stream, which is used to meet the display requirements when the viewing angle is switched. Compared with the 360-degree transmission solution, the bandwidth occupancy is reduced, and because the low-resolution full-view VR video stream covers the 360-degree viewing angle, the image data required when the viewing angle is switched does not need to be requested from the server, thereby reducing the display delay when the viewing angle is switched. Compared with the above example, the server does not need to repeatedly store multiple low-resolution full-view VR video streams, the storage overhead is reduced, and when the server determines that the viewing angle of the VR device has changed based on the viewing angle information, it can stop sending the VR video stream of the first perspective to the VR device and instead send the VR video stream of the second perspective to which it has changed, so there will be no problem of traffic bursts.

[0102] Combined with reference Figure 5A and Figure 5B , which shows a flow chart of a video playback method provided by another embodiment of the present application. This method can be applied to Figure 2A or Figure 2B In the application scenario shown in FIG. , the method may include the following steps.

[0103] exist Figure 5B , processes 1-10 represented in circles correspond to the following steps 501-510 respectively.

[0104] Step 501: The VR device sends a service request to the server, where the service request carries parameters for indicating the first perspective.

[0105] The service request is used to request VR video resources from the server.

[0106] Correspondingly, the server receives the service request sent by the VR device.

[0107] In step 502 , the server sends a first-perspective VR video stream of a first resolution to the VR device, and continuously sends a full-perspective VR video stream of a second resolution to the VR device, where the first resolution is greater than the second resolution.

[0108] After receiving the service request, the server obtains a first-viewpoint VR video stream at a first resolution based on the parameters indicating the first viewpoint carried in the service request and sends it to the VR device. Furthermore, the server continues to send a full-viewpoint VR video stream at a second resolution to the VR device until the VR device stops requesting VR video resources from the server, that is, until the service ends. Furthermore, the second-resolution VR video stream can be obtained by transcoding the first-resolution VR video stream, for example, with the second resolution being 10% of the first resolution.

[0109] Correspondingly, the VR device receives the first-perspective VR video stream of the first resolution sent by the server and the full-perspective VR video stream of the second resolution continuously sent by the server.

[0110] Combined with reference Figure 5B , the server is provided with two sending buffers, including a first sending buffer and a second sending buffer. Among them, the first sending buffer is used to send a VR video stream of a certain perspective of a first resolution (also called FOV video stream) to the VR device, and the second sending buffer is used to send a full-view VR video stream of a second resolution (also called 360-degree video stream) to the VR device. Correspondingly, the VR device is provided with two receiving buffers, including a first receiving buffer and a second receiving buffer. Among them, the first receiving buffer is also Figure 3 The first video buffer described in the embodiment is used to store the received VR video stream of a certain perspective of the first resolution (also called FOV video stream), and the second receiving buffer is also used to store the received VR video stream of a certain perspective of the first resolution (also called FOV video stream). Figure 3 The second video buffer introduced in the embodiment is used to store the received full-view VR video stream (also called 360-degree video stream) with a second resolution.

[0111] In step 503, the VR device uses a first-viewpoint VR video stream with a first resolution to play the first-viewpoint image.

[0112] After the VR video stream in the first video buffer is buffered to a certain length (for example, 1 second), the VR device reads the VR video stream of the first perspective with the first resolution from the first video buffer, and decodes and plays it.

[0113] Step 504: When the VR device determines that the viewing angle has changed, the VR device sends viewing angle information to the server. The viewing angle information carries parameters for indicating the second viewing angle to which the device has changed.

[0114] Optionally, the VR device sends a perspective switching request to the server, where the perspective switching request carries parameters for indicating a second perspective to change to, and the perspective switching request is used to request acquisition of a VR video stream of the second perspective to change to.

[0115] Correspondingly, the server receives the viewing angle information sent by the VR device.

[0116] In step 505 , the VR device performs data aging processing on the VR video stream of the first perspective at the first resolution cached in the first video buffer.

[0117] For example, the VR device clears the data stored in the first video buffer and prepares to receive a VR video stream of the second perspective with a first resolution.

[0118] In step 506 , the VR device uses a full-view VR video stream of a second resolution to play the image of the current view.

[0119] The VR device reads the full-view VR video stream of the second resolution from the second video buffer, decodes it to obtain a full-view picture, and extracts the current-view picture from the full-view picture for playback.

[0120] It should be noted that, in the embodiment of the present application, there is no limitation on the order in which the above steps 504, 505 and 506 are executed. The above three steps can be executed in sequence or simultaneously.

[0121] In step 507, the server sends a VR video stream of the second perspective at the first resolution to the VR device according to the parameter for indicating the second perspective to be changed to carried in the perspective information.

[0122] When the server determines that the VR device's perspective has changed based on the perspective information, it stops sending the VR video stream at the first perspective to the VR device and instead sends the VR video stream at the second perspective. Accordingly, the VR device receives the VR video stream at the second perspective at the first resolution sent by the server.

[0123] In addition, when the server determines that the viewing angle of the VR device has changed, it can clear the data in the first sending buffer and prepare to send the VR video stream of the second viewing angle with the first resolution.

[0124] In step 508 , the VR device detects whether the received VR video stream of the second perspective at the first resolution is synchronized with the timestamp of the currently playing picture; if so, execute the following step 509 ; if not, execute the following step 510 .

[0125] After receiving the VR video stream of the second perspective at the first resolution, the VR device compares it with the timestamp of the currently playing picture to determine whether the timestamps of the two are synchronized. Optionally, the synchronization means that the timestamp of the received VR video stream of the second perspective at the first resolution includes the timestamp of the currently playing picture. That is, if the timestamp of the received VR video stream of the second perspective at the first resolution includes the timestamp of the currently playing picture, it means that the received VR video stream of the second perspective at the first resolution has not expired; if the timestamp of the received VR video stream of the second perspective at the first resolution does not include the timestamp of the currently playing picture, it means that the received VR video stream of the second perspective at the first resolution may have expired, and the VR device may discard the expired VR video stream.

[0126] In step 509 , the VR device uses the VR video stream of the second perspective at the first resolution to play the image of the second perspective.

[0127] If the timestamps are synchronized, the VR device uses the VR video stream of the second perspective at the first resolution to play the image of the second perspective.

[0128] In step 510 , the VR device uses a full-view VR video stream with a second resolution to play the second-view image.

[0129] If the timestamps are not synchronized, the VR device still uses the second-resolution full-view VR video stream to play the second-view picture until the timestamps are synchronized, and then uses the first-resolution second-view VR video stream to play the second-view picture.

[0130] Optionally, if the timestamp is not synchronized, the VR device sends a synchronization request to the server, which is used to request the server to feedback a VR video stream that is synchronized with the timestamp of the currently playing picture. After receiving the synchronization request sent by the VR device, the server sends to the VR device a VR video stream with a first resolution and a second perspective that the VR device needs after a period of time. For example, the server can move back a period of time based on the timestamp of the originally sent VR video stream from the second perspective to obtain a new timestamp, and start sending a VR video stream with a first resolution and a second perspective to the VR device from the new timestamp. For another example, the synchronization request sent by the VR device to the server can carry the timestamp of the currently playing picture. The server can predict a new timestamp based on the timestamp of the currently playing picture and the time required for data transmission and encoding and decoding, and start sending a VR video stream with a first resolution and a second perspective to the VR device from the new timestamp. Of course, in other possible implementations, when the server sends the second-perspective VR video stream at the first resolution to the VR device in step 507, it may start sending the second-perspective VR video stream at the first resolution to the VR device, starting from the timestamp of the currently sent full-perspective VR video stream at the second resolution, or from a new timestamp obtained after a certain period of time after the timestamp. This approach can expedite the timestamp synchronization process, preventing the VR device from playing the second-resolution image for a prolonged period due to long-term timestamp desynchronization, thereby affecting the user's visual experience.

[0131] Furthermore, given that users' viewing angles may be constantly changing while watching VR videos, the viewing angle information fed back to the server by the VR device may not keep up with the actual viewing angle. Therefore, the server can also implement a corresponding viewing angle prediction mechanism. This mechanism predicts the actual viewing angle based on the viewing angle information fed back by the VR device, and then feeds back the video data of the desired viewing angle to the VR device, achieving the goal of viewing angle synchronization.

[0132] In the solution provided in the embodiment of the present application, the VR device is also used to detect whether the received VR video stream of the first resolution and the second perspective is synchronized with the timestamp of the current playback screen, ensuring smooth and accurate playback of the VR video to the user.

[0133] In the above method embodiment, the technical solution provided by this application is described only from the perspective of the interaction between the VR device and the server. The above steps on the VR device side can be independently implemented as a video playback method on the VR device side, and the steps on the server side can be independently implemented as a video playback method on the server side.

[0134] It is understandable that in order to implement the above functions, the device (such as VR device, server) includes a hardware structure and / or software module (or unit) corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiments disclosed in this application, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiments of the present application.

[0135] The embodiment of the present application can divide the device (such as VR device, server) into functional units according to the above method example. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0136] In the case of an integrated unit, Figure 6A A possible structural diagram of the device involved in the above embodiment is shown. The device 600 includes: a processing unit 602 and a communication unit 603. The processing unit 602 is used to control and manage the actions of the device 600. For example, when the device 600 is a VR device, the processing unit 602 is used to support the device 600 to execute Figure 3 Steps 302 to 304, step 306, and Figure 5A Steps 501, 503 to 506, and 508 to 510 in the embodiment of the present invention, and / or other steps for performing the technology described herein. When the device 600 is a server, the processing unit 602 is used to support the device 600 to perform Figure 3 Steps 301 and 305 in , and Figure 5A The communication unit 603 is used to support communication between the device 600 and other devices. The device 600 may also include a storage unit 601 for storing program codes and data of the device 600.

[0137] Among them, the processing unit 502 can be a processor or controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute the various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and so on. The communication unit 603 can be a communication interface, a transceiver, a transceiver circuit, etc., wherein the communication interface is a general term and can include one or more interfaces, such as an interface between a server and a VR device. The storage unit 601 can be a memory.

[0138] When the processing unit 602 is a processor, the communication unit 603 is a communication interface, and the storage unit 601 is a memory, the device involved in the embodiment of the present application can be Figure 6B Device shown.

[0139] See Figure 6B As shown, the device 610 includes: a processor 612, a communication interface 613, and a memory 611. Optionally, the device 610 may further include a bus 614. The communication interface 613, the processor 612, and the memory 611 may be interconnected via the bus 614; the bus 614 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 614 may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 6B Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0140] above Figure 6A or Figure 6B The device shown can be a server or a VR device.

[0141] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules (or units), and the software module (or unit) can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a device. Of course, the processor and the storage medium can also be present in a device as discrete components.

[0142] Those skilled in the art will appreciate that, in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. The embodiments of the present application also provide a computer program product that, when executed, is used to implement the above functions. In addition, the above computer program can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general-purpose or special-purpose computer can access.

[0143] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above description is only a specific implementation method of the embodiments of the present application and is not intended to limit the scope of protection of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the scope of protection of the embodiments of the present application.

Claims

1. A video playback method, characterized in that: The method comprises: The virtual reality VR device receives a VR video stream of a first perspective with a first resolution sent by a server and a full-perspective VR video stream of a second resolution sent by the server, wherein the first resolution is greater than the second resolution; The VR device uses the VR video stream of the first perspective at the first resolution to play the picture of the first perspective; When the VR device determines that the viewing angle has changed, the VR device uses the first one or both of the full-view VR video stream with the second resolution and the first-view VR video stream with the first resolution to play the picture of the current viewing angle; In the case where the VR device uses the full-view VR video stream of the second resolution to play the picture of the current view, before playing the picture of the current view, the method further includes: The VR device decodes the full-view VR video stream of the second resolution to obtain a full-view picture; The VR device extracts the picture of the current perspective from the full-perspective picture.

2. The method according to claim 1, characterized in that The method further comprises: The VR device sends viewing angle information to the server, wherein the viewing angle information carries a parameter for indicating a second viewing angle to be changed to, the current viewing angle is the viewing angle at the moment when the VR device reports the viewing angle information, and the current viewing angle includes the second viewing angle; The VR device receives the VR video stream of the second perspective at the first resolution sent by the server; The VR device uses the VR video stream of the second perspective at the first resolution to play the picture of the second perspective.

3. The method according to claim 1, characterized in that In the case where the VR device uses the full-view VR video stream of the second resolution and the first-view VR video stream of the first resolution to play the picture of the current view, before playing the picture of the current view, the method further includes: The VR device decodes the VR video stream of the first perspective at the first resolution to obtain a picture of the first perspective; The VR device decodes the full-view VR video stream of the second resolution to obtain a full-view picture; The VR device extracts an overlapping picture from the picture of the first perspective, and extracts a non-overlapping picture from the full-perspective picture, wherein the overlapping picture refers to a picture of an overlapping area between the picture of the first perspective and the picture of the current perspective, and the non-overlapping picture refers to a picture of the picture of the current perspective except the overlapping area; The VR device combines the overlapping images and the non-overlapping images to obtain the image of the current viewing angle.

4. The method according to claim 2, characterized in that: After the VR device receives the VR video stream of the second perspective at the first resolution sent by the server, the method further includes: The VR device detects whether the received VR video stream of the second perspective at the first resolution is synchronized with the timestamp of the current playback picture, wherein the synchronization means that the timestamp of the received VR video stream of the second perspective at the first resolution includes the timestamp of the current playback picture; If the received VR video stream of the second perspective at the first resolution is synchronized with the timestamp of the currently played picture, the VR device executes the step of using the VR video stream of the second perspective at the first resolution to play the picture of the second perspective.

5. The method according to claim 4, characterized in that After the VR device detects whether the received VR video stream of the second perspective at the first resolution is synchronized with the timestamp of the current playback picture, the method further includes: If the received second-perspective VR video stream of the first resolution is not synchronized with the timestamp of the currently played picture, the VR device uses the first or both of the full-perspective VR video stream of the second resolution and the first-perspective VR video stream of the first resolution to play the picture of the second perspective.

6. The method according to any one of claims 1 to 5, characterized in that: After the VR device receives the VR video stream of the first perspective with the first resolution sent by the server and the VR video stream of the full perspective with the second resolution sent by the server, the method further includes: The VR device stores the first-resolution first-perspective VR video stream in a first video buffer, and stores the second-resolution full-perspective VR video stream in a second video buffer.

7. The method according to claim 6, characterized in that The method further comprises: When the VR device determines that the viewing angle has changed, or when the VR device uses the VR video stream of the second viewing angle at the first resolution to play the image of the second viewing angle, the VR device performs data aging processing on the VR video stream of the first viewing angle at the first resolution cached in the first video buffer.

8. A video playback method, characterized in that: The method comprises: The server sends a VR video stream of a first perspective at a first resolution to a virtual reality VR device, and continuously sends a full-perspective VR video stream of a second resolution to the VR device, wherein the first resolution is greater than the second resolution, and the full-perspective VR video stream of the second resolution is used for the VR device to decode and obtain a full-perspective picture, and the full-perspective picture is used for the VR device to extract and play a picture of a current perspective; The server receives viewing angle information sent by the VR device, wherein the viewing angle information carries parameters for indicating a second viewing angle to be changed to, the current viewing angle is the viewing angle at the moment when the VR device reports the viewing angle information, and the current viewing angle includes the second viewing angle.

9. The method according to claim 8, characterized in that The method further comprises: The server sends a VR video stream of the second perspective at the first resolution to the VR device according to the parameters.

10. The method according to claim 8, characterized in that The full-view VR video stream of the second resolution is converted from the full-view VR video stream of the first resolution.

11. A virtual reality VR device, characterized in that: The VR device includes: a communication unit and a processing unit; The communication unit is configured to receive a VR video stream of a first perspective at a first resolution sent by a server and a full-perspective VR video stream of a second resolution continuously sent by the server, wherein the first resolution is greater than the second resolution; The processing unit is configured to use the VR video stream of the first perspective at the first resolution to play the picture of the first perspective; The processing unit is further configured to, when the VR device determines that the viewing angle has changed, use the first one or both of the full-view VR video stream of the second resolution and the first-view VR video stream of the first resolution to play the picture of the current viewing angle; The processing unit is further used to decode the full-view VR video stream with the second resolution to obtain a full-view picture when the full-view VR video stream with the second resolution is used to play the picture of the current view, and extract the picture of the current view from the full-view picture.

12. The VR device according to claim 11, characterized in that: The communication unit is further used to send viewing angle information to the server, wherein the viewing angle information carries a parameter for indicating a second viewing angle to be changed to, the current viewing angle is the viewing angle at the moment when the VR device reports the viewing angle information, and the current viewing angle includes the second viewing angle; The communication unit is further configured to receive a VR video stream of the second perspective at the first resolution sent by the server; The processing unit is further configured to use the VR video stream of the second perspective at the first resolution to play the image of the second perspective.

13. The VR device according to claim 11, characterized in that: The processing unit is further used for: When the full-view VR video stream of the second resolution and the first-view VR video stream of the first resolution are used to play the picture of the current view, the first-view VR video stream of the first resolution is decoded to obtain the picture of the first view; Decoding the full-view VR video stream of the second resolution to obtain a full-view picture; Extracting overlapping pictures from the pictures of the first viewing angle, and extracting non-overlapping pictures from the pictures of the full viewing angle, wherein the overlapping pictures refer to pictures of overlapping areas of the pictures of the first viewing angle and the pictures of the current viewing angle, and the non-overlapping pictures refer to pictures of the pictures of the current viewing angle except the overlapping areas; The overlapping picture and the non-overlapping picture are combined to obtain the picture of the current viewing angle.

14. The VR device according to claim 12, characterized in that: The processing unit is further used to detect whether the received VR video stream of the second perspective at the first resolution is synchronized with the timestamp of the current playback picture, wherein the synchronization means that the timestamp of the received VR video stream of the second perspective at the first resolution includes the timestamp of the current playback picture; The processing unit is further used to execute the step of using the second perspective VR video stream with the first resolution to play the picture of the second perspective if the received VR video stream with the second perspective of the first resolution is synchronized with the timestamp of the currently played picture.

15. The VR device according to claim 14, characterized in that: The processing unit is also used to use the first or both of the full-view VR video stream with the second resolution and the first-view VR video stream with the first resolution to play the picture of the second perspective if the received VR video stream with the second perspective of the first resolution is not synchronized with the timestamp of the currently played picture.

16. The VR device according to any one of claims 11 to 15, characterized in that: The VR device further includes: a storage unit; The storage unit is used to store the VR video stream of the first perspective with the first resolution in the first video buffer, and store the VR video stream of the full perspective with the second resolution in the second video buffer.

17. The VR device according to claim 16, characterized in that: The processing unit is further used to perform data aging processing on the VR video stream of the first perspective with the first resolution cached in the first video buffer when it is determined that the perspective has changed, or when the VR video stream of the second perspective with the first resolution is used to play the picture of the second perspective.

18. A server, characterized in that: The server comprises: a communication unit and a processing unit; The communication unit is used to send a VR video stream of a first perspective with a first resolution to a virtual reality VR device, and continuously send a full-perspective VR video stream of a second resolution to the VR device, wherein the first resolution is greater than the second resolution, and the full-perspective VR video stream of the second resolution is used for the VR device to decode to obtain a full-perspective picture, and the full-perspective picture is used for the VR device to extract and play a picture of a current perspective; The communication unit is further used to receive viewing angle information sent by the VR device, the viewing angle information carries parameters for indicating a second viewing angle to which the VR device changes, the current viewing angle is the viewing angle at the moment when the VR device reports the viewing angle information, and the current viewing angle includes the second viewing angle.

19. The server according to claim 18, characterized in that The processing unit is further configured to obtain a VR video stream of a second perspective at the first resolution according to the parameter; The communication unit is also used to send a VR video stream of the second perspective at the first resolution to the VR device.

20. The server according to claim 18, characterized in that The full-view VR video stream of the second resolution is converted from the full-view VR video stream of the first resolution.

21. A video playback system, characterized in that: The system includes: a VR device and a server; The VR device is a VR device as claimed in any one of claims 11 to 17; The server is a server as claimed in any one of claims 18 to 20.

22. A computer storage medium, characterized in that The computer storage medium stores a computer program, which can be executed by a processor to implement the method according to any one of claims 1 to 7.

23. A computer storage medium, characterized in that The computer storage medium stores a computer program, which can be executed by a processor to implement the method according to any one of claims 8 or 10.

24. A video playback method, characterized in that: The method comprises: The first device receives a full-view video stream with a second resolution sent by a server and a first-view video stream with a first resolution sent by the server, where the first resolution is greater than the second resolution; The first device uses the first-viewing angle video stream of the first resolution to play the first-viewing angle picture; When the first device determines that the viewing angle has changed, the first device uses the first one or both of the full-view video stream with the second resolution and the first-view video stream with the first resolution to play the picture of the current viewing angle; In the case where the first device uses the full-view video stream of the second resolution to play the picture of the current view, before playing the picture of the current view, the method further includes: Decoding the full-view video stream of the second resolution to obtain a full-view picture; The first device extracts a picture of a current perspective from the full-perspective picture.

25. The method according to claim 24, characterized in that The method further comprises: The first device sends viewing angle information to the server, where the viewing angle information carries a parameter for indicating a second viewing angle to be changed to, the current viewing angle is the viewing angle at the moment when the first device reports the viewing angle information, and the current viewing angle includes the second viewing angle; The first device receives a video stream of a second perspective with the first resolution sent by the server; The first device uses the second perspective video stream of the first resolution to play the second perspective picture.

26. The method according to claim 24, characterized in that In the case where the first device uses the full-view video stream of the second resolution and the first-view video stream of the first resolution to play the picture of the current view, before playing the picture of the current view, the method further includes: The first device decodes the video stream of the first perspective at the first resolution to obtain a picture of the first perspective; The first device decodes the full-view video stream of the second resolution to obtain a full-view picture; The first device extracts overlapping pictures from the pictures of the first viewing angle, and extracts non-overlapping pictures from the pictures of the full viewing angle, wherein the overlapping pictures refer to pictures of overlapping areas of the pictures of the first viewing angle and the pictures of the current viewing angle, and the non-overlapping pictures refer to pictures of the pictures of the current viewing angle except the overlapping areas; The first device combines the overlapping pictures and the non-overlapping pictures to obtain the picture of the current viewing angle.

27. The method according to claim 25, characterized in that After the first device receives the video stream of the second perspective with the first resolution sent by the server, the method further includes: The first device detects whether the received video stream of the second perspective at the first resolution is synchronized with the timestamp of the current playback picture, wherein the synchronization means that the timestamp of the received video stream of the second perspective at the first resolution includes the timestamp of the current playback picture; If the received video stream of the second perspective with the first resolution is synchronized with the timestamp of the currently played picture, the first device executes the step of using the video stream of the second perspective with the first resolution to play the picture of the second perspective.

28. The method according to claim 27, characterized in that After the first device detects whether the received video stream of the second perspective at the first resolution is synchronized with the timestamp of the current playback picture, the method further includes: If the received second-perspective video stream of the first resolution is not synchronized with the timestamp of the currently played picture, the first device uses first or both of the full-perspective video stream of the second resolution and the first-perspective video stream of the first resolution to play the picture of the second perspective.

29. The method according to any one of claims 24 to 28, characterized in that After the first device receives the full-view video stream with the second resolution sent by the server and the first-view video stream with the first resolution sent by the server, the method further includes: The first device stores the first-perspective video stream of the first resolution in a first video buffer, and stores the full-perspective video stream of the second resolution in a second video buffer.

30. The method according to claim 29, characterized in that The method further comprises: When the first device determines that the viewing angle has changed, or when the first device uses the second viewing angle video stream with the first resolution to play the second viewing angle image, the first device performs data aging processing on the first viewing angle video stream with the first resolution cached in the first video buffer.

31. A video playback method, characterized in that: The method comprises: The server sends a video stream of a first perspective at a first resolution to a first device, and continuously sends a full-perspective video stream of a second resolution to the first device, wherein the first resolution is greater than the second resolution, and the full-perspective video stream of the second resolution is used by the first device to decode to obtain a full-perspective picture, and the full-perspective picture is used by the first device to extract and play a picture of a current perspective; The server receives the viewing angle information sent by the first device, the viewing angle information carries a parameter for indicating a second viewing angle to be changed to, the current viewing angle is the viewing angle at the moment when the first device reports the viewing angle information, and the current viewing angle includes the second viewing angle.

32. The method according to claim 31, characterized in that The method further comprises: The server sends the video stream of the second perspective at the first resolution to the first device according to the parameters.

33. The method according to claim 31, characterized in that The full-view video stream of the second resolution is obtained by converting the full-view video stream of the first resolution.

34. A video playback device, characterized in that: The device comprises: a communication unit and a processing unit; The communication unit is configured to receive a video stream of a first perspective with a first resolution sent by a server and a full-perspective video stream of a second resolution continuously sent by the server, wherein the first resolution is greater than the second resolution; The processing unit is configured to use the video stream of the first perspective at the first resolution to play the picture of the first perspective; The processing unit is further configured to, when the device determines that the viewing angle has changed, use the first one or both of the full-view video stream of the second resolution and the first-view video stream of the first resolution to play the picture of the current viewing angle; The processing unit is also used to decode the full-view video stream of the second resolution to obtain a full-view picture when using the full-view video stream of the second resolution to play the picture of the current view, and extract the picture of the current view from the full-view picture.

35. The device according to claim 34, characterized in that The communication unit is further used to send viewing angle information to the server, wherein the viewing angle information carries a parameter for indicating a second viewing angle to be changed to, the current viewing angle is the viewing angle at the moment when the video playback device reports the viewing angle information, and the current viewing angle includes the second viewing angle; The communication unit is further configured to receive a video stream of the second viewing angle at the first resolution sent by the server; The processing unit is further configured to use the video stream of the second perspective at the first resolution to play the picture of the second perspective.

36. The device according to claim 34, characterized in that The processing unit is further used for: When the full-view video stream of the second resolution and the first-view video stream of the first resolution are used to play the picture of the current view, the first-view video stream of the first resolution is decoded to obtain the picture of the first view; Decoding the full-view video stream of the second resolution to obtain a full-view picture; Extracting overlapping pictures from the pictures of the first viewing angle, and extracting non-overlapping pictures from the pictures of the full viewing angle, wherein the overlapping pictures refer to pictures of overlapping areas of the pictures of the first viewing angle and the pictures of the current viewing angle, and the non-overlapping pictures refer to pictures of the pictures of the current viewing angle except the overlapping areas; Combining the overlapping pictures with the non-overlapping pictures to obtain the picture of the current viewing angle; Play the picture of the current perspective.

37. The device according to claim 35, characterized in that The processing unit is further configured to detect whether the received video stream of the second perspective at the first resolution is synchronized with the timestamp of the current playback picture, wherein the synchronization means that the timestamp of the received video stream of the second perspective at the first resolution includes the timestamp of the current playback picture; The processing unit is further configured to execute the step of using the second perspective video stream with the first resolution to play the picture of the second perspective if the received video stream of the second perspective with the first resolution is synchronized with the timestamp of the currently played picture.

38. The device according to claim 37, characterized in that The processing unit is also used to use the first or both of the full-perspective video stream at the second resolution and the first-perspective video stream at the first resolution to play the picture of the second perspective if the received video stream of the second perspective at the first resolution is not synchronized with the timestamp of the currently played picture.

39. Apparatus according to any one of claims 34 to 38, characterised in that The device further includes: a storage unit; The storage unit is used to store the first-resolution first-perspective video stream in a first video buffer, and store the second-resolution full-perspective video stream in a second video buffer.

40. The device according to claim 39, characterized in that The processing unit is further used to perform data aging processing on the video stream of the first perspective with the first resolution cached in the first video buffer when it is determined that the perspective has changed, or when the video stream of the second perspective with the first resolution is used to play the picture of the second perspective.

41. A server, characterized in that: The server comprises: a communication unit and a processing unit; The communication unit is used to send a video stream of a first perspective with a first resolution to a first device, and continuously send a full-perspective video stream with a second resolution to the first device, wherein the first resolution is greater than the second resolution, and the full-perspective video stream with the second resolution is used by the first device to decode to obtain a full-perspective picture, and the full-perspective picture is used by the first device to extract and play a picture of a current perspective; The communication unit is also used to receive perspective information sent by the first device, the perspective information carries parameters for indicating the change to a second perspective, the current perspective is the perspective at the moment when the first device reports the perspective information, and the current perspective includes the second perspective.

42. The server according to claim 41, characterized in that The processing unit is further configured to obtain a video stream of the second viewing angle at the first resolution according to the parameter; The communication unit is further configured to send a video stream of the second perspective at the first resolution to the first device.

43. The server according to claim 41, characterized in that The full-view video stream of the second resolution is obtained by converting the full-view video stream of the first resolution.

44. A video playback system, characterized in that: The system comprises: a device and a server; The device is a device as claimed in any one of claims 34 to 40; The server is a server as described in any one of claims 41 to 43.

45. A computer storage medium, characterized in that The computer storage medium stores a computer program, which can be executed by a processor to implement the method according to any one of claims 24 to 30.

46. ​​A computer storage medium, characterized in that The computer storage medium stores a computer program, which can be executed by a processor to implement the method as claimed in any one of claims 31 or 33.

Images