VR online watching method and device based on SUVC hierarchical coding

Abstract

The invention discloses a VR online watching method and device based on SUVC hierarchical coding, and belongs to the field of video processing, and the method comprises the steps: dividing a two-dimensional video into a plurality of window video blocks according to a VR window; carrying out SUVC layered coding on the divided window video; according to the current VR viewpoint information, determining an FOV area of the current video; transmitting the base layer videos of all the windows, and decoding to obtain a high-definition SDR full-breadth video; according to the FOV, decoding to obtain a high-definition HDR picture and a 4K HDR picture; and performing back projection transformation on the decoded video, transforming a two-dimensional video into a three-dimensional spherical video, clipping the video according to the optimal viewing angle, and finally playing the clipped video. According to the invention, network bandwidth resources can be effectively utilized, and the VR online watching experience is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and more specifically, to a VR online viewing method and device based on SUVC layered coding. Background Art

[0002] In a stereoscopic viewing scenario such as VR video, the user watches a portion of the video in the middle, which is called the FOV viewing angle. The area is about 120°, and the optimal viewing area is 90°. Therefore, even if the video itself is 8K, the pixels in the user's viewing area may only be 2K. Therefore, in order to ensure the viewing experience of VR videos, 8K or even higher resolution videos are often required to support it. At the same time, the industry believes that 90° FOV is considered to be the passing line for VR immersive experience, and 120° FOV is the standard for achieving partial immersive experience. The larger the FOV, the more immersive the picture is, but the bandwidth and computing costs will be higher.

[0003] Limited by the current network bandwidth and the computing power of video processing chips, VR videos usually transmit a high bitrate to decode the FOV perspective content for users to watch, and then transmit a low bitrate to decode the low-resolution full-face and render the entire sphere to prevent black fields when users turn their perspectives, and then decode the high bitrate to update the screen according to the user's new perspective. The disadvantage of this method is that the video source needs to be encoded with both low and high bitrates, and low bitrate videos may have blurred images and unnatural colors, which may affect the immersive feeling of the picture. At the same time, the high bitrate and low bitrate have exactly the same picture content, and there is repeated data encoding, resulting in a waste of calculation and bandwidth. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a VR online viewing method and device based on SUVC layered coding, which can effectively utilize network bandwidth resources and significantly improve the VR online viewing experience.

[0005] The object of the present invention is achieved through the following solutions:

[0006] A VR online viewing method based on SUVC layered coding comprises the following steps:

[0007] S1, stitching video sources into spherical video;

[0008] S2, performs projection transformation operation on the spherical video to convert it into a two-dimensional video;

[0009] S3, dividing the two-dimensional video into multiple window video blocks according to the VR window;

[0010] S4, SUVC layered encoding is performed on the divided window video. Each video block is divided into a base layer, an enhancement layer 1 and an enhancement layer 2 using a transform. The base layer is decoded to obtain a high-definition SDR picture, the base layer and the enhancement layer 1 are decoded to obtain a high-definition HDR picture, and the base layer, the enhancement layer 1 and the enhancement layer 2 are decoded to obtain a 4K HDR picture;

[0011] S5, confirming the FOV area of ​​the current video according to the current VR viewpoint information;

[0012] S6, transmits the basic layer video of all windows and decodes it to obtain high-definition SDR full-frame video;

[0013] S7, according to the FOV, requesting the enhanced layer 1 data for the picture window between the first angle and the second angle, and decoding to obtain a high-definition HDR picture; according to the FOV, requesting the enhanced layer 1 data and the enhanced layer 2 data for the picture window of the content seen by the visual center, and decoding to obtain a 4K HDR picture;

[0014] S8, performing back-projection transformation on the decoded video, transforming the two-dimensional video into a three-dimensional spherical video, then cropping the video according to the best viewing angle, and finally playing the cropped video.

[0015] Further, in step S1, the video source includes a 4K video source.

[0016] Furthermore, in step S1, the video source is obtained by shooting ultra-high-definition video with multiple cameras.

[0017] Further, in step S4, the transformation specifically includes DWT transformation.

[0018] Further, in step S7, the first angle is 90°.

[0019] Further, in step S7, the second angle is 120°.

[0020] A VR online viewing device based on SUVC layered coding includes: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, any of the methods described above is executed.

[0021] The beneficial effects of the present invention include:

[0022] (1) The method of the present invention saves more bandwidth by reducing the resolution instead of the bit rate.

[0023] (2) The method of the present invention only needs to encode the original video once, and then obtain videos of different definition by decoding the basic layer, the basic layer and the enhancement layer 1, the basic layer, the enhancement layer 1 and the enhancement layer 2. Compared with encoding and transmitting two-way videos of high and low bit streams at the same time, there is no repeated calculation and repeated transmission, and the resource utilization rate is higher.

[0024] (3) The method of the present invention obtains videos of different clarity through SUVC layered decoding for different FOV areas. Compared with decoding the entire FOV area into a high bit rate, the pressure on bandwidth and calculation is smaller. Compared with decoding the peripheral vision area into a low bit rate video for the FOV area, it ensures that the peripheral vision area and the best visible area are consistent in brightness and color, and the picture immersion is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 This is a schematic diagram of the FOV viewing angle of traditional spherical video;

[0027] Figure 2 is a flow chart of a method according to an embodiment of the present invention;

[0028] Figure 3 It is a schematic diagram of dividing a two-dimensional video into multiple window video blocks according to a VR window in a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0030] In order to solve the problem of improving the VR video viewing experience, based on the existing research technology foundation (SUVC, Scalable UHD Video Coding, 4K ultra-high-definition images can be layered encoded into a base layer, enhancement layer 1, and enhancement layer 2 through DWT transformation), the present invention provides a VR online viewing method based on SUVC layered coding, such as Figure 2 As shown, the specific steps include:

[0031] S1, stitching video sources into a spherical video; specifically, the video source may be obtained by shooting ultra-high-definition videos with multiple cameras, including a 4K video source, and the 4K video source is stitched into a spherical video;

[0032] S2, performs projection transformation operation on the spherical video to convert it into a two-dimensional video;

[0033] S3, divides the two-dimensional video into multiple window video blocks according to the VR window, such as Figure 3 As shown;

[0034] S4, SUVC layered encoding is performed on the divided window video, each video block is divided into a base layer, an enhancement layer 1 and an enhancement layer 2 using a transform, the base layer is decoded to obtain a high-definition SDR picture, the base layer and the enhancement layer 1 are decoded to obtain a high-definition HDR picture, and the base layer, the enhancement layer 1 and the enhancement layer 2 are decoded to obtain a 4K HDR picture; optionally, the transform specifically includes a DWT transform. Compared with SDR, HDR can provide a wider color space, greater color depth and higher brightness.

[0035] S5, confirming the FOV area of ​​the current video according to the current VR viewpoint information;

[0036] S6 transmits the basic layer video of all windows and decodes it to obtain high-definition SDR full-frame video, which can prevent black fields from appearing when the user rotates the viewing angle;

[0037] S7, based on the FOV, request the enhanced layer 1 data for the picture window between the first angle and the second angle, and decode to obtain a high-definition HDR picture; based on the FOV, request the enhanced layer 1 data and the enhanced layer 2 data for the picture window of the content seen by the center of vision, and decode to obtain a 4K HDR picture, which can ensure that the video content in the area seen by the peripheral vision of the human eye and the best visible area are consistent in brightness and color, and improve the picture immersion; in a preferred manner, the first angle is 90° and the second angle is 120°.

[0038] S8, performing back-projection transformation on the decoded video, transforming the two-dimensional video into a three-dimensional spherical video, then cropping the video according to the best viewing angle, and finally playing the cropped video.

[0039] In other embodiments, the present invention provides a VR online viewing device based on SUVC layered coding, including: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method described in the above embodiment is executed.

[0040] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0041] According to one aspect of an embodiment of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations.

[0042] As another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

Claims

1. A VR online viewing method based on SUVC layered coding, characterized in that: The steps include: S1, stitching video sources into spherical video; S2, performs projection transformation operation on the spherical video to convert it into a two-dimensional video; S3, dividing the two-dimensional video into multiple window video blocks according to the VR window; S4, SUVC layered encoding is performed on the divided window video. Each video block is divided into a base layer, an enhancement layer 1 and an enhancement layer 2 using a transform. The base layer is decoded to obtain a high-definition SDR picture, the base layer and the enhancement layer 1 are decoded to obtain a high-definition HDR picture, and the base layer, the enhancement layer 1 and the enhancement layer 2 are decoded to obtain a 4K HDR picture; S5, confirming the FOV area of ​​the current video according to the current VR viewpoint information; S6, transmits the basic layer video of all windows and decodes it to obtain high-definition SDR full-frame video; S7, according to the FOV, requesting the enhanced layer 1 data for the picture window between the first angle and the second angle, and decoding to obtain a high-definition HDR picture; according to the FOV, requesting the enhanced layer 1 data and the enhanced layer 2 data for the picture window of the content seen by the visual center, and decoding to obtain a 4K HDR picture; S8, performing back-projection transformation on the decoded video, transforming the two-dimensional video into a three-dimensional spherical video, then cropping the video according to the best viewing angle, and finally playing the cropped video.

2. The VR online viewing method based on SUVC layered coding according to claim 1 is characterized in that: In step S1, the video source includes a 4K video source.

3. The VR online viewing method based on SUVC layered coding according to claim 1 is characterized in that: In step S1, the video source is obtained by shooting ultra-high-definition video with multiple cameras.

4. The VR online viewing method based on SUVC layered coding according to claim 1 is characterized in that: In step S4, the transformation specifically includes DWT transformation.

5. The VR online viewing method based on SUVC layered coding according to claim 1 is characterized in that: In step S7, the first angle is 90°.

6. The VR online viewing method based on SUVC layered coding according to claim 1, characterized in that: In step S7, the second angle is 120°.

7. A VR online viewing device based on SUVC layered coding, characterized in that: include: A processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to any one of claims 1 to 6 is executed.

Images