View angle prediction method of panoramic video multicast stream based on limited view field feedback

Abstract

The invention discloses a view angle prediction method of a panoramic video multicast stream based on limited view field feedback, which comprises the following steps of: 1, constructing a significance detection model based on spherical convolution, and extracting significance features from a panoramic video by using the significance detection model; 2, constructing a memory network model based on spherical convolution, and mining causal relevance and features of user view sequences of a small number of users by using the memory network model based on spherical convolution; and 3, finally, combining the space-time saliency characteristics of the video and the visual angle characteristics of a small number of users to obtain a predicted visual angle of all the users. According to the method, the mapping distortion problem of the panoramic video is considered, and saliency detection and a small number of visual angles are combined to obtain the final predicted user visual angle, so that relatively high prediction accuracy can be realized.

Description

technical field [0001] The invention relates to the field of multimedia video transmission, in particular to a viewing angle prediction method for a panoramic video multicast stream based on limited field of view feedback. Background technique [0002] Immersive multimedia, including panoramic video and virtual / augmented reality (VR / AR) video, has recently become more popular as the demand for interactive applications rises. Panoramic video, with the help of head-mounted displays (HMDs), allows users to change the direction of view freely, providing an extraordinary immersive experience. However, there is a large gap between the bandwidth capabilities of traditional wireless technologies and the bandwidth requirements of 360-degree video streaming. Due to the limitations of the head-mounted display device HMD, the user can only see a small part of the entire video, so selecting the video area that the user is interested in for transmission can save bandwidth and computing r...

AI Technical Summary

Problems solved by technology

[0006] In order to avoid the disadvantages of the above-mentioned prior art, the present invention provides a viewing angle prediction method for panoramic video multicast streams based on limited field of view feedback, in order to combine saliency detection with a small number of viewing angles to obtain the final predicted user Viewing angle, so as to overcome the mapping distortion problem of panoramic video, and improve the accuracy of the saliency detection model and improve the accuracy of viewing angle prediction

Images