360° Video Streaming System Based on Saliency Detection

Abstract

The invention discloses a 360° video stream transmission system based on saliency detection, comprising: a server end, in the offline stage of the system, obtains a saliency matrix according to eye movement data and calculates the variance of each video, that is, the saliency matrix The average variance of the video can be divided into two types: salient video and non-salient video; after the acquisition and analysis of the saliency matrix, the video can be divided into different quality and corresponding forms by segmenting in time and space dimensions Exist; the client, when the user watches the salient video, downloads the corresponding saliency matrix from the server to the client player according to the actual network conditions, and the saliency-based rate control algorithm controls the download and transmission of the video stream during the playback process , for non-salient video, the quality of the video stream is adjusted through the MPC algorithm. The invention effectively alleviates the disadvantages of too small cache length due to the non-linearity of the user's viewing behavior and the limitation of the prediction window.

Description

technical field [0001] The invention belongs to the field of network systems, and in particular relates to a 360° video stream transmission system based on saliency detection. Background technique [0002] With the increasing popularity of 360° videos, virtual reality (Virtual Reality, VR) and augmented reality (Augmented Reality, AR) technologies have begun to receive attention from all walks of life. In recent years, with the maturity of VR equipment, network and terminal technologies, 360° videos are increasingly used in applications such as film and television, games, medical care, and education. According to Cisco's statistical report, the average monthly traffic of VR applications reached 13.3PB in 2016, and is expected to reach 140PB by 2021, with an annual growth rate of 60%. Such a wide range of 360° videos has brought new experiences to people's life and work, and a large number of network infrastructure and smart wearable devices have also promoted the developmen...

AI Technical Summary

Problems solved by technology

However, the 360° video stream transmission scheme based on head movement trajectory prediction has certain disadvantages due to the nonlinearity of user viewing behavior and the limitation of the prediction window, which makes the buffer length too small.

Images