Dynamic video tactile feature extraction and rendering method

Abstract

The invention relates to a dynamic video tactile feature extraction and rendering method, and belongs to the field of virtual reality and human-computer interaction. The method comprises the steps ofdecompressing a received video, preprocessing the video, segmenting shots based on inter-frame color histogram features, extracting saliency maps fused with time-space domain tactile saliency featuresfrom all the segmented frames in each shot, and performing pixel-level tactile rendering according to the saliency maps of the video frames. According to the method, the video content is divided intothe salient region and the non-salient region by extracting the video frame salient features fused with the space-time characteristics. And pixel-level tactile stimulation is applied to the video frame by adopting a one-to-one mapping relationship between the visual channels and the tactile channels. And real-time tactile feedback is generated through the terminal, so that the realistic experience of watching the video by the user is enriched. The method can be widely applied to video education, multimedia entertainment and human-computer interaction.

Description

technical field [0001] The invention belongs to the field of virtual reality and human-computer interaction, and in particular relates to a method for dynamic video feature extraction and tactile reproduction. Background technique [0002] The development of 4G technology has made the application of video streaming grow exponentially. The advent of the 5G era will support the construction of another important human body sensory channel on the basis of the original audio and video channels: the tactile channel, making the performance and experience of human-computer interaction be enriched. Therefore, it will become an important technical difficulty to realize feature extraction and tactile rendering of video media. [0003] Existing feature extraction methods for tactile rendering usually use static images as the extraction objects, while videos are composed of image sequences, adding a dimension of time to the original two-dimensional space of images. Due to the temporal ...

AI Technical Summary

Problems solved by technology

In 2014, Kim et al. proposed a vibration-based saliency-driven video for 4D movies: tactile conversion, but the tactile feedback generated by vibration has the disadvantage of low resolution.

but it doesn't take into account the dynamic nature of the video

[0006] The Chinese patent "A Video Chatting Method and Terminal Integrating Tactile Sensing Functions" (publication number CN104717449A) discloses a method of tactile feedback based on real-time video communication for mobile terminals. Similarly, it only considers spatial intra-frame features, Dynamic features between frames are not taken into account

Images