Method for clustering videos by using brain imaging space features and bottom layer vision features

Abstract

The invention relates to a method for clustering videos by using brain functional imaging space features and bottom layer vision features. The method is characterized by comprising the following steps of: extracting a brain signal vector in a functional magnetic resonance image sequence, calculating a Pearson relevant coefficient matrix of the signal vector, extracting the brain function imaging space features from the Pearson relevant coefficient matrix by using a single-factor variance analysis and relevant feature selection method, establishing a Gaussian process regression model according to the bottom layer vision features of partial videos and the brain function imaging space features, mapping the bottom layer vision features of the rest videos into the brain function imaging space features, and performing multi-modal spectrum clustering on the brain function imaging space features and the bottom layer vision features of all the videos. By the method, the brain function imaging space features and the bottom layer vision features can be combined and clustered; and compared with the conventional video clustering method based on the bottom layer vision features such as colors and shapes as well as the conventional space clustering method by independently using the brain functional features, the method has the advantage that the clustering accuracy is greatly improved.

Description

technical field [0001] The invention belongs to image processing and application technology, in particular to a method for video clustering using brain imaging spatial features and underlying visual features, Background technique [0002] With the explosive growth of digital multimedia data, the number of videos on the network is increasing day by day. What kind of features are used to represent videos is becoming more and more important. At present, the most popular is to extract the features of video color, texture and shape. These features are collectively referred to as underlying visual features. However, these traditional video features are difficult to accurately describe the video, which brings difficulties to the subsequent video processing. In contrast, humans can know the specific content of the video being played at a glance. This phenomenon has given scientists a lot of Great inspiration, at present, some scholars have extracted relevant features from the brain...

AI Technical Summary

Problems solved by technology

However, these traditional video features are difficult to accurately describe the video, which brings difficulties to the subsequent video processing. In contrast, humans can know the specific content of the video being played at a glance. This phenomenon has given scientists a lot of Great inspiration, at present, some scholars have extracted relevant features from the brain signals collected when the testers watch the video as the features of the video, and these features are used in video classification. These features extracted from the brain signals are called High-level features, which use functional magnetic resonance imaging technology to collect brain signals and extract features from these signals are called functional brain imaging spatial features. Related research only focuses on how to extract spatial features of brain functional imaging and use this feature to perform Video classification and retrieval, these studies are still in the exploratory stage

Images