Video semantic scene segmentation method based on convolutional neural network

Abstract

The invention discloses a video semantic scene segmentation method based on a convolutional neural network, which is mainly divided into two parts, wherein one part is that a convolutional neural network is built on the basis of shot segmentation, and then semantic feature vectors of video key frames are obtained by using the built convolutional neural network; and the other part is that the Bhattacharyya distance between the semantic feature vectors of two shot key frames is calculated by using the time continuity of the front and back key frames according to the semantic feature vectors, andthe semantic similarity of the shot key frames is obtained through measuring the Bhattacharyya distance. Probability estimation values of different semantics are outputted by using the convolutionalneural network to act as a semantic feature vector of the frame. Considering a time sequence problem of scene partition in the continuous time, the shot similarity is compared by combining semantic features of the two shot key frames and the time sequence feature distance between the shots, and thus a final scene segmentation result is obtained. The method disclosed by the invention has certain universality and has a good scene segmentation effect under the condition that training sets are sufficient.

Description

technical field [0001] The invention belongs to the technical field of video scene segmentation in image processing and machine vision, and more specifically relates to a video semantic scene segmentation method based on a convolutional neural network. Background technique [0002] Usually, people do not understand video content from the level of video shots, but are more accustomed to understanding video content from the level of scenes. This is because a shot is only a component unit in the video structure and cannot fully express semantic information, which is likely to cause information lost. However, a scene is a collection of shots and contains a lot of semantic information, so it is more in line with people's understanding habits, and it also makes the research on video scene segmentation more realistic. [0003] Combining a series of shots with related content to describe events or activities with certain semantic information in the video is called shot clustering. ...

AI Technical Summary

Problems solved by technology

[0006] Aiming at the above defects or improvement needs of the prior art, the present invention provides a video semantic scene segmentation method based on a convolutional neural network, thereby solving the problem that the existing video scene segmentation method is prone to information loss during the process of feature extraction and fusion. The loss of the scene, and the technical problem of poor scene classification caused by not taking into account the timing problem of video scene segmentation

Images