A video summarization method based on dynamic and static feature fusion
A technology of feature fusion and video summarization, applied in neural learning methods, special data processing applications, instruments, etc., can solve the problems of unable to reflect multi-level information of video content and single feature information, and achieve video summarization results improvement and generalization Good results
Active Publication Date: 2022-03-18
SHANDONG UNIV
View PDF6 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] At present, video image features extracted through deep neural network models are widely used in the field of computer vision, but most video summarization methods only extract image feature information through convolutional neural networks, resulting in feature information that is too single to reflect the richness of video content. hierarchical information
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0020] The present invention will be further described below in conjunction with accompanying drawing and example.
[0021] like figure 1 As shown, the schematic block diagram of the video summarization method based on deep learning of the present invention is as follows:
[0022] (1) Training phase: Firstly, the training data is preprocessed, the basic information of the video data is collected, and standardized operations such as sampling and clipping are performed on the video according to the requirements of the feature extraction module. Then the model is initialized, the dynamic and static features of video clips and video images are extracted respectively, and the two are fused together and then input into the importance score prediction network to calculate the error between the prediction result and the manual labeling result, and then update the parameters until Reaching the preset maximum number of iterations makes the algorithm output close to the manual labeling ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a video summarization method based on the fusion of dynamic and static features, which is used for video content understanding and processing, and extracts important parts of the original video as video summaries, so that the summarized video contains the most effective information for people and at the same time The total length of the video is shortened to save time for people to obtain effective information. In order to obtain visual features with richer hierarchical information, the method first extracts dynamic and static features from video clips and corresponding video images, then fuses the dynamic and static features, and finally sends the fused features to the video clip importance score prediction network, according to the score High and low selected partial video clips are concatenated in chronological order into the final video summary. Experiments were completed using two datasets, SumMe and TVSum, and 5-fold cross-validation was performed. Compared with the deep learning model that does not use feature fusion, the accuracy of the video summarization method proposed by the present invention is improved, which shows the effectiveness of the dynamic and static feature fusion proposed by the present invention.
Description
technical field [0001] The invention relates to a video summarization method based on fusion of dynamic and static features, belonging to the field of video understanding and processing. Background technique [0002] Video summarization is one of the basic problems in the field of computer vision. It belongs to the sub-direction of video content understanding. Its goal is to use computers to automatically extract key image frames or video clips from long videos as key summary content, so that the summarized video contains Shorten the total length of the video while providing the most effective information to people, so as to save the time for people to obtain effective information. It can be used in subsequent video classification, video retrieval, and video efficient storage and transmission technologies. [0003] Traditional video summarization methods use the underlying features of video images such as optical flow features, gray-level co-occurrence matrix features, and ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/738G06N3/04G06N3/08
CPCG06F16/739G06N3/049G06N3/08G06N3/045
Inventor 刘琚张昱刘晓玺赵雪圻张杰鲁昱顾凌晨
Owner SHANDONG UNIV