Multi-feature fusion behavior identification method based on key frame

Abstract

A multi-feature fusion behavior identification method based on a key frame comprises the following steps of firstly, extracting a joint point feature vector x (i) of a human body in a video through anopenpose human body posture extraction library to form a sequence S = {x (1), x (2),..., x (N)}; secondly, using a K-means algorithm to obtain K final clustering centers c '= {c' | i = 1, 2,..., K},extracting a frame closest to each clustering center as a key frame of the video, and obtaining a key frame sequence F = {Fii | i = 1, 2,..., K}; and then obtaining the RGB information, optical flow information and skeleton information of the key frame, processing the information, and then inputting the processed information into a double-flow convolutional network model to obtain the higher-levelfeature expression of the RGB information and the optical flow information, and inputting the skeleton information into a space-time diagram convolutional network model to construct the space-time diagram expression features of the skeleton; and then fusing the softmax output results of the network to obtain a final identification result. According to the process, the influences, such as the timeconsumption, accuracy reduction, etc., caused by redundant frames can be well avoided, and then the information in the video can be better utilized to express the behaviors, so that the recognition accuracy is further improved.

Description

technical field [0001] The invention belongs to the technical fields of computer graphics and human-computer interaction, and in particular relates to a multi-feature fusion behavior recognition method based on key frames of human motion sequences. Background technique [0002] Vision is the most important medium for information transmission in human activities. Studies have found that about 80% of information is obtained through vision. In recent years, with the development of computer technology, especially the rapid popularization of the Internet, the subject of computer vision has become one of the most active and popular subjects in the computer field. Computer Vision (Computer Vision) refers to the machine vision that uses cameras and computers to simulate human vision to identify, track, measure, etc. targets, and further image processing through recognition and analysis. Human action recognition, as an emerging research field in computer vision, has been extensively...

AI Technical Summary

Problems solved by technology

However, with the continuous improvement of video complexity, a single video feature can no longer accurately express the required video information.

Moreover, with the continuous increase of the amount of video data and information, in the process of behavior recognition, we miss important target information due to the existence of redundant data, and the detection of huge amounts of data one by one is contrary to Principles of Video Analysis and Difficult to Implement

Images