Group behavior identification method based on multi-modal information fusion and decision optimization

Abstract

The invention discloses a group behavior recognition method based on multi-modal information fusion and decision optimization, and the method comprises the steps: firstly obtaining a group member candidate box sequence for a to-be-recognized video, extracting the corresponding optical flow features, and extracting the human body posture segmentation features as a third visual clue; then acquiringa double-flow model of the spatial and temporal features of the human body target and performing multi-modal information fusion (MMF) on the double-flow model; and finally, connecting the two branchesobtained after MMF fusion with a GRU, and performing decision optimization by adopting a multi-classifier fusion method based on adaptive category weight, thereby obtaining a group behavior label. According to the scheme of the invention, during feature fusion, an MMF feature fusion algorithm is designed, so that space-time features supplement each other, information supplements each other, and finally better feature representation is obtained; in the aspect of decision optimization, a multi-classification fusion method based on self-adaptive class weights is designed, classifier acceptance and rejection and each class weight are calculated more accurately, and therefore high recognition precision is obtained.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a group behavior recognition method based on multimodal information fusion and decision optimization, so as to realize group behavior recognition in video sequences. Background technique [0002] In recent years, human action recognition in videos has achieved remarkable achievements in the field of computer vision. Human behavior recognition has also been widely used in real life, such as intelligent video surveillance, abnormal event detection, sports analysis, understanding social behavior, etc. These applications make group behavior recognition have important scientific practicability and huge economic value. As deep learning has gradually achieved great success in the field of computer vision, various neural network architectures have gradually been applied to video-based human behavior recognition, and have achieved remarkable results. [0003] The inve...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the shortcomings of the existing group behavior recognition methods that consider few factors, are easy to misjudgment, and affect the recognition accuracy, a group behavior recognition method based on multi-modal information fusion and decision-making optimization is proposed, which can achieve higher recognition precision

Images