The invention discloses a group behavior recognition method based on multi-modal information fusion and decision optimization, and the method comprises the steps: firstly obtaining a group member candidate box sequence for a to-be-recognized video, extracting the corresponding optical flow features, and extracting the human body posture segmentation features as a third visual clue; then acquiringa double-flow model of the spatial and temporal features of the human body target and performing multi-modal information fusion (MMF) on the double-flow model; and finally, connecting the two branchesobtained after MMF fusion with a GRU, and performing decision optimization by adopting a multi-classifier fusion method based on adaptive category weight, thereby obtaining a group behavior label. According to the scheme of the invention, during feature fusion, an MMF feature fusion algorithm is designed, so that space-time features supplement each other, information supplements each other, and finally better feature representation is obtained; in the aspect of decision optimization, a multi-classification fusion method based on self-adaptive class weights is designed, classifier acceptance and rejection and each class weight are calculated more accurately, and therefore high recognition precision is obtained.