The invention discloses an abnormal behavior detection method based on a space-time automatic encoder, and belongs to the technical field of image processing and mode recognition. The method comprisesthe steps of firstly obtaining a video image of a to-be-detected area; i; inputting the optical flow information and the RGB information of the video into a space-time auto-encoder respectively; p; passing through a 3D convolution layer in an automatic encoder; a; a pooling layer, an LSTM layer and a deconvolution layer, DERIVING RECONSTRUCTION INFORMATION, T, the method comprises the steps thatinput information is compared with reconstructed information, abnormal scores are obtained according to an abnormal scoring formula, t, then the two abnormal scores are fused to obtain a comprehensiveabnormal score, t, the comprehensive abnormal score is compared with a threshold value, whether abnormal behaviors exist or not and the occurrence time of abnormal conditions are judged, and the higher the abnormal score is, t, the higher the probability of occurrence of the abnormal conditions is. According to the invention, t, through the space-time automatic encoder fusing multi-modal input information, abnormal behaviors in crowds are detected in public areas such as banks, and early warning is sent to security personnel, so that emergencies such as major abnormal events are reduced.