The invention discloses a vision-based human body state judgment algorithm, which belongs to the technical field of computer vision, and comprises the following specific steps of moving human body detection, feature extraction and description, behavior recognition and feature extraction and description: S1, obtaining an original image, and obtaining a foreground binary image of the original imagethrough a motion detection algorithm to serve as an optical flow calculation region; s2, performing black edge filling on the optical flow calculation area to obtain a square optical flow graph, dividing the square optical flow graph into 16 equal parts by using a 4 * 4 grid, taking an optical flow average value of each area, and finally representing an optical flow vector of the frame of pictureby using a 16-dimensional vector; assuming that the silhouette feature vector obtained from the frame is s and the optical flow feature vector is 0, the fusion feature V of the two is equal to [s; 0].The method is simple in structure and convenient to use, improves the accuracy of behavior recognition, reduces the complexity and feature dimension of feature extraction through a quick feature fusion method, is high in practicality, and is suitable for popularization and application.