The invention provides an unmanned warehouse monitoring system, and the system comprises (1), an image collection module which is used for collecting a video monitoring image; (2), an image initial processing module which is used for the initial detection processing of a moving object of original video image sequence information, and the outputting of effective video image sequence information comprising the moving object; (3), a filtering module which is used for receiving the effective video image sequence information and carrying out the filtering of the background shape of the effective video image sequence information; (4), a background modeling module which is used for building a background model after filtering, wherein the background model is set to consist of N shape context histograms which represent edge points and have weights; (5), a background reduction module which is used for carrying out the attribute classification of the edge points of a current image frame, and reducing the edge points belonging to the background; (6), a feature matching module; (7), an alarm module. The device can reduce the wrong detection, caused by background noise and the jittering of a camera, of the moving object, and is good in instantaneity.