The invention discloses a moving object detection method with background reconstruction based on neighborhood correlation, which comprises the following steps of: inputting an image sequence, and sequencing data; dividing gray scale stable region classes; calculating the occurrence frequency of each gray scale stable region class; dividing background unstable areas, and determining a candidate background for pixel points; determining a background of pixel points; and detecting a moving object. The invention has the advantages that the amount of calculation is less; a model is not required for the background and objects in a scene, and condition assumption is not required for the background; the background can be reconstructed from a scene image with a moving prospect, and thus, a mixing phenomenon can be avoided effectively; a satisfied result can be obtained in a large range of parameter variation; a background can be reconstructed accurately for an area of which the background does not occur in the maximum frequency; and the robustness is good. The invention has wide application potential in the field of real-time systems, such as machine vision, video monitor, military science, urban traffic monitoring, resident routine safety monitoring, and the like.