The invention discloses a fixed area night fog all-weather color video imaging system and a building method thereof. The system comprises a visible light video collection system, a visible light image collection module, a background image updating module, a visible light background image bank, an infrared light video collection system, an infrared light image collection module, an image preprocessing module, an infrared light background image bank, an infrared light target detection module, an image registration module, a video fusion module, a video signal emitting module, an RTSP transmission module, and a display module. The system uses an improved algorithm integrating a mixed Gaussian model and a symmetric difference method as well as an imaging concept of fusion of the infrared video and visible light background image. After processing by a data processing center, the system can output a clear color video image at night, in fog weather or other weather with poor visibility, so as to realize the color effect of the infrared light video truly. The system is easy to realize, is low in cost, is easy to install, and realizes long-distance real-time monitoring.