The invention discloses a sampling device for water quality detection and a using method of the sampling device. The sampling device is a portable or on-line sampling and separating system capable oftaking out a plurality of samples from a water body to be monitored in situ by utilizing the principle of a liquid communicating vessel,a main body of the system consists of a filter, a flow control valve, a first three-way ball valve, a portable sample separation pipe, a second three-way ball valve and an online sample separation pipe, and a portable colorimetric component or an online sample preparation component is combined to finish the sampling, sample preparation and detection tasks together. The device is in modular design, realizes the instrument design concept of separating the sampletaking and matching from detection and analysis, and is convenient for the daily maintenance and troubleshooting. On one device, the manual portable sampling and automatic online sampling modes are realized at the same time, the spectrophotometry principle can be used, and the water quality state can be conveniently detected by adopting the electrochemical sensor principle. According to the present invention, the automatic cleaning step can be set, and the memory effect of two times of detection is reduced; a portable or on-line sampling mode can be selected and applied according to the use condition, and the multi-purpose, multi-mode and multi-parameter detection requirements are met.