The invention provides a coupling permeation efficiency optimization system of a near field scanning optical microscope (NSOM) and an optimization method thereof, relating to the field of scanning probe microtechnique. The optimization system comprises a computer, a detector, a collection mirror, a probe, a focusing mirror, a spatial light modulation instrument and a driver, wherein the computer,the detector, the spatial light modulation instrument and the driver form a closed loop circuit. In the system, the spatial light modulation instrument is introduced to modulate the light field distribution of an incident light field, so that the modulated light field is coupled with the probe, thereby having highest coupling permeation efficiency, and being incapable of reducing other performances of a microscope system.