The invention discloses a polarization frequency domain optical coherence tomography system based on a single detector. The polarization frequency domain optical coherence tomography system based on the single detector comprises a low-coherence optical source, a first collimator, a polarizer, a michelson interferometer, a polarization beam splitter, a second collimator, a first optical fiber, a third collimator, a fourth collimator, a second optical fiber, a fifth collimator, a grating, a cylindrical lens and a detector. According to the invention, the single detector is adopted, so that the system structure is simplified; the problem of time asynchronism when two detectors collect signals is prevented; meanwhile, two paths of light radiates the grating at the incident angle with highest diffraction efficiency; the improvement of the diffraction efficiency increases the signal to noise ratio of the system, and the image quality is improved.