The invention relates to a compressing three-dimension calculation ghost imaging system. The system comprises a light source, a spatial light modulator, at least four sets of convergence light receiving lenses, at least four sets of point detectors corresponding to the convergence light receiving lenses and an algorithm module. Light emitted by the light source is projected on the spatial light modulator, the spatial light modulator modulates the light randomly, the modulated light is projected on an object, the object reflects the light in different directions, and one set of convergence light receiving lens and point detector are arranged in each of at least four reflecting directions. Total light intensity in all the directions are compressed and sampled by the point detectors, a compressing and sampling result is input into the algorithm module, the processes of the compression and sampling of the total light intensity and the input of the result are repeated for many times, the spatial light modulator modulates different patterns every time, the algorithm module inverts a two-dimension image corresponding to the direction of each point detector by the application of the compressed sensing algorithm according to a measurement matrix and the measurement results obtained by the repeated compression and sampling, the information of shadow parts of the images is compared to construct a three-dimension surface gradient, and finally a three-dimension object shape is reconstructed.