The invention provides a device for realizing high-sensitivity distributed sound wave sensing based on a small-diameter multimode optical fiber. The device comprises the components of a light source first optical fiber coupler, a pulse modulator, an optical amplifier, a frequency shifter, an optical circulator, a single-mode optical fiber, a fusion part, a small-diameter multimode optical fiber, a second optical fiber coupler, a double-balance detector, and an information acquisition and processing unit. After amplification by means of the optical amplifier, a signal is input into the small-diameter multimode optical fiber through the optical circulator. Backward Rayleigh scattering light in the small-diameter multimode optical fiber is transmitted back through the optical circulator, thereby realizing heterodyne beat with the other path of intrinsic light which is shifted by the frequency shifter. A beat frequency optical signal is converted to an electric signal through the detector, and then data signal acquisition and processing are performed. The small-diameter multimode optical fiber according to the invention can greatly improve sensing signal strength and greatly improves sound wave sensing sensitivity. A distributed sound wave sensing system based on the small-diameter multimode optical fiber can realize real-time high-sensitivity monitoring and positioning of a sound wave signal in the range of an ultra-long-distance sensing optical fiber.