An interactive sound playback device includes two speakers, two microphones, a motion sensor, and an audio processing unit. The speakers and the microphones are disposed at two sides of the interactive sound playback device respectively. The audio processing unit is electrically connected to the speakers, the microphones, and the motion sensor, and has a recording mode and a playing mode. In the recording mode, the audio processing unit receives a motion sensing signal from the motion sensor and a first audio signal from the microphones, stores the first audio signal, and stores the motion sensing signal as position information. In the playing mode, the audio processing unit outputs the first audio signal to the speakers through a first path, or adjusts a second audio signal by referring to the motion sensing signal and the position information, and outputs the adjusted second audio signal to the two speakers through a second path.