The invention discloses a hand movement tracking system and a hand movement tracking method. The invention comprises an attitude and heading reference system based on an accelerometer, a gyroscope and a magnetic sensor, and a hand movement tracking method based on the attitude and heading reference system. The hand movement tracking method comprises the following steps: firstly, obtaining a triaxial acceleration measured by the accelerometer, a triaxial angular velocity measured by the gyroscope and a triaxial magnetic-field component measured by the magnetic sensor, performing error compensation on the magnetic sensor by adopting a least square method to establish an error model after an upper computer receives sensor data, eliminating high-frequency noise of the triaxial acceleration by virtue of a window low-pass filter, and establishing an error model for the gyroscope so as to perform error compensation on random drift of the gyroscope; secondly, effectively integrating the gyroscope, the accelerometer and the magnetic sensor by virtue of an improved adaptive complementary filtering algorithm to obtain an attitude angle and a path angle; and finally, performing gravity compensation and discrete digital integration on acceleration signals to obtain a velocity and a track of a hand movement. The tracking system and the tracking method disclosed by the invention can be applied to a man-machine interactive system, is convenient to operate, and is strong in experience feeling.