The invention discloses a panorama video automatic stitching method based on SURF feature tracking matching. The method comprises the following steps: a first frame distinguishing type processing method, the first frame serving to select a video reference coordinate system, directly performing video fusion on successive frames; in accordance with the first frame of a video to be stitched, extracting SURF feature points, generating feature point description vectors; based on the Hash mapping and a barrel storage method, searching for similar SURF feature vector point symmetries, constituting a similar feature set; using the vector point symmetries in the similar SURF feature set, resolving an optimal data relational degree coordinate system; conducting dynamic weighted summation on coordinate-converted pixel values of the video frames to be stitched, implementing seamless stitching fusion of the video. According to the invention, the method can implements omnibearing, multi-visual angle, and stereoscopic video seamless stitching fusion, and not only overcomes traces, fuzziness and double image brought by image translation, rotation, zooming and radiation transformation, and also increases efficiency and accuracy in conducting image calibration based on feature matching.