The invention discloses a method and device for calculating a sub pixel motion vector based on an H264 protocol. The device is used for calculating the sub pixel motion vector of a macro block, and comprises a buffer unit, a main control unit, a reference frame reuse and maintenance unit, an SAD cost calculation unit and an SAD cost selection unit. In the process of calculating the sub pixel motion vector of the H264 protocol, an original block pixel is obtained based on the size of a 16 x 16 block, so that the reusability of original block pixels of a 8 x 8 block and a 4 x 4 block can be guaranteed; and when acquiring a reference block pixel, a corresponding relationship between integer pixel motion vectors is recorded according to the reference frame reuse and maintenance unit, reference block pixels of different sub-blocks of the integer pixel motion vectors are acquired selectively, and reference block pixels of the same integer pixel motion vector are not acquired repeatedly, so that the reusability of a reference pixel can be greatly improved, the calculation efficiency of the sub pixel motion vector can be improved, the power consumption can be saved, and the hardware cost can also be saved.