The present invention relates to a face detection and tracking method based on FPGA. Firstly, video is collected, and then skin color threshold detection is performed, and the collected video data is converted from YUV format into a binary image, and then spatial filtering is performed to determine skin color pixels, and then Time filtering, to obtain the average value of the pixels of the current frame, to further determine the skin color pixels, and then to filter other skin color interference, to perform face tracking, to obtain real-time face detection results, and finally to use VGA for real-time display. The invention overcomes the cost of existing human face detection and the real-time problem of detection and tracking, the algorithm is relatively simple, the accuracy, speed and real-time performance are high, and the system cost is low.