The invention relates to a moving workpiece recognition method based on spatiotemporal contexts and a fully convolutional network, and belongs to the fields of digital image processing and object detection and recognition. According to the method, an object image database is utilized to train the fully convolutional neural network to obtain a classifier of a to-be-classified object; then a background difference method and a morphological method of digital image processing are utilized to obtain an initial position of the object in a first frame of a video sequence, an object tracking method of spatiotemporal context models is utilized to track the to-be-tracked object according to the initial position, and object tracking precision is verified through a precision graph; and finally, the trained classifier is utilized to carry out classification recognition on a tracking result, semantic-level segmentation is realized, and thus an object category is obtained. According to the method, the initial position of the moving object can be effectively and automatically acquired by using the background difference method and the morphological method of digital image processing, tracking and recognition for the moving workpiece on a conveyor belt can be realized, and an automation degree and an intelligence degree of an industrial robot are increased.