The invention discloses a video-based agricultural product source tracing method and a video-based agricultural product source tracing system. The method includes the following steps: A, mounting a camera in the production, processing and dispatching environment of agricultural products, and shooting the production, processing and dispatching processes of agricultural products through the camera; B, storing the video shot by the camera in a data base, and recording the batch and shooting time of agricultural products related to the video; C, receiving the history video query requests from users, wherein the history video query requests include the batch and shooting time of agricultural products; and D, retrieving the corresponding video from the data base and displaying and representing the video to users as per the history video query requests from users. According to the invention, the video monitoring is introduced in the agricultural product source tracing system, and is combined with the conventional agricultural product source tracing manner, so that video dimensionality information of the agricultural product source tracing information can be provided, as a result, the integrity, intuitive performance and facticity of the agricultural product source tracing information are realized, and the agricultural product source tracing information is vivider.