The present invention discloses a computer vision based mobile terminal application testing system and method. The system comprises an image script test module and an automatic test module. The image script test module obtains screen images and screen information simultaneously, reads a template image in a script and information in the template image, and executes a template matching algorithm; if the match is successful, an image matching result is returned, and if the match fails, a feature point matching algorithm is executed; and if the match is successful, the image matching result is returned, and if the matching fails, failure codes are returned. The automatic test module obtains an original image, calculates a saliency grayscale image of the original image by using a saliency detection algorithm, converts the saliency image into a binary image, and applies a random algorithm or a K-means algorithm to the saliency binary image to determine a saliency point set to automatically finish clicking and recording. The system and the method disclosed by the present invention improve test efficiency, are applicable to a wide variety of mobile devices, and greatly reduce the learning cost and provide test convenience for a tester by using two manners of a given script and full-automation.