The invention discloses pearl real-time detection and grading system based on machine vision which includes: computer vision recognition component which is equipped under the charging conduit of pearl charging device, and pearl sorting device is equipped under the computer vision recognition component. Using the computer vision technology, collecting image of sequential free-falling pearl surface, sending the collected image to DSP and computers and other hardware apparatus to dealing, then distinguishing the pearl classification standard instantaneously through the dealing results, last sending the distinguishing information to sorting device to achieving classification of pearl with different characteristics automatically using devices. It can get pearl whole surface image, can complete the testing of whole apparent quality index such as pearl size, shape, smooth, defect and color, etc. simultaneously, achieve the rapid real-time detection and grading. The system can adapt to different classification speed, and the structure is simple, adaptability is strong.