The invention provides a robot casting grinding self-adaption method based on a vision system. The robot casting grinding self-adaption method includes the following steps that casting surface morphology characteristics are extracted through machine vision; by means of a method for depicting the casting surface roughness, the first-moment characteristic, the second-moment characteristic, the third-moment characteristic and the fourth-moment characteristic of the casting height represent the casting surface average deviation degree, the standard deviation, the twisting degree and the kurtosis correspondingly; according to the casting surface roughness, by combining the vision measuring and positioning method, the kinematics analysis method, the force control method and the position control servo method, vision self-adaption robot grinding parameters are generated and a grinding execution element is controlled to conduct grinding; and in the grinding process, the workpiece surface real-time dynamic morphology characteristic parameter is compared with target morphology characteristic parameter in real time till the workpiece surface roughness reaches the target morphology characteristic parameter. According to the robot casting grinding self-adaption method, casting surface morphology characteristics can be more comprehensively and more finely depicted through the workpiece surface quality obtained by depicting the casting surface morphology characteristics through moments.