The invention discloses a multi-modal data-oriented vehicle insurance fraud behavior prediction system, method and device, and the method comprises the steps: extracting risk factors from picture data, combining the risk factors with corresponding structured data fields, constructing a vehicle insurance fraud risk prediction model based on the algorithms of feature engineering, machine learning, deep learning and the like, and predicting the fraud behavior of the vehicle insurance. And carrying out early warning on a risk behavior. After prediction, risk assessment and importance ranking are carried out on the picture factors, and visual expression is carried out on the factors with high risks and high weights. According to the method, manual risk assessment can be effectively assisted, and visual causal relationship expression of the model and the prediction result is realized by using data of different types of pictures. According to the method, a computer vision algorithm is utilized to perform factor extraction on some picture data difficult to use, and a prediction model and a result are visually displayed by means of factors analysis, causal inference and other algorithms.