The invention provides a multi-view naked eye 3D video conference system, comprising a server terminal and a client, wherein the server terminal and the client establish communication, the client is provided with at least one camera, a single camera is used for shooting a client user, the client is further provided with an image uploading module and an image display module, and the image display module decodes and converts a multi-view image synthesis encoding result returned by the server terminal to display a 3D naked eye video; and the server terminal is provided with at least three cameras, a plurality of cameras are used for shooting server images, the server terminal is further provided with a camera input module, a face detection module and an image processing module, and the image processing module carries out synthesis encoding on a face of the client user detected by the face detection module and a multi-view image obtained by the camera input module and transmits the same to the client. By adopting the multi-view naked eye 3D video conference system provided by the invention, the usability and the application field of multi-view scenes are greatly expanded, and meanwhile, the user experience and the performance can be improved.