The invention discloses a UAV (unmanned aerial vehicle) fleet bilateral remote control system and method thereof based on vision and force sense feedback. The system comprises a master end, a slave end and a communication module, wherein the master end comprises a hand controller and a control computer, the slave end comprises a UAV virtual proxy point, a UAV fleet, a distance sensor and an image acquiring module. An operator can control all virtual structural points by the single hand controller, and the virtual structural points and the UAV adopt bidirectional tracking mechanism to control formation movement of the UAV fleet. When the UAV fleet encounters environment obstacles, force feedback signals are provided for the UAV to change posture and control change of fleet formation on one hand and for the master end so that the operator can obtain force sense immediacy sense on the other hand. Meanwhile, cameras are installed on each UAV, vision ranges of the cameras can be adjusted by a steering engine, overall view of the slave end can be provided for the operator, and finally the operator can obtain the force sense and the vision immediacy sense at the same time.