The invention relates to a device for preventing faulty operation for using an accelerator as a brake by mistake based on computer vision, which comprises a vision sensor, an embedded system, a vehicle speed measuring unit, a vehicle driving angle measuring unit, a distance measuring unit, a voice alarm unit, an accelerator control unit, a brake control unit and a power supply unit, wherein the embedded system is used for reading the information of driving speed of a vehicle, a vehicle steering wheel angle, a video image of a front road of the vehicle, and the like, estimating a barrier distance on the front road and calculating a driving risk factor and driving risk change tendency according to the barrier distance, the vehicle steering wheel angle and the speed of the vehicle. The invention makes a corresponding driving decision according to the driving risk factor and the driving risk change tendency, controls the voice alarm unit to produce drive warning sound according to the driving decision, controls the accelerator control unit to allow the action effectiveness of stepping the accelerator by a driver and controls the brake control unit to control the automatic brake of the vehicle.