The invention discloses an unmanned aerial vehicle invasion detection and identification system based on a visual sense and a method thereof. The system comprises a plurality of cameras, a target detection module and a target identification module. The target identification module comprises a movement track determination device, an optical flow characteristic determination device, a zooming controller and a characteristic matcher. The cameras are deployed around a monitoring area. After the target detection module detects moving targets, through track determination and optical flow characteristic determination, preliminary screening is performed on the targets and parts of non-unmanned-aerial vehicle targets are eliminated. The cameras are controlled to zoom so as to acquire a sharp image. Through a scale-invariant characteristic transform matching algorithm, matching identification is performed on the targets. In the invention, an optical detection device is adopted, a visual sense detection and identification algorithm is used, unmanned aerial vehicle invasion in a monitoring range can be rapidly and accurately detected and identified, the system and the method are suitable for places which have an anti-unmanned-aerial-vehicle demand, such as an airport, a prison and the like, and detection and identification capabilities of unmanned aerial vehicle invasion are increased.