The invention provides an early friction instability fault identification method for a sleeve tooth connection structure. Firstly, a critical rotating speed of a rotor system with a sleeve tooth connection structure is determined, the critical rotating speed of the rotor system can be calculated through adopting a finite element method, and a sleeve tooth friction early-stage instability fault characteristic frequency band is determined. In actual operation, a first-order critical rotating speed of the rotor system can be obtained through adopting an experimental method or a commercial software calculation method. Vibration signals of the rotor system are collected, the rotating speed is collected, whether the rotating speed is slightly higher than the first-order critical rotating speed or not is further judged, vibration signals in the characteristic frequency band are collected, the sleeve tooth friction early-stage instability fault characteristic value is calculated, and whether the sleeve tooth friction early-stage instability fault occurs or not is judged according to the characteristic value. According to the early friction instability fault identification method, heavy loss caused by instability faults can be effectively avoided, the identification process is simple, the operand is small, the response time is at the millisecond level under a mainstream computing platform, the response is rapid, the method can be completely expanded and applied to airborne equipment, and the method has high engineering application value.