The invention discloses a failure mode recognition method of an epicyclic gearbox using a mixed domain feature vector and grey correlation analysis. The method comprises the steps that (1) the feature parameters ft, ff, fs and ftf of the time domain, the frequency domain, the order domain and the time-frequency domain are extracted to constitute the mixed domain feature vector Fm, and the weighted values of wt, wf, ws and wtf of all the feature parameters are calculated; (2) the vibration monitoring health state of the epicyclic gearbox and the historical data of various failure mode states are acquired, and the corresponding mixed domain feature vector value mj(k) is calculated to be used as a reference state matrix; (3) the feature vector t of a signal to be detected and the row vector of the reference state matrix m are both used as the input of a grey correlation analysis algorithm; (4) the correlation vector of the feature vector t of the signal to be detected and the standard mode matrix m is calculated to be used as an output vector of the grey correlation analysis algorithm. The failure mode recognition method of the epicyclic gearbox using the mixed domain feature vector and grey correlation analysis has the advantages of being simple in principle, simple and convenient to operate, stable and reliable, high in precision, suitable for high interference and varied working conditions and the like.