The invention discloses a pumped storage unit fault diagnosis method based on multi-
modal data fusion, and the method comprises the steps: collecting a voiceprint
signal, an
infrared thermal imaging image and historical operation data of a pumped storage unit, and carrying out the
noise reduction of a voiceprint set through employing an improved
unscented Kalman filtering algorithm, a COMRes + model is used to
train the voiceprint
signal after
noise reduction and historical operation data, a future voiceprint
signal is predicted, an improved Deeplabv3 + model is used to
train an image set, and features of an
infrared thermal imaging image are extracted; the voiceprint feature, the historical operation data text feature and the image feature are input into an improved CentraNet model for multi-
modal data fusion, and a fault category is output through a classification recognition module, so that diagnosis of the pumped storage unit fault is completed; according to the method, through fusion of the multi-
modal data and optimization of the
deep learning model, the accuracy and efficiency of fault diagnosis can be effectively improved, and a powerful guarantee is provided for
safe operation of the pumped storage unit.