The invention relates to the technical field of
data analysis, and discloses a
data analysis method and device based on multi-
modal data fusion, equipment and a medium, and the method comprises the steps: collecting visual data, audio data and process text data, carrying out the preprocessing and
standardization processing of different types of data, constructing a multi-
modal data set with aligned timestamps, and storing the multi-
modal data set in a
database; the method comprises the following steps: extracting a time-frequency
dynamic feature and a
semantic vector feature, extracting a map structure feature, a time-frequency
dynamic feature and a
semantic vector feature, fusing the features by using a cross-modal attention mechanism to generate a fused
feature vector, finally performing analysis
processing based on the fused
feature vector, and outputting an analysis result. According to the method, the multi-
modal data set with consistent time is constructed, the structural features of various modals are extracted, and the cross-modal attention mechanism is introduced to realize deep fusion of the feature level, so that the problems of single information utilization and weak
feature relevance of the existing detection means are solved, and the comprehensiveness of defect detection and the accuracy of fault diagnosis are improved.