The invention belongs to the technical field of
pilot training evaluation, and discloses a
flight simulator multi-
modal data-based
feature fusion model construction method, which specifically comprises the following steps of 1, selecting a
pilot training video, obtaining flight parameter data, and establishing a
flight simulator multi-
modal data set; 2, data preprocessing; step 3, providing a key
point sequence by the ViTPose; step 4, embedding key
point sequence features; 5, extracting flight parameter
time sequence and
pilot operation behavior
frequency domain features; step 6, carrying out cross-
modal feature fusion; and 7, performing comparative learning. According to the method, signals of different
modes in the use process of the
flight simulator are fully captured, and operation behavior characteristics in a
pilot training video and data
time sequence characteristics generated by the flight simulator are fused, so that more accurate and efficient pilot driving training behavior representation and recognition are realized; and meanwhile, the whole learning process is processed in a layered manner, and the information of different signals can be more deeply and effectively combined through the interaction effect of the mode in the main features of different stages of each layer of model, so that more advanced and abstract feature representation can be obtained in the subsequent hierarchical learning.