The invention relates to the technical field of
data analysis, can be applied to business
system platforms of financial science and technology,
medical health and the like, and discloses a
key frame feature extraction method, device, equipment and medium for multi-
modal data, and the method comprises the following steps: carrying out differentiable neural architecture search on a plurality of multi-
modal original data streams to obtain multi-
modal undetermined features; carrying out importance analysis on the multi-modal undetermined features to obtain importance scores, screening out an effective modal
feature set from the multi-modal undetermined features, carrying out dynamic sparse connection on the effective modal
feature set to obtain fused modal features, carrying out
time sequence attention
distillation on the fused modal features to obtain frame-level attention weights, and carrying out
time sequence attention
distillation on the fused modal features to obtain frame-level attention weights; and carrying out frame-level key analysis on the multi-modal
original data stream to obtain a
key frame of the multi-modal
original data stream. According to the method, the unified modeling capability of the multi-modal
time sequence is improved, dynamic adjustment of the modal weight and redundant
information compression are realized, and the overall
processing efficiency is remarkably improved while the calculation burden is reduced.