The invention discloses a video conference multi-
modal data alignment method and device based on a causal
mask, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: carrying out the
feature extraction and fusion of an original audio, an original video
stream and an original document in an
online video conference,
time sequence division is carried out based on the obtained multi-
modal fusion features to obtain a triple
time sequence window; determining an initial
weight value corresponding to the triple
time sequence window, and performing normalization adjustment on the initial
weight value by using a preset constraint condition to obtain an adjusted weight; indexing a preset time sequence offset matrix by using a speaking identifier of a speaking party, correcting an original time sequence of the triple time sequence window based on an indexing result, and determining a target attention result corresponding to the triple time sequence window by using a preset causal
mask mechanism, and performing multi-level alignment fusion on the multi-
modal fusion features based on the target attention result to obtain a multi-modal alignment result. The precision of the multi-mode alignment technology is improved, and future
information leakage is avoided.