The invention discloses a
fire smoke image early recognition method based on multi-
modal fusion. The method comprises the following steps: S1, collecting multi-
modal data and carrying out
time alignment; s2, constructing a visible light
branch, an
infrared branch and a photoacoustic
branch, generating multi-
modal features, and storing the multi-modal features in a shared feature
pool; s3, embedding a space-time tracking module in the shared feature
pool, and generating global feature representation; s4, generating a dynamic background template by adopting a low-rank
matrix decomposition algorithm, and separating and removing
background noise; s5, multi-modal features are fused in the shared feature
pool through a lightweight self-attention mechanism, and missing features are compensated; s6, enhancing an early
smoke signal through a sparse feature enhancement
algorithm, and performing fire state classification and
smoke diffusion path prediction by applying a multi-task classification network; and S7, updating the shared feature pool and the space-time tracking module through a dynamic feedback mechanism. According to the invention, early-stage accurate identification of
fire smoke is realized by using multi-modal fusion, space-time tracking and dynamic feedback methods.