The invention belongs to the technical field of
hydropower engineering construction safety intelligent monitoring, and particularly provides a
hydropower engineering construction potential safety
hazard image-text multi-mode intelligent identification method,
system and equipment, and the method comprises the steps: collecting
hydropower engineering construction potential safety
hazard text and image data, and building a corresponding relation between a potential
hazard text and a potential hazard image;
processing the hidden danger text by using a BERT model, extracting text features,
processing hidden danger image data by using a Vision Transform model of a multi-head self-attention mechanism, and extracting image features; fusing the extracted text features and image features through Gate Net to obtain fused feature expression; a feature with a higher level is extracted from the fused features through a Transform model; and classifying the processed features by using a multi-layer sensor, and outputting results of various hidden dangers. According to the method, association between a construction potential safety hazard text and an image is fully considered, an image-text multi-
modal joint characterization principle based on gating circulation fusion is provided, and a construction potential safety hazard multi-
modal classification identification method is provided.