The invention discloses a fish school photoacoustic
feature recognition method based on a multi-
modal deep network, and the method comprises the steps: 1, obtaining data: obtaining acoustic data and visual data; 2, acoustic data preprocessing and
feature engineering; 3, visual data preprocessing and labeling alignment are carried out; 4, constructing a sample and organizing a
data set; step 5, designing a multi-mode
branch network; step 6, carrying out cross-
modal fusion and joint representation; step 7, performing multi-task output and a
loss function; and step 8, model training and evaluation. According to the invention, by fusing acoustic, optical and
environmental sensor data, an acoustic-vision-environment three-
branch feature extraction network is constructed, adaptive fusion of multi-
modal features is realized, the problems of low fish school recognition precision, difficulty in alignment and fusion of multi-
modal data and the like in a complex marine environment are effectively solved, and the accuracy of fish school recognition is improved. The method is suitable for real-time fish school monitoring and analysis tasks of unmanned ships, buoys and
fishery administration monitoring platforms.