Multiband fusion model training and identification method and device based on semantic consistency

Abstract

The embodiment of the invention discloses a multi-band fusion target recognition model training method based on semantic consistency, and the method comprises the steps: extracting a feature pyramid of each band image through a target recognition model, taking the region of interest of each band image as a constraint, carrying out the feature extraction on the feature pyramid of the corresponding band image, and obtaining a target recognition model; obtaining region-of-interest features of each wave band; optimizing the features of the region of interest of each wave band, and obtaining multi-wave band semantic consistency comparison loss of the optimized features of the region of interest of each wave band according to a multi-wave band semantic consistency comparison learning strategy; and using a target identification model to determine the fused multiband features of each feature pyramid, predicting the target contained in each band image according to the fused multiband features, and obtaining the target prediction loss according to the prediction result. And obtaining the total loss according to the multi-band semantic consistency comparison loss and the target prediction loss, performing back propagation according to the total loss, and updating the parameters of the target identification model.

Images