Dual-channel-based remote sensing image open vocabulary directed target detection method

Abstract

The invention provides a dual-channel-based remote sensing image open vocabulary directed target detection method. Firstly, multi-modal features of an input image are extracted by freezing an image encoder and a text encoder at the same time, the strong characterization capability of a pre-training model on remote sensing image features is reserved to the maximum extent, and knowledge loss caused by task data deviation is avoided; secondly, a multi-scale feature extraction network and a category-independent directed candidate box generation network are designed, the multi-scale feature extraction network can relieve the problem of scale difference generally existing in remote sensing images, and the category-independent directed candidate box generation network ensures the generalization ability of the network to a target category through category-independent candidate box learning; and finally, a dual-channel detection network is constructed to realize dual decoupling of the base class and the target class in a feature level and a classification task, and feature learning confusion caused by feature sharing of the base class and the target class is avoided. According to the method, high base class detection precision can be kept while an excellent remote sensing image new class detection effect is achieved.

Images