Domain generalization open vocabulary object detection method and device based on cross-modal alignment invariance learning

Abstract

The application provides a domain generalization open vocabulary object detection method and device based on cross-modal alignment invariance learning, comprising the following steps: defining a domain generalization open vocabulary object detection task, which requires the model to realize robust object detection on unseen categories without target domain supervision in the test stage; when the visual appearance changes, the reliable image-text correlation cannot be maintained, the application proposes a cross-modal alignment invariance learning mechanism, discards the traditional uniform training strategy, and introduces a multi-level difficulty-reliability curriculum learning framework; on this basis, an adaptive pseudo-word prototype is constructed, which is dynamically optimized through sample confidence and visual consistency double constraints to strengthen the stability of modal alignment in the cross-domain scene; and systematic experimental verification is carried out on the real distribution offset dataset. The application establishes the first systematic domain generalization open vocabulary object detection benchmark, and indicates that the cross-modal alignment invariance is the key to realizing robust open vocabulary generalization.

Images