This invention discloses a two-stage unsupervised
adaptation method,
system, device, and medium for visual language models. These are corresponding solutions. The solutions include: constructing a high-quality auxiliary dataset through adaptive retrieval, achieving both cost-effectiveness and versatility, and providing an efficient and reliable new paradigm for cross-
domain knowledge transfer; furthermore, decomposing the complex
adaptation task into a two-stage optimization process, controlling the distribution of auxiliary data to be similar to the pre-training or
target distribution through adjustable data
distribution control parameters, significantly reducing the difficulty of single-step
adaptation; simultaneously, during category filtering, using weak supervision signals contained in the image-text pair data, matching frequency statistics, and predictive information entropy to filter samples of specific categories from the large-scale image-text dataset, avoiding the introduction of noisy category samples; and furthermore, based on two-stage training, the model can gradually adapt to the target task and target data distribution, improving adaptation performance and achieving higher classification accuracy.