Text-guided person re-identification model construction method and device based on confidence perception, and retrieval method

Abstract

The present application relates to the field of computer vision and multi-modal retrieval technology, and particularly relates to a text-guided person re-identification model construction method and retrieval method and device based on confidence perception. The model construction method comprises: obtaining a text-image pair training sample composed of a person image and a corresponding text description; constructing a text-guided person re-identification baseline model comprising an image encoder, a text encoder and a cross-modal alignment module; fitting a Gaussian mixture model according to the single-sample cross-modal alignment loss of each text-image pair sample in a batch to obtain a sample confidence weight, and weighting the cross-modal alignment loss based on the confidence weight; performing image block division and random masking on the input image, and combining the text features to perform cross-modal reconstruction on the masked image blocks; jointly optimizing the model according to the weighted cross-modal alignment loss, the reconstruction loss and the baseline loss to obtain a trained text-guided person re-identification model. The retrieval method comprises: using the trained text-guided person re-identification model to respectively encode the query text and the candidate person image, calculating the similarity and outputting the retrieval result. The present application can reduce the interference of noisy text-image pair samples on model training, enhance the fine-grained alignment capability between text semantics and image local regions, and thus improve the accuracy and robustness of text-guided person re-identification.

Images