A fine-grained clothing retrieval method based on CNN-Transformer dual-flow network

Abstract

The application discloses a fine-grained clothing retrieval method based on a CNN-Transformer double-flow network, which comprises the following steps: first, inputting an image into a CNN network to extract features; mapping clothing attributes into feature vectors, and then guiding an attention module to extract coarse-grained image features related to the attributes, wherein the generated attention matrix is used for positioning local areas related to the attributes in the image; inputting the local areas in the image and the clothing attributes into a Transformer network to extract fine-grained image features; finally, fusing the features of the two branches to obtain a robust expression of the clothing image and performing clothing retrieval by using similarity calculation; and introducing a Dilated-Transformer variant on the basis of the original Transformer to reduce the model calculation amount and accelerate the training and reasoning speed. The application uses a novel CNN-Transformer double-flow structure, utilizes the complementarity of the two network structures to perform coarse-to-fine feature representation on the clothing image, and finally fuses the coarse-grained and fine-grained features to realize high-performance retrieval.

Images