Sketch image retrieval method based on collaborative attention

Abstract

The invention discloses a sketch image retrieval method based on collaborative attention. The sketch image retrieval method comprises the following steps: constructing a three-branch network consisting of a natural image branch, an edge image branch and a sketch branch, and performing pre-training; constructing attention modules of a three-branch network, wherein the weights of the attention modules of the natural image branches and the edge graph branches are heterogeneous, and the attention modules of the edge graph branches and sketch branches share the weight; learning a common attention mask on two different data domains to capture a common channel-level dependency relationship between the two domains, learning common information features from the two different domains in a focused mode, and reducing the difference between a query data domain and a retrieval data domain; respectively carrying out channel corresponding weighting on the natural image and the output feature map of the last pooling layer of the edge image branch through a cross-domain public attention mask, and outputting a natural image feature map and an edge feature map after re-calibrating the importance degree of a feature channel; and performing joint training on the three-branch network to obtain cross-domain representation of the sketch and the natural image.

Description

technical field [0001] The invention relates to the technical fields of image retrieval and computer vision, in particular to a sketch image retrieval method based on collaborative attention. Background technique [0002] With the sharp increase of Internet media image data, content-based image retrieval technology has become a hot topic in the field of computer vision. Early text-based retrieval methods rely on manual annotation and are ambiguous, so they cannot be applied to large-scale image retrieval. In recent years, with the popularity of touch-screen devices, Sketch-based Image Retrieval (SBIR) has attracted extensive attention and achieved remarkable performance. [0003] In recent years, convolutional neural networks have been widely used in many fields such as face recognition, object detection, and image retrieval. Compared with traditional methods of handcrafting features, convolutional neural networks can automatically aggregate shallow features learned from b...

AI Technical Summary

Problems solved by technology

Early text-based retrieval methods relied on manual annotation and were ambiguous, making them unsuitable for large-scale image retrieval

Images