A fine-grained hand-drawn sketch image retrieval method based on enhanced attention
An attention, fine-grained technology, applied in the field of cross-media retrieval, which can solve the problems of inconsistent target objects and low image accuracy.
Active Publication Date: 2022-03-18
FUDAN UNIV
View PDF3 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The problems solved by the present invention include: the accuracy rate of images retrieved by the existing hand-drawn sketch image retrieval model in the hand-drawn sketch image retrieval results is low; the retrieved image is inconsistent with the target object in the query sketch; Research
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0051] The specific implementation details of the present invention will be introduced in detail below.
[0052] (1) Image semantic feature extraction
[0053] A CNN is used to extract visual features of each hand-drawn sketch and color image. Compared with traditional feature extraction methods, CNN has more powerful feature learning and extraction capabilities; ResNet network with attention mechanism is used as image semantic extractor, and the output features of the last layer represent visual global features. Thus, for each input image, the network outputs its global visual feature representation.
[0054] In the present invention, for the input hand-drawn sketches and color images, the corresponding modal network branches are used to extract semantic features, that is, for the input query sketches, the sketch network branches are used to extract the semantic features of the sketches; Color images, using image network branches to extract semantic features of images.
[...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention belongs to the technical field of cross-media retrieval, in particular to a fine-grained hand-drawn sketch image retrieval method based on enhanced attention. The present invention proposes a deep fine-grained hand-drawn sketch image retrieval FG-SBIR model, which uses different attention mechanisms to further focus on the fine-grained details between the sketch and the image. The new model not only focuses on the correlation information between the two modalities of sketch and image, but also focuses on the discriminative information within a single modality. The present invention proposes a mutual loss method to enhance the traditional triplet loss and improve the model's discriminative ability of fine-grained features within a single modality. For a given query sketch, the present invention can return its related images with fine-grained instance-level similarity in a specific category, meeting the strict requirements of instance-level retrieval for fine-grained hand-drawn sketch image retrieval.
Description
technical field [0001] The invention belongs to the technical field of cross-media retrieval, and in particular relates to a fine-grained hand-drawn sketch image retrieval method based on enhanced attention. Background technique [0002] With the ubiquity of mobile devices these days, it's becoming ever more convenient for people to sketch on-screen. Especially when searching for a specific image, people need to give a long textual description, but in fact they can also draw a sketch containing fine-grained visual features to express the retrieval needs. Therefore, Fine-grained Sketch-based Image Retrieval (FG-SBIR) has aroused extensive research interest of scholars. In recent years, although fruitful research results have been obtained in this field, it still faces many challenging problems, and more in-depth research work is needed to obtain better solutions. These challenging issues include, how to reduce the semantic difference between sketches and images from differe...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/53G06N3/04
CPCG06F16/53G06N3/045
Inventor 张玥杰王燕飞
Owner FUDAN UNIV