Zero sample sketch retrieval method based on mask and matching

By using mask and matching methods in zero-sample sketch retrieval, Vision Transformer and visual-cross-language sampler for feature processing and semantic matching, the problem of semantic alignment mismatch between sketches and natural images is solved, and efficient and accurate sketch image retrieval is achieved.

CN119848283AActive Publication Date: 2025-04-18ROBOTICS RESEARCH CENTER OF YUYAO CITY +1
View PDF 6 Cites 1 Cited by

Patent Information

Application Number
CN202411815334.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-11
Publication Date
2025-04-18
Estimated Expiration
2044-12-11

AI Technical Summary

Technical Problem

The semantic alignment mismatch between sketches and natural images leads to distracting and semantic blurring of the model, making it difficult to achieve effective zero-sample sketch retrieval.

Method used

The zero-sample sketch retrieval method based on mask and matching is adopted, feature embedding is performed through the Vision Transformer encoder, semantic mask is generated using the visual-cross-language sampler, feature reconstruction is purified, and cross-domain semantic interaction is performed through the Transformer decoder, and model training is performed based on triple loss, reconstruction loss and interaction loss.

Benefits of technology

Effectively reduce semantic noise, enhance fine semantic matching between sketches and images, and improve the accuracy and efficiency of zero-sample sketch retrieval.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119848283A_ABST
    Figure CN119848283A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of zero-sample sketch retrieval, and discloses a mask and matching-based zero-sample sketch retrieval method, which comprises the following steps of: firstly, designing a vision-cross-language sampler which generates a mask by using a semantic tag, shields semantic information irrelevant to a sketch target in an image and optimizes a semantic alignment effect of cross-domain matching; then, a purification mask matching module is designed, the purification mask matching module comprises a feature reconstruction part and a semantic interaction part, redundant semantics are inhibited by forcing an image encoder to reconstruct and cover features, cross-domain interaction between sketch and image features is promoted by using a converter decoder, and thus finer semantic matching is realized. And finally, in combination with training mechanisms of triple loss, reconstruction loss and interaction loss, the retrieval accuracy of the model in the purified semantic space can be remarkably improved. According to the method, interference elements in the image are masked through the mask, and pure sketch-image matching is realized, so that the problem of semantic difference between the sketch and the natural image is effectively solved.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Sketch image retrieval method based on joint space attention and metric learning

    CN114186088A

  • Freehand sketch recognition method based on multi-level stroke sequence feature extraction

    CN117877050A

  • Zero sample reference image segmentation method based on hierarchical prompt and directional clue

    CN119049057A

  • Method for fine-grained sketch-based scene image retrieval

    US20220058429A1

  • Unified scene text detection and layout analysis

    US20240062560A1

Cited By

  • Text-driven semantic consistency zero sample sketch retrieval method

    CN122489784A