Dual Gate Network for Semantic Image Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image search engines require users to input exact or pre-defined query terms to find desired images, making it challenging to locate images taken days, months, or years ago.
Innovation Solution
A method using a dual gate network that processes query text into a multi-dimensional word vector and performs weighted operations with text and visual gated networks to search for target images, allowing images to be retrieved without exact query terms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exact or pre-defined query terms are used for image search, then search precision is improved, but user operation complexity increases and search flexibility deteriorates
Solution Approach 1:
The patent replaces the mechanical keyword-matching system with a neural network-based semantic understanding system. The dual gate network (comprising text gated network and visual gated network) automatically processes natural language queries and image features to compute semantic similarity, eliminating the need for users to formulate exact query terms while maintaining high search precision through learned semantic representations
Solution Approach 2:
The patent transforms the search parameter from exact keyword matching to continuous multi-dimensional vector space comparison. By representing both queries and images as multi-dimensional vectors and computing similarity in this continuous space, the system enables flexible semantic search where users can input natural language descriptions rather than precise keywords, thereby improving ease of operation while maintaining search precision
2Measurement precision
If exact query terms are required for image search, then search accuracy is improved, but system adaptability deteriorates
Solution Approach 1:
The patent creates a universal image search system that handles multiple types of queries (exact matches, synonyms, fuzzy queries, natural language descriptions) through a single dual gate network architecture. The text gated network and visual gated network work together to process diverse input types and compute semantic similarity in a unified vector space, enabling the system to adapt to various search scenarios without requiring separate processing mechanisms for different query types
Solution Approach 2:
The patent introduces multi-dimensional vector representations as an intermediary between user queries and image database. The dual gate network transforms both queries and images into this intermediate vector space where semantic meaning is preserved and compared, allowing the system to adapt to different query formulations while maintaining consistent search accuracy through the mediating vector representation layer
Data Source
AI summary
A method for image search, an electronic device, and a non-transitory computer-readable storage medium are provided. The method includes the following. Receive a query text. Process the query text to obtain a first multi-dimensional word vector. Perform, with a text gated network in a dual gate network, a first weighted operation on the first multi-dimensional word vector to obtain a second multi-dimensional word vector. Search for at least one target image according to the second multi-dimensional word vector and a second multi-dimensional visual vector for each image in an image file. The second multi-dimensional visual vector for each image is obtained by performing, with a visual gated network in the dual gate network, a second weighted operation on a first multi-dimensional visual vector for each image, and the second multi-dimensional word vector and the second multi-dimensional visual vector for each image are in a same space.