Cross-Media Target Search with Unified Image-Text Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in effectively cross-media retrieval, particularly in matching image and text data due to the complexity and variability in representing objects across different media types, leading to difficulties in accurately identifying and retrieving target objects.
Innovation Solution
A cross-media feature extraction network is employed to project image and text features into a common feature space using pre-trained neural networks, incorporating graph attention mechanisms and adversarial learning to enhance feature matching and discrimination, enabling accurate retrieval of target objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image-text matching methods are used, then the system can process media data, but the accuracy of cross-media retrieval is insufficient due to complexity and variability in representing objects across different media types
Solution Approach 1:
The patent transforms image and text features into a unified feature space by changing the parameter representation of different media types. The cross-media feature extraction network projects image features and text features into the same dimensional space, enabling direct comparison and matching while reducing representation complexity.
Solution Approach 2:
The patent introduces a cross-media feature extraction network as an intermediary that bridges image and text data. This network contains an image processing branch and a text processing branch that both output to a common feature space, acting as a mediator that reconciles the differences between media types.
2Productivity
If simple feature matching is used, then the processing is fast, but the comprehensiveness of cross-media retrieval is insufficient
Solution Approach 1:
The patent segments the feature extraction process into distinct branches for images and text, with each branch specialized for its media type. The image processing branch extracts visual features while the text processing branch extracts semantic features, allowing comprehensive feature capture while maintaining efficient parallel processing.
Solution Approach 2:
The patent creates a composite feature representation by combining multiple feature types (image features, text features) into a unified feature space. This composite approach enables comprehensive retrieval by considering multiple media types simultaneously while maintaining processing efficiency through vector operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed in embodiments of the present disclosure are a method and apparatus for searching for a target. A specific implementation mode of the method comprises: obtaining at least one image and a description text of a specified object; extracting image features of the image and text features of the description text by using a pre-trained cross-media feature extraction network; and matching the image features and the text features to determine an image that contains the specified object. Thus, features are extracted by using cross-media features, and the image features and the text features are projected to an image and text common feature space for feature matching, thereby achieving cross-media target search.