Image-Text Mutual Retrieval Model Training with LSTM Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately performing image-text mutual retrieval, which is essential for scenarios where correlations between text and image data need to be leveraged for efficient data retrieval.
Innovation Solution
The proposed solution involves training an image-text mutual retrieval model using a method that includes acquiring training data pairs consisting of text and image data, extracting text and image coding features using multi-layer LSTM networks and image coding modules, respectively, and adjusting model parameters based on a calculated training loss to achieve convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-layer LSTM networks are used to extract text coding features, then the accuracy of text feature extraction is improved, but the device complexity increases
Solution Approach 1:
The text coding module is segmented into multiple LSTM network layers, where each layer processes text features at different levels of abstraction. The first LSTM network layer extracts basic text features, while subsequent layers capture more complex semantic patterns, enabling progressive feature refinement without requiring a single overly complex model
Solution Approach 2:
The patent implements a nested structure where multiple LSTM networks are arranged in sequence, with each network processing the output of the previous layer. This nested arrangement allows the model to capture text features at multiple hierarchical levels, from word-level representations to sentence-level semantics, thereby improving extraction accuracy through layered processing
2Quantity of substance
If training data pairs with long text data are used, then the comprehensiveness of text information is improved, but the processing time increases
Solution Approach 1:
The text processing is segmented into multiple stages corresponding to different LSTM network layers. Each layer processes portions of the text data independently, allowing parallel computation and reducing the sequential processing time while maintaining comprehensive text information extraction across all layers
Solution Approach 2:
The model performs preliminary feature extraction at earlier LSTM layers before processing more complex semantic patterns in subsequent layers. This staged approach allows the system to quickly capture basic text information while progressively refining it, reducing overall processing time compared to processing all text information simultaneously
Data Source
AI summary
Disclosed in the present disclosure are an image-text mutual retrieval model training method and apparatus, an image-text mutual retrieval method, and a device, applied to the technical field of retrieval. The image-text mutual retrieval model training method includes: acquiring training data pairs; inputting the training data pair in an initial model, and extracting text coding features of text training data and image coding features of image training data by using a text coding module and an image coding module in the initial model, respectively; calculating a training loss based on the text coding features and the image coding features, and performing parameter adjustment on the initial model based on the training loss; and in response to the training loss meeting a convergence condition, determining the initial model after the parameter adjustment as an image-text mutual retrieval model.