Text-Image Feature Codebooks for Accurate Cross-Modal Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing technologies face challenges in achieving accurate and efficient data retrieval and compression, particularly in handling large volumes of data with deep learning, where user queries often yield insufficiently accurate results.
Innovation Solution
A bidirectional text-image association model is trained using image and text feature codebooks to enhance data processing, enabling efficient data compression and retrieval by learning enhanced associations between images and text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If keyword retrieval is used for data processing, then ease of operation is improved, but measurement precision deteriorates
Solution Approach 1:
The patent introduces feature codebooks as intermediary structures between keyword queries and data retrieval. The query processing unit transforms keywords into query feature codebooks, which then match with stored feature codebooks of data. This intermediary representation enables more precise matching while maintaining keyword-based ease of operation.
Solution Approach 2:
The patent transforms the retrieval process from direct keyword matching to feature codebook matching. By changing the parameter representation from raw keywords to learned feature codebooks through bidirectional training, the system achieves higher measurement precision while preserving the operational simplicity of keyword-based queries.
2Productivity
If dimensionality reduction is applied to compress data, then productivity is improved, but loss of information increases
Solution Approach 1:
The patent employs bidirectional training with feedback mechanisms where the encoder and decoder continuously adjust their parameters based on reconstruction error. The feature codebooks are trained to preserve essential information by minimizing the difference between original and reconstructed data, thus reducing information loss while achieving compression.
Solution Approach 2:
The system performs preliminary encoding of data into feature codebooks before storage or transmission. This preliminary action captures the essential characteristics of the data in a compressed form, enabling efficient storage and retrieval while preserving key information through the learned feature representations.
Data Source
AI summary
Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for data processing. A method for data processing includes acquiring an image input and text input pair, the image input and text input pair comprising image input and text input, obtaining an image feature codebook corresponding to the image input and a text feature codebook corresponding to the text input, and training a text-image association model by using the image feature codebook and the text feature codebook, the text-image association model implementing the data processing based on the association between the image input and the text input. For example, a model may be adopted to learn an association between an image feature codebook corresponding to image input and a text feature codebook corresponding to text input, and data processing, such as data compression or data retrieval, is implemented using the learned association.