Learning Unpaired Multimodal Feature Matching for Semi-Supervised Learning
The method trains an image encoder and a common classifier with a text encoder aligned to a Gaussian distribution, addressing overfitting and paired data requirements, enhancing multimodal matching and cross-modal tasks.
JP7751951B2Active Publication Date: 2025-10-09INTERNATIONAL BUSINESS MACHINE CORPORATION
Patent Information
- Application Number
- JP2023528059
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-12-02
- Filing Date
- 2021-11-02
- Publication Date
- 2025-10-09
- Estimated Expiration
- 2041-11-02
Smart Images
Figure 0007751951000012 
Figure 0007751951000013 
Figure 0007751951000014
Abstract
A computer-implemented method for learning multimodal feature matching is provided. The method includes training an image encoder to obtain encoded images. The method further includes training a common classifier on the encoded images by using labeled images. The method also includes training a text encoder while keeping the common classifier in a fixed setting by using trained text embeddings and labels corresponding to the trained text embeddings. The text encoder is further trained to match distances of predicted text embeddings encoded by the text encoder to a Gaussian distribution fitted on the encoded images.
Need to check novelty before this filing date? Find Prior Art
Citation Information
Patent Citations
Improved font recognition using triplet loss neural network training
JP2019083002A
Caption generation device, caption generation method and program
JP2020140050A
Semantic Class Localization Digital Environment
US20190279074A1
Gaussian autoencoder detection of network flow anomalies
US20200106805A1