Unlock instant, AI-driven research and patent intelligence for your innovation.

A multi-level image feature fusion method based on image-text matching

A technology of image features and fusion methods, which is applied in the directions of instruments, calculations, character and pattern recognition, etc., can solve the problems of insufficient and reasonable use, and achieve the effect of reducing interference

Active Publication Date: 2022-06-24
GUANGDONG UNIV OF TECH
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, directly using or fine-tuning a single-level feature of the pre-trained network does not fully and reasonably use this pre-trained feature

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A multi-level image feature fusion method based on image-text matching
  • A multi-level image feature fusion method based on image-text matching
  • A multi-level image feature fusion method based on image-text matching

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings:

[0021] like figure 1 As shown, a multi-level image feature fusion method based on image and text matching includes the following steps:

[0022] S1), text representation, default m 1 word segmentation to process the text data, respectively Each text has a corresponding image, and text analysis techniques are used to generate corresponding feature vectors for all text data;

[0023] S2), multi-level image representation, specifically:

[0024] S201), preset m 2 image data, respectively Each image has its corresponding text. Under the guidance of the image classification learning target, a pre-trained convolutional neural network is obtained by pre-training using the rich image classification data set ImageNet data set;

[0025] S202), input the image into this pre-training convolutional neural network, and splicing the n-layer feature...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention discloses a multi-level image feature fusion method based on image-text matching, by using the multi-layer features in the pre-training network as the multi-level total pre-training features of the image, and under the guidance of the learning objectives of image-text matching, using Multi-Layer Perceptron (Multi-Layer Perceptron) supervised fusion and dimensionality reduction of multi-level total pre-training features of images to generate fused image features. In this way, more useful and different levels of pre-training features can be fully utilized, and features useful for image-text matching tasks can be summarized and useless features can be removed, reducing the interference of noise features. Then, the cosine similarity of the fused image features and text features in the feature space can be used for image-text matching.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to a multi-level image feature fusion method based on graphic and text matching. Background technique [0002] In recent years, image-text matching tasks have gradually become popular in the fields of artificial intelligence and machine learning. We can now build an image-text matching system that recommends suitable images based on textual content, and vice versa. This eliminates the need for tedious and repetitive manual searches and reduces work stress. As an image-text matching system, it must simultaneously focus on text and images, two research objects belonging to different modalities. Therefore, image-text matching is a multimodal task, which requires accurate extraction of text and image features. . Especially for images, it is especially difficult to obtain the characteristics of images due to their richer ways of expressing the same thing. [0003] In fact, in ord...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06V10/80G06V10/764G06V10/82
CPCG06F18/241G06F18/253
Inventor 郝志峰李俊峰蔡瑞初温雯王丽娟陈炳丰
Owner GUANGDONG UNIV OF TECH