Image-text mixed retrieval method, device, storage medium, computer equipment

A technology for image-text and information retrieval, applied in the field of information retrieval, can solve problems such as difficult sorting, high storage cost and high calculation cost

Active Publication Date: 2021-03-19
GUANGZHOU HUADUO NETWORK TECH
View PDF18 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are two problems with this method: one is that a separate index system needs to be maintained for text and image retrieval. When the amount of data is large, the storage cost and calculation cost of maintaining two index systems are very high ; Second, after obtaining the set of candidate image-text pairs through text index and image index retrieval, it is difficult to find a sorting strategy that can combine the two sets of retrieval results to sort and get the final query result

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image-text mixed retrieval method, device, storage medium, computer equipment
  • Image-text mixed retrieval method, device, storage medium, computer equipment
  • Image-text mixed retrieval method, device, storage medium, computer equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0053] Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, Integers, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention relates to a method, device, storage medium, and computer equipment for a mixed image-text retrieval method. The image-text mixed retrieval method includes: acquiring information to be retrieved input by a user, if the information to be retrieved includes both text data and image data , then perform an inverted index query operation after the text data is segmented to obtain the first candidate set; input the image data into a pre-trained neural network model to obtain a feature vector corresponding to the image data, and After the feature vector encoding process, perform an inverted index query operation to obtain a second candidate set; merge the documents in the first candidate set and the second candidate set, and count the merged documents in the first candidate set set and / or the number of hits in the second candidate set, sort the merged documents according to the statistical results, and return the sorted results to the display page; while improving the retrieval accuracy, the system maintenance is effectively reduced cost.

Description

technical field [0001] The present invention relates to the technical field of information retrieval, in particular to a method, device, storage medium, and computer equipment for mixed image and text retrieval. Background technique [0002] The previous retrieval systems mainly include two modes: single-modal retrieval and cross-modal retrieval. Whether it is single-modal retrieval or cross-modal retrieval, the query data only contains data of one modality. However, in recent years, with the development of the mobile Internet, people's query requirements have also changed, and new retrieval requirements have gradually emerged, and mixed-modal retrieval is one of them. Mixed-modal retrieval refers to the retrieval of data that includes multiple modalities when inputting query data into the retrieval system. For example, in the field of education, when inputting query data, it generally includes a text description of a topic and related graphic images. At this time, Retrieva...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/31G06F16/51G06F16/53G06F40/289G06N3/04G06N3/08
CPCG06F16/51G06F16/53G06F16/319G06F40/289G06N3/08G06N3/045
Inventor 熊智江琳陈科第张铨王耿鑫庞磊郭志伟
Owner GUANGZHOU HUADUO NETWORK TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products