Cross-modal information retrieval method and device
An information retrieval, cross-modal technology, applied in the computer field, can solve problems such as poor retrieval performance, unable to compensate for the difference in information characteristics of different modalities, and achieve the effect of improving the accuracy of information retrieval
Pending Publication Date: 2021-06-25
TAIKANG LIFE INSURANCE CO LTD
View PDF5 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The existing retrieval method is to independently embed different modal information into a joint space, and then measure the similarity between different modal information, which cannot compensate for the feature differences between different modal information, resulting in poor retrieval performance
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0039] Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
[0040] figure 1 is a schematic diagram of main steps of a cross-modal information retrieval method according to an embodiment of the present invention. Such as figure 1 As shown, the cross-modal information retrieval method in the embodiment of the present invention mainly includes the following steps:
[0041] Step S101: Obtain the first modality information to be retr...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a cross-modal information retrieval method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: acquiring to-be-retrieved first modal information, and performing feature extraction on the first modal information to obtain a first feature matrix; performing multi-modal fusion on the first feature matrix and a pre-constructed second feature matrix to obtain a first aggregation feature matrix and a second aggregation feature matrix; performing gating fusion on the first feature matrix and the second aggregated feature matrix to obtain a first mixed feature matrix; performing gating fusion on the second feature matrix and the first aggregation feature matrix to obtain a second mixed feature matrix; and calculating a matching degree between the first modal information and second modal information according to the first mixed feature matrix and the second mixed feature matrix, and determining a retrieval result of the first modal information according to the matching degree. According to the embodiment, the features with the high matching degree can be integrated in a concentrated mode, the features with the low matching degree are not integrated, and the information retrieval precision is improved.
Description
technical field [0001] The invention relates to the field of computer technology, in particular to a cross-modal information retrieval method and device. Background technique [0002] Cross-modal information retrieval is to find the relationship between different modal information, realize the use of a certain modal information, and search for other modal information with similar semantics. For example, using an image to retrieve a corresponding text, or using a text to retrieve a corresponding image. Among them, modality refers to the existence form of information, such as text, image, audio, video and other file formats. [0003] The existing cross-modal information retrieval methods mainly include binary representation learning and real-valued representation learning. Among them, binary representation learning is to map the extracted image features and text features to the Hamming binary space, and then learn in this space. Real-valued representation learning is to lea...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/583G06F16/55G06N3/04
CPCG06F16/5846G06F16/55G06N3/045
Inventor 于吉鹏侯博严李驰刘岩
Owner TAIKANG LIFE INSURANCE CO LTD