Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Modal independent retrieval method and system based on intermediate text semantic enhancement space

A text and semantic technology, applied in the field of cross-media retrieval, can solve the problems of unreachable, high performance, and subspace method without strong discrimination of text features to improve retrieval results, and achieve the effect of improving accuracy and low noise impact

Inactive Publication Date: 2019-02-22
SHANDONG NORMAL UNIV
View PDF6 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, since images often contain a lot of messy information, there will be a large number of semantic gaps in the process of image feature extraction, so text modality features are often more discriminative than image features, and traditional subspace methods do not Effectively utilize the strong discriminative properties of text features to improve retrieval results
At the same time, most methods only learn a pair of mapping matrices for different retrieval tasks, and this mapping mechanism cannot achieve the highest performance in a single retrieval task.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Modal independent retrieval method and system based on intermediate text semantic enhancement space
  • Modal independent retrieval method and system based on intermediate text semantic enhancement space
  • Modal independent retrieval method and system based on intermediate text semantic enhancement space

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0050] This embodiment discloses a modality-independent retrieval method based on an intermediate text semantic enhancement space, such as figure 1 As shown, the method includes the following steps:

[0051] Step 1: Obtain the underlying features of the paired text and image, construct the training set and test set of the text, and the training set and test set of the image;

[0052] The step 1 specifically includes:

[0053] Step 1: Input three data sets, and obtain the underlying features of each data set image and text, expressed as (i i ,t i ) represents the i-th pair of text and image features representing the same semantics, n is the number of samples, and there are k classes in the data set. Let I=[i 1 ,i 2 ,... i n ]∈R n×p is the underlying feature matrix of the image, and p is the dimension of the image. Let T = [t 1 ,t 2 ,...t n ]∈R n×q is the underlying feature matrix of the text, and q is the dimension of the text. Y=[y 1 ,y 2 ,...y n ]∈R n×k is ...

Embodiment 2

[0127] The purpose of this embodiment is to provide a computer system.

[0128] A computer system, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, when the processor executes the program, the following steps are implemented, including:

[0129] Obtain paired text and image underlying features, construct training data set and test data set;

[0130] Establish an intermediate text semantic enhancement space based on linear discriminant analysis; calculate the similarity matrix of image-text pairs;

[0131] Construct the projection matrix model of image retrieval text and text retrieval image respectively according to described intermediate text semantic enhancement space and similarity matrix;

[0132] Using the training data set to solve the projection matrix parameters in the projection matrix model of the image retrieval text and the text retrieval image respectively;

[0133] Based on the projection matrix parame...

Embodiment 3

[0135] The purpose of this embodiment is to provide a computer-readable storage medium.

[0136] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the following steps are performed:

[0137] Obtain paired text and image underlying features, construct training data set and test data set;

[0138] Establish an intermediate text semantic enhancement space based on linear discriminant analysis; calculate the similarity matrix of image-text pairs;

[0139] Construct the projection matrix model of image retrieval text and text retrieval image respectively according to described intermediate text semantic enhancement space and similarity matrix;

[0140] Using the training data set to solve the projection matrix parameters in the projection matrix model of the image retrieval text and the text retrieval image respectively;

[0141] Based on the projection matrix parameters of Image Retrieval Text / Text Retriev...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a modal independent retrieval method and system based on an intermediate text semantic enhancement space. The method comprises the following steps of obtaining the bottom features of pairs of text and image, constructing a training data set and a test data set; establishing the intermediate text semantic enhancement space based on linear discriminant analysis; calculating asimilarity matrix of image text pairs; respectively constructing a projection matrix model of an image retrieval text and a text retrieval image according to the intermediate text semantic enhancement space and the similarity matrix; using the training dataset to solve the projection matrix parameters of the retrieved text and the retrieved image respectively; and based on the projection matrix parameters of the text / text retrieval images, retrieving the text / text retrieval images by using the test data set. The method respectively learns mapping matrix for image retrieval text and retrievaltask of text retrieval image, the retrieval is more pertinent, and the retrieval accuracy is greatly improved.

Description

technical field [0001] The disclosure belongs to the technical field of cross-media retrieval, and in particular relates to a modality-independent retrieval method and system based on an intermediate text semantic enhancement space. Background technique [0002] With the development of society and the popularity of the Internet, multimedia data on the Internet, such as pictures, videos, and audios, has shown exponential growth. Many different types of multimedia data will express the same semantics. Users need to retrieve data from these massive data. to the information you need. With the rapid development of machine learning and pattern recognition theory and the upgrading of hardware, the calculation speed has been greatly improved. How to realize the retrieval of cross-media data has become an urgent problem to be solved. [0003] Cross-media retrieval refers to submitting data of any type of media and obtaining the same semantic results of different media types. The cu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/58G06K9/62
CPCG06F18/2132G06F18/22G06F18/214
Inventor 张化祥郑顺心李静吴泓辰王琳孙建德
Owner SHANDONG NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com