Course field-oriented image text aggregation method and system

A text and field technology, applied in the computer field, can solve problems such as automatic matching of images and text descriptions in difficult courses, and achieve the effect of promoting global similarity and improving accuracy

Active Publication Date: 2021-08-06
XI AN JIAOTONG UNIV
View PDF3 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the huge difference between course domain images and natural images, traditional natural image text aggregation methods cannot deeply mine the info

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Course field-oriented image text aggregation method and system
  • Course field-oriented image text aggregation method and system
  • Course field-oriented image text aggregation method and system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0059] Next, the technical solutions in the embodiments of the present invention will be described in contemplation in the embodiment of the present invention, and it is clear that the described embodiments are intended to be in an embodiment of the invention, not all of the embodiments of the invention. Based on the embodiments of the present invention, there are all other embodiments obtained without making creative labor without making creative labor premises.

[0060] It should be understood that when used in this specification and the appended claims, the terms "include" and "including" indicating the description of the features, overall, steps, operations, elements, and / or components, but does not exclude one or Multiple other features, overall, steps, operations, elements, components, and / or their set presence or addition.

[0061] It should also be understood that the terms used in the specification of the present invention are merely the purposes of describing the par...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a course field-oriented image text aggregation method and system, and the method comprises the steps of taking object features and an adjacent matrix as input to construct an object relation graph corresponding to an image, and updating the features of nodes in the relation graph through a graph convolutional neural network; taking a set of feature vectors of all the language chunks as feature representation of the whole text description; taking the object feature set and the step language block feature set as input, and constructing a local similarity matrix between the image-text pairs; calculating the global similarity between the image and the text description in the whole course field; and training parameters of the image-text matching model through a gradient descent method, obtaining a section of text description with the highest global similarity through the learned parameters, and taking the text description as a description text matched with the image, so as to realize image text aggregation. According to the invention, the features of image and text description in the course field can be effectively extracted, so that schematic diagrams and text knowledge fragments in the course field are aggregated, and cross-modal knowledge fragments are automatically constructed.

Description

Technical field [0001] The present invention belongs to the technical field of computer, and more particularly to an image text aggregation method and system for the course. Background technique [0002] In recent years, with the rapid development of Internet technology, various knowledge-related website platforms continue to emerge, users who have acquired knowledge through the Internet have increased rapidly, thus promoting various types of knowledge resource explosive growth in the network. A variety of courses and text knowledge containing knowledge concepts are created by different platform organizations, which are distributed in various websites. The complement between these multi-modal knowledge helps achieve efficient expression of complex things, knowledge, and concepts. [0003] At the same time, the multi-mode knowledge is in line with the multi-channel information processing mode of the human brain, which can effectively improve the learner's cognitive efficiency. Sin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/32G06K9/46G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V10/25G06V20/62G06V10/44G06V30/10G06N3/048G06N3/045G06F18/2132
Inventor 张玲玲吴文俊杨宽刘均郑庆华魏笔凡胡欣王绍伟王佳欣
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap