Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Image question and answer method and device

A technology in images and images, applied in the fields of computer vision and natural language processing, can solve problems such as few remote sensing images, and achieve the effect of accurate answers

Pending Publication Date: 2022-04-29
AEROSPACE INFORMATION RES INST CAS
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] At present, there are few related studies on visual question answering of remote sensing images, and there is an urgent need for a method that can realize visual question answering of remote sensing images

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image question and answer method and device
  • Image question and answer method and device
  • Image question and answer method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0031] like figure 1 As shown in FIG. 2 , it is a schematic flowchart of training a question answering model provided by an embodiment of the present invention. In this example, if figure 2 As shown, the question answering model includes an image feature extraction model, a text feature extraction model, a semantic analysis model, a fusion model and an answer prediction model, and the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of computer vision and natural language processing, and provides an image question and answer method and device.In the embodiment, the method comprises the steps that an image is determined; determining a problem vector corresponding to the problem of the image; performing text feature extraction on the problem vector, and determining a text feature corresponding to the problem vector; performing image feature extraction on the image, and determining image features corresponding to the image; the text features and the image features are fused, and fusion features are determined; wherein the fusion feature indicates the correlation between different regions and problems in the image; and based on the fusion features and the image features, performing classification to understand semantics of regions related to the questions in the images, and determining answers to the questions corresponding to the images. By understanding and analyzing the region related to the question in the image, the question of the image can be accurately solved.

Description

technical field [0001] The invention relates to the technical fields of computer vision and natural language processing, in particular to a method and device for image question answering. Background technique [0002] Visual Question Answering (Visual Question Answering, VQA), as a kind of interactive system combining computer vision and natural language processing, aims to provide an interactive question answering mode, and intelligently predict the corresponding questions according to the input pictures and corresponding questions. Answer. At present, although visual question answering has achieved some results, most of them revolve around natural images, and the application scenarios of remote sensing images are different from natural images. There are differences in depth of field in natural images, and people tend to focus on the salient objects in the image, but in remote sensing images, each object is in the same depth of field, and the existing visual question answe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/583G06F40/30G06K9/62G06V10/80
CPCG06F16/5846G06F40/30G06F18/253
Inventor 张美美陈方
Owner AEROSPACE INFORMATION RES INST CAS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com