Unlock instant, AI-driven research and patent intelligence for your innovation.

Intelligent image automatic description method based on deep neural network

A deep neural network and intelligent image technology, applied in the field of intelligent image automatic description, can solve the problems of not considering the scene information, not considering the hierarchy of semantic information, etc.

Active Publication Date: 2022-05-06
XIAMEN UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to solve the above-mentioned problems existing in the current automatic image description method, and to provide a new deep learning network design under the attention mechanism based on the factorization of scene information, which can solve the problem that the semantic information hierarchy is not considered in the automatic image description And an intelligent image automatic description method based on deep neural network without considering scene information and other issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Intelligent image automatic description method based on deep neural network
  • Intelligent image automatic description method based on deep neural network
  • Intelligent image automatic description method based on deep neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] The following embodiments will further illustrate the present invention in conjunction with the accompanying drawings.

[0050] Such as figure 2 As shown, the embodiment of the present invention includes the following steps:

[0051] 1. Describe data preprocessing

[0052] Step 1 removes stop words from the text content in all training data, and lowercases all English words. Then the text content is divided into words according to spaces, and 9487 words are obtained, and the words that appear less than five times in the description of the data set are eliminated, using " "Replace, and add the start character at the same time" " and terminator" " at the beginning and end of the description sentence, respectively.

[0053] 2. Image depth convolution feature and semantic information extraction

[0054] Step 1 uses the residual deep convolutional network to process the image convolution features, and obtains the feature map of each image, denoted as F I...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

An intelligent image automatic description method based on a deep neural network relates to an intelligent image automatic description in the field of artificial intelligence. The method includes the following steps: 1) description data preprocessing; 2) extracting image depth convolution features and semantic information; 3) intelligent image automatic description based on multi-level visual semantic embedding. The factorized attention mechanism module can solve the problems of not considering the hierarchy of semantic information and scene information in automatic image description, and can explicitly embed scene-related semantic information to guide the embedding of object-related semantic information and embedding of image features. The research on automatic image description based on multi-level visual semantic embedding can facilitate the promotion and use of automatic image description in the industry.

Description

technical field [0001] The present invention relates to intelligent image automatic description in the field of artificial intelligence, in particular to an intelligent image automatic description method based on a deep neural network to describe the objective content of an image in natural language based on a picture. Background technique [0002] Automatic image description (Image Captioning) is an ultimate machine intelligence task proposed by the computer science community this year. Its task content is based on a given image, using natural language to describe the objective content of the image, such as figure 1 shown. With the development of computer vision, it is not limited to allowing machines to complete tasks such as detection, recognition, and segmentation, but also requires computers to automatically describe the objective content of images. Different from image classification or target detection tasks, automatic image description needs to describe the importa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06V10/40G06V10/70G06K9/62G06N3/04G06N3/08
CPCG06N3/049G06N3/08G06V10/40G06N3/045G06F18/00
Inventor 纪荣嵘陈福海沈忱
Owner XIAMEN UNIV