An Image Description Method Fused with Visual Context

Abstract

The invention discloses an image description method for fusion of visual context, comprising the following steps: 1) preprocessing; 2) image description label preprocessing; 3) feature extraction; 4) mean value pooling; 5) convolution and mean value sampling Pooling; 6) Obtaining detected image entities; 7) Obtaining entity attributes; 8) Convolution; 9) Obtaining entity attribute features; 10) Convolution; 11) Convolution; 12) Convolution; 13) Acquiring entities and attributes Relationship; 14) The relationship between entities and attributes; 15) LSTM training; 16) Solve the exposure bias; 17) Reduce the dimension; 18) Normalize; 19) Get the description sentence of the current image, that is, the model; Descriptive statement; 21) Test and verify the training effect and performance of the model. This method can ensure the accuracy of image feature extraction, avoid visual errors, make the generated description more fluent to conform to human grammar rules, and obtain higher scores for evaluation indicators.

Description

technical field [0001] The invention relates to the technical field of computer vision and the field of natural language processing, in particular to an image description method integrating visual context in a deep neural network and a reinforcement learning method. Background technique [0002] Image description can be understood as giving a picture and generating a text described in natural language. Image description and visual question answering belong to the intersection of computer vision and natural language processing, and are more effective than target detection, image classification and semantic segmentation. It is challenging because it extracts image entities and attributes while inferring the relationship between entities and attributes. Image description has broad application prospects in blind navigation, early childhood education, and image-text retrieval. [0003] Image description needs to use encoding network and decoding network. The proposal of residual ...

AI Technical Summary

Problems solved by technology

Therefore, sentences that have not appeared will seriously affect the performance of the test during the test

Images