An Image Description Method Fused with Visual Context

A technology of image description and context, applied to instruments, biological neural network models, calculations, etc., can solve problems affecting test performance, etc.
CN110991515BActive Publication Date: 2022-04-22北京般芸聚合科技有限公司
7 Cites 1 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
北京般芸聚合科技有限公司
Publication Date
2022-04-22

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses an image description method for fusion of visual context, comprising the following steps: 1) preprocessing; 2) image description label preprocessing; 3) feature extraction; 4) mean value pooling; 5) convolution and mean value sampling Pooling; 6) Obtaining detected image entities; 7) Obtaining entity attributes; 8) Convolution; 9) Obtaining entity attribute features; 10) Convolution; 11) Convolution; 12) Convolution; 13) Acquiring entities and attributes Relationship; 14) The relationship between entities and attributes; 15) LSTM training; 16) Solve the exposure bias; 17) Reduce the dimension; 18) Normalize; 19) Get the description sentence of the current image, that is, the model; Descriptive statement; 21) Test and verify the training effect and performance of the model. This method can ensure the accuracy of image feature extraction, avoid visual errors, make the generated description more fluent to conform to human grammar rules, and obtain higher scores for evaluation indicators.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of computer vision and the field of natural language processing, in particular to an image description method integrating visual context in a deep neural network and a reinforcement learning method. Background technique

[0002] Image description can be understood as giving a picture and generating a text described in natural language. Image description and visual question answering belong to the intersection of computer vision and natural language processing, and are more effective than target detection, image classification and semantic segmentation. It is challenging because it extracts image entities and attributes while inferring the relationship between entities and attributes. Image description has broad application prospects in blind navigation, early childhood education, and image-text retrieval.

[0003] Image description needs to use encoding network and decoding network. The proposal of residual ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More