Image description generation method, system and device and storage medium
A technology for image description and information description, applied in the field of data processing, can solve problems such as loss function value, loss of detailed information, and insufficient sentence information, so as to achieve the effect of accurate sentence description and improved effect
Active Publication Date: 2020-07-03
SUN YAT SEN UNIV
View PDF5 Cites 9 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
At present, in the establishment of a model for image description, in order to avoid causing a large loss function value, the model always tends to generate some relatively broad sentences during the training process, so that some of the images in the generated sentences are often lost. Detailed information, resulting in insufficient information in the sentence
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0046] Such as figure 1 As shown, this embodiment provides a method for generating an image description including the following steps:
[0047] S101. Obtain picture information;
[0048] S102. Obtain description information after processing the image information by using an image description model based on fusion of multiple features;
[0049] The various features include block features, object features and text features.
[0050] In order to avoid the loss of detailed information in the image, this embodiment adopts and effectively integrates the features of different modalities, specifically including block features, target features and text features, so that the generated sentence structure is more diversified and the description information is more abundant. , can generate sentences with different content as the theme, wherein the block feature is a feature extracted from an image using a preset image classification model, the target feature is a feature extracted from a...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an image description generation method, system and device and a storage medium. The method comprises the following steps: acquiring picture information; processing the pictureinformation by adopting an image description model based on fusion of multiple features to obtain description information; wherein the multiple features include block features, target features and text features. According to the invention, the image description model is used in the training process; block features, target features and texts are fused; fusion based on multiple features, wherein thesemantic hierarchies represented by different features are different; the feature vector of the high semantic hierarchy can play a role in information guidance for the feature vector of the bottom layer, and the feature vector of the low semantic hierarchy can play a role in information guidance for the feature vector of the high layer, so that the generated sentence description is more accurate,the image description effect is improved, and the method can be widely applied to the technical field of data processing.
Description
technical field [0001] The present invention relates to the technical field of data processing, in particular to an image description generation method, system, device and storage medium. Background technique [0002] Image description refers to taking images as input, and outputting natural language (English, Chinese, etc.) sentences corresponding to the image to describe the content of the image through models and calculations. The generated sentences are sentences capable of describing the content of the image. At present, in the establishment of a model for image description, in order to avoid causing a large loss function value, the model always tends to generate some relatively broad sentences during the training process, so that some of the images in the generated sentences are often lost. Detailed information, resulting in insufficient information in the sentence. Contents of the invention [0003] In order to solve the above technical problems, the object of the...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/58G06F16/583G06N3/04G06N3/08
CPCG06F16/583G06F16/5866G06N3/08G06N3/045Y02T10/40
Inventor 陈弟虎王铎沣
Owner SUN YAT SEN UNIV