Picture visual reasoning method and device for complex relation description, and storage medium
A reasoning method and relational technology, applied in neural learning methods, electrical digital data processing, still image data retrieval, etc., can solve the problems of troublesome process, low accuracy, complicated and lengthy relations, etc., to improve the effect of visual reasoning, solve the Visual reasoning inaccuracy problem, the effect of realizing visual relational reasoning
Active Publication Date: 2020-07-31
SOUTH CHINA UNIV OF TECH
View PDF3 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
In practical applications, since the description sentence contains the description of the relationship between objects in the picture, and the described relationship may be complicated and lengthy, multi-step reasoning is required, the process is often cumbersome and the accuracy is not high
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
[0053] The present invention provides a picture visual reasoning method aimed at describing complex relationships, such as figure 1 shown, including the following steps:
[0054] S101. Obtain description sentences and pictures to be processed;
[0055] S102. Input the picture into the deep neural network to obtain object features in the picture; specifically, the object features in the picture may include visual features and positional features of the object;...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a picture visual reasoning method and a picture visual reasoning device for complex relation description, and a storage medium. The picture visual reasoning method comprises the steps of: acquiring a description statement to be processed and a picture to be processed; inputting the picture into a deep neural network to obtain object features in the picture; dividing the description statement into statements of different types of information by utilizing an attention mechanism, and acquiring statement features corresponding to the statements of different types of information; in combination with the obtained object features and statement features, enabling the statements to correspond to objects in the picture, using a graphic neural network model to construct relationships between the objects in the picture, enabling the statements to correspond to the relationships between the objects in the picture, and updating the object features; and predicting a description object in the picture according to the obtained statement features and the updated object features. According to the picture visual reasoning method, the information contained in the description statement is reasonably divided, and the complex relationships between the objects are modeled through using the graph neural network model, so that the visual relationship reasoning is realized, and thevisual reasoning effect under the complex relationship description is effectively improved.
Description
technical field [0001] The invention relates to the technical field of visual reasoning of artificial intelligence, in particular to a picture visual reasoning method, device and storage medium for describing complex relationships. Background technique [0002] At present, artificial intelligence, as a future development trend in the field of science and technology, has received extensive attention. With the introduction of labeled data and the continuous development of hardware devices such as GPUs in recent years, deep learning has made great progress in both computer vision and natural language understanding. In the direction of computer vision, the deep convolutional neural network can effectively improve the performance of image classification and recognition tasks such as image classification and face recognition by superimposing multiple convolution layers and extracting features from the shallow layer to the deep layer of the picture. In the direction of natural lan...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/583G06F16/58G06K9/00G06K9/32G06K9/62G06N3/04G06N3/08
CPCG06F16/583G06F16/5866G06N3/08G06V20/00G06V10/25G06V2201/07G06N3/045G06F18/22G06F18/2415
Inventor 谭明奎郑逸涵
Owner SOUTH CHINA UNIV OF TECH