Image target detection method based on natural language semantics

Abstract

The invention discloses an image target detection method based on natural language semantics. Input of the method includes natural language phrase description of an image to be detected and a target to be detected; the image target detection method includes the steps that a global feature graph of the image to be detected is calculated through a convolutional neural network, then the global feature graph is input into an RPN network to calculate an alternative target set, a regional feature graph of an alternative target is extracted from the calculated alternative target set through an RoI pooling layer, the global feature graph of the image, the regional feature graph of the alternative target region and position information are used as context and combined with query phrase word vectors to represent input of an LSTM module and calculate the conditional probability of query phrases generated in the target region, and a detection result is returned according to the conditional probability. The natural language processing module LSTM model is fused into the Faster-RCNN frame, the shared computation characteristic of the Faster-RCNN frame and the image characteristics extraction advantage of the convolutional network are used for improving target detection efficiency and accuracy based on natural language semantics.

Description

technical field [0001] The invention belongs to the technical field of image analysis and recognition, in particular to an image target detection method based on natural language semantics. Background technique [0002] Image target recognition is one of the core tasks in the field of computer vision research. In recent years, with the successful application of deep learning in the image field, the research on target recognition has also made breakthrough progress, and the detection accuracy has achieved a lot compared with traditional methods. It has been greatly improved, and has been commercially applied to people's lives in some fields, such as Alibaba's facial recognition payment, intelligent traffic road target recognition, etc. However, the recognition results of traditional target recognition methods are often some pre-defined objects of a certain category, such as faces, cars, etc., and the content contained in an image is far more than some independent objects. Th...

AI Technical Summary

Problems solved by technology

[0003] The image target recognition task includes four basic sub-tasks, including the generation of target candidate sets, feature extraction of candidate targets, classification of candidate targets, and position correction of candidate targets. The Faster-RCNN model is a typical representative of traditional target recognition methods. , this method uses a deep convolutional neural network combined with an RPN network to solve these four subtasks. The RPN network used to generate target candidate sets is essentially a deep convolutional neural network, so the entire model can be used in an end-to-end manner. Compared with the previous target recognition method that solves these subtasks in different ways, Faster-RCNN has greatly improved the efficiency of training and the accuracy of recognition, but the recognition object of the Faster-RCNN model It is still only a pre-defined class of objects, and it cannot be combined with natural language for target recognition

Existing methods for target detection combined with natural language, such as the SCRC model (spatial context recurrent convnet), mostly use non-deep learning methods to generate target candidate sets, such as selective search, etc., and then use convolutional neural networks The long-short-term memory model (LSTM) extracts image and natural language features for target detection. The entire framework does not support end-to-end training, and the efficiency and accuracy of detection need to be improved.

Images