Cross-modal-based rapid multi-label image classification method and system

Abstract

The invention discloses a cross-modal-based rapid multi-label image classification method, and the method can mine the co-occurrence relation between different objects in an image, and achieves the high-efficiency fusion of image features and label co-occurrence relations to generate an end-to-end multi-label image classification model. According to the method, the dependency relationship betweenmodel objects is modeled by counting the co-occurrence probability between tags, and the co-occurrence relationship between image features and the tags is fused by adopting a bilinear multi-modal factor pooling component, so that the convergence rate of the model is increased, and the image classification performance is improved. The method provided by the invention comprises the following steps:firstly, respectively generating co-occurrence relation word vectors of features and labels of an image in combination with a convolutional neural network and a graph convolutional network, then fusing the vectors of the two modes by adopting MFB, and finally generating an end-to-end classification model through a multi-label classification function. According to the method, the co-occurrence relation word vectors of the features and the labels of the images are efficiently fused, and the convergence speed of the model is greatly increased.

Description

technical field [0001] The invention belongs to the technical field of pattern recognition and image classification, and more particularly relates to a fast multi-label image classification method and system based on cross-modality, which utilizes dependencies between labels. Background technique [0002] Nowadays, Multi-label image classification (Multi-label image classification) has been widely used in the field of computer vision, including multi-target recognition, sentiment analysis, medical diagnosis recognition, etc. Since each image contains multiple objects, how to effectively learn the relationship between these objects and how to fuse these relationships with image features is still full of challenges. [0003] Generally speaking, the existing multi-label image classification methods first use the convolutional neural network to obtain the feature vector of the image, then use the graph convolutional network (Graph Convolutional Network, GCN) to obtain the co-occ...

AI Technical Summary

Problems solved by technology

[0003] Generally speaking, the existing multi-label image classification methods first use the convolutional neural network to obtain the feature vector of the image, then use the graph convolutional network (Graph Convolutional Network, GCN) to obtain the co-occurrence relation word vector between the labels, and finally directly The dot product operation of vectors is used to fuse image features and co-occurrence relationship word vectors of labels, but this will seriously affect the convergence efficiency of the image classification model, which in turn affects the image recognition performance of the image classification model

Images