Image scene classification method based on converter model and convolutional neural network

Abstract

The invention relates to an image scene classification method based on a converter model and a convolutional neural network. The method comprises the steps that two convolutional neural networks and a converter model are used in the scene classification process of an image, the converter model is used for establishing interrelation between global elements at the bottom layer of the scene image, the defects of the convolutional neural networks in scene image feature extraction are effectively overcome. Therefore, the network can pay attention to the key area in the scene from the beginning, so that the scene classification effect is improved. Features extracted by the convolutional neural network are used as additional input of a converter model to guide a coding unit to focus on an area which can form good supplementation with depth features, and a relation between a network bottom layer and a network high layer is established from the other aspect, so that model expression is more comprehensive and sufficient.

Description

technical field [0001] The present application relates to the technical field of image scene classification, in particular to an image scene classification method based on a transformer model and a convolutional neural network. Background technique [0002] With the development of Internet multimedia technology and the growth of visual data, how to deal with these massive data has become a difficult problem in the new era. Scene classification technology, as a key technology to solve image retrieval and image recognition problems, has become a very important and challenging research topic in the field of computer vision. At the same time, scene classification has a wide range of applications in remote sensing image analysis, video surveillance, robot perception and other fields. Therefore, it is of great significance to carry out corresponding research on scene classification technology and improve the ability of computer scene recognition. [0003] The so-called image sce...

AI Technical Summary

Problems solved by technology

In the early scene classification methods based on deep learning, these two types of features were often optimized, such as removing redundancy, adding details, and supplementing with more scene features. However, no matter which aspect of optimization, They are all based on the convolutional neural network, which also brings limitations to this type of algorithm.

The convolution kernel is the core of the convolutional neural network. It has the advantages of local connection and parameter sharing. However, it also has the disadvantage of not being able to model the global image at the bottom of the image. It is limited by the size of the convolution kernel. For image features The extraction of the global feature extraction of the entire image is often done step by step through the stacking of multi-layer convolutions, which will cause two areas that are too far apart and connected to each other to be associated in a deeper layer, which is not an efficient method. way, and cannot make full use of global information to complete feature extraction

The content of the scene image is richer than that of a single object image, so it is particularly important to grasp the relationship between the elements. If you only rely on the convolutional neural network to correlate the elements at a deep level, it will easily lead to the loss of some information, resulting in Decline in classification performance

Images