Image Scene Classification Method Based on Transformer Model and Convolutional Neural Network

Abstract

This application relates to an image scene classification method based on a transformer model and a convolutional neural network. The method includes: in the process of classifying the scene of the image, two kinds of convolutional neural networks and a transformer model are used, wherein the transformer model is used to establish the interconnection between the global elements at the bottom layer of the scene image, effectively compensating It overcomes the shortcomings of convolutional neural network in scene image feature extraction, so that the network can focus on key areas in the scene from the beginning, thereby improving the effect of scene classification. Using the features extracted by the convolutional neural network as an additional input to the transformer model aims to guide the coding unit to focus on areas that can form a good complement to the deep features, and on the other hand, establish a connection between the bottom layer and the top layer of the network. Contact, so that the model expression is more comprehensive and sufficient.

Description

technical field [0001] The present application relates to the technical field of image scene classification, in particular to an image scene classification method based on a transformer model and a convolutional neural network. Background technique [0002] With the development of Internet multimedia technology and the growth of visual data, how to deal with these massive data has become a difficult problem in the new era. Scene classification technology, as a key technology to solve image retrieval and image recognition problems, has become a very important and challenging research topic in the field of computer vision. At the same time, scene classification has a wide range of applications in remote sensing image analysis, video surveillance, robot perception and other fields. Therefore, it is of great significance to carry out corresponding research on scene classification technology and improve the ability of computer scene recognition. [0003] The so-called image sce...

AI Technical Summary

Problems solved by technology

In the early scene classification methods based on deep learning, these two types of features were often optimized, such as removing redundancy, adding details, and supplementing with more scene features. However, no matter which aspect of optimization, They are all based on the convolutional neural network, which also brings limitations to this type of algorithm.

The convolution kernel is the core of the convolutional neural network. It has the advantages of local connection and parameter sharing. However, it also has the disadvantage of not being able to model the global image at the bottom of the image. It is limited by the size of the convolution kernel. For image features The extraction of the global feature extraction of the entire image is often done step by step through the stacking of multi-layer convolutions, which will cause two areas that are too far apart and connected to each other to be associated in a deeper layer, which is not an efficient method. way, and cannot make full use of global information to complete feature extraction

The content of the scene image is richer than that of a single object image, so it is particularly important to grasp the relationship between the elements. If you only rely on the convolutional neural network to correlate the elements at a deep level, it will easily lead to the loss of some information, resulting in Decline in classification performance

Images