Remote sensing image content description method based on variational self-attention reinforcement learning

Abstract

The invention discloses a remote sensing image content description method based on variational self-attention reinforcement learning, and belongs to the crossing field of computer vision and natural language processing. The method comprises the following steps: pre-training a convolutional neural network fused with a variational auto-encoder by using a remote sensing image classification data set;extracting spatial features and semantic features of the remote sensing image by using a pre-trained convolutional neural network; fusing the spatial features with contextual information using self-attention; describing the data set by using remote sensing image content, decoding spatial features and semantic features by using Transformer, fusing the features, and outputting text description of the remote sensing image content; and improving text description quality by using reinforcement learning; pre-training a convolutional neural network by utilizing the remote sensing image classification data set and fusing the variational auto-encoder, and the quality of the remote sensing image content description text is optimized by using a self-attention mechanism, feature fusion and reinforcement learning.

Description

technical field [0001] The invention relates to the field of remote sensing image processing technology and natural language generation technology, in particular to a method for describing remote sensing image content based on variational self-attention reinforcement learning. Background technique [0002] Remote sensing is a non-contact, long-distance detection technology. Generally speaking, it is used to detect and identify electromagnetic waves, infrared rays and visible light emitted or reflected by the target object itself through the sensor. With the rapid development of remote sensing technology, especially the emergence of high-resolution remote sensing images in recent years, this technology has become an important means for timely global or regional earth observation. The scale of remote sensing images is also gradually expanding, and the information provided by image content is becoming more and more abundant. [0003] The goal of image content description is t...

AI Technical Summary

Problems solved by technology

This leads to a mismatch between the training phase and the testing phase, and the performance during training does not fully reflect the performance during testing.

Images