Image processing method and device based on attention mechanism

Abstract

The application relates to the technical field of data processing, and provides an image processing method and device based on an attention mechanism, which comprises the following steps: acquiring a query feature vector sequence, a key feature vector sequence and a value feature vector sequence of an image to be processed, and extracting a representative key feature vector sequence from the key feature vector sequence; based on a Laplace kernel function, a similarity matrix between the query feature vector sequence and the representative key feature vector sequence and a kernel matrix between the representative key feature vector sequences are constructed; based on pseudo-inverses of the similarity matrix and the kernel matrix, an approximate global similarity matrix is constructed, and the approximate global similarity matrix is subjected to centering and whitening treatment to obtain a normalized feature mapping matrix; and based on the normalized feature mapping matrix and the value feature vector sequence, an attention output feature of the image to be processed is obtained, so that the image task precision and stability are effectively improved while the calculation cost is greatly reduced.

Images