Attention-based model training method and device and electronic equipment

Abstract

The invention provides an attention-based model training method and device and electronic equipment, relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and computer vision, and can be applied to scenes such as image processing and image detection. The specific implementation scheme is as follows: obtaining an attention output matrix of an attention module in a neural network model, performing dimension reduction calculation on a sample dimension and a data block dimension of the attention output matrix based on a pooling layer of the neural network model, and determining a pooled first output matrix; carrying out convolution operation on the first output matrix based on a convolution layer of a neural network model, determining a second output matrix after convolution, and carrying out normalization processing and weighting processing on an output value of each header in the second output matrix to obtain an updated second output matrix; and obtaining an updated attention output matrix based on the updated second output matrix, and training the neural network model based on the updated attention output matrix.

Description

technical field [0001] The present disclosure relates to the field of artificial intelligence technology, specifically the field of deep learning and computer vision technology, which can be applied to image processing, image detection and other scenarios, and specifically relates to an attention-based model training method, device and electronic equipment. Background technique [0002] With the continuous development of computer technology, various neural network models have been widely used in fields such as images, texts, and voices. For example, Convolutional Neural Network (CNN) is a feedforward neural network with a deep structure , which extracts features through convolution calculations, captures features from local to global by deepening the network structure, and realizes the superposition of multi-dimensional features by adding channels. At present, technicians need to have a lot of experience in neural network structure design and parameter adjustment, and consum...

AI Technical Summary

Problems solved by technology

At present, technicians need to have a lot of experience in neural network structure design and parameter adjustment, and consume a lot of hardware resources to obtain the neural network structure through repeated replacement and experimentation with different structures of neural networks.

Images