A sparse
processing method and apparatus for a sparse
attention network, and an electronic device. The method specifically comprises: in a
memory cell, using constraint conditions to construct a masked sparse representation (S301), wherein the constraint conditions are: the dimension of a
mask matrix is [B,a,S], B represents the batch size, a represents the number of heads, S represents the sequence length, and each element in the S dimension represents a masked starting row of each column in the
mask matrix; and a calculation unit acquiring the masked sparse representation from the
memory cell, using the masked sparse representation to perform sparse
processing on input data, and storing the sparse
processing result into the
memory cell (S302). According to the method, the constraint conditions are used to construct the masked sparse representation, so that memory consumption can be reduced from a quadratic order of the sequence length to a linear order of the sequence length, thereby remarkably reducing memory requirements during
large model training, and improving the training efficiency.