Sparse matrix compression method and device, equipment and storage medium

Abstract

The invention relates to a sparse matrix compression method and device, equipment and a storage medium. The sparse matrix compression method comprises the steps: constructing an original feature set;acquiring sample data of a user, and determining a sparse feature set of the user from the original feature set according to the sample data of the user; carrying out compressed encoding on the elements in the sparse feature set to obtain a compressed feature set, and establishing a one-to-one correspondence relationship between the elements in the sparse feature set and the elements in the compressed feature set; and constructing a user feature compression matrix according to the sample data of the user, the sparse feature set, the compression feature set and the corresponding relationship. According to the sparse matrix compression method, the feature space during feature vectorization can be reduced, and the memory space needing to be opened up is reduced, and the whole calculation process is accelerated.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a sparse matrix compression method, device, equipment and storage medium. Background technique [0002] In the Internet scenario, hundreds of millions of users generate large-scale user data every day, forming super-large-scale training samples. How to use these data to train better models and use these models to serve users has brought huge challenges to machine learning platforms. [0003] Taking the recommended business scenario as an example, the current daily page views can reach up to tens of billions. If one month’s samples are required for training, the sample size will be at the hundreds of billions level. Engineering cross combinations yields an explosion of feature dimensions. When training the model, the user sample features obtained each time are sparse, and the sparse matrix representation is generally used to vectorize the user's features. Here, differe...

AI Technical Summary

Problems solved by technology

When training the model, the user sample features obtained each time are sparse, and the sparse matrix representation is generally used to vectorize the user's features. Here, different sparse matrix representations affect the speed of feature vectorization. There are different effects. The sparse matrix representation in the prior art takes up a lot of memory space and affects the speed of feature vectorization.

Images