Recommendation model distributed training method based on double-layer index embedding layer and GPU (Graphics Processing Unit)

Abstract

The invention provides a recommendation model distributed training method based on a double-layer index embedded layer and a GPU, the method comprises the following steps: constructing two layers of indexes for an embedded layer vector at a server node and a computing node, the server node being a static index, and the computing node being a dynamic index; introducing a sampler into the server node, and formulating a fragmentation strategy according to sampling data; and introducing a ping-pong buffer area into the computing node to read and write an embedded layer vector, and prefetching the embedded layer vector based on the input of adjacent iterations to form a data prefetching assembly line. According to the method, on the premise of ensuring that the prediction performance of the model is not reduced, the total throughput of recommendation model training is improved, the expandability of distributed training is enhanced, and the training of a large-scale embedded layer recommendation model is effectively supported.

Description

technical field [0001] The invention relates to the technical field of distributed computing and machine learning, in particular to a recommended model distributed training method based on a double-layer index embedding layer and a GPU. Background technique [0002] With the widespread popularity of the Internet, the scale of users and the amount of information are increasing day by day, and the problem of information overload is becoming more and more obvious. The recommendation system is one of the solutions to the problem of information overload. The recommendation system guides users to discover their own information needs by analyzing the user's historical behavior information, and provides users with personalized recommendation services. Industrial recommendation models usually have the characteristics of huge training data, complex models, and strong timeliness. Distributed training and computing hardware acceleration have become necessary conditions for solving large...

AI Technical Summary

Problems solved by technology

[0004] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a recommended model distributed training method based on a double-layer index embedding layer and a GPU, which are used to solve the problems of low model training throughput and low throughput in existing centralized distributed scenarios. Technical issues with poor scalability

Images