Graph-based large-scale embedding model training method and system for click-through rate prediction

Abstract

The invention discloses a graph-based large-scale embedding model training method and system for click-through rate prediction. The system includes a dense parameter module and a client-side module, adopts a hybrid communication architecture, and distributes the click-through rate prediction input data set to different jobs. Each worker node maintains a client, and the local model parameters are directly stored in the GPU memory; each worker node holds a copy of the model parameters and synchronizes them during training. The present invention adopts the Embedding model parameter to represent the importance of the feature value of the corresponding category of the input data of click-through rate prediction, expresses the click-through rate prediction data and the embedding model vector as a binary graph model, and uses the graph locality and degree skew characteristics to perform parallel training of the model; Design graph-based partitioning and bounded synchronization to improve scalability and parallel computing efficiency for training large embedding models.

Description

technical field [0001] The invention belongs to the technical field of distributed machine learning, and relates to a large-scale embedding model training method and system, in particular to a graph-based large-scale embedding model training method and system for click rate prediction. Background technique [0002] Embeddings are often used to deal with representation learning problems on high-dimensional data, such as words in text corpora, users, and items in recommender systems. Deep Embedding techniques use continuous vectors to represent discrete variables and have numerous practical applications, such as click-through rate (CTR) prediction systems, graph processing, and information extraction. However, as the scale of deep embedding models continues to expand and the amount of input data increases, building a huge embedding model training system is more challenging in terms of effectiveness and efficiency. For example, Facebook's production platform proposes a true de...

AI Technical Summary

Problems solved by technology

[0009] In summary, the existing click-through rate prediction technology cannot be applied to deep learning models, and there is expensive network communication overhead in large-scale distributed training scenarios; existing graph processing algorithms are not suitable for the embedding model used for click-through rate prediction. Training; the existing traditional consensus protocols and learning and training systems do not take into account the update dependencies between embeddings; high overhead and low efficiency

Images