Recommendation model distributed training method based on double-layer index embedding layer and GPU (Graphics Processing Unit)

A training method and embedding layer technology, applied in the field of distributed computing and machine learning, can solve problems such as poor scalability and low throughput of model training, and achieve the effect of improving total throughput and enhancing scalability

Pending Publication Date: 2022-02-08
SHANGHAI JIAO TONG UNIV
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a recommended model distributed training method based on a double-layer index em

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Recommendation model distributed training method based on double-layer index embedding layer and GPU (Graphics Processing Unit)
  • Recommendation model distributed training method based on double-layer index embedding layer and GPU (Graphics Processing Unit)
  • Recommendation model distributed training method based on double-layer index embedding layer and GPU (Graphics Processing Unit)

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention.

[0021] The purpose of the embodiments of the present invention is to provide a recommended model distributed training method and server based on a double-layer index embedding layer, which are used to solve the technical problems of low throughput and poor scalability of model training in existing centralized and distributed scenarios.

[0022] Embodiments of the present invention relate to better learning long-term dependencies and hi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a recommendation model distributed training method based on a double-layer index embedded layer and a GPU, the method comprises the following steps: constructing two layers of indexes for an embedded layer vector at a server node and a computing node, the server node being a static index, and the computing node being a dynamic index; introducing a sampler into the server node, and formulating a fragmentation strategy according to sampling data; and introducing a ping-pong buffer area into the computing node to read and write an embedded layer vector, and prefetching the embedded layer vector based on the input of adjacent iterations to form a data prefetching assembly line. According to the method, on the premise of ensuring that the prediction performance of the model is not reduced, the total throughput of recommendation model training is improved, the expandability of distributed training is enhanced, and the training of a large-scale embedded layer recommendation model is effectively supported.

Description

technical field [0001] The invention relates to the technical field of distributed computing and machine learning, in particular to a recommended model distributed training method based on a double-layer index embedding layer and a GPU. Background technique [0002] With the widespread popularity of the Internet, the scale of users and the amount of information are increasing day by day, and the problem of information overload is becoming more and more obvious. The recommendation system is one of the solutions to the problem of information overload. The recommendation system guides users to discover their own information needs by analyzing the user's historical behavior information, and provides users with personalized recommendation services. Industrial recommendation models usually have the characteristics of huge training data, complex models, and strong timeliness. Distributed training and computing hardware acceleration have become necessary conditions for solving large...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N20/00G06F9/48G06F9/54G06F9/52
CPCG06N20/00G06F9/4881G06F9/544G06F9/526
Inventor 陈全白铠豪
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap