Modeling method and device for distributed data routing for neural network computing
A technology of distributed data and modeling methods, applied in the field of computer systems, can solve problems such as the lack of distributed routing strategies, and achieve the effect of low difficulty
Active Publication Date: 2022-08-09
ZHEJIANG LAB
View PDF2 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
In the process of deploying the tensor data generated by the global logical computing graph to the physical computing graph of different device processes when compiling the neural network distributed model, the existing deep learning compiler lacks a unified distributed routing strategy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0058] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention, and not to limit the scope of the present invention. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present invention.
[0059] The embodiment of the present invention provides a modeling method for distributed data routing oriented to neural network computing, and provides a dynamic graph execution method for neural network model computing in a deep learning training system. The modeling method for distributed data routing for neural network computing includes three main processes: the design of physical tensor distributed...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The present invention provides a method and device for modeling distributed data routing for neural network computing, including the following steps: S1: Designing distributed attributes of physical tensors: abstracting the mapping relationship between logical tensors and physical tensors as There are three distributed attributes: broadcasting attribute, spreading attribute and local reduction attribute; S2: infer the distributed attribute of the output tensor: specify the distributed attribute of the input tensor, and then deduce the output tensor according to the known distributed attribute of the input tensor The legal distributed attribute of the quantity; S3: according to the distributed attribute situation, judge whether it is necessary to insert the intermediate communication primitive to obtain the distributed attribute of the local physical tensor; use the described modeling method and The device builds the model, and the difficulty of distributed design and development is low, which promotes the development of the application of large-scale deep neural network models.
Description
technical field [0001] The invention relates to the field of computer systems based on a specific computing model, in particular to a modeling method and device for distributed data routing oriented to neural network computing. Background technique [0002] With the rapid development of artificial intelligence industrial applications, distributed training systems for large-scale deep neural network models have increasingly become a research hotspot in academia and industry. The existing deep learning compilers lack a unified distributed routing strategy in the process of deploying the tensor data generated by the global logical computing graph to the physical computing graph of different device processes when compiling the distributed model of the neural network. SUMMARY OF THE INVENTION [0003] The purpose of the present invention is to provide a modeling method and device for distributed data routing oriented to neural network computing, so as to overcome the deficienci...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L45/00H04L45/44G06N3/08
CPCH04L45/08H04L45/44G06N3/08Y02D10/00H04L41/145H04L41/0806H04L41/16
Inventor 王宏升何水兵鲍虎军陈光
Owner ZHEJIANG LAB