Unlock instant, AI-driven research and patent intelligence for your innovation.

Generation method, optimization method and device of deep learning model, equipment and medium

A deep learning and model technology, applied in the computer field, can solve a lot of labor costs, learning costs, trial and error costs, etc.

Active Publication Date: 2021-08-20
上海燧原科技有限公司
View PDF8 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

For artificial intelligence enterprises, the model codes used in business need a lot of intrusive modification to convert from the traditional data parallel mode to the model parallel mode, and it requires a lot of labor costs, learning costs, and trial and error costs. A certain project cycle is required to migrate the business model code

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Generation method, optimization method and device of deep learning model, equipment and medium
  • Generation method, optimization method and device of deep learning model, equipment and medium
  • Generation method, optimization method and device of deep learning model, equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0040] figure 1 It is a flowchart of a deep learning model generation method provided by Embodiment 1 of the present invention. This embodiment is applicable to the case where the deep learning model supports model parallelism. This method can be generated by the deep learning model provided by the embodiment of the present invention device, which can be implemented in the form of software and / or hardware, and can generally be integrated into electronic equipment.

[0041] Such as figure 1 As shown, the generation method of the deep learning model provided by this embodiment includes:

[0042] S110. Obtain the parallel configuration parameters of the business model code and the target model.

[0043] The business model code refers to the model code used to set the business in the setting scenario, for example, it can be the model code used to set the business (such as natural language processing, etc.) in the artificial intelligence scenario. Exemplarily, the business model...

Embodiment 2

[0088] image 3 It is a flowchart of a method for generating a deep learning model provided by Embodiment 2 of the present invention. This embodiment is embodied on the basis of the foregoing embodiments, wherein the initial abstract syntax tree is constructed according to the business model code, and the initial abstract syntax tree is updated to the target abstract syntax tree according to the parallel configuration parameters of the target model, which may be specifically as follows:

[0089] Construct the module dependency graph of the business model code, and the initial abstract syntax tree corresponding to each module on the module dependency graph; based on the PASS mechanism, update the initial abstract syntax tree corresponding to each module to the corresponding target abstraction according to the parallel configuration parameters of the target model syntax tree.

[0090] Such as image 3 As shown, the generation method of the deep learning model provided by this ...

Embodiment 3

[0149] Figure 7 It is a flowchart of a method for generating a deep learning model provided by Embodiment 3 of the present invention. This embodiment provides a specific implementation manner on the basis of the foregoing embodiments.

[0150] Such as Figure 7 As shown, the generation method of the deep learning model provided by this embodiment includes:

[0151] S310. Obtain a business model code.

[0152] S320. Determine whether the business model code uses model parallelism. If yes, execute S330. If not, execute S3100.

[0153] S330. Initialize and acquire parallel configuration parameters of the target model.

[0154] Optionally, the parallel configuration parameters of the target model include: configuration parameters corresponding to the fragmentation mode, and / or configuration parameters corresponding to the pipeline mode, which may specifically include fragmentation size, fragmentation dimension, number of pipeline segments, number of pipeline aggregations, equ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a generation method and device of a deep learning model, an optimization method and device, equipment and a medium. The deep learning model generation method comprises the steps of obtaining a business model code and a target model parallel configuration parameter; constructing an initial abstract syntax tree according to the business model code, and updating the initial abstract syntax tree into a target abstract syntax tree according to the target model parallel configuration parameters; loading the target abstract syntax tree to a target deep learning framework, and performing computational graph compiling on the target abstract syntax tree through the target deep learning framework to generate a deep learning model which is executed in parallel on a plurality of devices. The technical scheme mainly acts on a model code compiling stage, intrusive modification of business model codes can be avoided, and automation of enabling the deep learning model to support model parallelism is achieved.

Description

technical field [0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a method for generating a deep learning model, an optimization method, a device, a device, and a medium. Background technique [0002] With the development of deep learning, the scale of deep learning models has shown a trend of rapid growth, similar to Bert (Bidirectional Encoder Representations from Transformers, bidirectional encoder representation based on transformers)-Large, GPT3 (General Pre-trained Transformer-3) For new ultra-large-scale deep learning models, the storage capacity of mainstream artificial intelligence chips or GPGPU (General-Purpose computing on Graphics Processing Units, general-purpose graphics processors) is far from enough to accommodate the storage requirements of a single model for training. The clustered deployment of resources has become an inevitable trend. [0003] At present, mainstream model parallel frameworks...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N20/00G06F40/253
CPCG06N20/00G06F40/253
Inventor 石恒姜天雨刘育良
Owner 上海燧原科技有限公司