A fast deep learning model building system compatible with multiple frameworks

Abstract

The present invention discloses a deep learning model rapid construction system compatible with multiple frameworks, including: a device layer, a backend layer and a user layer, the device layer includes several hardware modules, the backend layer includes several deep learning frameworks, each deep learning framework includes several underlying operators, each underlying operator has a unified operator interface, the user layer includes a model abstract module, the model abstract module includes model training parameters and calculation logic corresponding to each deep learning framework, the user layer is used to call the underlying operators in the backend layer to build a deep learning model, and train the deep learning model through the hardware modules of the device layer, wherein the deep learning model is built based on an API provided by the user layer. The construction system of the present invention is compatible with different deep learning frameworks and different hardware, so that users can quickly and efficiently build deep learning models, reducing tedious code development and code migration work for switching different deep learning frameworks.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a deep learning model rapid construction system compatible with multiple frameworks. Background Art

[0002] In recent years, deep learning has been widely used in speech recognition, machine translation, computer vision and other fields, and has achieved good results. With the rise of deep learning theory, a variety of deep learning programming frameworks have emerged. Deep learning frameworks such as TensorFlow, Pytorch, MindSpore and other open source frameworks have different naming rules and usage rules for operator API parameters. When developers develop deep learning applications, they need to spend a lot of energy to learn the corresponding framework language and refactor the code when migrating from one framework to another. In addition, TensorFlow only provides a lower-level operator API, which is more complicated to use and has a high learning cost.

[0003] Although the existing open source framework Keras can achieve compatibility with multiple deep learning frameworks, it unifies the operator APIs of the deep learning frameworks into the same operator API function interface when achieving compatibility with different deep learning frameworks, and then calls the underlying operator API through a high-level interface to implement each neural network component. Finally, the various components are assembled through the Models package. This double encapsulation has a large performance loss, and the unified interface API can only solve the function interface provided by the underlying operators of the framework. The class-based interface provided by the underlying framework of MindSpore will not be compatible.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a deep learning model rapid building system compatible with multiple frameworks is provided, aiming to solve the problem that the existing model building system has a significant performance damage after being encapsulated twice and is not compatible with deep learning frameworks with class-based interfaces.

[0006] The technical solution adopted by the present invention to solve the problem is as follows:

[0007] An embodiment of the present invention provides a deep learning model rapid construction system compatible with multiple frameworks, which includes: a device layer, a backend layer and a user layer, the device layer includes a plurality of hardware modules, the backend layer includes a plurality of deep learning frameworks, each of the deep learning frameworks includes a plurality of underlying operators, each of the underlying operators has a unified operator interface, the user layer includes a model abstraction module, the model abstraction module includes model training parameters and calculation logic corresponding to each of the deep learning frameworks, the user layer is used to call the underlying operators in the backend layer to build a deep learning model, and train the deep learning model through the hardware modules of the device layer, wherein the deep learning model is built based on the model training parameters and calculation logic in the model abstraction module.

[0008] The deep learning model rapid construction system compatible with multiple frameworks, wherein the user layer also includes a model construction module, a model training module, a parameter configuration module and a data processing module;

[0009] The model building module is used to call the underlying operator in the backend layer to build a deep learning model;

[0010] The model training module is used to call the underlying operator in the backend layer to determine the hardware module used to train the deep learning model;

[0011] The model parameter configuration module is used to call the underlying operator in the backend layer to configure the parameters of the deep learning model;

[0012] The data processing module is used to call the underlying operators in the backend layer to perform data processing.

[0013] The deep learning model rapid construction system compatible with multiple frameworks, wherein the model construction module includes a model construction component for building the deep learning model, and the model construction component includes a convolutional layer, a pooling layer, a fully connected layer, a noise layer, a quantization layer, a merging layer, an expansion layer, an embedding layer and a Dropout layer.

[0014] The deep learning model rapid building system compatible with multiple frameworks, wherein the model abstract module includes a module base class corresponding to each of the deep learning frameworks, and the model building algorithm inherits from the module base class.

[0015] The deep learning model rapid construction system compatible with multiple frameworks, wherein the parameter configuration module includes a loss function, an optimization function, an activation function, a normalization function and an initialization function.

[0016] The deep learning model rapid construction system compatible with multiple frameworks, wherein the data processing module includes a sequence processing unit, a text processing unit and an image processing unit;

[0017] The sequence processing unit is used to call the underlying operator in the backend layer to process the sequence;

[0018] The text processing unit is used to call the bottom layer operator in the backend layer to process the text;

[0019] The image processing unit is used to call the underlying operator in the backend layer to process the image.

[0020] The deep learning model rapid construction system compatible with multiple frameworks, wherein the model training module includes model training units corresponding to each of the deep learning frameworks.

[0021] The system for rapidly building a deep learning model compatible with multiple frameworks, wherein some of the hardware modules include GPU, CPU, Ascend and TPU.

[0022] The system for rapidly building a deep learning model that is compatible with multiple frameworks, wherein the system further includes: an application layer, wherein the application layer is used to store the built deep learning model.

[0023] The deep learning model rapid construction system compatible with multiple frameworks, wherein the construction system also includes: a language layer, which provides a programming language for calling the underlying operator.

[0024] Beneficial effects of the present invention: The rapid construction system of the embodiment of the present invention includes a device layer, a backend layer and a user layer, the device layer includes several hardware modules, the backend layer includes several deep learning frameworks, each of the deep learning frameworks includes several underlying operators, each of the underlying operators has a unified operator interface, the user layer includes a model abstraction module, the model abstraction module includes model training parameters and calculation logic corresponding to each of the deep learning frameworks, the user layer is used to call the underlying operators in the backend layer to build a deep learning model, and train the deep learning model through the hardware modules of the device layer, wherein the deep learning model is built based on the model training parameters and calculation logic in the model abstraction module. The present invention standardizes the underlying operators of different deep learning frameworks by designing a unified operator API, which can be compatible with deep learning frameworks that provide function interfaces and class interfaces, and designs model abstraction modules according to the characteristics of different deep learning frameworks, so as to be compatible with different deep learning frameworks and different hardware, so that users can quickly and efficiently build deep learning models, reduce tedious code development and code migration work for switching different deep learning frameworks. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is an overall design framework diagram of a deep learning model rapid construction system compatible with multiple frameworks provided by an embodiment of the present invention;

[0027] Figure 2 It is a detailed structural diagram of a deep learning model rapid construction system compatible with multiple frameworks provided by an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the implementation principle of a system for rapidly building a deep learning model that is compatible with multiple frameworks provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0031] Although the existing open source framework Keras can achieve compatibility with multiple deep learning frameworks, it unifies the operator APIs of deep learning frameworks TensorFlow, Theano, and CNTK into the same operator API function interface, and then calls the underlying operator API through a high-level interface to implement each neural network component. Finally, the various components are assembled through the Models package to achieve compatibility with different framework training. In the Keras implementation, the underlying API is called to implement the network components and then assembled through Models. This double encapsulation has a large performance loss, and the unified interface API can only solve the function interface provided by the underlying operator of the framework. For example, the underlying interface provided by MindSpore is incompatible with the class interface.

[0032] In order to solve the problems of the prior art, this embodiment provides a deep learning model fast building system compatible with multiple frameworks, such as Figure 1As shown, the construction system (TensorLayer) includes from bottom to top: device layer (Serves), backend layer (Backends) and user layer (APIs), the device layer includes several hardware modules, the backend layer includes several deep learning frameworks, each of the deep learning frameworks includes several underlying operators, each of the underlying operators has a unified operator interface, the user layer includes a model abstraction module, the model abstraction module includes the model training parameters and calculation logic corresponding to each of the deep learning frameworks, the user layer is used to call the underlying operator corresponding to the deep learning framework in the backend layer to build a deep learning model, and train the deep learning model on the hardware module of the device layer, wherein the deep learning model is built based on the model training parameters and calculation logic in the model abstraction module. In this embodiment, the construction system standardizes the underlying operators of different deep learning frameworks by designing a unified operator API, which can be compatible with deep learning frameworks that provide function interfaces and class interfaces, and designs model abstraction modules according to the characteristics of different deep learning frameworks, so as to be compatible with different deep learning frameworks and different hardware, so that users can quickly and efficiently build deep learning models, reduce tedious code development and code migration work for switching different deep learning frameworks.

[0033] The deep learning framework includes existing open source frameworks such as TensorFlow, Pytorch, MindSpore, etc. When building a deep learning model, users can use the default deep learning framework of the building system, or select a deep learning framework as needed. After confirming the construction of the deep learning framework, the underlying operator under the determined deep learning framework can be called through the user layer to build the model. The following is the implementation code for users to set the deep learning framework and call the interface:

[0034] import numpy as np

[0035] import time

[0036] import os

[0037] #Set the training framework called by the backend

[0038] #os.environ['TL_BACKEND']='tensorflow'

[0039] os.environ['TL_BACKEND']='mindspore'

[0040] import tensorflow as tf

[0041] import tensorlayer as tl

[0042] from tensorlayer.layers import Module

[0043] from tensorlayer.layers import Dense,Dropout#This line provides Layers with the interface for users, Dense and Dropout.

[0044] like Figure 1 and Figure 2 As shown, in a specific implementation, the user layer also includes a model building module (Layers), a model training module (Models), a parameter configuration module (Configuration) and a data processing module (Dataflow), wherein the model building module is used to call the underlying operators of each deep learning framework in the backend layer to build a deep learning model; the model training module is used to call the underlying operators of each deep learning framework in the backend layer to determine the hardware module used to train the deep learning model; the model parameter configuration module is used to call the underlying operators of each deep learning framework in the backend layer to perform parameter configuration on the deep learning model; the data processing module is used to call the underlying operators corresponding to the deep learning framework in the backend layer to perform data processing.

[0045] Furthermore, if Figure 2As shown, the model building module includes model building components for implementing commonly used neural network layers such as convolutional layers, pooling layers, and fully connected layers, and the model building components include convolutional layers, pooling layers, fully connected layers, noise layers, quantization layers, merging layers, extension layers, embedding layers, and Dropout layers. The backend layer includes the underlying operators of various deep learning frameworks such as TensorFlow, MindSpore, and Dragon, each of which has a unified operator interface. The model building module can implement the construction of commonly used neural network layers such as convolutional layers, pooling layers, and fully connected layers by calling the underlying operators of various deep learning frameworks in the backend layer. For example, when using TensorFlow to build a fully connected layer, you need to use TensorFlow's underlying operators matmul and bias_add. The model building module can build a fully connected layer by calling TensorFlow's underlying operators matmul and bias_add in the backend layer. Since the underlying operators of each deep learning framework in the backend layer have a unified operator interface, when you need to use other frameworks such as MindSpore, Dragon, etc. to build a fully connected layer, there is no need to modify the code. You can complete the underlying operator call of the corresponding framework by switching the backend, thereby achieving fast and efficient construction of deep learning models and reducing tedious code development and code migration work for switching different frameworks.

[0046] In order to solve the problem of different framework model building methods, the building system (TensorLayer) in this embodiment also abstracts the model according to the characteristics of different deep learning frameworks, such as Figure 3As shown, the user layer includes a model abstract module (Core), and the model abstract module includes module base classes (Module base class) corresponding to different deep learning frameworks, for example, the module base class Tensorflow_Module corresponding to Tensorflow, and the module base class Mindspore_Module corresponding to Mindspore. The Module base class includes model training parameters and calculation logic, and TensorLayer performs model abstraction through the Module base class. Each model building component in the model building module (Layer), such as convolution layer (Convolution), pooling layer (Pooling), fully connected layer (Dense), noise layer (Noise), quantization layer (Quantize), merging layer (Merge), extension layer (Extend), embedding layer (Embedding), Dropout layer (Dropout), etc., all inherit the Module base class. The following is the implementation code for building a three-layer perceptron neural network model using the model building model of this embodiment:

[0047]

[0048]

[0049] Furthermore, in the Module base class, registered Parameters objects are stored through self._params, Layer objects are stored through self._layers, attribute assignment and management are implemented through _setattr__ and __getattr__, the construction of training parameters and the instantiation of the underlying operators of the backend layer are implemented in the build function, and the calculation logic is defined in the forward function.

[0050] Continue to refer to Figure 2As shown, the parameter configuration module includes a model building function for building and training a deep learning model, and the model building function includes a loss function, an optimization function, an activation function, a normalization function, and an initialization function. The back-end layer includes the underlying operators of various deep learning frameworks such as TensorFlow, MindSpore, and Dragon, each of which has a unified operator interface. The parameter configuration module can implement functions such as loss functions, optimization functions, activation functions, normalization functions, and initialization functions for building and training deep learning models by calling the underlying operators of various deep learning frameworks in the back-end layer.

[0051] In a specific implementation, the data processing module is used to call the underlying operators of each deep learning framework in the backend layer to implement sequence, text and image data processing. Figure 2 As shown, the data processing module includes a sequence processing unit (Sequence), a text processing unit (Text) and an image processing unit (Image). The sequence processing unit is used to call the underlying operators of each deep learning framework in the backend layer to process the sequence data; the text processing unit is used to call the underlying operators of each deep learning framework in the backend layer to process the text data; the image processing unit is used to call the underlying operators of each deep learning framework in the backend layer to process the image data.

[0052] In a specific embodiment, the model training module includes model training units corresponding to each of the deep learning frameworks, for example, the model training unit corresponding to Mindspore is Mindspore_train, and the model training unit corresponding to Tensorflow is Tensorflow_train, and each of the model training units is used to call the underlying operator of the corresponding deep learning framework in the backend layer to determine the hardware module used to train the deep learning model. For example, Mindspore_train is used to call the underlying operator of Mindspore in the backend layer to determine the hardware module used to train the deep learning model, and Tensorflow_train is used to call the underlying operator of Tensorflow in the backend layer to determine the hardware module used to train the deep learning model.

[0053] In a specific embodiment, the device layer includes several hardware modules for training, and the several hardware modules include a graphics processing unit (GPU), a central processing unit (CPU), an Ascend, and a tensor processing unit (TPU). During the deep learning model construction process, the deep learning model training can be performed on a suitable hardware module according to the user's hardware environment.

[0054] In a specific embodiment, the construction system also includes: an application layer (Applications) arranged above the user layer, and the application layer is used to store the constructed deep learning model for direct use by the user when needed. For example, the application layer includes a deep convolutional generative adversarial network (DCGAN), a super-resolution generative adversarial network (SRGAN), a chatbot (Chatbot), a style transfer (Style Transfer), etc.

[0055] In a specific implementation, the construction system further includes: a language layer (Language) disposed between the user layer and the application layer, the language layer providing a programming language for calling the underlying operator, and the programming language provided by the language layer in this embodiment is Python.

[0056] In summary, the present invention discloses a deep learning model rapid construction system compatible with multiple frameworks, including: a device layer, a backend layer and a user layer, the device layer includes several hardware modules, the backend layer includes several deep learning frameworks, each of the deep learning frameworks includes several underlying operators, each of the underlying operators has a unified operator interface, the user layer includes a model abstract module, the model abstract module includes model training parameters and calculation logic corresponding to each of the deep learning frameworks, the user layer is used to call the underlying operators in the backend layer to build a deep learning model, and train the deep learning model through the hardware modules of the device layer, wherein the deep learning model is built based on the model training parameters and calculation logic in the model abstract module. The present invention standardizes the underlying operators of different deep learning frameworks by designing a unified operator API, can be compatible with deep learning frameworks that provide function interfaces and class interfaces, designs model abstract modules according to the characteristics of different deep learning frameworks, and is compatible with different deep learning frameworks and different hardware, so that users can quickly and efficiently build deep learning models, reduce tedious code development and code migration work for switching different deep learning frameworks.

[0057] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A deep learning model rapid construction system compatible with multiple frameworks, characterized by: include: The device layer includes several hardware modules, the backend layer includes several deep learning frameworks, each of which includes several underlying operators, and each of which has a unified operator interface. The user layer includes a model abstraction module, which includes model training parameters and calculation logic corresponding to each of the deep learning frameworks. The user layer is used to call the underlying operators in the backend layer to build a deep learning model, and train the deep learning model through the hardware modules of the device layer, wherein the deep learning model is built based on the model training parameters and calculation logic in the model abstraction module. The model abstract module includes module base classes corresponding to each of the deep learning frameworks; In the module base class, self._params is used to store registered Parameters objects, self._layers is used to store Layer objects, _setattr__ and __getattr__ are used to implement attribute assignment and management, the construction of training parameters and the instantiation of the underlying operators of the backend layer are implemented in the build function, and the calculation logic is defined in the forward function.

2. The multi-framework compatible deep learning model rapid construction system according to claim 1, characterized in that: The user layer also includes a model building module, a model training module, a parameter configuration module and a data processing module; The model building module is used to call the underlying operator in the backend layer to build a deep learning model; The model training module is used to call the underlying operator in the backend layer to determine the hardware module used to train the deep learning model; The parameter configuration module is used to call the underlying operator in the backend layer to perform parameter configuration on the deep learning model; The data processing module is used to call the underlying operators in the backend layer to perform data processing.

3. The multi-framework compatible deep learning model rapid construction system according to claim 2, characterized in that: The model building module includes a model building component for building the deep learning model, and the model building component includes a convolution layer, a pooling layer, a fully connected layer, a noise layer, a quantization layer, a merging layer, an expansion layer, an embedding layer and a Dropout layer.

4. The multi-framework compatible deep learning model rapid construction system according to claim 3 is characterized in that: The model building components all inherit from the module base class.

5. The multi-framework compatible deep learning model rapid construction system according to claim 2, characterized in that: The parameter configuration module includes a loss function, an optimization function, an activation function, a normalization function and an initialization function.

6. The multi-framework compatible deep learning model rapid construction system according to claim 2, characterized in that: The data processing module includes a sequence processing unit, a text processing unit and an image processing unit; The sequence processing unit is used to call the underlying operator in the backend layer to process the sequence data; The text processing unit is used to call the underlying operator in the backend layer to process the text data; The image processing unit is used to call the underlying operator in the backend layer to process the image data.

7. The multi-framework compatible deep learning model rapid construction system according to claim 2, characterized in that: The model training module includes model training units corresponding to each of the deep learning frameworks.

8. The multi-framework compatible deep learning model rapid construction system according to claim 1, characterized in that: Some of the hardware modules include GPU, CPU, Ascend and TPU.

9. The multi-framework compatible deep learning model rapid construction system according to claim 1, characterized in that: The construction system also includes: an application layer, and the application layer is used to store the constructed deep learning model.

10. The multi-framework compatible deep learning model rapid construction system according to claim 1, characterized in that: The construction system also includes: a language layer, which provides a programming language for calling the underlying operator.

Images