Method, data processing method, device and storage medium for constructing model inference network

Abstract

The present application relates to a method, data processing method, apparatus, computer equipment and storage medium for constructing a model inference network. The method includes: acquiring a deep learning network, where the deep learning network includes a plurality of network layers; acquiring test data; performing compilation and detection on each network layer according to the test data, obtaining compilation results and detection results, and according to the compilation of each network layer The results and / or detection results determine the resource allocation strategy of each network layer, build a model inference network according to the resource allocation strategy of each network layer, and obtain the data to be processed; input the data to be processed into the model inference network, and pass the resources in the model inference network through the model inference network. The network layer whose configuration strategy is to optimize the configuration strategy processes the data to be processed, and the network layer whose resource allocation strategy is the original configuration strategy through the model inference network processes the data to be processed, and obtains the processing result of the data to be processed, which improves the performance of the entire network. Data processing efficiency.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular, to a method, data processing method, apparatus, device and storage medium for constructing a model inference network. Background technique [0002] At present, the network structure of deep learning is becoming more and more complex, and the network often contains thousands of layers of computing layers. However, due to the decentralization of front-end AI development frameworks and the diversity of computing layers, mainstream inference acceleration frameworks are both from the network optimization compiler side and the model accelerator side. There are certain limitations. For example, network optimization compilers often encounter unsupported AI computing layers and cause compilation failures, resulting in the failure of the overall model to be optimized. In addition, even if the compilation is successful, sometimes the model accelerator will not be able to use ...

AI Technical Summary

Problems solved by technology

[0002] At present, the deep learning network structure is becoming more and more complex, and the network often contains thousands of computing layers. However, due to the decentralization of front-end AI development frameworks and the diversity of computing layers, mainstream inference acceleration frameworks are optimized from the network optimization compiler side to the model accelerator side. There are certain limitations

For example, the network optimization compiler often encounters unsupported AI computing layers, causing compilation failures, which in turn leads to the inability to optimize the overall model

In addition, even if the compilation is successful, sometimes the model accelerator will not be able to use the accelerated model due to excessive calculation errors after optimization of some calculation layers.

Images