Method and device for inserting conversion operator

Abstract

The invention provides a method for inserting a conversion operator. The method comprises the following steps: determining an initial data format of a first operator in a neural network; determining a current data format of the first operator; and when the initial data format of the first operator is different from the current data format, inserting at least one first conversion operator and at least one second conversion operator around the first operator, wherein the first conversion operator is used for converting the input data of the first operator into the current data format from the initial data format, and the second conversion operator is used for converting the output data of the first operator from the current data format to the initial data format. In the method, the initial data format is a data format beneficial to hardware performance, and the current data format is a data format beneficial to improving the processing efficiency of the first operator, therefore, the conversion operator is inserted according to the initial data format and the current data format of the first operator, and the requirements of the operator and the requirements of hardware can be met at the same time.

Description

Technical field [0001] The present application relates to the field of artificial intelligence and in particular to a method and apparatus for inserting conversion operator. Background technique [0002] Neural networks are artificial intelligence tools (artificial intelligence, AI), which has a large computation when processing the input data, large-scale memory access and the frequency characteristics. Neural network chip direct memory access speed tend to be small, and the amount of input data and a neural network using a large, relatively low speed in the cache needs large memory, therefore, the data throughput has become a bottleneck in computing neural network. [0003] A method for improving neural network processing speed is reasonable strategy planning data slice, slice the data copied to the high-speed memory to improve memory access efficiency. Data slice strategy relies on hardware supported data arrangement (layout), different hardware applicable data arrangement may...

AI Technical Summary

Problems solved by technology

The high-speed memory directly accessed by the neural network chip is often very small, and the amount of input data used by the neural network is large, which needs to be cached in a relatively low-speed large memory. Therefore, data throughput has become the computing bottleneck of the current neural network.

Images