Inference method and device

Abstract

The invention discloses an inference method and device, and belongs to the field of data processing. Operators which are not supported by the target hardware and do not have corresponding replaceable operators can be determined, namely, the operators of the first type in the neural network model are determined, then the registration operators corresponding to the operators of the first type determined based on the domain-specific language are acquired, and the registration operators are processed. A target executable code supported by the target hardware is acquired, and forward Inference is performed by the target hardware according to the target executable code and the neural network model. Due to the fact that the design of the specific language in the field is irrelevant to hardware, a user can compile the registration operator without knowing hardware characteristics of equipment, and development difficulty is low. Besides, aiming at the same first type of operators possibly corresponding to different hardware, the user only needs to write the operator once to obtain the corresponding registration operator, and the registration operator can be applied to different hardware, so that the development workload is relatively low.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a reasoning method and device. Background technique [0002] Operators are used to indicate a data processing operation and describe a calculation method. For example, neural networks usually include basic convolution operators and pooling operators. Convolution operators are used to indicate a convolution operation. Pooling An operator is used to indicate a pooling operation. Users can also define their own calculation methods according to their needs, and the calculation methods defined by themselves can be called custom operators. A neural network model can be constructed and trained based on basic operators and custom operators. Afterwards, the neural network model can be deployed on various devices for forward inference. [0003] Since the hardware conditions of different devices may be different, the operators supported by different devices may be different. Gen...

AI Technical Summary

Problems solved by technology

After a neural network model is trained, in order to enable the custom operators in the neural network model to adapt to devices with various hardware conditions, the user needs to manually convert the operators that are not supported by each device into The executable code supported by the corresponding device requires a huge development workload, and users need to understand the hardware characteristics of various devices, making development difficult

Images