Compiler-based tensor data calculation and reasoning method, device, and storage medium

Abstract

The present invention relates to the field of deep learning technology, and discloses a compiler-based tensor data computational reasoning method, device, and storage medium. The method comprises: obtaining a specified input model file, and obtaining a target unified format intermediate representation according to the specified input model file; generating a target self-decoding according to the target unified format intermediate representation and an intermediate layer optimization strategy; constructing a target device executable file according to the target self-decoding, a hardware driver, and a development tool library; reasoning the target tensor data according to the target device executable file and a model type; through the above-mentioned method, format unification is performed based on the front-end access layer, and then data optimization is performed based on the intermediate conversion layer and the intermediate layer optimization strategy, and computational reasoning is performed based on the target device executable file and the model type constructed by the terminal execution layer, thereby being able to adapt to computational reasoning of various frameworks and platforms and simplifying the development workload of developers for model reasoning.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a compiler-based tensor data computation and inference method, device, and storage medium. Background Art

[0002] With the continuous development of artificial intelligence, deep learning technology has been widely used in various fields, and the application of deep learning is inseparable from the inference framework, such as TensorFlow, PyTorch, and TNN frameworks. However, different inference frameworks have different functions. For example, TensorFlow and PyTorch are platform-level frameworks that can be used for training and inference, while the TNN framework can only be used for inference. Regardless of the function of the framework, some related acceleration devices are adapted behind it, such as NVIDIA GPU, Apple M1, and ARM, etc. The adaptation and optimization degree of different inference frameworks are not the same. In addition, different types of frameworks need to adapt to multiple model files because of different training platforms, and the model files include multiple data types, operation flows, and data flows, which ultimately leads to the fragmentation of deep learning inference. This fragmentation adds additional development work for developers when performing model inference, and also requires developers to understand the conversion methods between various frameworks and platforms, resulting in a more cumbersome development workload.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a compiler-based tensor data computational reasoning method, device and storage medium, aiming to solve the technical problem that the existing technology cannot adapt to the computational reasoning of various frameworks and platforms, resulting in a relatively cumbersome development workload for developers to perform model reasoning.

[0005] To achieve the above object, the present invention provides a compiler-based tensor data calculation and reasoning method, which includes the following steps:

[0006] Obtaining a specified input model file, and obtaining a target unified format intermediate representation element according to the specified input model file;

[0007] Generate a target self-decoding according to the target unified format intermediate representation element and the intermediate layer optimization strategy;

[0008] Building a target device executable file based on the target self-decoding, hardware driver and development tool library;

[0009] Perform inference on target tensor data based on the target device executable file and model type.

[0010] Optionally, the acquiring of a specified input model file and obtaining a target unified format intermediate representation according to the specified input model file includes:

[0011] Obtain a specified input model file, and obtain corresponding numerical weights, tensor structures, and computational graphs according to the specified input model file;

[0012] Respectively obtaining the numerical weight, the tensor structure, and the data format of the computation graph;

[0013] When the condition that the data formats of any two of the numerical weights, the tensor structure and the computational graph are inconsistent is met, the numerical weights, the tensor structure and the computational graph are converted into a target unified format intermediate representation through the front-end input layer.

[0014] Optionally, generating a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy includes:

[0015] Obtaining the number of layers of the intermediate conversion layer, and obtaining a data optimization strategy set at each layer according to the number of layers of the intermediate conversion layer and the intermediate layer optimization strategy;

[0016] Optimizing the target unified format intermediate representation element according to the data optimization strategy of each layer to obtain a tensor calculation model file;

[0017] Generate target self-decoding according to the tensor calculation model file.

[0018] Optionally, generating a target self-decoding according to the tensor calculation model file includes:

[0019] Obtaining a corresponding file type and tensor calculation model data according to the tensor calculation model file;

[0020] Selecting a target encoding strategy from an encoding strategy set according to the file type;

[0021] The tensor calculation model data is encoded using the target encoding strategy to obtain target self-decoding.

[0022] Optionally, the step of constructing a target device executable file based on the target self-decoding, hardware driver, and development tool library includes:

[0023] Obtain a certain number of hardware driver collections and development tool library collections;

[0024] Determining a corresponding self-decoding type according to the target self-decoding;

[0025] Matching the self-decoding type with the hardware driver set to obtain a hardware driver;

[0026] Matching the self-decoding type with the development tool library set to obtain a development tool library;

[0027] The target self-decoding is translated according to the hardware driver and the development tool library to obtain a target device executable file.

[0028] Optionally, translating the target self-decoding according to the hardware driver and the development tool library to obtain a target device executable file includes:

[0029] Compile the target self-decoding according to the hardware driver and the development tool library to obtain a current assembly file;

[0030] Assembling the current assembly file to obtain a current binary file;

[0031] The current binary file is linked with the database to be called to obtain a target device executable file.

[0032] Optionally, reasoning on target tensor data according to the target device executable file and model type includes:

[0033] Acquire information of the device to be operated, and determine the target operating device according to the information of the device to be operated;

[0034] Executing the target device executable file through the target running device to obtain a file execution result;

[0035] Determining a model type according to the target device executable file;

[0036] Inference is performed on target tensor data according to the file execution result and the model type.

[0037] In addition, to achieve the above-mentioned purpose, the present invention further proposes a compiler-based tensor data calculation and reasoning device, the compiler-based tensor data calculation and reasoning device comprising:

[0038] An acquisition module, configured to acquire a specified input model file and obtain a target unified format intermediate representation element according to the specified input model file;

[0039] A generation module, configured to generate a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy;

[0040] A construction module, configured to construct a target device executable file based on the target self-decoding, hardware driver, and development tool library;

[0041] An inference module is configured to infer target tensor data based on the target device executable file and the model type.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also proposes a compiler-based tensor data computation and reasoning device, wherein the compiler-based tensor data computation and reasoning device includes: a memory, a processor, and a compiler-based tensor data computation and reasoning program stored on the memory and runnable on the processor, wherein the compiler-based tensor data computation and reasoning program is configured to implement the compiler-based tensor data computation and reasoning method as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a compiler-based tensor data calculation and reasoning program is stored. When the compiler-based tensor data calculation and reasoning program is executed by a processor, the compiler-based tensor data calculation and reasoning method described above is implemented.

[0044] The compiler-based tensor data computational reasoning method proposed in the present invention obtains a specified input model file, obtains a target unified format intermediate representation according to the specified input model file; generates a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy; builds a target device executable file according to the target self-decoding, hardware driver and development tool library; and infers the target tensor data according to the target device executable file and model type. In the above manner, format unification is performed based on the front-end access layer, and then data optimization is performed based on the intermediate conversion layer and the intermediate layer optimization strategy, and computational reasoning is performed based on the target device executable file and model type constructed based on the terminal execution layer, thereby being able to adapt to computational reasoning of various frameworks and platforms and simplifying the development workload of developers for model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural diagram of a compiler-based tensor data computing and inference device in the hardware operating environment involved in the embodiment of the present invention;

[0046] Figure 2 1 is a flowchart of a first embodiment of a compiler-based tensor data calculation and reasoning method according to the present invention;

[0047] Figure 3 1 is a flow chart of a second embodiment of a compiler-based tensor data calculation and reasoning method according to the present invention;

[0048] Figure 4 Schematic diagram of the functional modules of the first embodiment of the compiler-based tensor data calculation and inference device of the present invention.

[0049] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a compiler-based tensor data calculation and inference device in the hardware operating environment involved in the embodiment of the present invention.

[0052] like Figure 1 As shown, the compiler-based tensor data computing and inference device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the compiler-based tensor data computing inference device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a compiler-based tensor data calculation and reasoning program.

[0055] exist Figure 1In the compiler-based tensor data computing and reasoning device shown, the network interface 1004 is mainly used for data communication with the network integration platform workstation; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the compiler-based tensor data computing and reasoning device of the present invention can be set in the compiler-based tensor data computing and reasoning device, and the compiler-based tensor data computing and reasoning device calls the compiler-based tensor data computing and reasoning program stored in the memory 1005 through the processor 1001, and executes the compiler-based tensor data computing and reasoning method provided by the embodiment of the present invention.

[0056] Based on the above hardware structure, an embodiment of the tensor data calculation and reasoning method based on the compiler of the present invention is proposed.

[0057] Reference Figure 2 , Figure 2 2 is a flowchart of the first embodiment of the compiler-based tensor data calculation and reasoning method of the present invention.

[0058] In a first embodiment, the compiler-based tensor data computation inference method includes the following steps:

[0059] Step S10: obtaining a designated input model file, and obtaining a target unified format intermediate representation element according to the designated input model file.

[0060] It should be noted that the execution subject of this embodiment is a compiler-based tensor data calculation and inference device, and can also be other devices that can achieve the same or similar functions, such as a compiler for tensor calculation, etc. This embodiment does not limit this. In this embodiment, a compiler for tensor calculation is used as an example for explanation.

[0061] It should be understood that the compiler for tensor computing supports model files of multiple deep learning frameworks. The design structure of the compiler for tensor computing includes three layers, namely the front-end access layer, the intermediate conversion layer and the terminal execution layer, and is applied to the field of deep learning reasoning.

[0062] It can be understood that the specified input model file refers to the file corresponding to the model that needs to be deployed on the acceleration device. The specified input model file can be of multiple types. The target unified format intermediate representation element refers to the intermediate representation element constructed by the various data in the specified input model file. The target unified format intermediate representation element is for use by the intermediate conversion layer of the next layer.

[0063] Furthermore, step S10 includes: obtaining a specified input model file, and obtaining corresponding numerical weights, tensor structures, and calculation graphs according to the specified input model file; respectively obtaining the data formats of the numerical weights, the tensor structures, and the calculation graphs; and when the condition that the data formats of any two of the numerical weights, the tensor structures, and the calculation graphs are inconsistent is met, converting the numerical weights, tensor structures, and calculation graphs into target unified format intermediate representation elements through the front-end input layer.

[0064] It is understandable that after receiving the specified input model file, the compiler for tensor calculation needs to unify the format of the specified input model file because the model file format of each deep learning framework is different. Specifically, the corresponding numerical weights, tensor structure and calculation graph are obtained according to the specified input model file, and then it is determined whether the data format of the numerical weights, tensor structure and calculation graph are the same. If so, they are directly transmitted to the intermediate conversion layer for processing. If not, the numerical weights, tensor structure and calculation graph are unified into the same format of intermediate representation elements through the front-end access layer, that is, the target unified format intermediate representation elements.

[0065] Step S20: generating a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy.

[0066] It can be understood that target self-decoding refers to deep self-decoding, and the target self-decoding is for use by the terminal execution layer of the next layer. The intermediate layer optimization strategy refers to the optimization strategy set in each layer of the intermediate conversion layer, and each layer can set one or more optimization strategies, such as operator fusion optimization strategy and matrix decomposition optimization strategy. For example, the operator fusion optimization strategy is used to reduce the number of data exchanges between the video memory and the internal memory, and the matrix decomposition optimization strategy is used to increase parallelism. After obtaining the target unified format intermediate representation, the target unified format intermediate representation is optimized through the intermediate layer optimization strategy to generate the target self-decoding.

[0067] Step S30: constructing a target device executable file based on the target self-decoding, hardware driver and development tool library.

[0068] It should be understood that the target device executable file refers to a file that supports execution on a compiler for tensor computing. The target device executable file can be a binary model file. The hardware driver refers to the driver required by the hardware at runtime. The development tool library refers to the tool library for developing different functions that comes with the hardware platform. The hardware driver and development tool library have the highest matching degree with the target self-decoding. After obtaining the target self-decoding, the target device executable file is constructed according to the target self-decoding, hardware driver and development tool library.

[0069] Furthermore, step S30 includes: obtaining a number of hardware driver sets and development tool library sets; determining a corresponding self-decoding type according to the target self-decoding; matching the self-decoding type with the hardware driver set to obtain a hardware driver; matching the self-decoding type with the development tool library set to obtain a development tool library; translating the target self-decoding according to the hardware driver and the development tool library to obtain a target device executable file.

[0070] It can be understood that the self-decoding type refers to the type of target self-decoding. Since different types of self-decoding correspond to different hardware drivers and development tool libraries, it is necessary to match the hardware drivers and development tool libraries from several hardware driver sets and development tool library sets according to the self-decoding type. At this time, the hardware driver and development tool library have the highest matching degree with the target self-decoding, and then the target self-decoding is translated into the target device executable file according to the hardware driver and development tool library.

[0071] Furthermore, the target self-decoding is translated according to the hardware driver and the development tool library to obtain a target device executable file, including: compiling the target self-decoding according to the hardware driver and the development tool library to obtain a current assembly file; assembling the current assembly file to obtain a current binary file; and linking the current binary file with the database to be called to obtain a target device executable file.

[0072] It should be understood that after obtaining the target self-decoding, first, use the hardware driver and developer tool library to assemble the target self-decoding. Specifically, you can use the -S command. After the assembly is completed, get the current assembly file, and then compile the current assembly file. Specifically, you can use the -c command. After the compilation is completed, get the current binary file. The binary file at this time does not support execution on a compiler oriented to tensor calculations. Therefore, it is necessary to link the database to be called. The database to be called refers to the database that needs to be called when the file is set to executable format. Specifically, link the current binary file with the database to be called to obtain the target device executable file. Specifically, you can use the -o command.

[0073] Step S40: inferring the target tensor data according to the target device executable file and the model type.

[0074] It can be understood that the model type refers to the type of model corresponding to the specified input model file. After obtaining the target device executable file, the target device executable file is executed on the target running device. After the execution is completed, the inference result of the target tensor data is obtained. For example, if the model type is an image recognition model, after the execution is completed, the output inference result is the image feature.

[0075] Furthermore, step S40 includes: obtaining information of the device to be run, and determining the target running device based on the information of the device to be run; executing the target device executable file through the target running device to obtain a file execution result; determining a model type based on the target device executable file; and inferring target tensor data based on the file execution result and the model type.

[0076] It should be understood that the information of the device to be run refers to the device information that the developer needs to run. The target running device is determined based on the information of the device to be run, and then the target device executable file is imported into the target running device for execution to obtain the file execution result. Then, the target tensor data is inferred based on the file execution result and the model type, that is, the type of the inference result of the target tensor data is obtained according to the model type, and then the inference result of the target tensor data is obtained according to the file execution result and the model type.

[0077] This embodiment obtains a specified input model file, obtains a target unified format intermediate representation according to the specified input model file; generates a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy; builds a target device executable file according to the target self-decoding, hardware driver and development tool library; infers the target tensor data according to the target device executable file and model type; through the above method, format unification is performed based on the front-end access layer, and then data optimization is performed based on the intermediate conversion layer and the intermediate layer optimization strategy, and then computational reasoning is performed based on the target device executable file and model type constructed by the terminal execution layer, so that it can adapt to computational reasoning of various frameworks and platforms, and simplify the development workload of developers for model reasoning.

[0078] In one embodiment, if Figure 3 Based on the first embodiment, a second embodiment of the compiler-based tensor data calculation and reasoning method of the present invention is proposed. Step S20 includes:

[0079] Step S201 : obtaining the number of intermediate conversion layers, and obtaining data optimization strategies set at each layer according to the number of intermediate conversion layers and the intermediate layer optimization strategy.

[0080] It should be understood that the number of layers refers to the total number of layers in the intermediate conversion layer, and the intermediate-level optimization strategy refers to the optimization strategy set in the intermediate conversion layer. There can be multiple intermediate-level optimization strategies, and then the data optimization strategies set in each layer are obtained according to the number of layers and the intermediate-level optimization strategies. For example, the data optimization strategy set in the first layer is the operator fusion optimization strategy, and the data optimization strategy set in the second layer is the operator fusion optimization strategy and the matrix decomposition optimization strategy.

[0081] Step S202: Optimize the target unified format intermediate representation element according to the data optimization strategy of each layer to obtain a tensor calculation model file.

[0082] It is understandable that due to the differences between each model file and the target device, direct operation will not result in adaptation. Even if adaptation is possible, the best performance solution cannot be obtained. Therefore, format conversion is required. Specifically, after obtaining the target unified format intermediate representation element given by the front-end access layer, the intermediate conversion layer will use data optimization strategies according to the various layers of the specified target to perform multi-level optimization on the target unified format intermediate representation element.

[0083] Step S203: Generate target self-decoding according to the tensor calculation model file.

[0084] Furthermore, step S203 includes: obtaining the corresponding file type and tensor calculation model data according to the tensor calculation model file; selecting a target encoding strategy from the encoding strategy set according to the file type; encoding the tensor calculation model data through the target encoding strategy to obtain target self-decoding.

[0085] It can be understood that the file type refers to the type of tensor computing model file, the tensor computing model data refers to the individual data in the tensor computing model file, the target encoding strategy refers to the strategy for encoding the data, and the target encoding strategy is to select the encoding strategy that is suitable for the tensor computing model data in the encoding set through the file type, and then encode the tensor computing model data through the target encoding strategy to obtain the target self-decoding.

[0086] This embodiment obtains the number of layers of the intermediate conversion layer, and obtains the data optimization strategy set at each layer according to the number of layers of the intermediate conversion layer and the intermediate level optimization strategy; optimizes the target unified format intermediate representation element according to the data optimization strategy of each layer to obtain a tensor calculation model file; generates a target self-decoding according to the tensor calculation model file; through the above method, the data optimization strategy set at each layer is obtained according to the number of layers of the intermediate conversion layer and the intermediate level optimization strategy, and then the target unified format intermediate representation element is optimized according to the data optimization strategy of each layer according to the specified target. After the optimization is completed, the tensor calculation model file is obtained, and then the target self-decoding is automatically generated according to the tensor calculation model file, thereby effectively improving the accuracy of generating the target self-decoding, and then improving the adaptability between the model file and the target device.

[0087] In addition, an embodiment of the present invention also proposes a storage medium, on which a compiler-based tensor data calculation and reasoning program is stored. When the compiler-based tensor data calculation and reasoning program is executed by a processor, the steps of the compiler-based tensor data calculation and reasoning method described above are implemented.

[0088] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0089] In addition, refer to Figure 4 , an embodiment of the present invention further proposes a compiler-based tensor data calculation and reasoning device, the compiler-based tensor data calculation and reasoning device comprising:

[0090] The acquisition module 10 is used to acquire a specified input model file and obtain a target unified format intermediate representation element according to the specified input model file.

[0091] The generating module 20 is configured to generate a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy.

[0092] The construction module 30 is used to construct a target device executable file according to the target self-decoding, hardware driver and development tool library.

[0093] The inference module 40 is configured to infer target tensor data based on the target device executable file and the model type.

[0094] This embodiment obtains a specified input model file, obtains a target unified format intermediate representation according to the specified input model file; generates a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy; builds a target device executable file according to the target self-decoding, hardware driver and development tool library; infers the target tensor data according to the target device executable file and model type; through the above method, format unification is performed based on the front-end access layer, and then data optimization is performed based on the intermediate conversion layer and the intermediate layer optimization strategy, and then computational reasoning is performed based on the target device executable file and model type constructed by the terminal execution layer, so that it can adapt to computational reasoning of various frameworks and platforms, and simplify the development workload of developers for model reasoning.

[0095] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0096] In addition, for technical details not fully described in this embodiment, please refer to the compiler-based tensor data calculation and reasoning method provided in any embodiment of the present invention, and will not be repeated here.

[0097] In one embodiment, the acquisition module 10 is also used to obtain a specified input model file, and obtain corresponding numerical weights, tensor structures and calculation graphs according to the specified input model file; obtain the data formats of the numerical weights, the tensor structures and the calculation graphs respectively; when the condition that the data formats of any two of the numerical weights, the tensor structures and the calculation graphs are inconsistent is met, the numerical weights, tensor structures and calculation graphs are converted into target unified format intermediate representation elements through the front-end input layer.

[0098] In one embodiment, the generation module 20 is also used to obtain the number of layers of the intermediate conversion layer, and obtain the data optimization strategy set at each layer according to the number of layers of the intermediate conversion layer and the intermediate level optimization strategy; optimize the target unified format intermediate representation element according to the data optimization strategy of each layer to obtain a tensor calculation model file; and generate a target self-decoding according to the tensor calculation model file.

[0099] In one embodiment, the generation module 20 is also used to obtain the corresponding file type and tensor calculation model data based on the tensor calculation model file; select a target encoding strategy from the encoding strategy set based on the file type; encode the tensor calculation model data through the target encoding strategy to obtain target self-decoding.

[0100] In one embodiment, the construction module 30 is further used to obtain a number of hardware driver sets and development tool library sets; determine the corresponding self-decoding type according to the target self-decoding; match the self-decoding type with the hardware driver set to obtain a hardware driver; match the self-decoding type with the development tool library set to obtain a development tool library; translate the target self-decoding according to the hardware driver and the development tool library to obtain a target device executable file.

[0101] In one embodiment, the construction module 30 is further used to compile the target self-decoding according to the hardware driver and the development tool library to obtain a current assembly file; assemble the current assembly file to obtain a current binary file; and link the current binary file with the database to be called to obtain a target device executable file.

[0102] In one embodiment, the inference module 40 is also used to obtain information about the device to be run, determine the target running device based on the information about the device to be run; execute the target device executable file through the target running device to obtain a file execution result; determine the model type based on the target device executable file; and infer the target tensor data based on the file execution result and the model type.

[0103] Other embodiments or implementation methods of the compiler-based tensor data calculation and inference device described in the present invention can refer to the above-mentioned method embodiments, which will not be repeated here.

[0104] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0105] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0106] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, an integrated platform workstation, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0107] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A compiler-based tensor data computation and reasoning method, characterized in that: The compiler-based tensor data calculation and reasoning method includes the following steps: Obtaining a specified input model file, and obtaining a target unified format intermediate representation element according to the specified input model file; Generate a target self-decoding according to the target unified format intermediate representation element and the intermediate layer optimization strategy; Building a target device executable file based on the target self-decoding, hardware driver and development tool library; Performing inference on target tensor data based on the target device executable file and model type; The step of obtaining a designated input model file and obtaining a target unified format intermediate representation element according to the designated input model file includes: Obtain a specified input model file, and obtain corresponding numerical weights, tensor structures, and computational graphs according to the specified input model file; Respectively obtaining the numerical weight, the tensor structure, and the data format of the computation graph; When the condition that the data formats of any two of the numerical weights, the tensor structure and the computational graph are inconsistent is met, the numerical weights, the tensor structure and the computational graph are converted into a target unified format intermediate representation through the front-end input layer.

2. The compiler-based tensor data computation and reasoning method according to claim 1, wherein: Generating a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy includes: Obtaining the number of layers of the intermediate conversion layer, and obtaining a data optimization strategy set at each layer according to the number of layers of the intermediate conversion layer and the intermediate layer optimization strategy; Optimizing the target unified format intermediate representation element according to the data optimization strategy of each layer to obtain a tensor calculation model file; Generate target self-decoding according to the tensor calculation model file.

3. The compiler-based tensor data computation and reasoning method according to claim 2, wherein: Generating a target self-decoding according to the tensor calculation model file includes: Obtaining a corresponding file type and tensor calculation model data according to the tensor calculation model file; Selecting a target encoding strategy from an encoding strategy set according to the file type; The tensor calculation model data is encoded using the target encoding strategy to obtain target self-decoding.

4. The compiler-based tensor data computation and reasoning method according to claim 1, wherein: The target device executable file is constructed according to the target self-decoding, hardware driver and development tool library, including: Obtain a certain number of hardware driver collections and development tool library collections; Determining a corresponding self-decoding type according to the target self-decoding; Matching the self-decoding type with the hardware driver set to obtain a hardware driver; Matching the self-decoding type with the development tool library set to obtain a development tool library; The target self-decoding is translated according to the hardware driver and the development tool library to obtain a target device executable file.

5. The compiler-based tensor data calculation and reasoning method according to claim 4, wherein: The step of translating the target self-decoding according to the hardware driver and the development tool library to obtain a target device executable file includes: Compile the target self-decoding according to the hardware driver and the development tool library to obtain a current assembly file; Assembling the current assembly file to obtain a current binary file; The current binary file is linked with the database to be called to obtain a target device executable file.

6. The compiler-based tensor data computation and reasoning method according to any one of claims 1 to 5, wherein: The reasoning on the target tensor data according to the target device executable file and the model type includes: Acquire information of the device to be operated, and determine the target operating device according to the information of the device to be operated; Executing the target device executable file through the target running device to obtain a file execution result; Determining a model type according to the target device executable file; Inference is performed on target tensor data according to the file execution result and the model type.

7. A compiler-based tensor data computing and reasoning device, characterized in that: The compiler-based tensor data calculation and reasoning device includes: An acquisition module, configured to acquire a specified input model file and obtain a target unified format intermediate representation element according to the specified input model file; A generation module, configured to generate a target self-decoding according to the target unified format intermediate representation and the intermediate layer optimization strategy; Creating a module for building a target device executable file based on the target self-decoding, hardware driver and development tool library; an inference module, configured to infer target tensor data based on the target device executable file and model type; The acquisition module is further used to acquire a specified input model file, and obtain corresponding numerical weights, tensor structures and calculation graphs according to the specified input model file; Respectively obtaining the numerical weight, the tensor structure, and the data format of the computation graph; When the condition that the data formats of any two of the numerical weights, the tensor structure and the computational graph are inconsistent is met, the numerical weights, the tensor structure and the computational graph are converted into a target unified format intermediate representation through the front-end input layer.

8. A compiler-based tensor data computing and inference device, characterized in that: The compiler-based tensor data computation and inference device includes: a memory, a processor, and a compiler-based tensor data computation and inference program stored in the memory and executable on the processor, wherein the compiler-based tensor data computation and inference program is configured to implement the compiler-based tensor data computation and inference method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a compiler-based tensor data calculation and reasoning program, which, when executed by a processor, implements the compiler-based tensor data calculation and reasoning method according to any one of claims 1 to 6.

Images