Virtual machine-based tensor data computation inference method and device, and storage medium

Abstract

The application relates to the technical field of deep learning, and discloses a tensor data calculation and reasoning method and device based on a virtual machine and a storage medium. The method comprises the following steps: obtaining a specified input model file, obtaining a target uniform format intermediate representation element according to the specified input model file, generating a target self-decoding according to the target uniform format intermediate representation element and an intermediate layer optimization strategy, constructing a target device executable file according to the target self-decoding and acceleration device operation information, and performing reasoning on target tensor data according to the target device executable file and a model type. In the foregoing manner, format unification is performed based on a front-end access layer, then data optimization is performed based on an intermediate conversion layer and an intermediate layer optimization strategy, and finally calculation reasoning is performed based on a target device executable file and a model type constructed based on a terminal execution layer, so that the calculation reasoning of various frameworks and platforms can be adapted, and the development workload of model reasoning of developers can be simplified.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus and storage medium for tensor data computation and inference based on virtual machines. Background Technology

[0002] With the continuous development of artificial intelligence, deep learning technology has been widely applied in various industries. The application of deep learning is inseparable from inference frameworks, such as TensorFlow, PyTorch, and TNN. However, different inference frameworks have different functions. For example, TensorFlow and PyTorch are platform-level frameworks that can be used for both training and inference, while the TNN framework can only be used for inference. Regardless of the function of the framework, it is adapted to some related acceleration devices, such as NVIDIA GPUs, Apple M1, and ARM. However, the degree of adaptation and optimization of different inference frameworks is not the same. Moreover, different types of frameworks require adaptation to various model files due to different training platforms. These model files include various data types, operation flows, and data flows, ultimately resulting in the fragmentation of deep learning inference. This fragmentation adds extra development work for developers when performing model inference. Developers also need to understand the conversion methods between various frameworks and platforms, as well as the optimization methods of acceleration devices, making the development workload quite cumbersome.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, and storage medium for tensor data computation and inference based on a virtual machine, aiming to solve the technical problem that existing technologies cannot adapt to computation and inference on various frameworks and platforms, resulting in a relatively cumbersome workload for developers in model inference.

[0005] To achieve the above objectives, the present invention provides a virtual machine-based tensor data computation and inference method, which includes the following steps:

[0006] Obtain a specified input model file, and obtain the target unified format intermediate representation based on the specified input model file;

[0007] The target self-decoder is generated based on the target unified format intermediate representation and intermediate layer optimization strategy;

[0008] Construct an executable file for the target device based on the target self-decoding and accelerated device operation information;

[0009] Inference is performed on the target tensor data based on the target device's executable file and model type.

[0010] Optionally, obtaining a specified input model file and obtaining a target unified format intermediate representation based on the specified input model file includes:

[0011] Obtain a specified input model file, and obtain the corresponding numerical weights, tensor structure, and computational graph based on the specified input model file;

[0012] The data formats of the numerical weights, the tensor structure, and the computation graph are obtained respectively.

[0013] When the data formats of any two of the numerical weights, tensor structures, and computation graphs are inconsistent, the numerical weights, tensor structures, and computation graphs are converted into intermediate representations of the target unified format through the front-end input layer.

[0014] Optionally, generating the target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategy includes:

[0015] Obtain the number of intermediate transformation layers, and based on the number of intermediate transformation layers and the intermediate layer optimization strategy, obtain the data optimization strategy set for each layer;

[0016] Based on the data optimization strategies of each layer, the intermediate representation of the target unified format is optimized to obtain the tensor computation model file;

[0017] The target self-decoder is generated based on the tensor computation model file.

[0018] Optionally, generating the target self-decoder based on the tensor computation model file includes:

[0019] Based on the tensor computation model file, obtain the corresponding file type and tensor computation model data;

[0020] Select a target encoding strategy from the set of encoding strategies based on the file type;

[0021] The tensor computation model data is encoded using the target encoding strategy to obtain the target self-decoding.

[0022] Optionally, constructing the target device executable file based on the target self-decoding and acceleration device operation information includes:

[0023] Obtain information on a number of processors and acceleration devices;

[0024] A number of computational databases are obtained based on the processor information and the acceleration device information;

[0025] The target operating device code is generated layer by layer based on the target self-decoding using the aforementioned quantity calculation database and preset database;

[0026] An executable file for the target device is constructed based on the target device code and the acceleration device operation information.

[0027] Optionally, constructing the target device executable file based on the target operating device code and the acceleration device operation information includes:

[0028] Based on the acceleration device operation information, we obtain acceleration device supported operation information and acceleration device unsupported operation information.

[0029] Based on the operation information supported by the acceleration device, the target running device code is imported into the acceleration device for execution, and the execution result of the acceleration device is obtained.

[0030] Based on the operation information that the acceleration device does not support, the target device code is imported into the target processing device for execution, and the execution result of the processing device is obtained.

[0031] The target device executable file is constructed based on the execution results of the acceleration device and the execution results of the processing device.

[0032] Optionally, the step of reasoning about the target tensor data based on the target device executable file and model type includes:

[0033] Obtain information about the equipment to be run, and determine the target equipment to be run based on the information about the equipment to be run;

[0034] The executable file of the target device is executed by the target running device to obtain the file execution result;

[0035] The model type is determined based on the executable file of the target device;

[0036] Inference is performed on the target tensor data based on the file execution results and the model type.

[0037] Furthermore, to achieve the above objectives, the present invention also proposes a virtual machine-based tensor data computation and inference device, the virtual machine-based tensor data computation and inference device comprising:

[0038] The acquisition module is used to acquire a specified input model file and obtain the target unified format intermediate representation based on the specified input model file;

[0039] The generation module is used to generate a target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategy;

[0040] The construction module is used to construct the target device executable file based on the target self-decoding and acceleration device operation information;

[0041] The inference module is used to perform inference on the target tensor data based on the target device executable file and model type.

[0042] Furthermore, to achieve the above objectives, the present invention also proposes a virtual machine-based tensor data computation and inference device, the virtual machine-based tensor data computation and inference device comprising: a memory, a processor, and a virtual machine-based tensor data computation and inference program stored in the memory and executable on the processor, the virtual machine-based tensor data computation and inference program being configured to implement the virtual machine-based tensor data computation and inference method described above.

[0043] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a virtual machine-based tensor data computation and inference program, which, when executed by a processor, implements the virtual machine-based tensor data computation and inference method described above.

[0044] The proposed method for tensor data computation and inference based on a virtual machine involves: acquiring a specified input model file; obtaining a target unified format intermediate representation based on the specified input model file; generating a target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategies; constructing a target device executable file based on the target self-decoder and accelerated device operation information; and performing inference on the target tensor data based on the target device executable file and model type. Through this approach, format unification is achieved at the front-end access layer, data optimization is performed based on the intermediate conversion layer and intermediate layer optimization strategies, and computation and inference are performed based on the target device executable file and model type constructed at the terminal execution layer. This method is adaptable to computation and inference on various frameworks and platforms, and simplifies the development workload for developers performing model inference. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of a virtual machine-based tensor data computing and inference device in the hardware operating environment involved in the embodiments of the present invention;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the tensor data computation and inference method based on a virtual machine according to the present invention.

[0047] Figure 3 This is a flowchart illustrating the second embodiment of the tensor data computation and inference method based on a virtual machine according to the present invention.

[0048] Figure 4This is a schematic diagram of the functional modules of the first embodiment of the tensor data computation and inference device based on a virtual machine according to the present invention.

[0049] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0051] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a virtual machine-based tensor data computation and inference device in the hardware operating environment involved in the embodiments of the present invention.

[0052] like Figure 1 As shown, the virtual machine-based tensor data computation and inference device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on a virtual machine-based tensor data computing inference device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0054] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a virtual machine-based tensor data computation and inference program.

[0055] exist Figure 1In the virtual machine-based tensor data computation and inference device shown, the network interface 1004 is mainly used for data communication with the network integrated platform workstation; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the virtual machine-based tensor data computation and inference device of the present invention can be set in the virtual machine-based tensor data computation and inference device. The virtual machine-based tensor data computation and inference device calls the virtual machine-based tensor data computation and inference program stored in the memory 1005 through the processor 1001 and executes the virtual machine-based tensor data computation and inference method provided in the embodiment of the present invention.

[0056] Based on the above hardware structure, an embodiment of the tensor data computation and inference method based on a virtual machine is proposed in this invention.

[0057] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the tensor data computation and inference method based on a virtual machine according to the present invention.

[0058] In the first embodiment, the virtual machine-based tensor data computation and inference method includes the following steps:

[0059] Step S10: Obtain the specified input model file, and obtain the target unified format intermediate representation based on the specified input model file.

[0060] It should be noted that the execution subject of this embodiment is a tensor data computing and inference device based on a virtual machine. It can also be other devices that can achieve the same or similar functions, such as a virtual machine for tensor computing. This embodiment does not limit this. In this embodiment, a virtual machine for tensor computing is used as an example for explanation.

[0061] It should be understood that the BayesVM (BVM) virtual machine for tensor computation supports model files from various deep learning frameworks and various acceleration devices. The BVM's design structure includes three layers: the front-end access layer, the intermediate transformation layer, and the terminal execution layer, and it is applied in the field of deep learning inference.

[0062] It is understandable that the specified input model file refers to the file corresponding to the model that needs to be deployed to the acceleration device. The specified input model file can be of various types. The target uniform format intermediate representation refers to the intermediate representation constructed from the data in the specified input model file. The target uniform format intermediate representation is for use by the next layer of intermediate transformation layer.

[0063] Further, step S10 includes: obtaining a specified input model file; obtaining the corresponding numerical weights, tensor structure, and computation graph based on the specified input model file; obtaining the data formats of the numerical weights, tensor structure, and computation graph respectively; and when the data formats of any two of the numerical weights, tensor structure, and computation graph are inconsistent, converting the numerical weights, tensor structure, and computation graph into intermediate representations of a target unified format through the front-end input layer.

[0064] Understandably, after receiving a specified input model file, BVM needs to unify the format of the specified input model file because the model file formats of each deep learning framework are different. Specifically, it obtains the corresponding numerical weights, tensor structures, and computation graphs based on the specified input model file, and then determines whether the data formats of the numerical weights, tensor structures, and computation graphs are all the same. If they are, they are directly transmitted to the intermediate conversion layer for processing. If not, the front-end access layer unifies the numerical weights, tensor structures, and computation graphs into intermediate representations of the same format, i.e., the target unified format intermediate representation.

[0065] Step S20: Generate target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategy.

[0066] It is understandable that target self-decoding refers to deep self-decoding. Target self-decoding is intended for use by the next layer, the terminal execution layer. Intermediate layer optimization strategies refer to the optimization strategies set in each layer of the intermediate transformation layer. Each layer can set one or more optimization strategies, such as operator fusion optimization strategies and matrix factorization optimization strategies. For example, operator fusion optimization strategies reduce the number of data exchanges between video memory and main memory, and matrix factorization optimization strategies increase parallelism. After obtaining the target unified format intermediate representation, the target unified format intermediate representation is optimized through intermediate layer optimization strategies to generate target self-decoding.

[0067] Step S30: Construct the target device executable file based on the target self-decoding and acceleration device operation information.

[0068] It should be understood that the acceleration device operation information includes operation information supported by the acceleration device and operation information not supported by the acceleration device. The target device executable file refers to the file that can be executed normally on the target device. After executing the acceleration device operation information, the generated target self-decoder will automatically build the target device executable file. The target device is the device that needs to be run when the system where BVM is located is running.

[0069] Further, step S30 includes: acquiring a number of processor information and acceleration device information; obtaining a number of computational databases based on the processor information and acceleration device information; generating target running device code layer by layer according to the target self-decoding through the number of computational databases and a preset database; and constructing a target device executable file based on the target running device code and the acceleration device operation information.

[0070] Understandably, since target devices include different architectures, they will use processors and acceleration devices during operation, such as CPUs, GPUs, and NPUs. Processor information refers to the processor information used by the target device during operation, including but not limited to the number and type of processors. Similarly, acceleration device information refers to the information on the target device deployed to acceleration devices, including but not limited to the number and type of acceleration devices. Acceleration devices are encapsulated into corresponding computing libraries, i.e., several computing databases, including but not limited to CUDA, TensorRT, Compute Library, and ROCm. Preset databases refer to the computing libraries unique to the hardware chips of different manufacturers. Based on several computing databases and preset databases, the target self-decodes and generates the target running device code layer by layer.

[0071] Further, the step of constructing the target device executable file based on the target running device code and the acceleration device operation information includes: obtaining acceleration device supported operation information and acceleration device unsupported operation information based on the acceleration device operation information; importing the target running device code into the acceleration device for execution based on the acceleration device supported operation information to obtain the acceleration device execution result; importing the target device code into the target processing device for execution based on the acceleration device unsupported operation information to obtain the processing device execution result; and constructing the target device executable file based on the acceleration device execution result and the processing device execution result.

[0072] It should be understood that the accelerated device supported operation information refers to the operation information supported by the accelerated device. Similarly, the accelerated device unsupported information refers to the operation information that the accelerated device does not support. After generating the target running device code, the accelerated device supported operation information will be used first to execute the target running device code to obtain the accelerated device execution result. Then, based on the accelerated device unsupported operation information, the target device code will be left to be executed on the CPU to obtain the processing device execution result. This achieves both acceleration and versatility. After execution, the target device executable file is constructed based on the accelerated device execution result and the processing device execution result.

[0073] Step S40: Infer the target tensor data based on the target device executable file and model type.

[0074] Understandably, the model type refers to the type of model corresponding to the specified input model file. After obtaining the target device executable file, the target device executable file is executed on the target running device. After execution, the inference result of the target tensor data is obtained. For example, if the model type is an image recognition model, the output inference result after execution is image features.

[0075] Further, step S40 includes: obtaining information about the device to be run; determining the target running device based on the information about the device to be run; executing the executable file of the target device through the target running device to obtain the file execution result; determining the model type based on the executable file of the target device; and performing inference on the target tensor data based on the file execution result and the model type.

[0076] It should be understood that the device information to be run refers to the device information that the developers need to run. The target device is determined based on the device information to be run, and then the executable file of the target device is imported into the target device for execution to obtain the file execution result. Then, the target tensor data is inferred based on the file execution result and the model type. That is, the type of inference result of the target tensor data is obtained based on the model type, and the inference result of the target tensor data is obtained based on the file execution result and the model type.

[0077] This embodiment obtains a specified input model file, and then obtains a target unified format intermediate representation based on the specified input model file. It generates a target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategies. It constructs a target device executable file based on the target self-decoder and accelerated device operation information. Finally, it performs inference on the target tensor data based on the target device executable file and model type. Through this method, format unification is performed based on the front-end access layer, data optimization is performed based on the intermediate conversion layer and intermediate layer optimization strategies, and computational inference is performed based on the target device executable file and model type constructed by the terminal execution layer. This enables it to adapt to computational inference on various frameworks and platforms, and simplifies the development workload for developers performing model inference.

[0078] In one embodiment, such as Figure 3 The second embodiment of the virtual machine-based tensor data computation and inference method of the present invention, based on the first embodiment, includes step S20:

[0079] Step S201: Obtain the number of intermediate conversion layers, and obtain the data optimization strategy set for each layer based on the number of intermediate conversion layers and the intermediate layer optimization strategy.

[0080] It should be understood that the number of layers refers to the total number of layers in the intermediate transformation layer, and the intermediate layer optimization strategy refers to the optimization strategy set in the intermediate transformation layer. There can be multiple intermediate layer optimization strategies. Then, based on the number of layers and the intermediate layer optimization strategies, the data optimization strategies set in each layer are obtained. For example, the data optimization strategy set in the first layer is the operator fusion optimization strategy, and the data optimization strategy set in the second layer is the operator fusion optimization strategy and the matrix factorization optimization strategy.

[0081] Step S202: Optimize the intermediate representation of the target unified format according to the data optimization strategy of each layer to obtain the tensor calculation model file.

[0082] Understandably, due to the differences between each model file and the target device, direct execution may result in incompatibility. Even if it is compatible, the best performance solution cannot be obtained. Therefore, format conversion is required. Specifically, after obtaining the target unified format intermediate representation provided by the front-end access layer, the intermediate conversion layer will use data optimization strategies to optimize the target unified format intermediate representation in multiple layers according to the specified target.

[0083] Step S203: Generate target self-decoder based on the tensor calculation model file.

[0084] Further, step S203 includes: obtaining the corresponding file type and tensor computation model data based on the tensor computation model file; selecting a target encoding strategy from the encoding strategy set based on the file type; and encoding the tensor computation model data using the target encoding strategy to obtain a target self-decoder.

[0085] It is understandable that file type refers to the type of tensor computation model file, tensor computation model data refers to the various data in the tensor computation model file, and target encoding strategy refers to the strategy for encoding the data. The target encoding strategy selects an encoding strategy that is suitable for the tensor computation model data from the encoding set based on the file type, and then encodes the tensor computation model data through the target encoding strategy to obtain the target self-decoding.

[0086] This embodiment obtains the number of intermediate conversion layers, and based on the number of intermediate conversion layers and intermediate layer optimization strategies, obtains data optimization strategies for each layer. It then optimizes the target unified format intermediate representation based on the data optimization strategies for each layer, resulting in a tensor computation model file. Finally, it generates a target self-decoder based on the tensor computation model file. By obtaining the data optimization strategies for each layer based on the number of intermediate conversion layers and intermediate layer optimization strategies, and then optimizing the target unified format intermediate representation based on the data optimization strategies for each layer according to the specified target, a tensor computation model file is obtained after optimization. The target self-decoder is then automatically generated based on the tensor computation model file, thereby effectively improving the accuracy of the generated target self-decoder and thus enhancing the compatibility between the model file and the target device.

[0087] Furthermore, this embodiment of the invention also proposes a storage medium storing a virtual machine-based tensor data computation and inference program, which, when executed by a processor, implements the steps of the virtual machine-based tensor data computation and inference method described above.

[0088] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0089] In addition, refer to Figure 4 This invention also proposes a virtual machine-based tensor data computation and inference device, which includes:

[0090] The acquisition module 10 is used to acquire a specified input model file and obtain the target unified format intermediate representation based on the specified input model file.

[0091] The generation module 20 is used to generate a target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategy.

[0092] The construction module 30 is used to construct an executable file for the target device based on the target self-decoding and acceleration device operation information.

[0093] The inference module 40 is used to infer the target tensor data based on the target device executable file and model type.

[0094] This embodiment obtains a specified input model file, and then obtains a target unified format intermediate representation based on the specified input model file. It generates a target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategies. It constructs a target device executable file based on the target self-decoder and accelerated device operation information. Finally, it performs inference on the target tensor data based on the target device executable file and model type. Through this method, format unification is performed based on the front-end access layer, data optimization is performed based on the intermediate conversion layer and intermediate layer optimization strategies, and computational inference is performed based on the target device executable file and model type constructed by the terminal execution layer. This enables it to adapt to computational inference on various frameworks and platforms, and simplifies the development workload for developers performing model inference.

[0095] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0096] In addition, for technical details not described in detail in this embodiment, please refer to the virtual machine-based tensor data computation and inference method provided in any embodiment of the present invention, which will not be repeated here.

[0097] In one embodiment, the acquisition module 10 is further configured to acquire a specified input model file, obtain corresponding numerical weights, tensor structures, and computation graphs based on the specified input model file; acquire the data formats of the numerical weights, tensor structures, and computation graphs respectively; and when the data formats of any two of the numerical weights, tensor structures, and computation graphs are inconsistent, convert the numerical weights, tensor structures, and computation graphs into intermediate representations of a target unified format through the front-end input layer.

[0098] In one embodiment, the generation module 20 is further configured to obtain the number of intermediate conversion layers, obtain data optimization strategies set for each layer based on the number of intermediate conversion layers and intermediate layer optimization strategies, optimize the target unified format intermediate representations according to the data optimization strategies of each layer to obtain tensor computation model files, and generate target self-decoders based on the tensor computation model files.

[0099] In one embodiment, the generation module 20 is further configured to obtain the corresponding file type and tensor computation model data according to the tensor computation model file; select a target encoding strategy from the encoding strategy set according to the file type; and encode the tensor computation model data through the target encoding strategy to obtain a target self-decoder.

[0100] In one embodiment, the construction module 30 is further configured to acquire a number of processor information and acceleration device information; obtain a number of computing databases based on the processor information and acceleration device information; generate target running device code layer by layer according to the target self-decoding through the number of computing databases and a preset database; and construct a target device executable file based on the target running device code and the acceleration device operation information.

[0101] In one embodiment, the construction module 30 is further configured to: obtain acceleration device supported operation information and acceleration device unsupported operation information based on the acceleration device operation information; import the target running device code into the acceleration device for execution based on the acceleration device supported operation information to obtain acceleration device execution results; import the target device code into the target processing device for execution based on the acceleration device unsupported operation information to obtain processing device execution results; and construct a target device executable file based on the acceleration device execution results and the processing device execution results.

[0102] In one embodiment, the inference module 40 is further configured to acquire information about the device to be run, determine the target running device based on the information about the device to be run, execute the executable file of the target device through the target running device to obtain the file execution result, determine the model type based on the executable file of the target device, and infer the target tensor data based on the file execution result and the model type.

[0103] Other embodiments or implementation methods of the virtual machine-based tensor data computation and inference device described in this invention can be referred to the above-described method embodiments, and will not be repeated here.

[0104] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0105] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, all-in-one platform workstation, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0107] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for tensor data computation and inference based on a virtual machine, characterized in that, The virtual machine-based tensor data computation and inference method includes the following steps: Obtain a specified input model file, and obtain the target unified format intermediate representation based on the specified input model file; The target self-decoder is generated based on the target unified format intermediate representation and intermediate layer optimization strategy; Construct an executable file for the target device based on the target self-decoding and accelerated device operation information; Inference is performed on the target tensor data based on the target device's executable file and model type; The step of obtaining a specified input model file and obtaining a target unified format intermediate representation based on the specified input model file includes: Obtain a specified input model file, and obtain the corresponding numerical weights, tensor structure, and computational graph based on the specified input model file; The data formats of the numerical weights, the tensor structure, and the computation graph are obtained respectively. When the data formats of any two of the numerical weights, tensor structures, and computation graphs are inconsistent, the numerical weights, tensor structures, and computation graphs are converted into intermediate representations of the target unified format through the front-end input layer.

2. The tensor data computation and inference method based on a virtual machine as described in claim 1, characterized in that, The step of generating the target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategy includes: Obtain the number of intermediate transformation layers, and based on the number of intermediate transformation layers and the intermediate layer optimization strategy, obtain the data optimization strategy set for each layer; Based on the data optimization strategies of each layer, the intermediate representation of the target unified format is optimized to obtain the tensor computation model file; The target self-decoder is generated based on the tensor computation model file.

3. The tensor data computation and inference method based on a virtual machine as described in claim 2, characterized in that, The step of generating the target self-decoder based on the tensor computation model file includes: Based on the tensor computation model file, obtain the corresponding file type and tensor computation model data; Select a target encoding strategy from the set of encoding strategies based on the file type; The tensor computation model data is encoded using the target encoding strategy to obtain the target self-decoding.

4. The tensor data computation and inference method based on a virtual machine as described in claim 1, characterized in that, The step of constructing the target device executable file based on the target self-decoding and acceleration device operation information includes: Obtain information on a number of processors and acceleration devices; A number of computational databases are obtained based on the processor information and the acceleration device information; The target operating device code is generated layer by layer based on the target self-decoding using the aforementioned quantity calculation database and preset database; An executable file for the target device is constructed based on the target device code and the acceleration device operation information.

5. The tensor data computation and inference method based on a virtual machine as described in claim 4, characterized in that, The step of constructing the target device executable file based on the target operating device code and the acceleration device operation information includes: Based on the acceleration device operation information, we obtain acceleration device supported operation information and acceleration device unsupported operation information. Based on the operation information supported by the acceleration device, the target running device code is imported into the acceleration device for execution, and the execution result of the acceleration device is obtained. Based on the operation information that the acceleration device does not support, the target device code is imported into the target processing device for execution, and the execution result of the processing device is obtained. An executable file for the target device is constructed based on the execution results of the acceleration device and the execution results of the processing device.

6. The virtual machine-based tensor data computation and inference method according to any one of claims 1 to 5, characterized in that, The inference of the target tensor data based on the target device executable file and model type includes: Obtain information about the equipment to be run, and determine the target equipment to be run based on the information about the equipment to be run; The executable file of the target device is executed through the target running device to obtain the file execution result; The model type is determined based on the executable file of the target device; Inference is performed on the target tensor data based on the file execution results and the model type.

7. A tensor data computation and inference device based on a virtual machine, characterized in that, The virtual machine-based tensor data computation and inference device includes: The acquisition module is used to acquire a specified input model file and obtain the target unified format intermediate representation based on the specified input model file; The generation module is used to generate a target self-decoder based on the target unified format intermediate representation and intermediate layer optimization strategy; The construction module is used to construct the target device executable file based on the target self-decoding and acceleration device operation information; The inference module is used to infer the target tensor data based on the executable file and model type of the target device; The acquisition module is further configured to acquire a specified input model file, obtain the corresponding numerical weights, tensor structures, and computation graphs based on the specified input model file; acquire the data formats of the numerical weights, tensor structures, and computation graphs respectively; and when the data formats of any two of the numerical weights, tensor structures, and computation graphs are inconsistent, convert the numerical weights, tensor structures, and computation graphs into intermediate representations of a target unified format through the front-end input layer.

8. A tensor data computation and inference device based on a virtual machine, characterized in that, The virtual machine-based tensor data computation and inference device includes: a memory, a processor, and a virtual machine-based tensor data computation and inference program stored in the memory and executable on the processor. The virtual machine-based tensor data computation and inference program is configured to implement the virtual machine-based tensor data computation and inference method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a virtual machine-based tensor data computation and inference program, which, when executed by a processor, implements the virtual machine-based tensor data computation and inference method as described in any one of claims 1 to 6.

Images