Method and system for executing virtual instruction set

Abstract

The invention discloses a method for executing a virtual instruction set so as to construct an operator library to realize a deep learning accelerator and solve the problem of cross-generation compatibility. The method provides a virtual instruction set execution unit, namely a virtual instruction set, then compiles the virtual instruction set into a vector instruction and / or a dedicated instruction by a compiler, and finally executes the vector instruction and / or the dedicated instruction by the virtual instruction set execution unit. Wherein the virtual instruction set execution unit is used for executing a virtual instruction set and comprises a general vector calculation unit and a special instruction set calculation unit, the general vector calculation unit is used for executing a vector instruction set, and the special instruction set calculation unit is used for executing a special instruction set. The virtual instruction set comprises a first C + + template and a second C + + template, the first C + + template defines a first C + + function, the first C + + function can be compiled by the compiler into vector instructions in the vector instruction set, the second C + + template defines a second C + + function, and the second C + + function can be compiled by the compiler into dedicated instructions in the dedicated instruction set.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and system for executing a virtual instruction set. Background Art

[0002] Deep learning is one of the core technologies in the field of artificial intelligence. However, due to the complexity and large amount of computation in deep learning models, traditional computing devices often cannot meet the requirements of efficient operation. Therefore, deep learning accelerators are often used to accelerate deep learning calculations. Deep learning calculators provide efficient computing resources by optimizing key computing processes in deep learning models. Specifically, deep learning accelerators can utilize parallel computing features to process multiple computing tasks at the same time. In addition, they are customized for deep learning computing needs, providing specific instruction sets and hardware optimization to accelerate common deep learning computing operations such as matrix operations and convolution operations.

[0003] Existing deep learning accelerators mainly include three categories: deep learning dedicated accelerator AS IC, programmable deep learning accelerator based on tensor instructions, and programmable deep learning accelerator based on vector instructions. Among them, the deep learning dedicated accelerator AS IC is customized to design hardware instructions according to deep learning, and the operation granularity of its instructions is relatively coarse. For example, conv2d is an instruction, and layernorm is an instruction. From the perspective of data granularity, it supports tensor operation semantics and belongs to coarse-grained instructions. The programmability and versatility of the deep learning dedicated accelerator AS IC instruction set are intertwined. When the algorithm changes, the hardware cannot support it. The programmable deep learning accelerator based on tensor instructions is customized to design hardware instructions for deep learning, which refines the operation granularity of some instructions, such as providing GEMM acceleration instead of cov2d, and providing basic operations such as add sub di v mu l instead of coarse-grained operators such as layernorm and si gmod. Compared with SIMD instructions in traditional CPUs, the refined IS operation can support multi-dimensional tensor semantics. From the perspective of data granularity, it is coarser than traditional one-dimensional SIMD instructions. It is usually difficult for IS with tensor semantics to be completely Turing complete. It is programmable but difficult to program. There are a large number of parameters with tensor semantics in the instructions, which are difficult for users to master and have poor programmability. It is difficult to automatically generate these tensor instructions through AI compilers. The commercial implementation of such AI compilers also consumes a lot of manpower and time resources. Programmable deep learning accelerators based on vector instructions customize the design of matrix acceleration instructions according to deep learning, and use SIMD instructions of DSP or CPU core to support deep learning and machine learning. The current SIMD DLEN generally does not exceed 2048bit it or 1024bit it. It is a fine-grained data parallelism. From the perspective of operation, it is also relatively fine-grained, not a coarse-grained operator operation like onnx. However, the vector instruction set loses the domain semantics of ML and AI in the dimensions of data granularity and operation granularity. It is a great challenge for software and compilers to be able to efficiently use SIMD IS. In addition, the AS IC instruction set and tensor instruction set of the deep learning dedicated accelerator need to rely on other programmable cores in the soc or host cpu for support for programs that are not programmable or have poor performance on tensor instructions. For model compilation, it is necessary to support heterogeneous model deployment. The communication overhead between heterogeneous cores will affect the overall performance of the model. At the same time, the existing instruction set software and hardware are deeply coupled, so after the hardware evolution, it will have a greater impact on the software. Summary of the invention

[0004] In view of some or all of the problems in the prior art, the present invention provides a method for executing a virtual instruction set in a first aspect, comprising:

[0005] A virtual instruction set execution unit is provided, the virtual instruction set execution unit is used to execute the virtual instruction set, and the virtual instruction set execution unit includes:

[0006] A general purpose vector computing unit for executing a vector instruction set; and

[0007] A dedicated instruction set computing unit, which is used to execute a dedicated instruction set;

[0008] A virtual instruction set is provided, wherein the virtual instruction set comprises:

[0009] A first C++ template, wherein the first C++ template defines a first C++ function, and the first C++ function can be compiled by a compiler into a vector instruction in a vector instruction set; and

[0010] A second C++ template, wherein a second C++ function is defined in the C++ template, and the second C++ function can be compiled by a compiler into a dedicated instruction in a dedicated instruction set;

[0011] The compiler compiles the virtual instruction set into vector instructions and / or special instructions; and the virtual instruction set execution unit executes the vector instructions and / or special instructions.

[0012] Further, the dedicated instruction set includes one or more of the following:

[0013] Tensor instruction set, and deep learning dedicated accelerator AS IC instruction set.

[0014] Further, providing a virtual instruction set includes:

[0015] Define an operator using one or more of the following:

[0016] ONNX, staticHLO, and principaltorch.

[0017] Further, compiling the virtual instruction set into vector instructions and / or special instructions by the compiler includes:

[0018] The data granularity of the tensor is determined based on the hardware's SRAM specifications, the data segmentation by the operator developer or AI compiler, and the form of operator fusion.

[0019] Further, the dedicated instruction set computing unit includes an acceleration unit in the AI ​​processor, and the acceleration unit is configured to execute one or more of the following:

[0020] Data movement, matrix operations, and convolution operations.

[0021] Furthermore, compiling the virtual instruction set into vector instructions and / or special instructions by the compiler also includes:

[0022] generating an operator or model based on the virtual instruction set in a compiler; and

[0023] The compiler performs instruction scheduling based on the instruction granularity of the operator or model to optimize the performance of the operator or model.

[0024] Furthermore, the method further comprises:

[0025] The first C++ template and / or the second C++ template are replaced or modified to adapt to different virtual instruction set execution units, wherein the virtual instruction set execution units include ARM NEON and DSP SIMD, such as commercial DSP IPs such as TI and CEVA.

[0026] A second aspect of the present invention provides a system for executing a virtual instruction set, comprising:

[0027] A virtual instruction set execution unit, which is used to execute the virtual instruction set, and the virtual instruction set execution unit includes:

[0028] A general purpose vector computing unit for executing a vector instruction set; and

[0029] A dedicated instruction set computing unit, which is used to execute a dedicated instruction set;

[0030] A virtual instruction set, which includes:

[0031] A first C++ template, wherein the first C++ template defines a first C++ function, and the first C++ function can be compiled by a compiler into a vector instruction in a vector instruction set; and

[0032] A second C++ template, wherein a second C++ function is defined in the C++ template, and the second C++ function can be compiled by a compiler into a dedicated instruction in a dedicated instruction set; and

[0033] A compiler is configured to compile the virtual instruction set into vector instructions and / or special instructions.

[0034] The present invention provides a method and system for executing a virtual instruction set, which realizes deep learning tasks such as AI accelerators by providing and executing the virtual instruction set. Specifically, a vector instruction set in the form of a general-purpose vector processor hardware is used as the basis to construct a virtual instruction set in the form of a field-specific software, which ensures the versatility and programmability of the hardware. At the same time, the virtual instruction set is externally dedicated through the design of the interface, ensuring efficiency. By fixing the virtual instruction set, the upper-level AI compiler or operator library can be built on the virtual instruction set, thereby decoupling the software and hardware, so the hardware evolution will not have a significant impact on the software, and the software can also expand the virtual instruction set to meet the acceleration support for more special scenarios. In addition, the execution efficiency of the model can be optimized by arranging the coarse-grained virtual instructions at compile time and dynamically scheduling them at runtime. The arrangement at compile time is completed by the AI ​​compiler or the writing of the operator library. The support of dynamic scheduling requires hardware support for the management and scheduling of virtual instructions. The scheduling unit is a function composed of a general-purpose vector instruction set, which is a virtual instruction. The support of dynamic scheduling can further reduce the programming burden of users and improve programmability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] To further illustrate the above and other advantages and features of various embodiments of the present invention, a more specific description of various embodiments of the present invention will be presented with reference to the accompanying drawings. It will be understood that these drawings only depict typical embodiments of the present invention and are therefore not to be considered as limiting the scope thereof. In the accompanying drawings, for clarity, identical or corresponding parts will be represented by identical or similar reference numerals.

[0036] Figure 1 A flowchart showing a method for executing a virtual instruction set according to an embodiment of the present invention; and

[0037] Figure 2 A schematic diagram showing the structure of a system for executing a virtual instruction set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In the following description, the present invention is described with reference to various embodiments. However, those skilled in the art will recognize that various embodiments can be implemented without one or more specific details or with other replacement and / or additional methods, materials or components. In other cases, well-known structures, materials or operations are not shown or described in detail to avoid blurring the inventive point of the present invention. Similarly, for the purpose of explanation, specific quantities, materials and configurations are set forth to provide a comprehensive understanding of embodiments of the present invention. However, the present invention is not limited to these specific details. In addition, it should be understood that the various embodiments shown in the drawings are illustrative representations and are not necessarily drawn in correct proportions.

[0039] In this specification, reference to "one embodiment" or "the embodiment" means that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in various places in this specification does not necessarily all refer to the same embodiment.

[0040] In the present invention, each module of the system according to the present invention can be implemented using software, hardware, firmware or a combination thereof. When the module is implemented using software, the function of the module can be implemented by a computer program flow, for example, the module can be implemented by a code segment (such as a code segment of a language such as C, C++) stored in a storage device (such as a hard disk, a memory, etc.), wherein the corresponding function of the module can be implemented when the code segment is executed by a processor. When the module is implemented using hardware, the function of the module can be implemented by setting a corresponding hardware structure, for example, the function of the module can be implemented by hardware programming a programmable device such as a field programmable gate array (FPGA), or the function of the module can be implemented by designing an application-specific integrated circuit (ASIC) including electronic devices such as a plurality of transistors, resistors and capacitors. When the module is implemented using firmware, the function of the module can be written into a read-only memory such as an EPROM or EEPROM of the device in the form of a program code, and the corresponding function of the module can be implemented when the program code is executed by a processor. In addition, some functions of the module may need to be implemented by separate hardware or by collaboration with the hardware, for example, the detection function is implemented by corresponding sensors (such as proximity sensors, acceleration sensors, gyroscopes, etc.), the signal transmission function is implemented by corresponding communication devices (such as Bluetooth devices, infrared communication devices, baseband communication devices, Wi-Fi communication devices, etc.), the output function is implemented by corresponding output devices (such as displays, speakers, etc.), and so on.

[0041] The inventor has found through research that the existing deep learning accelerators are mostly implemented using dedicated instruction sets, tensor instruction sets or vector instruction sets, but the software and hardware of these instruction sets are deeply coupled, so after the hardware evolves, it will have a greater impact on the software. Specifically, some existing custom architectures mostly use vector instructions for multi-dimensional data, such as taking matrices as input, high-dimensional or compound operation vector instructions may be Turing incomplete, unable to describe logic, or have poor performance. In addition, such architectures often provide instructions for complex operations, such as treating an onnx operator as a hardware instruction, etc., which will also lead to more parameters and worse usability. Turing incompleteness requires the CPU on the SoC or the CPU on the host side to complete part of the calculation, which will generate additional communication overhead, which will lead to the promotion of the programming ecosystem and affect performance optimization. In addition, if an instruction set with relatively coarse data or operations is used, although the program can be guaranteed to be efficient in most scenarios, it cannot be guaranteed to be universal, and there will be efficiency problems in a small number of scenarios. Some other AI processors use general vector instructions, which ensures the versatility of the hardware. However, the hardware cannot obtain the domain-specific semantics, and a large amount of optimization work is handed over to the software. When writing operators, AI compilers or operator developers need to manage and arrange these fine-grained SIMD instructions, which increases the scale of the problem and affects the efficiency of the final operator. After further research, the inventors found that to solve the above problems, on the one hand, it is necessary to maintain the versatility of the hardware and provide a more programmable programming abstraction from the software level. On the other hand, it is also necessary to maintain the domain-specific semantics to help the compiler or hardware optimize the program more efficiently. At the same time, it is necessary to decouple the software and hardware so that the software and hardware can easily evolve and expand independently. Based on this, the present invention proposes a virtual instruction set, which implements the operator library through the virtual instruction set to reduce the difficulty of operator development or AI compiler development and improve performance. At the same time, it can decouple the hardware and software, solving the problem of cross-generational evolution of hardware and software.

[0042] In one embodiment of the present invention, the virtual instruction set is a domain-specific software form constructed based on the vector instruction set of the general vector processor hardware form, thereby ensuring the versatility and programmability of the hardware, while reflecting the specialization to the outside through the design of the interface, ensuring efficiency. In one embodiment of the present invention, a general-purpose vector core is used as a computing unit, and other computing units such as matrix acceleration in the AI ​​processor are used to form an AI kernel architecture. In terms of data granularity, the vector length of the vector core of the AI ​​kernel architecture constructed in this way is generally 512bit or 1024bit. At the same time, in terms of operation granularity, the instructions of the vector core of the AI ​​kernel architecture can be considered as fine-grained operations compared with the operators in the neural network. In addition, the general-purpose vector processor does not have the Turing incompleteness problem, and the versatility of the hardware can also be guaranteed.

[0043] In addition, the virtual instruction set can also improve the granularity of data, from one-dimensional SIMD to multi-dimensional, and improve the complexity of operations. At the same time, C++ template technology can be used to make the granularity of data adapt to different hardware configurations, such as changes in the number of cores and changes in SRAM capacity. In one embodiment of the present invention, the virtual instruction can be in the form of a C++ function, and the API supports an M*N matrix as in0 input, and out output is an M*N matrix. This is a CLIP operation of a two-dimensional matrix. This API format reflects the domain semantics of the AI ​​operator. The CLIP process is performed on a general vector core, which itself also supports scalars, and the M*N matrix operation can be iteratively processed using loops on the vector core. The internal processing of the virtual instruction can also be extremely optimized to make the virtual instruction optimal. The optimization method generally adopts loop unrolling, soft pipelining and other means, which can be manual optimization or automatic optimization of the compiler. At the same time, this is a CPU-like programming interface that can reduce the difficulty of custom operators.

[0044] In the abstraction of fine-grained SIMD instructions, the virtual instruction set provides a coarse-grained expression of data, operations and other field-specific semantics for AI scenarios, which reduces the basic units seen by AI compilers or operator developers, simplifies the scale of optimization problems, and thus ensures efficiency. The virtual instruction set has a limited optimization space. If it is A, the operator based on virtual instructions joins the optimization space is B. By optimizing A and B respectively, the optimal in the A*B optimization space can be achieved. In this scenario, the optimization space is A+B. If the virtual instruction set is not used, the optimization space of the operator is A*B. Therefore, the use of the virtual instruction set simplifies the scale of the optimization problem, reduces the difficulty of operator optimization, and makes it easier to improve program performance.

[0045] In one embodiment of the present invention, when defining virtual instructions in a virtual instruction set, the operation granularity and data strength can be selected according to actual needs. Specifically, in one embodiment of the present invention, ONNX / stabl eHLO / primtorch and other sets are used as virtual instruction set operation granularity support. Most of the operators between these sets overlap, a few formats are slightly different, and a few are complementary. Since the virtual instruction set is a software concept, it is easy to implement a superset covering these sets, and then it can be relatively easy to connect to mainstream AI frameworks or model formats. The operators of these sets are generally between 100-200, and the supersets they constitute are approximately between 200-250, so they are controllable and lightweight.

[0046] In one embodiment of the present invention, the virtual instruction API supports tensor semantic information such as M*N matrix and stride, which are all in the form of C++ templates, so it can be flexible and variable at compile time. The data granularity of the tensor depends on the SRAM specifications of the hardware, and also depends on the data segmentation, operator fusion form, etc. of the operator developer or AI compiler. The hardware SRAM specifications, data segmentation, operator fusion form, etc. can be determined at compile time to determine the data granularity of the tensor.

[0047] In one embodiment of the present invention, other acceleration units in the AI ​​processor, such as data movement, matrix / convolution calculation, etc., can also be encapsulated into virtual instructions, thereby shielding the configuration process of hardware parameters and only exposing the configuration information of operator semantics to the user, such as tensor data shape, segmentation information, etc. Through the encapsulation of C++ templates, hardware information can be shielded as much as possible, lowering the user's usage threshold.

[0048] Implementing the operator library based on the virtual instruction set mentioned above can reduce the difficulty of operator development or AI compiler development and improve performance. It can also decouple hardware and software and solve the problem of cross-generational evolution of hardware and software. Figure 1 A flowchart of a method for executing a virtual instruction set according to an embodiment of the present invention is shown. By executing the virtual instruction set, a deep learning task can be implemented, such as Figure 1 As shown, a method for executing a virtual instruction set includes:

[0049] First, in step 101, a virtual instruction set execution unit is provided. The virtual instruction set execution unit is used to execute a virtual instruction set. As mentioned above, in one embodiment of the present invention, the virtual instruction set is constructed based on a general-purpose vector core, and at the same time, for AI field scenarios, a coarse-grained expression of domain-specific semantics such as data and operations is provided. Based on this, in one embodiment of the present invention, the virtual instruction set execution unit includes a general-purpose vector computing unit and a special-purpose instruction set computing unit, wherein the general-purpose vector computing unit is used to execute a vector instruction set, and the special-purpose instruction set computing unit is used to execute a special-purpose instruction set. In one embodiment of the present invention, the special-purpose instruction set includes a tensor instruction set, and / or a deep learning special accelerator AS IC instruction set. In another embodiment of the present invention, the special-purpose instruction set computing unit includes an acceleration unit in an AI processor, and the acceleration unit can be used to perform operations such as data movement, matrix operations, and convolution operations;

[0050] Next, in step 102, a virtual instruction set is provided. The virtual instruction set is constructed using the method described above, for example, using pytorch / tensorflow high-level composite operators to define operators (or "operators"), wherein the pytorch / tensorflow high-level composite operators include one or more of the following: ONNX, stableHLO, and primtorch, etc. In one embodiment of the present invention, the virtual instruction set includes a first C++ template and a second C++ template, wherein the first C++ template defines a first C++ function, which can be compiled by a compiler into a vector instruction in a vector instruction set, and the C++ template defines a second C++ function, which can be compiled by a compiler into a special instruction in a special instruction set. An example of the C++ template is shown below, which implements a CLIP operation of a two-dimensional matrix, embodies the domain semantics of the AI ​​operator, and the API supports an M*N matrix as in0 input and an M*N matrix out output. The API format CLIP process is performed on a general vector core, which itself also supports scalars, and the M*N matrix operation can be iteratively processed using a loop on the vector core:

[0051]

[0052]

[0053] Next, in step 103, instruction compilation is performed. The compiler compiles the virtual instruction set into vector instructions and / or special instructions. The operator library may exist in the form of a header file, for example, and the program may call the operator in the corresponding operator library. After the operator is compiled, it becomes a machine-readable instruction, and the instruction may include vector instructions and / or special instructions. The operator library is implemented using virtual instructions. When developing operators, virtual instructions with special semantics are used, which reduces the development difficulty for users. Users only need to pay attention to optimization based on virtual instructions. Part of the optimization space of the virtual instructions themselves can also be exposed to users based on C++ template parameters. Based on the operator library, operators or models of virtual instructions can be automatically generated in the AI ​​compiler. The AI ​​compiler will expand the available range from operators to models, with greater optimization space. Based on virtual instructions, the optimization complexity of the AI ​​compiler can be reduced. The AI ​​compiler performs instruction scheduling based on the granularity of virtual instructions, which can better optimize the performance of models or operators. In one embodiment of the present invention, when the compiler compiles the virtual instruction set into vector instructions and / or special instructions, the data granularity of the tensor needs to be determined based on the SRAM specifications of the hardware, the data segmentation by the operator developer or the AI ​​compiler, and the form of operator fusion. In another embodiment of the present invention, during compilation, an operator or model based on the virtual instruction set is first generated in the compiler, and then the compiler performs instruction scheduling based on the instruction granularity of the operator or model to optimize the performance of the operator or model; and

[0054] Finally, in step 104, the instruction is executed. The vector instruction and / or the special instruction is executed by the virtual instruction set execution unit, that is, the operator is executed to realize the deep learning accelerator. In one embodiment of the present invention, different virtual instruction set execution units can be adapted by replacing or modifying the first C++ template and / or the second C++ template, and the virtual instruction set execution unit includes ARM NEON and DSP SIMD, such as commercial DSP IPs such as TI and CEVA.

[0055] The evolution of the general vector core architecture only requires adjusting the implementation of the virtual instruction set. For example, if an ARMNEON vector core is replaced with a DSP SIMD core, only about 200+ virtual instructions need to be re-implemented. Since the API is compatible, it will not affect the operator library based on virtual instructions or the AI ​​compiler. In practical applications, more virtual instruction sets can be designed to cover more application areas. At the same time, if there are efficiency issues with some virtual instructions supported by general vector instructions, the virtual instructions can also be implemented in hardware.

[0056] Figure 2 A schematic diagram of the structure of a system for executing a virtual instruction set according to an embodiment of the present invention is shown. Figure 2The system for executing a virtual instruction set includes a virtual instruction set execution unit 201, a virtual instruction set 202, and a compiler 203.

[0057] The virtual instruction set execution unit 201 is used to execute the virtual instruction set. In one embodiment of the present invention, the virtual instruction set execution unit includes a general vector calculation unit 211 and a special instruction set calculation unit 212, wherein the general vector calculation unit 211 is used to execute the vector instruction set, and the special instruction set calculation unit 212 is used to execute the special instruction set.

[0058] The virtual instruction set 202 includes a first C++ template and a second C++ template, wherein the first C++ template defines a first C++ function, which can be compiled by a compiler into a vector instruction in a vector instruction set, and the C++ template defines a second C++ function, which can be compiled by a compiler into a special instruction in a special instruction set.

[0059] The compiler 203 is used to compile the virtual instruction set into vector instructions and / or special instructions.

[0060] The present invention provides a method and system for executing a virtual instruction set, which constructs an operator library based on the virtual instruction set to implement deep learning accelerator tasks. The virtual instruction set has high hardware versatility and good scalability, and can adapt to the future evolution of AI algorithms or support acceleration in other fields. At the same time, its software is designed through API to achieve dedicated extensions. The application of the virtual instruction set simplifies the design complexity of hardware and software and has a better TTM. Specifically, from the software dimension, it simplifies the complexity of software programming and compiler design. From the hardware perspective, mature SIMD hardware IPs, such as DSP or CPU IP, can be used to reduce the workload of hardware design, especially verification time. In addition, the operator library constructed by the virtual instruction set has a custom operator experience, which is better than other AI accelerators in programmability and reduces the difficulty of custom operators. Based on the method and system, the hardware and software are effectively decoupled, and the problem of cross-generation compatibility is solved.

[0061] Although various embodiments of the present invention are described above, it should be understood that they are presented as examples only and not as limitations. It is obvious to those skilled in the relevant art that various combinations, modifications and changes can be made thereto without departing from the spirit and scope of the present invention. Therefore, the breadth and scope of the present invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should only be defined according to the attached claims and their equivalents.

Claims

1. A method for executing a virtual instruction set, characterized in that: Includes steps: A virtual instruction set execution unit is provided, wherein the virtual instruction set execution unit is configured to execute a virtual instruction set, and the virtual instruction set execution unit includes: a general purpose vector computing unit configured to execute a vector instruction set; and a dedicated instruction set computing unit configured to execute a dedicated instruction set; A virtual instruction set is provided, wherein the virtual instruction set comprises: A first C++ template, wherein the first C++ template defines a first C++ function, and the first C++ function can be compiled by a compiler into a vector instruction in a vector instruction set; and A second C++ template, wherein a second C++ function is defined in the C++ template, and the second C++ function can be compiled by a compiler into a dedicated instruction in a dedicated instruction set; Compiling the virtual instruction set into vector instructions and / or special instructions by a compiler; and The vector instructions and / or special instructions are executed by a virtual instruction set execution unit.

2. The method according to claim 1, characterized in that The dedicated instruction set includes one or more of the following: Tensor instruction set, and deep learning dedicated accelerator ASIC instruction set.

3. The method according to claim 1, characterized in that The virtual instruction sets provided include: Operators are defined using pytorch / tensorflow high-level composite operators, where the pytorch / tensorflow high-level composite operators include one or more of the following: ONNX, stableHLO, and primtorch.

4. The method according to claim 1, characterized in that Compiling the virtual instruction set into vector instructions and / or special instructions by the compiler comprises the steps of: The data granularity of the tensor is determined based on the hardware's SRAM specifications, the data segmentation by the operator developer or AI compiler, and the form of operator fusion.

5. The method according to claim 1, characterized in that The dedicated instruction set computing unit includes an acceleration unit in the AI ​​processor, wherein the acceleration unit is configured to perform one or more of the following: Data movement, matrix operations, and convolution operations.

6. The method according to claim 4, characterized in that Compiling the virtual instruction set into vector instructions and / or special instructions by the compiler also includes the steps of: generating an operator or model based on the virtual instruction set in a compiler; and The compiler performs instruction scheduling based on the instruction granularity of the operator or model to optimize the performance of the operator or model.

7. The method according to claim 1, characterized in that Also includes the steps: The first C++ template and / or the second C++ template are replaced or modified to adapt to different virtual instruction set execution units, wherein the virtual instruction set execution units include ARM NEON and DSP SIMD.

8. A system for executing a virtual instruction set, characterized in that: include: A virtual instruction set execution unit, configured to execute a virtual instruction set, the virtual instruction set execution unit comprising: a general purpose vector computing unit configured to execute a vector instruction set; and a dedicated instruction set computing unit configured to execute a dedicated instruction set; A virtual instruction set, which includes: A first C++ template, wherein the first C++ template defines a first C++ function, and the first C++ function can be compiled by a compiler into a vector instruction in a vector instruction set; and A second C++ template, wherein a second C++ function is defined in the C++ template, and the second C++ function can be compiled by the compiler into a dedicated instruction in a dedicated instruction set; and a compiler configured to compile the virtual instruction set into vector instructions and / or dedicated instructions.

Images