Hardware computing simulation method, system, and computer readable storage medium

Abstract

The application discloses a hardware computing simulation method and system and a computer readable storage medium, wherein the method is applied to a hardware computing simulation system, the hardware computing simulation system comprises a construction interface, a compiler and an executor, and the method comprises the following steps: acquiring a neural network through the construction interface; serializing the neural network into a software graph structure through the compiler; acquiring the to-be-processed data and the software graph structure through the executor, and calling a software operator corresponding to the software graph structure to perform software calculation on the to-be-processed data, so as to simulate hardware calculation of the hardware module on the to-be-processed data. The application aims to improve the adaptability and accuracy of the hardware computing simulation system in simulating hardware device calculation.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a hardware computing simulation method, system, and computer-readable storage medium. Background Technology

[0002] With the rapid development of deep learning and AI chips, deep learning systems are usually quite complex, and various frameworks and hardware devices have unique architectures. This makes the deployment, testing, and iterative improvement of accuracy and performance tuning of deep learning development environments cumbersome and time-consuming. In order to optimize the above processes, deep learning inference frameworks have been introduced.

[0003] Currently, common inference frameworks include open-source deep learning inference frameworks such as MNN / NCNN / TNN. These open-source deep learning inference frameworks can support common hardware devices on the market. However, for some hardware devices with unique designs and optimizations, using open-source deep learning inference frameworks may result in problems such as operator incompatibility and accuracy loss, making it impossible to fully simulate the performance and advantages of the hardware devices.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a hardware computing simulation method, system, and computer-readable storage medium, aiming to solve the problem of incompatibility of hardware computing simulation systems when using open-source deep learning hardware computing simulation systems for some hardware devices with unique designs and optimizations, such as operator incompatibility and accuracy loss, and to improve the adaptability and accuracy of simulating hardware device computing.

[0006] To achieve the above objectives, the present invention provides a hardware computing simulation method applied to a hardware computing simulation system. The hardware computing simulation system includes a construction interface, a compiler, and an executor. The construction interface is connected to the compiler, and the compiler is connected to the executor. The executor is used to call software operators or hardware modules to perform calculations on the data to be processed. The hardware computing simulation method includes:

[0007] The neural network is obtained through the aforementioned construction interface;

[0008] The compiler serializes the neural network into a software graph structure.

[0009] The actuator acquires the data to be processed and the software graph structure, and calls the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed.

[0010] Optionally, the compiler includes a hardware compilation unit, and after the step of serializing the neural network into a software graph structure by the compiler, the method further includes:

[0011] The software graph structure is serialized into a hardware graph structure by the hardware compilation unit;

[0012] The actuator acquires the data to be processed and the hardware graph structure, and calls the hardware module to perform the hardware calculation on the data to be processed.

[0013] Optionally, the step of serializing the software graph structure into a hardware graph structure through the hardware compilation unit includes:

[0014] If the operator corresponding to the hardware graph structure contains a software operator, then a corresponding data copy operator is added to the hardware graph structure.

[0015] Optionally, the step of adding a corresponding data copy operator to the hardware graph structure if the operator corresponding to the hardware graph structure contains a software operator includes:

[0016] A first data copy operator is added before the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and the corresponding software operator is called to perform software calculations on the data to be processed.

[0017] Optionally, the step of adding a corresponding data copy operator to the hardware graph structure if the operator corresponding to the hardware graph structure contains a software operator includes:

[0018] A second data copy operator is added after the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and the hardware module is called to perform software calculations on the data to be processed.

[0019] Optionally, the step of serializing the neural network into a software graph structure using the compiler includes:

[0020] Obtain the preset relationship between the components of the neural network and the software operator;

[0021] The neural network is serialized into the software graph structure according to the preset relationship.

[0022] Optionally, before the step of obtaining the neural network through the construction interface, the method further includes:

[0023] When a call instruction is received from the upper-level compiler, the call to the upper-level compiler is accepted;

[0024] The neural network constructed by the upper-level compiler is obtained through the construction interface.

[0025] Furthermore, to achieve the above objectives, the present invention also provides a hardware computing simulation system, the hardware computing simulation system comprising:

[0026] Build modules are used to obtain neural networks through build interfaces;

[0027] A compilation module is used to serialize the neural network into a software graph structure using a compiler;

[0028] The execution module is used to acquire the data to be processed and the software graph structure through the executor, and call the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed.

[0029] In addition, to achieve the above objectives, the present invention also provides a hardware computing simulation system, the hardware computing simulation system comprising: a memory, a processor, and a hardware computing simulation program stored in the memory and executable on the processor, wherein the hardware computing simulation program, when executed by the processor, implements the steps of the hardware computing simulation method as described above.

[0030] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a hardware computing simulation program, which, when executed by a processor, implements the steps of the hardware computing simulation method as described above.

[0031] This invention proposes a hardware computing simulation method, system, and computer-readable storage medium. The hardware computing simulation method is applied to a hardware computing simulation system. First, a neural network is obtained through a construction interface. Then, the neural network is serialized into a software graph structure through a compiler. Next, the executor obtains the data to be processed and the software graph structure, and calls the software operators corresponding to the software graph structure to perform software calculations on the data to be processed, thereby simulating the hardware calculations performed by the hardware module on the data to be processed. By serializing the neural network into a software graph structure composed of software operators, and then calling the corresponding software operators to simulate the hardware calculations of the hardware module, compared with traditional hardware computing simulation methods, the hardware computing simulation method proposed in this invention supports hardware devices with unique designs and optimizations. It solves the problems of operator incompatibility and accuracy loss when traditional inference frameworks simulate hardware device calculations, thus improving the adaptability and accuracy of simulating hardware device calculations. Attached Figure Description

[0032] Figure 1This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention;

[0033] Figure 2 This is a flowchart illustrating an embodiment of the hardware computing simulation method of the present invention;

[0034] Figure 3 This is a flowchart illustrating another embodiment of the hardware computing simulation method of the present invention;

[0035] Figure 4 This is a schematic diagram of the basic components of the neural network involved in the hardware computing simulation method of the present invention;

[0036] Figure 5 This is a schematic diagram of the basic components of the software graph structure involved in the hardware computing simulation method of the present invention;

[0037] Figure 6 This is a schematic diagram of the basic components of the hardware graph structure involved in the hardware computing simulation method of the present invention;

[0038] Figure 7 This is a schematic diagram of the software operator computation process involved in the hardware computation simulation method of the present invention;

[0039] Figure 8 This is a modular schematic diagram of the hardware computing simulation system involved in an embodiment of the present invention.

[0040] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0041] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0042] While open-source deep learning inference frameworks such as MNN / NCNN / TNN can support common hardware devices on the market, they may cause problems such as operator incompatibility and accuracy loss when using them on some hardware devices with unique designs and optimizations. These inference frameworks cannot fully simulate the performance characteristics and advantages of the hardware devices and cannot provide accurate testing support during the hardware device design phase.

[0043] To improve the adaptability and accuracy of simulated hardware computing, this invention proposes a hardware computing simulation method, system, and computer-readable storage medium. The main steps of the method include:

[0044] The neural network is obtained through the aforementioned construction interface;

[0045] The compiler serializes the neural network into a software graph structure.

[0046] The actuator acquires the data to be processed and the software graph structure, and calls the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed.

[0047] By serializing the neural network into a software graph structure and calling the corresponding software operators to perform software calculations on the data to be processed, the simulation of the hardware device's calculation process is realized, improving the adaptability and accuracy of the simulation.

[0048] The claims of this invention will be described in detail below with reference to the accompanying drawings.

[0049] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.

[0050] The hardware computing simulation system can run on the terminal in this embodiment of the invention.

[0051] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a memory 1003, and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The memory 1003 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1003 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] like Figure 1 As shown, the memory 1003, which serves as a computer storage medium, may include an operating system and a hardware computing simulation program.

[0054] exist Figure 1 In the terminal shown, the processor 1001 can be used to call the hardware computing simulation program stored in the memory 1003 and perform the following operations:

[0055] The neural network is obtained through the aforementioned construction interface;

[0056] The compiler serializes the neural network into a software graph structure.

[0057] The actuator acquires the data to be processed and the software graph structure, and calls the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed.

[0058] Furthermore, the processor 1001 can call the hardware computing simulation program stored in the memory 1003 and also perform the following operations:

[0059] The software graph structure is serialized into a hardware graph structure by the hardware compilation unit;

[0060] The actuator acquires the data to be processed and the hardware graph structure, and calls the hardware module to perform the hardware calculation on the data to be processed.

[0061] Furthermore, the processor 1001 can call the hardware computing simulation program stored in the memory 1003 and also perform the following operations:

[0062] If the operator corresponding to the hardware graph structure contains a software operator, then a corresponding data copy operator is added to the hardware graph structure.

[0063] Furthermore, the processor 1001 can call the hardware computing simulation program stored in the memory 1003 and also perform the following operations:

[0064] A first data copy operator is added before the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and the corresponding software operator is called to perform software calculations on the data to be processed.

[0065] Furthermore, the processor 1001 can call the hardware computing simulation program stored in the memory 1003 and also perform the following operations:

[0066] A second data copy operator is added after the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and the hardware module is called to perform software calculations on the data to be processed.

[0067] Furthermore, the processor 1001 can call the hardware computing simulation program stored in the memory 1003 and also perform the following operations:

[0068] Obtain the preset relationship between the components of the neural network and the software operator;

[0069] The neural network is serialized into the software graph structure according to the preset relationship.

[0070] Furthermore, the processor 1001 can call the hardware computing simulation program stored in the memory 1003 and also perform the following operations:

[0071] When a call instruction is received from the upper-level compiler, the call to the upper-level compiler is accepted;

[0072] The neural network constructed by the upper-level compiler is obtained through the construction interface.

[0073] With the rapid development of deep learning and AI chips, deep learning systems are usually quite complex, and various deep learning frameworks and hardware devices have unique architectures. This makes the deployment, testing, and iterative improvement of accuracy and performance tuning of deep learning development environments tedious and time-consuming. In order to simulate the computational process of deep learning, deep learning inference frameworks have been introduced.

[0074] There are many common deep learning inference frameworks, mainly divided into two categories. One category consists of inference frameworks developed by hardware manufacturers for their specific hardware, such as Intel's OpenVINO, NVIDIA's TensorRT, and Google's Mediapipe. Among them, OpenVINO and TensorRT can support model formats of mainstream deep learning frameworks such as TensorFlow, PyTorch, MXNet, and Caffe, while Mediapipe only supports the model format of the TensorFlow deep learning framework. Although frameworks like OpenVINO, TensorRT, and Mediapipe have good performance, they support only a single hardware platform, mostly supporting self-developed hardware platforms. Another type is open-source deep learning inference frameworks such as MNN / NCNN / TNN, which can support multiple training frameworks such as TensorFlow, PyTorch, MXNet, and Caffe. However, these open-source deep learning inference frameworks all define their own model format, which inevitably requires the use of the converters provided by each framework. Furthermore, for some hardware devices with unique designs and optimizations, or for some self-developed operators, using open-source frameworks may lead to problems such as operator incompatibility and accuracy loss, making it impossible to fully utilize the performance characteristics and advantages of self-developed hardware platforms and failing to provide accurate runtime testing for hardware devices.

[0075] Therefore, the aforementioned defects exist in relevant hardware computing simulation methods. To address these defects, this invention proposes a hardware computing simulation method for hardware computing simulation systems. This method aims to resolve issues such as operator incompatibility and accuracy loss when using open-source deep learning inference frameworks for some uniquely designed and optimized hardware devices, thereby improving the adaptability and accuracy of the simulated hardware device computation.

[0076] The following explanation, through specific exemplary solutions, clarifies the scope of protection claimed in the claims of this invention, so that those skilled in the art can better understand the scope of protection of the claims. It is understood that the following exemplary solutions do not limit the scope of protection of this invention, but are only used to explain this invention.

[0077] For example, refer to Figure 2 In one embodiment of the hardware computing simulation method of the present invention, the hardware computing simulation method includes the following steps:

[0078] S10. Obtain the neural network through the construction interface;

[0079] In this embodiment, the main body executing the hardware computing simulation method is the hardware computing simulation system. The hardware computing simulation system can schedule operators for computational inference. The hardware computing simulation system includes a construction interface, a compiler, and an executor. The construction interface is connected to the compiler, and the compiler is connected to the executor. The executor is used to call software operators or hardware modules to perform calculations on the data to be processed. The hardware module includes hardware devices, such as the CAISA chip. The CAISA chip is an artificial intelligence chip for edge and cloud inference. The CAISA chip supports all commonly used AI operators. Through different configurations and combinations of operators in the data flow network, the CAISA chip can support most CNN algorithms (Convolutional Neural Networks). The construction interface is a higher-level abstract interface in the hardware computing simulation system, used to obtain the neural network. A neural network is an algorithmic mathematical model for distributed parallel information processing. It achieves the purpose of processing information by adjusting the interconnections between a large number of internal nodes. The neural network is obtained through a construction window, such as... Figure 4 As shown, Figure 4 It is a basic component of the convolutional neural network MobileNet, which is a commonly used convolutional neural network in mobile devices or embedded systems. It is a small and efficient CNN model. The 3×3DepthWise Conv (3×3 depthwise convolution), BN (normalization), ReLU (activation function), and 1×1 Conv (1×1 convolution) are the calculation flow for the data to be processed.

[0080] Optionally, the construction interface can receive calls from the upper-level compiler. Before the step of obtaining the neural network through the construction interface, if the construction interface receives a call instruction from the upper-level compiler, it will accept the call from the upper-level compiler. The upper-level compiler can be the RainBuilder compiler, which is a general-purpose end-to-end automatic compilation tool. The RainBuilder compiler can build neural networks through the construction interface of the hardware computing simulation system, thereby decoupling the RainBuilder compiler from hardware computing, reducing the difficulty of building neural networks, and making them easier to maintain.

[0081] S20. The neural network is serialized into a software graph structure using the compiler;

[0082] In this embodiment, the compiler of the hardware computing simulation system is connected to the construction interface and is located in the lower layer of the construction interface. When the construction interface obtains the neural network, it serializes the components of the neural network into a software graph structure, which is the arrangement of pure software operators.

[0083] Optionally, the step of serializing a neural network into a software graph structure using a compiler includes: obtaining a preset relationship between the neural network components and the software operators; and serializing the neural network into the software graph structure according to the preset relationship.

[0084] There is a corresponding mapping relationship between neural network components and software operators. The mapping relationship can be set in advance. When the compiler loads the neural network, it parses the components of the neural network, identifies the corresponding software operators of the neural network according to the preset relationship, and arranges them in order into a software graph structure.

[0085] Reference Figure 4 and Figure 5 , Figure 4 These are the basic components of the MobileNet convolutional neural network. Taking single-batch processing as an example, after the hardware computing simulation system's construction interface obtains the MobileNet neural network, the compiler of the hardware computing simulation system represents the basic components of the MobileNet neural network as follows: Figure 5 The MobileNet software graph structure SW_GRAPH shown here, where op_m_gconv (depth 2D convolution operator) and op_m_conv (2D convolution operator) represent the computational flow of the software graph structure for the data to be processed.

[0086] S30. The actuator obtains the data to be processed and the software graph structure, and calls the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed.

[0087] In this embodiment, the executor of the hardware computing simulation system is connected to the compiler and is located below the compiler. After the compiler serializes the neural network into a software graph structure, it obtains the data to be processed and the software graph structure. Based on the software operators on the software graph structure, it calls the corresponding software operators to calculate the data to be processed. The data to be processed can be an image that needs feature extraction. The software operators are simulation operators that can simulate hardware modules. Its calculation process is completed on the CPU. The executor can call the software operators on the CPU. The software operators are simple to deploy and suitable for demonstration effects. The executor can also call hardware modules. The hardware modules calculate the data to be processed quickly and are suitable for actual production scenarios.

[0088] When the software operator performs calculations on the data to be processed, the calculation process is the same as that of the hardware module. The software can simulate the hardware module, and the hardware module called by the executor can be a CAISA chip. Through the unified scheduling and optimization of the executor in the hardware calculation simulation system provided in this embodiment of the invention, the computational performance advantages of the CAISA chip's dataflow architecture can be fully utilized, while also supporting INT8 / INT16 quantization precision modes. Operator calculations that simulate hardware behavior can provide a reference for accuracy verification during the chip design phase, and can be cross-verified with the calculation results of the CAISA chip, further improving the efficiency of verification and debugging.

[0089] In the technical solution disclosed in this embodiment, a neural network is first obtained through the construction interface; the neural network is serialized into a software graph structure through the compiler; the data to be processed and the software graph structure are obtained through the executor, and the software operator corresponding to the software graph structure is called to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed. By serializing the neural network components into a software graph structure and calling the corresponding software operator according to the software graph structure to perform software calculations on the data to be processed, the software calculation has a demonstration effect and can simulate the hardware calculations of the data to be processed. The software calculation can provide a reference for accuracy verification in the chip design stage and can be mutually verified with the calculation results of the CAISA chip, further improving the efficiency of verification and debugging. The hardware calculation simulation method proposed in this embodiment can not only simulate the hardware calculations of hardware modules, but also adapt to some hardware devices with unique designs and optimizations, or some self-developed operators. It can give full play to the performance and advantages of hardware modules, provide accurate running tests for the calculations of hardware modules, eliminate the need for complex hardware environment installation and deployment, and improve the adaptability and accuracy when simulating hardware device calculations.

[0090] Optionally, refer to Figure 3Based on any of the above embodiments, in another embodiment of the hardware computing simulation method of the present invention, the hardware computing simulation method includes:

[0091] S40. The software graph structure is serialized into a hardware graph structure by the hardware compilation unit;

[0092] In this embodiment, step S40 is performed after step S20. The compiler can not only serialize the neural network into a software graph structure, but also includes a hardware compilation unit that can serialize the software graph structure into a hardware graph structure. The hardware graph structure will be optimized by splitting and merging according to hardware features and the characteristics of specific operators, so as to achieve faster calculation speed.

[0093] Optionally, if the operator corresponding to the hardware graph structure contains a software operator, then a corresponding data copy operator is added to the hardware graph structure.

[0094] The hardware computation simulation method in this embodiment supports user-defined software operators. If there are software operators before and after the hardware operators, a data copy operator OP_DATA_CPY needs to be added accordingly to copy the data to be processed. Specifically, the data to be processed is copied from DDR (Double Data Rate Synchronous Dynamic Random Memory) to memory, or the data to be processed is copied from memory to DDR.

[0095] Specifically, a data copy operator is added before the software operator in the operator corresponding to the hardware graph structure to copy the data to be calculated from the hardware module to the memory where the software operator is located, so that the data to be calculated can be performed by software computation. Alternatively, a data copy operator is added after the software operator in the operator corresponding to the hardware graph structure to copy the data to be calculated from the memory where the software operator is located to the hardware module, so that the data to be calculated can be performed by hardware computation.

[0096] Understandably, the hardware module includes memory to store the operators needed for hardware computation, which can be DDR memory. Software operators are stored in the memory corresponding to the CPU. While software operators may exist in the hardware graph structure, they may not be stored in the hardware module's memory. Therefore, it's impossible to perform hardware operators on the hardware module based on the hardware graph structure. To address this, a data copy operator is added before the software operators in the hardware graph structure. This copies the data to be processed from DDR to memory, calls the corresponding software operator, and performs the computation on the CPU. If further computation on the hardware module is needed after the software operator, a data copy operator is added after the software operator in the hardware graph structure to copy the data to be processed from memory to DDR, allowing hardware computation to continue on the hardware module.

[0097] Reference Figure 5 and Figure 6 , Figure 5 It is the MobileNet software graph structure SW_GRAPH for neural networks. Figure 6 It is the hardware graph structure HW_GRAPH, which is the serialized result of the MobileNet software graph structure SW_GRAPH in the compiler's hardware compilation module. Figure 6 In the diagram, `op_m_gconv_HW` (depth 2D convolution operator) and `op_m_gconv_HW` (2D convolution operator) represent the computation flow of the data to be processed in the hardware graph structure. The dashed rectangle indicates that the operator may exist, and the operator within the dashed rectangle is the data copy operator `OP_DATA_CPY`. If there are software operators in the hardware graph structure, the data copy operator also exists; the specific presence depends on whether there are operators that need to be performed in the hardware module before or after the software operators.

[0098] S50. The actuator obtains the data to be processed and the hardware graph structure, and calls the hardware module to perform the hardware calculation on the data to be processed.

[0099] In this embodiment, after the hardware compilation unit of the compiler in the hardware computing simulation system transforms the software graph structure of the neural network into a hardware graph structure, the executor obtains the data to be processed and the hardware graph structure. The data to be processed can be an image that needs to be feature extracted. The hardware module called by the executor performs calculations on the data to be processed. Compared with software calculations, hardware calculations performed in the hardware module are faster and have stronger performance, making them suitable for actual production. The hardware module is located in the lower layer of the executor and can be a CAISA chip.

[0100] This embodiment is based on an existing embodiment. Step S50 can be performed simultaneously with step S30, or before step S30, or after step S30. This embodiment does not limit this.

[0101] It is understandable that the software computation mentioned in step S30 is actually simulating the hardware computation in this embodiment. The software operator can run on the CPU to show the expected effect and computation process on the hardware module. The results of the software computation can be mutually verified with the results of the hardware computation, which simplifies the deployment, testing, and iterative improvement of the hardware module's development environment, as well as the accuracy and performance tuning, and provides accurate testing support for hardware devices used for deep learning during the design phase.

[0102] To facilitate understanding, the following explanation uses specific application scenarios to illustrate this solution. Please refer to the provided examples. Figure 4 , Figure 5 , Figure 6 , Figure 7 .

[0103] Figure 4 These are the basic components of the MobileNet convolutional neural network. Taking single-batch processing as an example, after the hardware computing simulation system's graphing interface obtains the MobileNet neural network, the hardware computing simulation system's compiler represents the basic components of the MobileNet neural network as follows: Figure 5 The MobileNet software graph structure SW_GRAPH is shown. Figure 6 It is the hardware graph structure HW_GRAPH, which is the serialized result of the MobileNet software graph structure SW_GRAPH in the compiler's hardware compilation module. Figure 6 In the diagram, a dashed rectangle indicates that the operator may exist. The operator within the dashed rectangle is the data copy operator OP_DATA_CPY. If there is a software operator in the hardware diagram structure, the data copy operator will exist. The specific location depends on whether there are any operators that need to be performed in the hardware module before or after the software operator.

[0104] After the hardware computing simulation system converts SW_GRAPH to HW_GRAPH, the executor will execute the corresponding operators to perform calculations based on these two graph structures. The SW_GRAPH software graph structure will be used to execute operators that simulate hardware computing behavior in the CPU, while the HW_GRAPH hardware graph structure will be used by the executor to schedule the CAISA chip hardware module to perform calculations on the data.

[0105] The SW_GRAPH software graph structure will invoke software operators that simulate hardware computation behavior through the executor, here taking... Figure 5 The INT8 / INT16 quantization process of the software operator M_CONV is illustrated with an example.

[0106] The convolution formula corresponding to M_CONV is:

[0107]

[0108] f(x) represents the output of the convolution, x is the input of the convolutional layer, w is the weight matrix of the convolutional layer, and b is the bias unit. kh and kw represent the height and width of the convolutional kernel, and c is the number of input channels.

[0109] The quantization formula corresponding to M_CONV is:

[0110]

[0111] r represents a real number, q is the quantized value, S is the scaling factor from the real number's range to the quantization domain, and z is the zero point, which is the value of q when r=0, to ensure that r can take the value of 0.

[0112] Based on the two formulas above, the formula for the quantized convolution result can be derived:

[0113]

[0114] ro, ri, and rw represent the output, input, and weight matrices in the real domain; qo, qi, and qw represent the output, input, and weight matrices in the quantization domain; zo, zi, and zw represent the zeros of the output, input, and weight matrices; and so, si, and sw represent the scaling factors of the output, input, and weight matrices.

[0115] The specific calculation process in the CAISA chip hardware module is as follows: Figure 7 As shown, in a hardware computing simulation system, the computation process of calling software operators is also as follows: Figure 7 The diagram illustrates the computational behavior of the simulated hardware.

[0116] The above formula is implemented using the KRNL module and the ACTV module, with the calculation of the square brackets performed in the KRNL module:

[0117]

[0118] The constant part in formula (7) can be transferred to the ACTV module as a new bias for calculation, so formula (7) can be expressed as:

[0119]

[0120] The hardware simulation method provided in this embodiment can completely simulate the computation process and behavior of CAISA hardware, and can also obtain intermediate results of each module, providing a basis for hardware verification and debugging.

[0121] In the technical solution proposed in this embodiment, the software graph structure is serialized into a hardware graph structure by the hardware compilation unit; the executor obtains the data to be processed and the hardware graph structure, and calls the hardware module to perform hardware calculations on the data to be processed. On the one hand, data processing is completed by calling the hardware module, which improves the speed of data processing; on the other hand, the hardware calculation results can be mutually verified with the software calculation results, and the hardware calculation process can be displayed through software calculation, providing accurate testing support for each other, and also improving the adaptability and accuracy when simulating hardware device calculations.

[0122] Furthermore, this invention also proposes a hardware computing simulation system, which includes a memory, a processor, and a hardware computing simulation method program stored in the memory and executable on the processor. When the hardware computing simulation method program is executed by the processor, it implements the steps of the hardware computing simulation method described in the above embodiments.

[0123] Furthermore, embodiments of the present invention also propose a hardware computing simulation system, exemplarily referring to... Figure 8 The hardware computing simulation system 100 includes:

[0124] The system comprises a construction module 101, a compilation module 102, and an execution module 103. The construction module 101 is used to obtain a neural network through a construction interface; the compilation module 102 is used to serialize the neural network into a software graph structure through a compiler; and the execution module 103 is used to obtain the data to be processed and the software graph structure through an executor, and call the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed.

[0125] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a hardware computing simulation method program, which, when executed by a processor, implements the steps of the hardware computing simulation method as described in the above embodiments.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0127] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause the hardware computing simulation system to execute the methods described in the various embodiments of the present invention.

[0129] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A hardware computing simulation method, characterized in that, An application is made in a hardware computing simulation system, the hardware computing simulation system including a construction interface, a compiler, and an executor, the construction interface being connected to the compiler, the compiler being connected to the executor, the compiler including a hardware compilation unit, and the executor being used to call software operators or hardware modules to perform calculations on the data to be processed, the hardware computing simulation method including: The neural network is obtained through the aforementioned construction interface; The compiler serializes the neural network into a software graph structure, which is an arrangement of pure software operators. The software graph structure is serialized into a hardware graph structure by the hardware compilation unit; The actuator acquires the data to be processed and the hardware graph structure, and calls the hardware module to perform the hardware calculation on the data to be processed. The executor acquires the data to be processed and the software graph structure, and calls the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed. The hardware module does not store software operators in its memory. When the software operators perform software calculations on the data to be processed, the hardware module is not required. The software operators are simulation operators that simulate the hardware module. The software operators run on the CPU, the software calculations are performed on the CPU, and the expected effects and calculation processes on the hardware module are displayed. The hardware module is a CAISA chip. The step of serializing the neural network into a software graph structure using the compiler includes: Obtain the preset relationship between the components of the neural network and the software operator; The neural network is serialized into the software graph structure according to the preset relationship; The step of serializing the software graph structure into a hardware graph structure through the hardware compilation unit includes: If the operator corresponding to the hardware graph structure contains a software operator, a first data copy operator is added before the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and the corresponding software operator is called to perform software calculation on the data to be processed. A second data copy operator is added after the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and the hardware module is called to perform software calculations on the data to be processed.

2. The hardware computing simulation method as described in claim 1, characterized in that, Before the step of obtaining the neural network through the construction interface, the method further includes: When a call instruction is received from the upper-level compiler, the call to the upper-level compiler is accepted; The neural network constructed by the upper-level compiler is obtained through the construction interface.

3. A hardware computing simulation system, characterized in that, The hardware computing simulation system includes: Build modules are used to obtain neural networks through build interfaces; A compilation module is used to serialize the neural network into a software graph structure using a compiler. The software graph structure is an arrangement of pure software operators. The step of serializing the neural network into a software graph structure using the compiler includes: obtaining a preset relationship between the components of the neural network and the software operators; and serializing the neural network into the software graph structure according to the preset relationship. An execution module is used to acquire the data to be processed and the software graph structure through an executor, and call the software operator corresponding to the software graph structure to perform software calculations on the data to be processed, so as to simulate the hardware calculations of the hardware module on the data to be processed. The software operator is a simulation operator that simulates the hardware module. The software operator runs on the CPU, the software calculation is performed on the CPU, and the expected effect and calculation process on the hardware module are displayed. The hardware module is a CAISA chip. The execution module is further configured to serialize the software graph structure into a hardware graph structure through a hardware compilation unit; obtain the data to be processed and the hardware graph structure through the executor, and call the hardware module to perform the hardware calculation on the data to be processed; the step of serializing the software graph structure into a hardware graph structure through the hardware compilation unit includes: if the operator corresponding to the hardware graph structure contains a software operator, adding a first data copy operator before the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and calling the corresponding software operator to perform software calculation on the data to be processed; adding a second data copy operator after the software operator in the operator corresponding to the hardware graph structure to copy the data to be processed from the hardware module to the memory where the software operator is located, and calling the hardware module to perform software calculation on the data to be processed.

4. A hardware computing simulation system, characterized in that, The hardware computing simulation system includes: a memory, a processor, and a hardware computing simulation program stored in the memory and executable on the processor. When the hardware computing simulation program is executed by the processor, it implements the steps of the hardware computing simulation method as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a hardware computing simulation program, which, when executed by a processor, implements the steps of the hardware computing simulation method as described in any one of claims 1 to 2.

Images