Reconfigurable AI chip-oriented neural network reasoning deployment method and system

Abstract

The invention discloses a reconfigurable AI chip-oriented neural network reasoning deployment method, which comprises the following steps of: exporting a trained model into an ONNX format by using an ONNX library corresponding to a deep learning framework, and analyzing the ONNX format model to obtain a network topology structure, operator information and general control information; dividing each operator according to the operator information to obtain a division result; assembling and splicing operator division results in combination with a network topology structure to obtain task information, and the like. Hardware resources of the reconfigurable AI chip are fully utilized. And finally, organizing general control information and task information of all operators into a final executable file, and efficiently completing reasoning deployment of the neural network model on the reconfigurable AI chip. According to the method and the device, the technical problems of high development threshold and low efficiency of the existing specific AI chip architecture compiler can be solved, and the technical problem that the existing automatic code generation method possibly cannot fully utilize the chip performance can be solved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and more specifically, relates to a neural network reasoning deployment method and system for a reconfigurable AI chip. Background Art

[0002] In recent years, the rapid development of artificial intelligence has deeply penetrated into many key fields such as medical care, transportation, and finance, becoming the core force driving industrial transformation. As the hardware cornerstone of artificial intelligence computing, AI reconfigurable chips, with their unique architectural advantages, can flexibly configure hardware resources according to diverse algorithm requirements, showing excellent performance potential, and are highly favored in dealing with complex and changeable AI tasks.

[0003] There are two main existing neural network deployment methods. The first is to develop a compiler for a specific AI chip architecture. After parsing the representation of the neural network, it generates corresponding machine instructions from the operators in the model according to the chip instruction set and hardware resource limitations. The second is to use automatic code generation methods, using high-level programming languages ​​​​(such as Python) and code generation templates to automatically generate code according to the structure of the neural network model and the hardware parameters of the AI ​​chip. These tools can generate efficient code based on the type and number of operators in the model, combined with the computing power and storage limitations of the chip.

[0004] However, both of the above two existing neural network deployment methods have some defects that cannot be ignored: first, developing a compiler for a specific AI chip architecture has a high learning cost and development threshold for developers, and the development cycle is very long and the efficiency is low; second, the automatic code generation method has very high requirements for preset templates and code generation rules. If the details of the AI ​​chip architecture are not clear enough, it is easy to fail to generate high-performance instructions that adapt to the chip architecture. Summary of the invention

[0005] In response to the above defects or improvement needs of the prior art, the present invention provides a neural network reasoning deployment method and system for a reconfigurable AI chip, which aims to solve the technical problems of high threshold and low efficiency in developing compilers for specific AI chip architectures, and the technical problem that the existing automatic code generation method may not be able to fully utilize the chip performance.

[0006] To achieve the above object, according to one aspect of the present invention, a neural network reasoning deployment method for a reconfigurable AI chip is provided, comprising the following steps:

[0007] (1) Obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model;

[0008] (2) Parse the model in ONNX format obtained in step (1) to obtain the network topology of the model, separate the operator information of all operators (including three types of convolution type operators, pooling type operators and fully connected type operators) from the network topology, and obtain the master control information in the executable file corresponding to the model according to the network topology and operator information.

[0009] (3) Perform operator partitioning on each operator according to the operator information obtained in step (2) to obtain the operator partitioning result corresponding to the operator.

[0010] (4) Assemble and splice the partitioning results of all operators obtained in step (3) to obtain the task information of all operators.

[0011] (5) Organize the master control information obtained in step (2) and the task information of all operators obtained in step (4) into a final executable file.

[0012] Preferably, the deep learning framework is PyTorch or TensorFlow.

[0013] Operators include three types: convolution type operators, pooling type operators, and fully connected type operators;

[0014] For the convolution type operator, the operator information of the convolution type operator includes the size of the input feature map corresponding to the convolution type operator, the size of the output feature map, the convolution kernel size, the number of input channels, the number of output channels, the input feature map address, the output feature map address and the weight data address.

[0015] For the pooling type operator, the operator information of the pooling type operator includes the size of the input feature map corresponding to the pooling type operator, the size of the output feature map, the pooling window size, the pooling type and other information.

[0016] For a fully connected type operator, the operator information of the fully connected type operator includes information such as input dimension and output dimension.

[0017] Preferably, the process of obtaining the master control information in the executable file according to the network topology structure and operator information in step (2) is as follows: first, analyzing the connection relationship between each node in the network topology structure to obtain the data flow direction information of the network topology structure; then, according to the operator information, completing the data flow direction information, thereby obtaining the master control information in the executable file; the master control information is the operator outline of the executable file.

[0018] Preferably, step (3) comprises the following sub-steps:

[0019] (3-1) Obtain a reconfigurable AI chip operator library, and create a corresponding basic operator for each operator obtained in step (2) based on the reconfigurable AI chip operator library, and the basic operators corresponding to all operators constitute a basic operator library;

[0020] (3-2) Set counter i=1;

[0021] (3-3) Determine whether i is greater than the total number of all operators split from the network topology structure in step (2). If so, the process ends; otherwise, proceed to step (3-4);

[0022] (3-4) According to step (2), the i-th operator among all the operators split from the network topology structure is judged whether the basic operator corresponding to the i-th operator can be obtained from the basic operator library obtained in step (3-1). If so, proceed to step (3-6); otherwise, proceed to step (3-5);

[0023] (3-5) Select a basic operator with the same convolution kernel size and the same number of input channels as the i-th operator obtained in step (2) from the basic operator library obtained in step (3-1), and fill the part of the input feature map of the i-th operator that cannot be completely covered by the basic operator with 0;

[0024] (3-6) dividing the input feature map corresponding to the i-th operator into blocks according to the basic operator corresponding to the i-th operator to obtain the division result of the i-th operator;

[0025] (3-7) Set i=i+1 and return to step (3-3);

[0026] Preferably, step (3-4) comprises the following sub-steps:

[0027] (3-4-1) Set counter j = 1;

[0028] (3-4-2) Determine whether j is greater than the number of operators in the basic operator library. If so, the process ends; otherwise, proceed to step (3-4-3);

[0029] (3-4-3) Determine whether the jth basic operator in the basic operator library has the same convolution kernel size as the i-th operator obtained in step (2), whether the number of input channels of the jth basic operator is the same as the i-th operator obtained in step (2), and whether the jth basic operator can completely cover the input feature map corresponding to the i-th operator obtained in step (2). If so, the process ends, otherwise proceed to (3-4-4);

[0030] (3-4-4) Set j=j+1 and return to step (3-4-2).

[0031] Preferably, step (3-6) comprises the following sub-steps:

[0032] (3-6-1) According to the basic operator corresponding to the i-th operator, calculate the total number of blocks block_nums into which the input feature map corresponding to the i-th operator is divided;

[0033] (3-6-2) Set counter k = 1;

[0034] (3-6-3) Determine whether k is greater than the total number of blocks block_nums obtained in step (3-6-1). If so, assemble and splice all the instantiated basic operators obtained. Otherwise, proceed to step (3-6-4);

[0035] (3-6-4) Calculate the data input address input_addr in the input feature graph corresponding to the i-th operator according to k, and calculate the output address output_addr of the operation result of the i-th operator according to k;

[0036] (3-6-5) Write the input address obtained in step (3-6-4) to the corresponding position of the load instruction of the basic operator corresponding to the i-th operator, and write the output address of the obtained operation result to the corresponding position of the store instruction of the basic operator corresponding to the i-th operator, so as to obtain the instantiated basic operator;

[0037] (3-6-6) Set k=k+1 and return to step (3-6-3).

[0038] Preferably, the total number of blocks block_nums is equal to:

[0039] block_nums=inputFeatureMap_size÷kernelFeatureMap_size

[0040] Among them, inputFeatureMap_size represents the size of the input feature map corresponding to the i-th operator, and kernelFeatureMap_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator.

[0041] Preferably, the data input address input_addr is equal to:

[0042] input_addr=(k÷block_nums)×(input_featureMap_size×(operator_input_feature_map_size-operator_kernel_size+1))+k%block_nums)×(operator_input_feature_map_size-operator_kernel_size+1)

[0043] Where k represents the kth block, block_nums represents the total number of blocks obtained in step (3-6-1), input_featureMap_size represents the size of the input feature map corresponding to the i-th operator, operatorinput_feature_map_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator, and operator_kernel_size represents the convolution kernel size of the i-th operator.

[0044] The output address output_addr of the operation result is equal to:

[0045] output_addr=(k÷(output_size÷operator_output_size))×(output_size×operator_output_size)+(k%(output_size÷operator_output_size))×operator_output_size

[0046] Among them, output_size represents the size of the output feature map corresponding to the i-th operator, and operatoroutput_size represents the size of the output feature map corresponding to the basic operator corresponding to the i-th operator.

[0047] Preferably, step (5) is to write the task information of all operators obtained in step (4) into the executable file in sequence according to the network topology structure of the model obtained in step (2), and at the same time write the master control information obtained in step (2) into the corresponding position of the executable file, and finally obtain the executable file of the model trained in step (1).

[0048] According to another aspect of the present invention, a neural network reasoning deployment system for a reconfigurable AI chip is provided, comprising:

[0049] The first module is used to obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange ONNX library corresponding to the deep learning framework to export the trained model into an ONNX format model;

[0050] The second module is used to parse the ONNX format model obtained by the first module to obtain the network topology structure of the model, split the operator information of all operators from the network topology structure, and obtain the master control information in the executable file corresponding to the model based on the network topology structure and the operator information.

[0051] The third module is used to perform operator division on each operator according to the operator information obtained by the second module to obtain the operator division result corresponding to the operator.

[0052] The fourth module is used to assemble and splice the division results of all operators obtained by the third module to obtain the task information of all operators.

[0053] The fifth module is used to organize the master control information obtained by the second module and the task information of all operators obtained by the fourth module into a final executable file.

[0054] In general, the technical method conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0055] (1) Since the present invention adopts steps (2) to (4), it can enable developers to quickly understand the model network structure by efficiently parsing the ONNX format model and call a complete basic operator library to perform operator division, thereby solving the technical problems of high threshold and low efficiency in the existing development of compilers for specific AI chip architectures;

[0056] (2) Since the present invention adopts step (3), based on the AI ​​chip architecture characteristics, it compiles a series of basic neural network operators (convolution, pooling, full connection) that fully utilize the chip hardware characteristics, forming a complete basic operator library. The operator library is used to divide operators, thereby solving the technical problem that the existing automatic code generation method may not fully utilize the chip performance;

[0057] (3) Due to step (1), the present invention exports the network model algorithm into the ONNX format. Since many deep learning frameworks support converting their own models into the ONNX format, this makes the tool chain have a broad compatibility basis for cross-hardware deployment scenarios;

[0058] (4) Due to step (3), the present invention can reasonably divide convolution, pooling and full connection of different sizes through an efficient partitioning algorithm for the three basic operator types, making the entire tool chain more versatile and flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of a neural network reasoning deployment method for a reconfigurable AI chip of the present invention;

[0060] Figure 2 This is a flow chart of the convolution operator partitioning in step (3) of the method of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0062] The basic idea of ​​the present invention is to provide a neural network reasoning deployment method for reconfigurable AI chips based on the characteristics of reconfigurable AI chips. The trained model is exported to the ONNX format by using the ONNX library corresponding to the deep learning framework, and the ONNX format model is parsed to obtain the network topology, operator information and master control information. Each operator is divided according to the operator information to obtain the division result; the operator division results are assembled and spliced ​​in combination with the network topology to obtain task information and other steps, so as to make full use of the hardware resources of the reconfigurable AI chip; finally, the master control information and the task information of all operators are organized into a final executable file, so as to efficiently complete the reasoning deployment of the neural network model on the reconfigurable AI chip.

[0063] like Figure 1 As shown, the present invention provides a neural network reasoning deployment method for a reconfigurable AI chip, comprising the following steps:

[0064] (1) Obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model;

[0065] Specifically, the deep learning framework in this step is PyTorch or TensorFlow.

[0066] (2) Parse the model in ONNX format obtained in step (1) to obtain the network topology of the model, separate the operator information of all operators (including three types of convolution type operators, pooling type operators and fully connected type operators) from the network topology, and obtain the master control information in the executable file corresponding to the model according to the network topology and operator information.

[0067] Specifically, operators include three types: convolution type operators, pooling type operators and fully connected type operators.

[0068] For the convolution type operator, the operator information of the convolution type operator includes the size of the input feature map corresponding to the convolution type operator, the size of the output feature map, the convolution kernel size, the number of input channels, the number of output channels, the input feature map address, the output feature map address and the weight data address.

[0069] For the pooling type operator, the operator information of the pooling type operator includes the size of the input feature map corresponding to the pooling type operator, the size of the output feature map, the pooling window size, the pooling type and other information.

[0070] For a fully connected type operator, the operator information of the fully connected type operator includes information such as input dimension and output dimension.

[0071] The process of obtaining the master control information in the executable file based on the network topology and operator information in this step is as follows: first, analyze the connection relationship between the nodes in the network topology to obtain the data flow direction information of the network topology; then, based on the operator information, complete the data flow direction information to obtain the master control information in the executable file; the master control information is the operator outline of the entire executable file.

[0072] The advantage of the above steps (1) to (2) is that developers can use the deep learning framework they are familiar with to build and train the network. Secondly, the parsing of the ONNX format model makes the tool chain highly scalable and flexible.

[0073] (3) Perform operator partitioning on each operator according to the operator information obtained in step (2) to obtain the operator partitioning result corresponding to the operator.

[0074] like Figure 2 As shown, this step includes the following sub-steps:

[0075] (3-1) Obtain a reconfigurable AI chip operator library, and create a corresponding basic operator for each operator obtained in step (2) based on the reconfigurable AI chip operator library, and the basic operators corresponding to all operators constitute a basic operator library;

[0076] (3-2) Set counter i=1;

[0077] (3-3) Determine whether i is greater than the total number of all operators split from the network topology structure in step (2). If so, the process ends; otherwise, proceed to step (3-4);

[0078] (3-4) According to step (2), the i-th operator among all the operators split from the network topology structure is judged whether the basic operator corresponding to the i-th operator can be obtained from the basic operator library obtained in step (3-1). If so, proceed to step (3-6); otherwise, proceed to step (3-5);

[0079] Specifically, step (3-4) includes the following sub-steps:

[0080] (3-4-1) Set counter j = 1;

[0081] (3-4-2) Determine whether j is greater than the number of operators in the basic operator library. If so, the process ends; otherwise, proceed to step (3-4-3);

[0082] (3-4-3) Determine whether the jth basic operator in the basic operator library has the same convolution kernel size as the i-th operator obtained in step (2), whether the number of input channels of the jth basic operator is the same as the i-th operator obtained in step (2), and whether the jth basic operator can completely cover the input feature map corresponding to the i-th operator obtained in step (2). If so, the process ends, otherwise proceed to (3-4-4);

[0083] (3-4-4) Set j=j+1 and return to step (3-4-2);

[0084] (3-5) Select a basic operator with the same convolution kernel size and the same number of input channels as the i-th operator obtained in step (2) from the basic operator library obtained in step (3-1), and fill the part of the input feature map of the i-th operator that cannot be completely covered by the basic operator with 0;

[0085] The advantage of this step (3-5) is that it enables the system to adapt to operators of larger sizes and improves the scalability of the system;

[0086] (3-6) dividing the input feature map corresponding to the i-th operator into blocks according to the basic operator corresponding to the i-th operator to obtain the division result of the i-th operator;

[0087] Specifically, step (3-6) includes the following sub-steps:

[0088] (3-6-1) According to the basic operator corresponding to the i-th operator, calculate the total number of blocks block_nums into which the input feature map corresponding to the i-th operator is divided;

[0089] Specifically, the total number of blocks block_nums is equal to:

[0090] block_nums=inputFeatureMap_size÷kernelFeatureMap_size

[0091] Among them, inputFeatureMap_size represents the size of the input feature map corresponding to the i-th operator, and kernelFeatureMap_size represents the size of the input feature map corresponding to the base operator corresponding to the i-th operator;

[0092] (3-6-2) Set counter k = 1;

[0093] (3-6-3) Determine whether k is greater than the total number of blocks block_nums obtained in step (3-6-1). If so, assemble and splice all the instantiated basic operators obtained. Otherwise, proceed to step (3-6-4);

[0094] (3-6-4) Calculate the data input address input_addr in the input feature graph corresponding to the i-th operator according to k, and calculate the output address output_addr of the operation result of the i-th operator according to k;

[0095] Specifically, the data input address input_addr is equal to:

[0096] input_addr=(k÷block_nums)×(input_featureMap_size×(operator_input_feature_map_size-operator_kernel_size+1))+k%block_nums)×(operator_input_feature_map_size-operator_kernel_size+1)

[0097] Where k∈[1, total number of blocks block_nums], input_featureMap_size represents the size of the input feature map corresponding to the i-th operator, operator_input_feature_map_size represents the size of the input feature map corresponding to the base operator corresponding to the i-th operator, and operator_kernel_size represents the convolution kernel size of the i-th operator;

[0098] The output address output_addr of the operation result is equal to:

[0099] output_addr=(k÷(output_size÷operator_output_size))×(output_size×operator_output_size)+(k%(output_size÷operator_output_size))×operator_output_size

[0100] Among them, output_size represents the size of the output feature map corresponding to the i-th operator, and operator_output_size represents the size of the output feature map corresponding to the basic operator corresponding to the i-th operator;

[0101] (3-6-5) Write the input address obtained in step (3-6-4) to the corresponding position of the load instruction of the basic operator corresponding to the i-th operator, and write the output address of the obtained operation result to the corresponding position of the store instruction of the basic operator corresponding to the i-th operator, so as to obtain the instantiated basic operator;

[0102] (3-6-6) Set k=k+1 and return to step (3-6-3);

[0103] (3-7) Set i=i+1 and return to step (3-3);

[0104] (4) Assemble and splice the partitioning results of all operators obtained in step (3) to obtain the task information of all operators.

[0105] Specifically, this step is to splice the results of operator division according to the results of operator division in step (3) to obtain the task information of all operators.

[0106] (5) Organize the master control information obtained in step (2) and the task information of all operators obtained in step (4) into a final executable file.

[0107] Specifically, this step writes the task information of all operators obtained in step (4) into the executable file in sequence according to the network topology structure of the model obtained in step (2), and writes the master control information obtained in step (2) into the corresponding position of the executable file, and finally obtains the executable file of the model trained in step (1).

[0108] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A neural network reasoning deployment method for a reconfigurable AI chip, characterized in that: The following steps are involved: (1) Obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model; (2) Parse the model in ONNX format obtained in step (1) to obtain the network topology of the model, separate the operator information of all operators (including three types of convolution type operators, pooling type operators and fully connected type operators) from the network topology, and obtain the master control information in the executable file corresponding to the model according to the network topology and operator information. (3) Perform operator partitioning on each operator according to the operator information obtained in step (2) to obtain the operator partitioning result corresponding to the operator. (4) Assemble and splice the partitioning results of all operators obtained in step (3) to obtain the task information of all operators. (5) Organize the master control information obtained in step (2) and the task information of all operators obtained in step (4) into a final executable file.

2. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 1 is characterized in that: The deep learning framework is PyTorch or TensorFlow. Operators include three types: convolution type operators, pooling type operators, and fully connected type operators; For the convolution type operator, the operator information of the convolution type operator includes the size of the input feature map corresponding to the convolution type operator, the size of the output feature map, the convolution kernel size, the number of input channels, the number of output channels, the input feature map address, the output feature map address and the weight data address. For the pooling type operator, the operator information of the pooling type operator includes the size of the input feature map corresponding to the pooling type operator, the size of the output feature map, the pooling window size, the pooling type and other information. For a fully connected type operator, the operator information of the fully connected type operator includes information such as input dimension and output dimension.

3. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 1 or 2, characterized in that: The process of obtaining the master control information in the executable file based on the network topology and operator information in step (2) is as follows: first, the connection relationship between the nodes in the network topology is analyzed to obtain the data flow direction information of the network topology; then, according to the operator information, the data flow direction information is completed to obtain the master control information in the executable file; the master control information is the operator outline of the executable file.

4. The neural network reasoning deployment method for a reconfigurable AI chip according to any one of claims 1 to 3, characterized in that: Step (3) includes the following sub-steps: (3-1) Obtain a reconfigurable AI chip operator library, and create a corresponding basic operator for each operator obtained in step (2) based on the reconfigurable AI chip operator library, and the basic operators corresponding to all operators constitute a basic operator library; (3-2) Set counter i=1; (3-3) Determine whether i is greater than the total number of all operators split from the network topology structure in step (2). If so, the process ends; otherwise, proceed to step (3-4); (3-4) According to step (2), the i-th operator among all the operators split from the network topology structure is judged whether the basic operator corresponding to the i-th operator can be obtained from the basic operator library obtained in step (3-1). If so, proceed to step (3-6); otherwise, proceed to step (3-5); (3-5) Select a basic operator with the same convolution kernel size and the same number of input channels as the i-th operator obtained in step (2) from the basic operator library obtained in step (3-1), and fill the part of the input feature map of the i-th operator that cannot be completely covered by the basic operator with 0; (3-6) dividing the input feature map corresponding to the i-th operator into blocks according to the basic operator corresponding to the i-th operator to obtain the division result of the i-th operator; (3-7) Set i=i+1 and return to step (3-3).

5. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 4 is characterized in that: Step (3-4) includes the following sub-steps: (3-4-1) Set counter j = 1; (3-4-2) Determine whether j is greater than the number of operators in the basic operator library. If so, the process ends; otherwise, proceed to step (3-4-3); (3-4-3) Determine whether the jth basic operator in the basic operator library has the same convolution kernel size as the i-th operator obtained in step (2), whether the number of input channels of the jth basic operator is the same as the i-th operator obtained in step (2), and whether the jth basic operator can completely cover the input feature map corresponding to the i-th operator obtained in step (2). If so, the process ends, otherwise proceed to (3-4-4); (3-4-4) Set j=j+1 and return to step (3-4-2).

6. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 5, characterized in that: Step (3-6) includes the following sub-steps: (3-6-1) According to the basic operator corresponding to the i-th operator, calculate the total number of blocks block_nums into which the input feature map corresponding to the i-th operator is divided; (3-6-2) Set counter k = 1; (3-6-3) Determine whether k is greater than the total number of blocks block_nums obtained in step (3-6-1). If so, assemble and splice all the instantiated basic operators obtained. Otherwise, proceed to step (3-6-4); (3-6-4) Calculate the data input address input_addr in the input feature graph corresponding to the i-th operator according to k, and calculate the output address output_addr of the operation result of the i-th operator according to k; (3-6-5) Write the input address obtained in step (3-6-4) to the corresponding position of the load instruction of the basic operator corresponding to the i-th operator, and write the output address of the obtained operation result to the corresponding position of the store instruction of the basic operator corresponding to the i-th operator, so as to obtain the instantiated basic operator; (3-6-6) Set k=k+1 and return to step (3-6-3).

7. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 6 is characterized in that: The total number of blocks block_nums is equal to: block_nums=inputFeatureMap_size÷kernelFeatureMap_size Among them, inputFeatureMap_size represents the size of the input feature map corresponding to the i-th operator, and kernelFeatureMap_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator.

8. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 6, characterized in that: The data input address input_addr is equal to: input_addr=(k÷block_nums)×(input_feaqtureMap_size×(operator_input_feature_map_size―operator_kernel_size+1))+k% block_nums)×(operator_input_feature_map_size―operator_kernel_size+1) Among them, input_featureMap_size represents the size of the input feature map corresponding to the i-th operator, operator_input_feature_map_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator, and operator_kernel_size represents the convolution kernel size of the i-th operator. The output address output_addr of the operation result is equal to: output_addr=(k÷(output_size÷operator_output_size))×(output_size×operator_output_size)+(k%(output_size÷operator_output_size))×operator_output_size Among them, output_size represents the size of the output feature map corresponding to the i-th operator, and operator_output_size represents the size of the output feature map corresponding to the basic operator corresponding to the i-th operator.

9. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 1, characterized in that: Step (5) is to write the task information of all operators obtained in step (4) into the executable file in order according to the network topology structure of the model obtained in step (2), and write the master control information obtained in step (2) into the corresponding position of the executable file, and finally obtain the executable file of the model trained in step (1).

10. A neural network reasoning deployment system for reconfigurable AI chips, characterized in that: include: The first module is used to obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange ONNX library corresponding to the deep learning framework to export the trained model into an ONNX format model; The second module is used to parse the ONNX format model obtained by the first module to obtain the network topology structure of the model, split the operator information of all operators from the network topology structure, and obtain the master control information in the executable file corresponding to the model based on the network topology structure and the operator information. The third module is used to perform operator division on each operator according to the operator information obtained by the second module to obtain the operator division result corresponding to the operator. The fourth module is used to assemble and splice the division results of all operators obtained by the third module to obtain the task information of all operators. The fifth module is used to organize the master control information obtained by the second module and the task information of all operators obtained by the fourth module into a final executable file.

Images