A neural network inference deployment method and system for a reconfigurable AI chip

Abstract

The application discloses a neural network inference deployment method for a reconfigurable AI chip, and comprises the following steps: using a corresponding ONNX library of a deep learning framework to export a trained model into an ONNX format, analyzing the ONNX format model to obtain a network topology structure, operator information and total control information, and dividing each operator according to the operator information to obtain a division result; combining the network topology structure to assemble and splice the operator division result to obtain task information and the like, and fully utilizing hardware resources of the reconfigurable AI chip; and finally organizing the total control information and task information of all operators into a final executable file, and efficiently completing inference deployment of the neural network model on the reconfigurable AI chip. The application can solve the technical problems of high development threshold and low efficiency of a current development specific AI chip architecture compiler, and the technical problem that a current automatic code generation method may not fully utilize chip performance.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and more specifically, relates to a neural network reasoning deployment method and system for reconfigurable AI chips. Background Art

[0002] In recent years, the rapid development of artificial intelligence has deeply penetrated numerous key sectors, including healthcare, transportation, and finance, becoming a core force driving industrial transformation. As the hardware cornerstone of AI computing, AI reconfigurable chips, with their unique architectural advantages, can flexibly configure hardware resources based on diverse algorithm requirements, demonstrating outstanding performance potential and becoming highly favored for handling complex and ever-changing AI tasks.

[0003] There are two main existing methods for deploying neural networks. The first is to develop a compiler for a specific AI chip architecture. After parsing the representation of the neural network, it generates corresponding machine instructions for the operators in the model based on the chip instruction set and hardware resource limitations. The second is to adopt an automatic code generation method, using high-level programming languages ​​​​(such as Python) and code generation templates to automatically generate code based on the structure of the neural network model and the hardware parameters of the AI ​​chip. These tools can generate efficient code based on the type and number of operators in the model, combined with the computing power and storage limitations of the chip.

[0004] However, both of the above-mentioned existing neural network deployment methods have some non-negligible flaws: First, developing a compiler for a specific AI chip architecture requires a high learning cost and development threshold for developers, and the development cycle is very long and inefficient; second, the automatic code generation method has very high requirements for preset templates and code generation rules. If the details of the AI ​​chip architecture are not clear enough, it is easy to fail to generate high-performance instructions that adapt to the chip architecture. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a neural network reasoning deployment method and system for reconfigurable AI chips, which aims to solve the technical problems of high development threshold and low efficiency of existing compilers for developing specific AI chip architectures, and the technical problem that existing automatic code generation methods may not fully utilize chip performance.

[0006] To achieve the above objectives, according to one aspect of the present invention, a neural network reasoning deployment method for a reconfigurable AI chip is provided, comprising the following steps:

[0007] (1) Obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model;

[0008] (2) Parse the ONNX format model obtained in step (1) to obtain the network topology of the model, separate the operator information of all operators (including three types of operators: convolution type operators, pooling type operators, and fully connected type operators) from the network topology, and obtain the master control information in the executable file corresponding to the model based on the network topology and operator information.

[0009] (3) Perform operator division on each operator according to the operator information obtained in step (2) to obtain the operator division result corresponding to the operator.

[0010] (4) Assemble and splice the partitioning results of all operators obtained in step (3) to obtain the task information of all operators.

[0011] (5) Organize the master control information obtained in step (2) and the task information of all operators obtained in step (4) into the final executable file.

[0012] Preferably, the deep learning framework is PyTorch or TensorFlow.

[0013] Operators include three types: convolution type operators, pooling type operators and fully connected type operators;

[0014] For the convolution type operator, the operator information of the convolution type operator includes the size of the input feature map corresponding to the convolution type operator, the size of the output feature map, the convolution kernel size, the number of input channels, the number of output channels, the input feature map address, the output feature map address and the weight data address.

[0015] For the pooling type operator, the operator information of the pooling type operator includes information such as the size of the input feature map corresponding to the pooling type operator, the size of the output feature map, the pooling window size, and the pooling type.

[0016] For a fully connected type operator, the operator information of the fully connected type operator includes information such as input dimension and output dimension.

[0017] Preferably, the process of obtaining the master control information in the executable file based on the network topology structure and operator information in step (2) is as follows: first, analyzing the connection relationship between each node in the network topology structure to obtain the data flow direction information of the network topology structure; then, based on the operator information, completing the data flow direction information, thereby obtaining the master control information in the executable file; the master control information is the operator outline of the executable file.

[0018] Preferably, step (3) includes the following sub-steps:

[0019] (3-1) Obtain a reconfigurable AI chip operator library, and create a corresponding basic operator for each operator obtained in step (2) based on the reconfigurable AI chip operator library. The basic operators corresponding to all operators constitute the basic operator library;

[0020] (3-2) Set counter i=1;

[0021] (3-3) Determine whether i is greater than the total number of all operators split from the network topology structure in step (2). If so, the process ends; otherwise, proceed to step (3-4);

[0022] (3-4) According to step (2), the i-th operator among all the operators split from the network topology structure is judged whether the basic operator corresponding to the i-th operator can be obtained from the basic operator library obtained in step (3-1). If so, the process proceeds to step (3-6); otherwise, the process proceeds to step (3-5);

[0023] (3-5) Select a basic operator with the same convolution kernel size and the same number of input channels as the i-th operator obtained in step (2) from the basic operator library obtained in step (3-1), and fill the part of the input feature map of the i-th operator that cannot be completely covered by the basic operator with 0;

[0024] (3-6) dividing the input feature map corresponding to the i-th operator into blocks according to the basic operator corresponding to the i-th operator to obtain the division result of the i-th operator;

[0025] (3-7) Set i=i+1 and return to step (3-3);

[0026] Preferably, step (3-4) includes the following sub-steps:

[0027] (3-4-1) Set counter j = 1;

[0028] (3-4-2) Determine whether j is greater than the number of operators in the basic operator library. If so, the process ends; otherwise, go to step (3-4-3);

[0029] (3-4-3) Determine whether the j-th basic operator in the basic operator library has the same convolution kernel size as the i-th operator obtained in step (2), whether the number of input channels of the j-th basic operator is the same as the i-th operator obtained in step (2), and whether the j-th basic operator can completely cover the input feature map corresponding to the i-th operator obtained in step (2). If so, the process ends, otherwise go to (3-4-4);

[0030] (3-4-4) Set j=j+1 and return to step (3-4-2).

[0031] Preferably, steps (3-6) include the following sub-steps:

[0032] (3-6-1) According to the basic operator corresponding to the i-th operator, calculate the total number of blocks block_nums for dividing the input feature map corresponding to the i-th operator;

[0033] (3-6-2) Set counter k = 1;

[0034] (3-6-3) Determine whether k is greater than the total number of blocks block_nums obtained in step (3-6-1). If so, assemble and splice all the instantiated basic operators obtained. Otherwise, go to step (3-6-4);

[0035] (3-6-4) Calculate the data input address input_addr in the input feature graph corresponding to the i-th operator based on k, and calculate the output address output_addr of the operation result of the i-th operator based on k;

[0036] (3-6-5) Write the input address obtained in step (3-6-4) to the corresponding position of the load instruction of the basic operator corresponding to the i-th operator, and write the output address of the obtained operation result to the corresponding position of the store instruction of the basic operator corresponding to the i-th operator to obtain the instantiated basic operator;

[0037] (3-6-6) Set k=k+1 and return to step (3-6-3).

[0038] Preferably, the total number of blocks block_nums is equal to:

[0039] block_nums=inputFeatureMap_size÷kernelFeatureMap_size

[0040] Among them, inputFeatureMap_size represents the size of the input feature map corresponding to the i-th operator, and kernelFeatureMap_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator.

[0041] Preferably, the data input address input_addr is equal to:

[0042] input_addr=(k÷block_nums)×(input_featureMap_size×(operator_input_feature_map_size-operator_kernel_size+1))+k%block_nums)×(operator_input_feature_map_size-operator_kernel_size+1)

[0043] Where k represents the kth block, block_nums represents the total number of blocks obtained in step (3-6-1), input_featureMap_size represents the size of the input feature map corresponding to the i-th operator, operatorinput_feature_map_size represents the size of the input feature map corresponding to the base operator corresponding to the i-th operator, and operator_kernel_size represents the convolution kernel size of the i-th operator.

[0044] The output address output_addr of the operation result is equal to:

[0045] output_addr=(k÷(output_size÷operator_output_size))×(output_size×operator_output_size)+(k%(output_size÷operator_output_size))×operator_output_size

[0046] Among them, output_size represents the size of the output feature map corresponding to the i-th operator, and operatoroutput_size represents the size of the output feature map corresponding to the basic operator corresponding to the i-th operator.

[0047] Preferably, step (5) is to write the task information of all operators obtained in step (4) into the executable file in sequence according to the network topology structure of the model obtained in step (2), and at the same time write the master control information obtained in step (2) into the corresponding position of the executable file, and finally obtain the executable file of the model trained in step (1).

[0048] According to another aspect of the present invention, a neural network reasoning deployment system for a reconfigurable AI chip is provided, comprising:

[0049] The first module is used to obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model;

[0050] The second module is used to parse the ONNX format model obtained by the first module to obtain the network topology structure of the model, split the operator information of all operators from the network topology structure, and obtain the master control information in the executable file corresponding to the model based on the network topology structure and operator information.

[0051] The third module is used to perform operator division on each operator according to the operator information obtained by the second module to obtain the operator division result corresponding to the operator.

[0052] The fourth module is used to assemble and splice the division results of all operators obtained by the third module to obtain the task information of all operators.

[0053] The fifth module is used to organize the master control information obtained by the second module and the task information of all operators obtained by the fourth module into a final executable file.

[0054] In general, the technical method conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0055] (1) Since the present invention adopts steps (2) to (4), it can allow developers to quickly understand the model network structure by efficiently parsing the ONNX format model and call a complete basic operator library to perform operator division, thereby solving the technical problems of high threshold and low efficiency in the existing development of compilers for specific AI chip architectures;

[0056] (2) Since the present invention adopts step (3), based on the characteristics of the AI ​​chip architecture, it compiles a series of basic neural network operators (convolution, pooling, and full connection) that fully utilize the chip hardware characteristics, forming a complete basic operator library. The operator library is used to perform operator division, thereby solving the technical problem that the existing automatic code generation method may not fully utilize the chip performance;

[0057] (3) Due to step (1), the present invention exports the network model algorithm into the ONNX format. Since many deep learning frameworks support converting their own models into the ONNX format, this makes the tool chain have a broad compatibility basis for cross-hardware deployment scenarios;

[0058] (4) Due to step (3), the present invention can reasonably divide convolution, pooling and full connection of different sizes through an efficient partitioning algorithm for the three basic operator types, making the entire tool chain more versatile and flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flowchart of the neural network reasoning deployment method for a reconfigurable AI chip of the present invention;

[0060] Figure 2 This is a flow chart of the convolution operator partitioning in step (3) of the method of the present invention. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0062] The basic idea of ​​the present invention is to provide a neural network inference deployment method for reconfigurable AI chips based on the characteristics of reconfigurable AI chips. The trained model is exported to the ONNX format by using the ONNX library corresponding to the deep learning framework, and the ONNX format model is parsed to obtain the network topology structure, operator information and master control information. Each operator is divided according to the operator information to obtain the division result; the operator division results are assembled and spliced ​​in combination with the network topology structure to obtain task information and other steps, so as to make full use of the hardware resources of the reconfigurable AI chip; finally, the master control information and the task information of all operators are organized into a final executable file, and the inference deployment of the neural network model on the reconfigurable AI chip is efficiently completed.

[0063] like Figure 1 As shown, the present invention provides a neural network reasoning deployment method for a reconfigurable AI chip, comprising the following steps:

[0064] (1) Obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model;

[0065] Specifically, the deep learning framework in this step is PyTorch or TensorFlow.

[0066] (2) Parse the ONNX format model obtained in step (1) to obtain the network topology of the model, separate the operator information of all operators (including three types of operators: convolution type operators, pooling type operators, and fully connected type operators) from the network topology, and obtain the master control information in the executable file corresponding to the model based on the network topology and operator information.

[0067] Specifically, operators include three types: convolution type operators, pooling type operators and fully connected type operators.

[0068] For the convolution type operator, the operator information of the convolution type operator includes the size of the input feature map corresponding to the convolution type operator, the size of the output feature map, the convolution kernel size, the number of input channels, the number of output channels, the input feature map address, the output feature map address and the weight data address.

[0069] For the pooling type operator, the operator information of the pooling type operator includes information such as the size of the input feature map corresponding to the pooling type operator, the size of the output feature map, the pooling window size, and the pooling type.

[0070] For a fully connected type operator, the operator information of the fully connected type operator includes information such as input dimension and output dimension.

[0071] In this step, the process of obtaining the master control information in the executable file based on the network topology and operator information is as follows: first, the connection relationship between each node in the network topology is analyzed to obtain the data flow direction information of the network topology structure. Then, based on the operator information, the data flow direction information is completed to obtain the master control information in the executable file; this master control information is the operator outline of the entire executable file.

[0072] The advantage of the above steps (1) to (2) is that developers can use the deep learning framework they are familiar with to build and train the network. Secondly, the parsing of ONNX format models makes the tool chain highly scalable and flexible.

[0073] (3) Perform operator division on each operator according to the operator information obtained in step (2) to obtain the operator division result corresponding to the operator.

[0074] like Figure 2 As shown, this step includes the following sub-steps:

[0075] (3-1) Obtain a reconfigurable AI chip operator library, and create a corresponding basic operator for each operator obtained in step (2) based on the reconfigurable AI chip operator library. The basic operators corresponding to all operators constitute the basic operator library;

[0076] (3-2) Set counter i=1;

[0077] (3-3) Determine whether i is greater than the total number of all operators split from the network topology structure in step (2). If so, the process ends; otherwise, proceed to step (3-4);

[0078] (3-4) According to step (2), the i-th operator among all the operators split from the network topology structure is judged whether the basic operator corresponding to the i-th operator can be obtained from the basic operator library obtained in step (3-1). If so, the process proceeds to step (3-6); otherwise, the process proceeds to step (3-5);

[0079] Specifically, step (3-4) includes the following sub-steps:

[0080] (3-4-1) Set counter j = 1;

[0081] (3-4-2) Determine whether j is greater than the number of operators in the basic operator library. If so, the process ends; otherwise, go to step (3-4-3);

[0082] (3-4-3) Determine whether the j-th basic operator in the basic operator library has the same convolution kernel size as the i-th operator obtained in step (2), whether the number of input channels of the j-th basic operator is the same as the i-th operator obtained in step (2), and whether the j-th basic operator can completely cover the input feature map corresponding to the i-th operator obtained in step (2). If so, the process ends, otherwise go to (3-4-4);

[0083] (3-4-4) Set j = j + 1 and return to step (3-4-2);

[0084] (3-5) Select a basic operator with the same convolution kernel size and the same number of input channels as the i-th operator obtained in step (2) from the basic operator library obtained in step (3-1), and fill the part of the input feature map of the i-th operator that cannot be completely covered by the basic operator with 0;

[0085] The advantage of this sub-step (3-5) is that it enables the system to adapt to operators of larger sizes and improves the scalability of the system;

[0086] (3-6) dividing the input feature map corresponding to the i-th operator into blocks according to the basic operator corresponding to the i-th operator to obtain the division result of the i-th operator;

[0087] Specifically, steps (3-6) include the following sub-steps:

[0088] (3-6-1) According to the basic operator corresponding to the i-th operator, calculate the total number of blocks block_nums for dividing the input feature map corresponding to the i-th operator;

[0089] Specifically, the total number of blocks, block_nums, is equal to:

[0090] block_nums=inputFeatureMap_size÷kernelFeatureMap_size

[0091] Among them, inputFeatureMap_size represents the size of the input feature map corresponding to the i-th operator, and kernelFeatureMap_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator;

[0092] (3-6-2) Set counter k = 1;

[0093] (3-6-3) Determine whether k is greater than the total number of blocks block_nums obtained in step (3-6-1). If so, assemble and splice all the instantiated basic operators obtained. Otherwise, go to step (3-6-4);

[0094] (3-6-4) Calculate the data input address input_addr in the input feature graph corresponding to the i-th operator based on k, and calculate the output address output_addr of the operation result of the i-th operator based on k;

[0095] Specifically, the data input address input_addr is equal to:

[0096] input_addr=(k÷block_nums)×(input_featureMap_size×(operator_input_feature_map_size-operator_kernel_size+1))+k%block_nums)×(operator_input_feature_map_size-operator_kernel_size+1)

[0097] Where k∈[1, total number of blocks block_nums], input_featureMap_size represents the size of the input feature map corresponding to the i-th operator, operator_input_feature_map_size represents the size of the input feature map corresponding to the base operator corresponding to the i-th operator, and operator_kernel_size represents the convolution kernel size of the i-th operator;

[0098] The output address output_addr of the operation result is equal to:

[0099] output_addr=(k÷(output_size÷operator_output_size))×(output_size×operator_output_size)+(k%(output_size÷operator_output_size))×operator_output_size

[0100] Among them, output_size represents the size of the output feature map corresponding to the i-th operator, and operator_output_size represents the size of the output feature map corresponding to the basic operator corresponding to the i-th operator;

[0101] (3-6-5) Write the input address obtained in step (3-6-4) to the corresponding position of the load instruction of the basic operator corresponding to the i-th operator, and write the output address of the obtained operation result to the corresponding position of the store instruction of the basic operator corresponding to the i-th operator to obtain the instantiated basic operator;

[0102] (3-6-6) Set k = k + 1 and return to step (3-6-3);

[0103] (3-7) Set i=i+1 and return to step (3-3);

[0104] (4) Assemble and splice the partitioning results of all operators obtained in step (3) to obtain the task information of all operators.

[0105] Specifically, this step is to splice the results of operator division according to the results of operator division in step (3) to obtain the task information of all operators.

[0106] (5) Organize the master control information obtained in step (2) and the task information of all operators obtained in step (4) into the final executable file.

[0107] Specifically, this step writes the task information of all operators obtained in step (4) into the executable file in sequence according to the network topology structure of the model obtained in step (2), and writes the master control information obtained in step (2) into the corresponding position of the executable file, and finally obtains the executable file of the model trained in step (1).

[0108] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A neural network reasoning deployment method for reconfigurable AI chips, characterized in that: The following steps are involved: (1) Obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model; (2) Parsing the ONNX format model obtained in step (1) to obtain the network topology of the model, splitting the operator information of all operators from the network topology, and obtaining the master control information in the executable file corresponding to the model according to the network topology and operator information. The operators include three types: convolution type operators, pooling type operators and fully connected type operators; (3) Performing operator division on each operator according to the operator information obtained in step (2) to obtain an operator division result corresponding to the operator; step (3) includes the following sub-steps: (3-1) Obtain a reconfigurable AI chip operator library, and create a corresponding basic operator for each operator obtained in step (2) based on the reconfigurable AI chip operator library. The basic operators corresponding to all operators constitute the basic operator library; (3-2) Set counter i=1; (3-3) Determine whether i is greater than the total number of all operators split from the network topology structure in step (2). If so, the process ends; otherwise, proceed to step (3-4); (3-4) According to step (2), the i-th operator among all the operators split from the network topology structure is judged whether the basic operator corresponding to the i-th operator can be obtained from the basic operator library obtained in step (3-1). If so, the process proceeds to step (3-6); otherwise, the process proceeds to step (3-5); (3-5) Select a basic operator with the same convolution kernel size and the same number of input channels as the i-th operator obtained in step (2) from the basic operator library obtained in step (3-1), and fill the part of the input feature map of the i-th operator that cannot be completely covered by the basic operator with 0; (3-6) dividing the input feature map corresponding to the i-th operator into blocks according to the basic operator corresponding to the i-th operator to obtain the division result of the i-th operator; (3-7) Set i=i+1 and return to step (3-3); (4) Assemble and splice the partitioning results of all operators obtained in step (3) to obtain the task information of all operators; (5) Organize the master control information obtained in step (2) and the task information of all operators obtained in step (4) into the final executable file.

2. The neural network reasoning deployment method for reconfigurable AI chips according to claim 1 is characterized in that: The deep learning framework is PyTorch or TensorFlow; Operators include three types: convolution type operators, pooling type operators and fully connected type operators; For convolution type operators, the operator information of the convolution type operator includes the size of the input feature map corresponding to the convolution type operator, the size of the output feature map, the convolution kernel size, the number of input channels, the number of output channels, the input feature map address, the output feature map address and the weight data address information; For the pooling type operator, the operator information of the pooling type operator includes the size of the input feature map corresponding to the pooling type operator, the size of the output feature map, the pooling window size, and the pooling type information; For a fully connected type operator, the operator information of the fully connected type operator includes input dimension information and output dimension information.

3. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 1 or 2, characterized in that: The process of obtaining the master control information in the executable file based on the network topology and operator information in step (2) is as follows: first, the connection relationship between each node in the network topology is analyzed to obtain the data flow direction information of the network topology; then, based on the operator information, the data flow direction information is completed to obtain the master control information in the executable file; the master control information is the operator outline of the executable file.

4. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 3 is characterized in that: Step (3-4) includes the following sub-steps: (3-4-1) Set counter j = 1; (3-4-2) Determine whether j is greater than the number of operators in the basic operator library. If so, the process ends; otherwise, go to step (3-4-3); (3-4-3) Determine whether the j-th basic operator in the basic operator library has the same convolution kernel size as the i-th operator obtained in step (2), whether the number of input channels of the j-th basic operator is the same as the i-th operator obtained in step (2), and whether the j-th basic operator can completely cover the input feature map corresponding to the i-th operator obtained in step (2). If so, the process ends, otherwise go to (3-4-4); (3-4-4) Set j=j+1 and return to step (3-4-2).

5. The neural network reasoning deployment method for reconfigurable AI chips according to claim 4 is characterized in that: Steps (3-6) include the following sub-steps: (3-6-1) According to the basic operator corresponding to the i-th operator, calculate the total number of blocks block_nums for dividing the input feature map corresponding to the i-th operator; (3-6-2) Set counter k = 1; (3-6-3) Determine whether k is greater than the total number of blocks block_nums obtained in step (3-6-1). If so, assemble and splice all the instantiated basic operators obtained. Otherwise, go to step (3-6-4); (3-6-4) Calculate the data input address input_addr in the input feature graph corresponding to the i-th operator based on k, and calculate the output address output_addr of the operation result of the i-th operator based on k; (3-6-5) Write the input address obtained in step (3-6-4) to the corresponding position of the load instruction of the basic operator corresponding to the i-th operator, and write the output address of the obtained operation result to the corresponding position of the store instruction of the basic operator corresponding to the i-th operator to obtain the instantiated basic operator; (3-6-6) Set k=k+1 and return to step (3-6-3).

6. The neural network reasoning deployment method for reconfigurable AI chips according to claim 5, characterized in that: The total number of blocks block_nums is equal to: block_nums=inputFeatureMap_size÷kernelFeatureMap_size Among them, inputFeatureMap_size represents the size of the input feature map corresponding to the i-th operator, and kernelFeatureMap_size represents the size of the input feature map corresponding to the basic operator corresponding to the i-th operator.

7. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 6, characterized in that: The data input address input_addr is equal to: input_addr=(k÷block_nums)×( input_featureMap_size×(operator_input_feature_map_siz -operator_kernel_size+1)) +k%block_nums)×(operator_input_feature_map_size -operator_kernel_size+1) Where input_featureMap_size represents the size of the input feature map corresponding to the i-th operator, operator_input_feature_map_size represents the size of the input feature map corresponding to the base operator corresponding to the i-th operator, and operator_kernel_size represents the convolution kernel size of the i-th operator; The output address output_addr of the operation result is equal to: output_addr =(k÷(output_size ÷operator_output_size))×(output_size ×operator_output_size)+(k%(output_size ÷operator_output_size))×operator_output_size Among them, output_size represents the size of the output feature map corresponding to the i-th operator, and operator_output_size represents the size of the output feature map corresponding to the basic operator corresponding to the i-th operator.

8. The neural network reasoning deployment method for a reconfigurable AI chip according to claim 1, characterized in that: Step (5) is to write the task information of all operators obtained in step (4) into the executable file in sequence according to the network topology structure of the model obtained in step (2), and at the same time write the master control information obtained in step (2) into the corresponding position of the executable file, and finally obtain the executable file of the model trained in step (1).

9. A system using the neural network reasoning deployment method for reconfigurable AI chips according to claim 1, characterized in that: include: The first module is used to obtain the trained model and its corresponding executable file, and use the Open Neural Network Exchange (ONNX) library corresponding to the deep learning framework to export the trained model into an ONNX format model; The second module is used to parse the ONNX format model obtained by the first module to obtain the network topology of the model, split the operator information of all operators from the network topology, and obtain the master control information in the executable file corresponding to the model based on the network topology and operator information; The third module is used to perform operator division on each operator according to the operator information obtained by the second module to obtain the operator division result corresponding to the operator; The fourth module is used to assemble and splice the partitioning results of all operators obtained in the third module to obtain the task information of all operators; The fifth module is used to organize the master control information obtained by the second module and the task information of all operators obtained by the fourth module into a final executable file.

Images