A firmware runtime system for a neural network processor

Abstract

The present application belongs to the technical field of embedded processor firmware, and particularly relates to an efficient firmware running system for a neural network processor, aiming to solve the problems of slow firmware deployment speed, large occupied space and low processing efficiency of files generated after compilation by a neural network compiler of the existing neural network processor. The system comprises: a file preprocessing unit configured to obtain files generated after compilation by a neural network compiler and preprocess the files to obtain preprocessed files; an embedded hardware platform configured to store the preprocessed files to a set memory address; and an efficient firmware configured to read and parse the preprocessed files, obtain operator stream files corresponding to a neural network model, and sequentially execute each operator through a pre-built operator execution efficient optimization method until a complete calculation of the neural network model is completed. The present application improves the processing efficiency of files generated after compilation by a neural network compiler, improves the firmware deployment speed, and reduces the occupied space.

Description

Technical Field

[0001] This invention belongs to the field of embedded processor firmware technology, specifically relating to a firmware runtime system for neural network processors. Background Technology

[0002] In recent years, with the significant improvement in computing power, neural network technology has become the foundation for both academic research and industrial practice in artificial intelligence. It has been widely applied in fields such as computer vision, speech processing, natural language processing, and pattern recognition, achieving remarkable results. To address the high computational and data volume inherent in neural networks, neural network processors have undergone specific acceleration optimizations for their structure, algorithms, and operators. Compared to computations performed on general-purpose CPUs and GPUs, neural network processors offer advantages such as faster processing and lower power consumption. Consequently, in recent years, the miniaturization and integration of neural network processors have become increasingly mature, enabling the rapid deployment and application of neural networks on small terminal devices, playing a crucial role in daily life and industrial production.

[0003] The hardware design of a neural network processor determines its theoretical performance ceiling. Correspondingly, the firmware of the neural network processor directly determines the extent to which the processor can perform for upper-layer applications. Poor firmware performance not only results in slow deployment speed and large space consumption, but also low file processing efficiency. Therefore, it is crucial to implement a neural network processor firmware (i.e., file processing firmware compiled by the neural network compiler) that is specific to a particular neural network processor and compatible with its hardware platform. The technical challenges lie in several aspects. First, a clear understanding of the neural network processor itself is essential: its architecture, supported operators, operator execution flow, register configuration, etc. Second, a clear understanding of the hardware platform on which the neural network processor resides, especially the general-purpose CPU: its architecture, instruction set, interrupt handling, and ensuring the correct execution of the written firmware on the CPU. Third, a clear understanding of other hardware devices involved in the neural network processing process is necessary: ​​memory, DMA, timers, etc. Finally, a comprehensive understanding of the entire process, from upstream neural network applications to downstream neural network processor processing, is required: the use of typical neural network frameworks, generating and training neural networks, neural network compilers for this processor, compilation and conversion of neural network models and file preprocessing, the computation process on the neural network processor, and post-processing after computation.

[0004] Based on addressing the above-mentioned challenges, this application develops a firmware processing system tailored to the unique architecture of neural network processors and their respective hardware platforms. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, namely the slow firmware deployment speed, large space consumption, and low processing efficiency of files generated by neural network compilers in existing neural network processors, the first aspect of this invention proposes a firmware runtime system for neural network processors, used for processing files generated by neural network compilers. This system includes:

[0006] The file preprocessing unit is configured to, upon detecting that the neural network compiler has completed compilation, obtain the file generated by the neural network compiler after compiling the input neural network model, and use it as the input file; preprocess the input file to obtain a preprocessed file, and send it to the embedded hardware platform; the preprocessing includes file merging, file conversion, and file header design;

[0007] The embedded hardware platform is configured to store the received preprocessed file to a set memory address; it is also configured to load the compiled firmware to a set program entry address for startup and execution.

[0008] The firmware is configured to establish a connection with the embedded hardware platform and confirm whether each module of the embedded hardware platform is running normally. If it is running normally, the hardware devices are initialized. The initialized hardware devices include serial port transmission devices, neural network processors, interrupt devices, DMA, and timers.

[0009] The firmware is further configured to read and parse the preprocessed file stored on the embedded hardware platform, obtain the operator stream file corresponding to the neural network model, parse and initialize the neural network variables, and sequentially execute each operator in the operator stream file through a pre-built operator execution optimization method, thereby obtaining the output files of each intermediate layer of the neural network model and the final output file; the neural network variables include the number of data node operators, the list of data node operators, the number of data input nodes and the number of data output nodes.

[0010] In some preferred embodiments, the input file is preprocessed to obtain a preprocessed file, and the method is as follows:

[0011] The four files—the neural network processor operation instruction file, the neural network model operator stream file, the neural network model weight file, and the neural network model output configuration file—are merged to obtain the neural network model merged file.

[0012] The input file of the neural network model generated by the neural network compiler is converted into a neural network input file; the input file of the neural network model includes images and videos.

[0013] Select the necessary intermediate layer correct output files from all the correct output files of the neural network generated by the neural network compiler, and merge them into a single correct intermediate layer output file.

[0014] Select all the correct output files of the neural networks generated by the neural network compiler and merge them into a single final correct output file.

[0015] In some preferred embodiments, the neural network model merge file has a header of 32 bytes, with each 8 bytes describing a file. The first 4 bytes of each 8 bytes represent the bias of the file in the neural network model merge file, and the last 4 bytes represent the size of the file.

[0016] For each file in the merged neural network model file, its starting address is aligned to 16 bytes, and if its ending address cannot be aligned to 16 bytes, it is padded with corresponding 0 bytes to make it aligned.

[0017] In some preferred embodiments, the merged correct output file of the neural network intermediate layer has a header of 4 bytes indicating the number of correct output files of the neural network intermediate layer, followed by 8 bytes describing a selected correct output file of the neural network intermediate layer. The first 4 bytes of each 8 bytes are the data node number corresponding to the selected correct output file of the neural network intermediate layer, and the last 4 bytes are the offset of the selected correct output file of the neural network intermediate layer in the merged correct output file of the neural network intermediate layer. Since the end of the header cannot be aligned to 16 bytes, corresponding 0 bytes are added to the end of the header for alignment.

[0018] The starting address of each selected intermediate layer of the neural network is guaranteed to be aligned to 16 bytes. If the ending address cannot be aligned to 16 bytes, it is padded with corresponding 0 bytes to make it aligned.

[0019] In some preferred embodiments, the neural network ultimately outputs a merged file, the first 4 bytes of which indicate the number of final correct output files, followed by 8 bytes describing one final correct output file. The first 4 bytes of each 8 bytes represent the offset of this final correct output file in the neural network's final correct output merged file, and the last 4 bytes represent the size of this file. The end of the file header cannot be aligned to 16 bytes, so corresponding 0 bytes are added to the end of the file header for alignment.

[0020] Each final correct output file is guaranteed to start at a 16-byte aligned address. If the end of the file is not 16-byte aligned, it is padded with corresponding 0 bytes to make it aligned.

[0021] In some preferred embodiments, the operators in the operator stream file include instruction transfer operators, weight transfer operators, data input operators, data output operators, data concatenation operators, data preprocessing operators, data migration operators, lookup table transfer operators, data normalization operators, neural network processing operators, and data node operators.

[0022] In some preferred embodiments, the optimization method is implemented by sequentially executing each operator using pre-constructed operators, as follows:

[0023] The instruction transfer operator uses DMA to transfer some or all of the neural network processor operation instructions from the neural network processor operation instruction file in memory to the operation instruction space of the neural network processor according to the operator parameter settings.

[0024] The weight transfer operator uses DMA to transfer all weights from the weight file of the neural network model in memory to the weight FIFO space of the neural network processor at high speed. Since the size of each DMA transfer is limited, all weights are first divided into blocks of 16MB each, and the last block less than 16MB is recorded. During the last DMA, the interrupt operation of the embedded hardware platform is used to trigger an interrupt when each 16MB transfer is completed, and the transfer size is configured to be either the next block of 16MB or the last block less than 16MB, so as to realize the function of automatically and asynchronously transferring weights of any size in the background.

[0025] The data input operator and the data output operator transfer the input and output data of the neural network model into or out of the data space of the neural network processor using a multi-channel DMA transfer method. During the current DMA channel transfer, the next DMA transfer operation is configured for the next DMA channel, and the corresponding registers are configured for the next DMA transfer operation using the DMA transfer interval. Therefore, when the current DMA transfer time is greater than the next DMA configuration register time, it is equivalent to eliminating the next DMA transfer configuration register time. The final total transfer time is equal to the sum of the total DMA transfer time and the first DMA transfer configuration register time. Conversely, the total transfer time is equal to the sum of the total DMA transfer configuration register time and the last DMA transfer time, thus saving data transfer time.

[0026] The data splicing operator is optimized into the data input and data output operators through neural network compiler optimization, so no specific operation is required.

[0027] The data preprocessing operator uses DMA to quickly transfer the preprocessed weight data from the weight file of the neural network model in memory to the preprocessed weight space of the neural network processor, according to a portion of the preprocessed weights set by the operator parameters.

[0028] The data transfer operator is optimized into the data input and data output operators through neural network compiler optimization, so no specific operation is required.

[0029] The lookup table transfer operator uses DMA to transfer a portion of the lookup table data from the weight file of the neural network model in memory to the lookup table data space of the neural network processor at high speed, according to the operator parameter settings.

[0030] The data uniformization operator uses the CPU to perform data uniformization calculations; the data includes bytes, two-byte integers, four-byte integers, and floating-point numbers.

[0031] The neural network processing operator, through the neural network processor, calculates the currently prepared instructions and data, and triggers an interrupt as a sign of processing completion;

[0032] The data node operator refers to the data node information that needs to be referenced during the processing of the neural network model, and therefore no specific operation is performed on it.

[0033] In some preferred embodiments, the firmware runtime system further includes an intermediate layer output comparison unit;

[0034] The intermediate layer output comparison unit is configured to compare the intermediate layer output of the neural network model with the correct intermediate layer output during the calculation of a complete neural network model, after the data input operator is executed or before the data output operator is executed. If the comparison is correct, it means that all operations up to the current data input or output operator have been executed correctly.

[0035] In some preferred embodiments, the firmware runtime system further includes a final output comparison unit;

[0036] The final output comparison unit is configured to compare the final output of the neural network model with the final correct output after one or more complete calculations of the neural network model. If the comparison is correct, it means that the neural network model has run completely and correctly.

[0037] The beneficial effects of this invention are:

[0038] This invention improves the efficiency of processing files generated by neural network compilers, increases the firmware deployment and execution speed of neural network processors, reduces the operational complexity of deploying model files, and reduces the space occupied.

[0039] 1) In this application, the preprocessing script set is in the many files generated by the neural network compiler. Through selection, merging, and specific preprocessing operations, the subsequent transmission and loading of the model are simplified. At the same time, the complexity of the firmware program is reduced, making the entire neural network operation process easy to operate without having to pay attention to too many details.

[0040] 2) In this application, the preprocessing script set includes a set of model file generation scripts, a set of correct neural network output merging scripts, and a set of correct neural network intermediate layer output merging scripts, all of which are Python scripts. They can run across different operating system platforms, and only require placing one or more model files compiled by a neural network compiler in a specific folder and executing the scripts to quickly complete the preprocessing process, generating the necessary neural network model merging files, neural network input files, and optional correct neural network output merging files and correct neural network intermediate layer output merging files for debugging. It is simple and fast.

[0041] 3) The file header and file structure design of the merged file of the neural network model, the final correct output merged file of the neural network, and the correct output merged file of the intermediate layer of the neural network in this application have the advantages of being simple and clear, easy to parse, and saving space.

[0042] 4) In this application, during the model file parsing process, the fixed values ​​required for the entire neural network are also pre-calculated, so that the neural network does not need to be recalculated when it is executed multiple times, thus accelerating the computation process of the neural network.

[0043] 5) In this application, based on the characteristics of the hardware platform, a unique optimization method is implemented for each operator. These methods are the core technology and guarantee for the operation of the neural network processor firmware.

[0044] 6) In this application, the intermediate layer output comparison method and the final output comparison method are optional debugging tools. During firmware compilation, the parameters can be adjusted to conveniently select whether to perform comparison and debugging. If it has been confirmed that the neural network is running without errors, the debugging tool can be disabled during compilation to skip the comparison.

[0045] 7) In this application, the firmware of the neural network processor running the system using this firmware is compiled into a program of only about 50KB, which has the advantages of small space occupation, fast deployment and high running efficiency. Attached Figure Description

[0046] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0047] Figure 1This is a schematic diagram of the framework of a firmware running system for a neural network processor according to an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the file structure of a neural network model merging file according to an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the file structure of a merged file correctly output by the intermediate layer of a neural network according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the execution method of the weight transfer operator according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of a data input and output operator execution method using a multi-channel DMA transfer method according to an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0053] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0054] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0055] The first embodiment of the present invention provides a firmware running system for a neural network processor, used for processing files generated after compilation by a neural network compiler. The system includes: a file preprocessing unit, configured to, upon detecting that the neural network compiler has completed compilation, obtain the file generated by the neural network compiler after compiling the input neural network model, as an input file; preprocess the input file to obtain a preprocessed file, and send it to an embedded hardware platform; the preprocessing includes file merging, file conversion, and file header design;

[0056] The embedded hardware platform is configured to store the received preprocessed file to a set memory address; it is also configured to load the compiled firmware to a set program entry address for startup and execution.

[0057] The firmware is configured to establish a connection with the embedded hardware platform and confirm whether each module of the embedded hardware platform is running normally. If it is running normally, the hardware devices are initialized. The initialized hardware devices include serial port transmission devices, neural network processors, interrupt devices, DMA, and timers.

[0058] The firmware is further configured to read and parse the preprocessed file stored on the embedded hardware platform, obtain the operator stream file corresponding to the neural network model, parse and initialize the neural network variables, and sequentially execute each operator in the operator stream file through a pre-built operator execution optimization method, thereby obtaining the output files of each intermediate layer of the neural network model and the final output file; the neural network variables include the number of data node operators, the list of data node operators, the number of data input nodes and the number of data output nodes.

[0059] To more clearly illustrate the firmware runtime system for neural network processors of the present invention, the following description is in conjunction with the accompanying drawings. Figure 1 The modules of one embodiment of the system of the present invention will be described in detail below.

[0060] This invention proposes a firmware execution method for specific neural network processors and their associated hardware platforms. By implementing a preprocessing method, it integrates files generated by the neural network compiler, simplifying the process of executing neural network applications on specific neural network processors. Furthermore, by implementing firmware and operator optimization methods, it effectively utilizes hardware resources, fully leverages hardware performance, and achieves low-power, high-performance, and high-speed operation of various mainstream neural networks. Specifically:

[0061] The file preprocessing unit is configured to, upon detecting that the neural network compiler has completed compilation, obtain the file generated by the neural network compiler after compiling the input neural network model, and use it as the input file; preprocess the input file to obtain a preprocessed file, and send it to the embedded hardware platform; the preprocessing includes file merging, file conversion, and file header design;

[0062] In this embodiment, the script selects and merges the necessary neural network model merge file and neural network input file from numerous files related to a certain neural network model generated by the neural network compiler. It can also generate, as needed, a final correct output merge file and a correct output merge file for the intermediate layers of the neural network for debugging. That is, in this invention, the script includes a model file set generation script, a correct output merging script for the intermediate layers of the neural network, and a final correct output merging script for the neural network. Figure 1 As shown, all scripts are Python scripts. After preparing the relevant files for the neural network model generated by the compiler, you can execute the scripts using Python.

[0063] The model file set generation script identifies and selects four files from the files generated by the neural network compiler: the neural network processor operation instruction file (machine code that the neural network processor can process, controlling the operation of the neural network processor), the neural network model operator stream file (which the firmware can parse and execute, expressing the compiled neural network operator stream, controlling the execution of neural network operators), the neural network model weight file (the weights required when executing the neural network), and the neural network model output configuration file (which determines the number of output files, the size of the output files, and the memory address of the output files after the neural network execution is completed). These four files are then merged into a single firmware-supported neural network model merge file, which is a necessary file for neural network computation. In other words, the model file set generation script includes methods for identifying the four files (neural network processor operation instruction file, neural network model operator stream file, neural network model weight file, and neural network model output configuration file) from the neural network model files generated by the neural network compiler; file structure design when merging neural network model files; and methods for determining the endianness of neural network input files. The method for determining the four required files is comprehensively determined by the endianness settings of the hardware platform. Regarding the file structure design of the neural network model merge file, such as... Figure 2 As shown, after confirming the existence of the required 4 files, the file header is designed to be 32 bytes, with each 8 bytes describing one file: the first 4 bytes of each 8 bytes are the offset of this file in the entire merged file (i.e., the neural network model merged file), and the last 4 bytes are the size of this file; for each file, its starting address is guaranteed to be aligned to 16 bytes, and if its ending address cannot be aligned to 16 bytes, it is padded with corresponding 0 bytes to align; the endianness determination method for the neural network input file is determined by the endianness settings of the hardware platform.

[0064] The Neural Network Intermediate Layer Correct Output Merging Script selects the necessary Neural Network Intermediate Layer Correct Output Files from all the Neural Network Correct Output Files generated by the Neural Network Compiler and merges them into a single Neural Network Intermediate Layer Correct Output Merging File. This is an optional file for debugging Neural Network calculations and can be used to compare intermediate layer results during Neural Network calculations in the firmware. This refers to the script for merging the correct outputs of neural network intermediate layers. It includes methods for selecting the required correct output files for intermediate layers from all correct output files generated by the neural network compiler, and methods for merging these correct output files. The method for selecting the required correct output files is determined by reading the mapping file generated by the neural network compiler. Regarding the merged correct output files, the first 4 bytes of the file header indicate the number of correct output files for each intermediate layer. Each subsequent 8 bytes describe one correct output file: the first 4 bytes of each 8-byte header are the data node number corresponding to that file, and the last 4 bytes are the offset of that file within the entire merged file. The end of the file header cannot be 16-byte aligned, so corresponding 0 bytes are added to the end for alignment. The starting address of each correct output file is guaranteed to be 16-byte aligned; if its end cannot be 16-byte aligned, corresponding 0 bytes are added for alignment. Figure 3 As shown.

[0065] The final correct output merge script for the neural network is selected from all the correct output files generated by the neural network compiler and merged into a single final correct output file. This file is an optional file used for debugging neural network calculations and can be used to compare the final output results after the firmware has completed the neural network calculations. The aforementioned neural network final correct output merging script includes methods for selecting the required final correct output file from all correct output files generated by the neural network compiler, and methods for merging the final correct output files. Regarding the method for selecting the required final correct output file, the number and size of the final correct output files are confirmed by reading the neural network output configuration file in the model file. Regarding the method for merging the final correct output files, the first 4 bytes of the file header indicate the number of final correct output files, followed by 8 bytes describing each final correct output file. The first 4 bytes of each 8-byte header represent the offset of this file within the entire merged file, and the last 4 bytes represent the size of this file. Since the end of the file header cannot be 16-byte aligned, corresponding 0 bytes are added to the end for alignment. The starting address of each final correct output file is guaranteed to be 16-byte aligned; if its ending address cannot be 16-byte aligned, corresponding 0 bytes are added for alignment.

[0066] In addition, the preprocessing files also include neural network input files, which are essential for neural network calculations. These files are obtained by converting the input files of the neural network model generated by the neural network compiler to a specified size (i.e., endianness). The method for determining the endianness of the neural network input files is determined by the endianness settings of the hardware platform. The input files of the neural network model include images and videos.

[0067] The embedded hardware platform is configured to store the received preprocessed file to a set memory address; it is also configured to load the compiled firmware to a set program entry address for startup and execution.

[0068] In this embodiment, the embedded hardware platform (preferably a development board of a general-purpose processor SoC (which includes an NPU)) is used to transfer the generated neural network model merge file and neural network input file, as well as the optional final correct output merge file and the correct output merge file of the neural network intermediate layer, to a set memory address to ensure that the firmware can correctly read the corresponding file content; it is also used to load the neural network processor firmware compiled in the process into a specific program entry address to ensure that the firmware can start normally.

[0069] The firmware is configured to establish a connection with the embedded hardware platform and confirm whether each module of the embedded hardware platform is running normally. If it is running normally, the hardware devices are initialized. The initialized hardware devices include serial port transmission devices, neural network processors, interrupt devices, DMA, and timers.

[0070] The firmware is further configured to read and parse the preprocessed file stored on the embedded hardware platform, obtain the operator stream file corresponding to the neural network model, parse and initialize the neural network variables, and sequentially execute each operator in the operator stream file through a pre-built operator execution optimization method, thereby obtaining the output files of each intermediate layer of the neural network model and the final output file; the neural network variables include the number of data node operators, the list of data node operators, the number of data input nodes and the number of data output nodes.

[0071] In this embodiment, when using firmware for application programming, parameters can be passed according to application requirements based on the function interfaces exposed by the firmware to configure relevant firmware variables: By setting the firmware's net_addr (the address in memory where the neural network model merge file is loaded) and input_addr (the address in memory where the neural network model input file is loaded, to ensure the firmware program can correctly parse the neural network and perform correct calculations on the input), the addresses in memory of the necessary neural network model merge file and neural network input file are adjusted to ensure the firmware runs normally; by setting the firmware's run_times (the number of times the firmware executes the neural network to achieve the function of executing the neural network multiple times; if the set value is less than 0, the loaded neural network is executed an infinite number of times), the number of times the firmware executes the neural network model is adjusted; if the set value is less than 0, it means the neural network model is executed an infinite number of times; by setting the firmware's golden_mid_addr (the address in memory where the correct output merge file of the neural network intermediate layer is loaded) and golden_final_addr (the address in memory where the correct output merge file of the neural network final layer is loaded), the number of times the firmware executes the neural network model is adjusted; if the set value is less than 0, it means the neural network model is executed an infinite number of times; by setting the firmware's golden_mid_addr (the address in memory where the correct output merge file of the neural network intermediate layer is loaded) and golden_final_addr (the address in memory where the correct output merge file of the neural network final layer is loaded), the number of times the firmware executes the neural network model is adjusted to ensure the firmware runs normally; The `PRINT_LOG_LEVEL` parameter sets the memory address of the merged output file (for comparing intermediate layer results with the final output during neural network execution). It adjusts the memory addresses of the correct intermediate layer merged output file and the correct final output merged output file for debugging. The `PRINT_LOG_LEVEL` parameter, with values ​​from 1 to 6, allows adjustment of the print information level to six levels: FATAL, ERROR, WARN, INFO, DEBUG, and TRACE. This adjusts the granularity of the print information during firmware execution; the default value is 3. The `COMPARE_GOLDEN_MID` (controls whether intermediate layer comparison is performed) and `COMPARE_GOLDEN_FINAL` (controls whether final output comparison is performed) parameters control whether the comparison process between intermediate layers and the final output is enabled during debugging. Both parameters have selectable values: 0 for disabling the comparison process and 1 for enabling it; the default is 0.

[0072] After setup, a connection is established with the embedded hardware platform. The normal operation of all hardware modules, including the neural network processor and memory, is confirmed. Then, the configurations of hardware devices such as the serial port transmission device, neural network processor, interrupt device, DMA, and timers are initialized. The neural network processor operation instruction file, neural network model operator stream file, neural network model weight file, and neural network model output configuration file are correctly read and parsed. To avoid redundant calculations, reduce operator execution time, and achieve efficient neural network computation, the firmware pre-parses the neural network operator stream file, initializing variables such as the number of data node operators, the data node operator list, and the number of data input and output nodes. Furthermore, by parsing the data input and output node information, a corresponding DMA operation information list is generated. This allows for lookup-based omission of calculations during the actual execution of data input and output node operators.

[0073] The neural network computation process officially begins. The firmware parses the neural network operator stream file to obtain a sequentially executed stream of neural network operators. Operator stream parsing is the part of the firmware that correctly processes the execution of the neural network. By parsing and reconstructing the neural network operator stream file, the firmware obtains a stream of operators to be executed sequentially, and then performs specific processing on each operator.

[0074] Operator execution is the actual execution part of the firmware to correctly process the neural network. It includes eleven operators: instruction transfer operator, weight transfer operator, data input operator, data output operator, data concatenation operator, data preprocessing operator, data migration operator, lookup table transfer operator, data normalization operator, neural network processing operator, and data node operator. Each operator has specific operations. The operator execution optimization method for each operator in this application is as follows:

[0075] The instruction transfer operator uses DMA to transfer some or all of the neural network processor operation instructions from the neural network processor operation instruction file in memory to the operation instruction space of the neural network processor according to the operator parameter settings.

[0076] The weight transfer operator uses DMA to transfer all weights from the weight file of the neural network model in memory to the weight FIFO space of the neural network processor at high speed. Since each DMA transfer has a limited size, all weights are first divided into 16MB blocks, and the last block less than 16MB is recorded. During the last DMA transfer, the embedded hardware platform's interrupt operation is used to trigger an interrupt when each 16MB transfer is completed, configuring the transfer size to either the next block of 16MB or the last block less than 16MB. This achieves the function of automatically and asynchronously transferring weights of any size in the background. Figure 4 As shown;

[0077] The data input operator and the data output operator transfer the input and output data of the neural network model into or out of the neural network processor's data space using a multi-channel DMA transfer method. During the current DMA channel transfer, the next DMA transfer operation is configured for the next DMA channel, and the corresponding registers are configured for the next DMA transfer operation during the DMA transfer interval. Therefore, when the current DMA transfer time is greater than the next DMA configuration register time, it is equivalent to eliminating the next DMA configuration register time. The final total transfer time equals the sum of the total DMA transfer time and the first DMA configuration register time; conversely, the total transfer time equals the sum of the total DMA configuration register time and the last DMA transfer time. Compared to sequential DMA execution, this significantly shortens the operator execution time. Figure 5 As shown;

[0078] The data splicing operator is optimized into the data input and data output operators through neural network compiler optimization, so no specific operation is required.

[0079] The data preprocessing operator uses DMA to quickly transfer the preprocessed weight data from the weight file of the neural network model in memory to the preprocessed weight space of the neural network processor, according to a portion of the preprocessed weights set by the operator parameters.

[0080] The data transfer operator is optimized into the data input and data output operators through neural network compiler optimization, so no specific operation is required.

[0081] The lookup table transfer operator uses DMA to transfer a portion of the lookup table data from the weight file of the neural network model in memory to the lookup table data space of the neural network processor at high speed, according to the operator parameter settings.

[0082] The data uniformization operator uses the CPU to perform data uniformization calculations; the data includes bytes, two-byte integers, four-byte integers, and floating-point numbers.

[0083] The neural network processing operator, through the neural network processor, calculates the currently prepared instructions and data, and triggers an interrupt as a sign of processing completion;

[0084] The data node operator refers to the data node information that needs to be referenced during the processing of the neural network model (the data node information to be referenced is set according to the actual situation), so no specific operation is performed.

[0085] The high-speed firmware applies the aforementioned operator execution optimization method to quickly complete the execution of each operator and the entire operator flow, i.e., to complete a complete calculation of a neural network model. It obtains the output files of each intermediate layer of the neural network model (in this embodiment, each layer of the neural network model is output) and the final output file (the final output result of the neural network model), such as the feature map output by the convolutional layer, the pooling feature output by the pooling layer, etc. The specific output is based on the network structure of the combined model. The results of the execution can be further processed and applied. After the firmware runs, a neural network execution frame rate module can be built in, which can be set to calculate the frame rate every n seconds, with a default of 10 seconds. The execution frame rate will also be printed when the set number of neural network executions is completed.

[0086] In addition, the firmware running system also includes an intermediate layer output comparison unit and a final output comparison unit;

[0087] The intermediate layer output comparison unit is part of the debugging part that ensures the firmware correctly processes the neural network. It is configured to compare the intermediate layer output of the neural network model with the correct intermediate layer output during the calculation of a complete neural network model, after the data input operator is executed or before the data output operator is executed (that is, to fully compare the data currently stored in the data space of the neural network processor with the correct intermediate layer output). This verifies and ensures the correctness of the neural network execution. If the comparison is correct, it means that all operations before the current data input or output operator have been executed correctly.

[0088] The final output comparison unit is part of the debugging part that ensures the firmware correctly processes the neural network. It is configured to compare the final output of the neural network model with the final correct output after one or more complete calculations of the neural network model (that is, to fully compare the data nodes corresponding to the output with the final correct output) to verify and ensure the correctness of the neural network execution. If the comparison is correct, it means that the neural network model is running completely and correctly.

[0089] In other words, when debugging is required, during application programming using the firmware's runtime library, by setting the COMPARE_MID_GOLDEN parameter to 1 according to the function interfaces exposed by the firmware, and setting the golden_mid_addr parameter according to the memory address where the correctly merged output file of the neural network's intermediate layers is loaded, the outputs of the intermediate layers of the neural network can be compared each time the neural network is executed, and the results of the intermediate layer comparison can be printed. When debugging is also required, by setting the COMPARE_FINAL_GOLDEN parameter to 1, and setting the golden_final_addr parameter according to the memory address where the finally correctly merged output file of the neural network is loaded, the final output comparison can be performed n times as needed, and the final output comparison results can be printed.

[0090] It should be noted that the firmware runtime system for neural network processors provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.

[0091] The second embodiment of the present invention is based on the first embodiment, with reference to the appendix. Figure 1 - Appendix Figure 5 This invention applies a firmware runtime system designed for neural network processors to image or video file processing, and combines it with the YOLOv3 object detection neural network model to classify target objects in the image or video file. Through debugging, a connection is established with the neural network processor and its embedded hardware platform. This embedded hardware platform does not run an operating system; therefore, all programs, including the firmware of this invention's neural network processor, run directly on the general-purpose processor, neural network processor, and other hardware modules mounted on the hardware platform, as detailed below:

[0092] 1) Preprocess the model files generated by the neural network compiler.

[0093] Image or video files are input for target object detection. After input, a neural network compiler compiles and runs the neural network model. Once compilation is complete, the file generated by the neural network compiler is retrieved and used as the input file. Then, a script preprocesses the input file.

[0094] The preprocessing script set consists of the model file generation script `case_gen.py`, the neural network intermediate layer correct output merging script `golden_mid_gen.py`, the neural network final correct output merging script `golden_final_gen.py`, and an empty folder named `cases`. Confirm that the model files generated by the neural network compiler include the `~ / yolov3 / case` folder and the `~ / yolov3 / golden` folder. Place the root folder `yolov3` into the `cases` folder in the preprocessing script set. Then, open a terminal with a Python environment in the path where the scripts are located and run the command `python . / case_gen.py`. After execution, a folder named `ready` will be generated in the path of the preprocessing script set. Opening this folder reveals a folder named yolov3, which contains the preprocessed YOLOv3 model. This folder contains two files: net.bin and input.bin. These correspond to the neural network model merge file (including the neural network processor operation instruction file (e.g., after inputting an image or video file, the first step is to extract the feature map file of the image or video, and then process it sequentially through residual processing, average pooling, and a softmax layer to obtain the object detection result file), the neural network model operator stream file, the neural network model weight file (e.g., the bias, mean, variance, weights, etc. of each layer of the YOLOv3 model), and the neural network model output configuration file (e.g., the memory address corresponding to the object detection result file (i.e., the target category)) and the neural network input file (which is the file conversion of the input file (image or video file) generated by the neural network compiler to obtain the neural network input file).

[0095] 2) Compile the neural network processor firmware

[0096] The firmware's net_addr variable is set to 0x50000000, and the input_addr variable is set to 0x58000000, indicating that the neural network model merge file will be loaded at 0x50000000, and the neural network input file will be loaded at 0x58000000; the firmware's run_times variable is set to 10000, indicating that the Yolov3 neural network model will be executed 10000 times; the PRINT_LOG_LEVEL parameter is defined as 3, indicating that the printing level is controlled to WARN, that is, only WARN, ERROR, and FATAL information will be printed in addition to necessary printing information; the COMPARE_MID_GOLDEN and COMPARE_FINAL_GOLDEN parameters are defined as 0, indicating that no comparison between intermediate layer outputs and final outputs will be performed;

[0097] 3) Transfer the preprocessed neural network model file to the hardware platform.

[0098] Establish a connection with the embedded hardware platform and confirm that all modules of the hardware platform, including the neural network processor and memory, are running normally. Load the merged neural network model file generated in the first step into address 0x50000000, and load the neural network input file into address 0x58000000; load the neural network processor firmware compiled in the second step into the set program entry address;

[0099] 4) Firmware initialization and fully automatic operation

[0100] Upon running the firmware program, the firmware initializes hardware devices including serial port transmission, interrupts, and timers. It then correctly reads and parses files such as the neural network processor operation instruction file, neural network operator stream file, neural network weight file, and neural network output configuration file. Next, the firmware pre-parses the neural network operator stream file, initializing variables including the number of data node operators, the list of data node operators, and the number of data input and output nodes. By parsing the data input and output node information, it generates a corresponding DMA operation information list, which will be used to omit redundant calculations when executing data input and output node operators (i.e., the image or video file input to the model, and the categories or other results of object detection in the output image or video file) through table lookup. Afterward, the formal calculation process begins. Based on the parsed operator stream, operations are executed sequentially. The firmware applies and executes the optimization method described in this patent for each parsed operator to quickly complete the operator execution. During execution, the frame rate of the neural network is printed every 10 seconds, and the firmware program terminates after 10,000 executions based on the run_times variable set during compilation.

[0101] After completing the above steps, the processing of the mainstream object detection neural network Yolov3 on this hardware platform and neural network processor is realized, preparing for subsequent processing and applications. If any WARN, ERROR, FATAL, or other similar messages are printed during execution, it indicates that the firmware encountered a problem when executing the neural network and should be debugged and resolved; otherwise, it indicates that no problems were encountered and subsequent processing can proceed normally.

[0102] The third embodiment of this invention, based on the second embodiment, applies a firmware runtime system for neural network processors to image or video file processing. When combining the object detection neural network model Yolov3 for target object detection in images or video files, an intermediate layer output comparison unit and a final output comparison unit are added. Specifically, when the firmware prints any error-related information during execution, or when there are obvious errors in the firmware execution process or results, or when there is a need for debugging intermediate layer output comparison or final output comparison, or when more detailed runtime records need to be printed, a debugging module can be added during compilation by setting corresponding parameters, as detailed below:

[0103] 1) Preprocess the model files generated by the neural network compiler.

[0104] Image or video files are input for target object detection. After input, a neural network compiler compiles and runs the neural network model. Once compilation is complete, the file generated by the neural network compiler is retrieved and used as the input file. Then, a script preprocesses the input file.

[0105] The preprocessing script set consists of the model file generation script `case_gen.py`, the neural network intermediate layer correct output merging script `golden_mid_gen.py`, the neural network final correct output merging script `golden_final_gen.py`, and an empty folder named `cases`. Confirm that the model files generated by the neural network compiler include the `~ / yolov3 / case` folder and the `~ / yolov3 / golden` folder. Place the root folder `yolov3` into the `cases` folder in the preprocessing script set. Then, open a terminal with a Python environment in the path where the scripts are located and run the command `python . / case_gen.py`. After execution, a folder named `ready` will be generated in the path of the preprocessing script set. Continue running the command `python . / golden_mid_gen.py`; after execution, there is no obvious change. Continue running the command `python . / golden_final_gen.py`; after execution, there is no obvious change. Open the generated ready folder, and you will find a folder with the same name yolov3, which is the preprocessed Yolov3 model. It contains four files: net.bin and input.bin, which correspond to the neural network model merge file and the neural network input file, respectively; golden_mid.bin and golden_final.bin, which correspond to the neural network intermediate layer correct output merge file and the neural network final correct output merge file, respectively, which can be used for output comparison.

[0106] 2) Compile the neural network processor firmware

[0107] The firmware's net_addr variable is set to 0x50000000, and the input_addr variable is set to 0x58000000, indicating that the neural network model merge file will be loaded at 0x50000000, and the neural network input file will be loaded at 0x58000000; the firmware's golden_mid_addr is set to 0x68000000, and the golden_final_addr is set to 0x70000000, indicating that the correct output merge file of the neural network's intermediate layers will be loaded at 0x68000000, and the correct final output merge file of the neural network will be loaded at 0x70000000. The firmware's `run_times` variable is set to 1, indicating that the Yolov3 neural network model will be executed only once; the `PRINT_LOG_LEVEL` parameter is defined as any integer greater than or equal to 6, indicating that the print level is controlled to TRACE, i.e., all information set in the firmware is printed; the `COMPARE_MID_GOLDEN` and `COMPARE_FINAL_GOLDEN` parameters are set to 1, indicating that intermediate layer outputs and final outputs are compared.

[0108] 3) Transfer the preprocessed neural network model file to the hardware platform.

[0109] Establish a connection with the embedded hardware platform and confirm that all modules of the hardware platform, including the neural network processor and memory, are functioning correctly. Load the merged file of the neural network model generated in the first step into address 0x50000000, the neural network input file into address 0x58000000, the merged file of the correct output of the intermediate layers into address 0x68000000, and the merged file of the final correct result of the neural network into address 0x7000000; load the compiled neural network processor firmware from the second step into the designated program entry address.

[0110] 4) Firmware initialization and fully automatic operation

[0111] Upon running the firmware program, the firmware will print a large amount of information, which will be saved to a file using a log method for easy error localization and analysis during subsequent debugging. The firmware will initialize hardware devices including serial port transmission, interrupts, and timers, and then correctly read and parse the neural network processor operation instruction file, neural network operator stream file, neural network weight file, and neural network output configuration file. Next, the firmware pre-parses the neural network operator stream file, initializing variables such as the number of data node operators, the list of data node operators, and the number of data input and output nodes. Furthermore, by parsing the data input and output node information, it generates a corresponding DMA operation information list, which will omit redundant calculations during subsequent execution of data input and output node operators through table lookup. Afterwards, the formal debugging calculation process begins. Based on the parsed operator stream, operations are executed sequentially. The firmware applies and executes the optimization method described in this patent for each parsed operator, but due to the intervention of debugging print information, operator execution will be slower.

[0112] 5) Debugging: Comparison of intermediate layer outputs

[0113] The firmware automatically performs an intermediate layer output comparison process, comparing each byte based on the parameters of the corresponding data node operators in the intermediate layer. If the comparison is correct, it prints information showing that err_cnt is 0 and correct_cnt is an integer value; if the comparison is incorrect, it prints a large amount of information showing that the values ​​of err_output and golden are not equal, including the specific memory address where the problem occurred, its 32-bit value, and relevant information about the corresponding data node operator. Based on this debugging information, it is possible to first locate which intermediate layer and which operator the problem occurs in, then determine the number of errors, and also determine the type of error, and then analyze the cause of the error. For example, by obtaining the feature maps extracted from the convolutional layers of the image or video file used for debugging, and combining them with the given feature map labels, a comparison calculation can be performed. Based on the comparison results, the loss values ​​of each convolutional layer can be calculated, and then the network parameters of the neural network convolutional layers can be updated.

[0114] 6) Debugging: Final output comparison

[0115] After executing a complete Yolov3 model once, the firmware performs a final output comparison. If the comparison is correct, it prints information indicating that `err_cnt` is 0 and `correct_cnt` is an integer value. If the comparison is incorrect, it prints a large amount of information indicating that the values ​​of `err_output` and `golden` are not equal, including the specific memory address where the problem occurred, its 32-bit value, and information about the corresponding data node operator. Furthermore, if any errors occur during the intermediate layer output comparison process, the final output comparison will inevitably be incorrect; conversely, the correctness of the final output comparison is guaranteed. Therefore, this debugging method can be used to roughly verify the correctness of the neural network execution. For example, by taking a video or image file for debugging as input, obtaining the target detection results (i.e., the categories of each target or other detection results) from the object detection network, comparing them with the ground truth labels of the corresponding target detection results, calculating the loss value based on the comparison results, and then updating the network parameters.

[0116] After completing the above steps, the processing and debugging of the mainstream object detection neural network YOLOv3 on this hardware platform and neural network processor is achieved. After debugging and all errors have been identified and corrected, no WARN, ERROR, FATAL, or other error messages should appear in the firmware output, nor should there be any matching errors in the intermediate layer output comparison or the final output comparison. After ensuring the correct execution of the entire firmware program and neural network, Example 1 can be implemented to disable the debugging module and realize the neural network execution process based on a specific hardware platform and the firmware to which this patent pertains.

[0117] A firmware running method for a neural network processor according to a fourth embodiment of the present invention is based on a firmware running system for a neural network processor according to the first embodiment described above. Figure 2 As shown, the method includes the following steps:

[0118] Step S100: When the neural network compiler has finished compiling, obtain the file generated by the neural network compiler after compiling the input neural network model, and use it as the input file;

[0119] Step S200: Preprocess the input file to obtain a preprocessed file;

[0120] Step S300: The preprocessed file is parsed to obtain the operator stream file corresponding to the neural network model. Then, each operator is executed sequentially through the pre-built operator execution optimization method to obtain the output files of each intermediate layer of the neural network model and the final output file.

[0121] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related explanations of the method described above can be found in the corresponding process in the foregoing system embodiments, and will not be repeated here.

[0122] A fifth embodiment of the present invention provides an electronic device comprising at least one processor and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to implement the aforementioned firmware running method for a neural network processor.

[0123] A computer-readable storage medium according to a sixth embodiment of the present invention stores computer instructions, which are executed by the computer to implement the above-described firmware running method for a neural network processor.

[0124] Those skilled in the art will understand that, for the convenience and simplicity of the undescribed electronic devices and computer-readable storage media, the specific working process and related descriptions can be found in the corresponding processes in the aforementioned system examples, and will not be repeated here.

[0125] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.

[0126] The terms “first” and “second” are used to distinguish similar objects, rather than to describe or indicate a specific order or sequence.

[0127] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0128] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A firmware runtime system for neural network processors, used for processing files generated after compilation by a neural network compiler, characterized in that, The system includes: The file preprocessing unit is configured to, upon detecting that the neural network compiler has completed compilation, obtain the file generated by the neural network compiler after compiling the input neural network model, and use it as the input file; preprocess the input file to obtain a preprocessed file, and send it to the embedded hardware platform; the preprocessing includes file merging, file conversion, and file header design; The embedded hardware platform is configured to store the received preprocessed file to a set memory address; it is also configured to load the compiled firmware to a set program entry address for startup and execution. The firmware is configured to establish a connection with the embedded hardware platform and confirm whether each module of the embedded hardware platform is running normally. If it is running normally, the hardware devices are initialized. The initialized hardware devices include serial port transmission devices, neural network processors, interrupt devices, DMA, and timers. The firmware is further configured to read and parse the preprocessed file stored on the embedded hardware platform, obtain the operator stream file corresponding to the neural network model, parse and initialize the neural network variables, and sequentially execute each operator in the operator stream file through a pre-built operator execution optimization method, thereby obtaining the output files of each intermediate layer of the neural network model and the final output file; the neural network variables include the number of data node operators, the list of data node operators, the number of data input nodes and the number of data output nodes.

2. The firmware running system for a neural network processor according to claim 1, characterized in that, The input file is preprocessed to obtain a preprocessed file, and the method is as follows: The four files—the neural network processor operation instruction file, the neural network model operator stream file, the neural network model weight file, and the neural network model output configuration file—are merged to obtain the neural network model merged file. The input file of the neural network model generated by the neural network compiler is converted into a neural network input file; the input file of the neural network model includes images and videos. Select the necessary intermediate layer correct output files from all the correct output files of the neural network generated by the neural network compiler, and merge them into a single correct intermediate layer output file. Select all the correct output files of the neural networks generated by the neural network compiler and merge them into a single final correct output file.

3. The firmware running system for a neural network processor according to claim 2, characterized in that, The neural network model merge file has a 32-byte header, with each 8-byte section describing a file. The first 4 bytes of each 8-byte section represent the bias of the file in the neural network model merge file, and the last 4 bytes represent the size of the file. For each file in the merged neural network model file, its starting address is aligned to 16 bytes, and if its ending address cannot be aligned to 16 bytes, it is padded with corresponding 0 bytes to make it aligned.

4. A firmware running system for a neural network processor according to claim 2, characterized in that, The merged file of the correct output of the intermediate layer of the neural network has a header. The first 4 bytes of the header indicate the number of correct output files of the intermediate layer of the neural network. The next 8 bytes describe a selected correct output file of the intermediate layer of the neural network. The first 4 bytes of each 8 bytes are the data node number corresponding to the selected correct output file of the intermediate layer of the neural network, and the last 4 bytes are the offset of the selected correct output file of the intermediate layer of the neural network in the merged file. The end of the header cannot be aligned to 16 bytes, so corresponding 0 bytes are added to the end of the header for alignment. The starting address of each selected intermediate layer of the neural network is guaranteed to be aligned to 16 bytes. If the ending address cannot be aligned to 16 bytes, it is padded with corresponding 0 bytes to make it aligned.

5. A firmware running system for a neural network processor according to claim 2, characterized in that, The neural network ultimately outputs a merged file. The first 4 bytes of the file header indicate the number of final correct output files. Then, every 8 bytes describe a final correct output file. The first 4 bytes of each 8 bytes are the offset of this final correct output file in the neural network's final correct output merged file, and the last 4 bytes are the size of this file. The end of the file header cannot be aligned to 16 bytes, so corresponding 0 bytes are added to the end of the file header for alignment. Each final correct output file is guaranteed to start at a 16-byte aligned address. If the end of the file is not 16-byte aligned, it is padded with corresponding 0 bytes to make it aligned.

6. A firmware running system for a neural network processor according to claim 2, characterized in that, The operators in the operator stream file include instruction transmission operators, weight transmission operators, data input operators, data output operators, data concatenation operators, data preprocessing operators, data migration operators, lookup table transmission operators, data normalization operators, neural network processing operators, and data node operators.

7. A firmware running system for a neural network processor according to claim 6, characterized in that, The optimization method executes each operator sequentially using pre-built operators, as follows: The instruction transfer operator, from the neural network processor operation instruction file in memory, uses DMA to transfer some or all neural network processing operation instructions from the neural network processor operation instruction file according to the operator parameter settings to the operation instruction space of the neural network processor at high speed. The weight transfer operator uses DMA to transfer all weights from the weight file of the neural network model in memory to the weight FIFO space of the neural network processor at high speed. Since the size of each DMA transfer is limited, all weights are first divided into blocks of 16MB each, and the last block less than 16MB is recorded. During the last DMA, the interrupt operation of the embedded hardware platform is used to trigger an interrupt when each 16MB transfer is completed, and the transfer size is configured to be either the next block of 16MB or the last block less than 16MB, so as to realize the function of automatically and asynchronously transferring weights of any size in the background. The data input operator and the data output operator transfer the input and output data of the neural network model into or out of the data space of the neural network processor using a multi-channel DMA transfer method. During the current DMA channel transfer, the next DMA transfer operation is configured for the next DMA channel, and the corresponding registers are configured for the next DMA transfer operation using the DMA transfer interval. Therefore, when the current DMA transfer time is greater than the next DMA configuration register time, it is equivalent to eliminating the next DMA transfer configuration register time. The final total transfer time is equal to the sum of the total DMA transfer time and the first DMA transfer configuration register time. Conversely, the total transfer time is equal to the sum of the total DMA transfer configuration register time and the last DMA transfer time, thus saving data transfer time. The data splicing operator is optimized into the data input and data output operators through neural network compiler optimization, so no specific operation is required. The data preprocessing operator uses DMA to quickly transfer the preprocessed weight data from the weight file of the neural network model in memory to the preprocessed weight space of the neural network processor, according to a portion of the preprocessed weights set by the operator parameters. The data transfer operator is optimized into the data input and data output operators through neural network compiler optimization, so no specific operation is required. The lookup table transfer operator uses DMA to transfer a portion of the lookup table data from the weight file of the neural network model in memory to the lookup table data space of the neural network processor at high speed, according to the operator parameter settings. The data uniformization operator uses the CPU to perform data uniformization calculations; the data includes bytes, two-byte integers, four-byte integers, and floating-point numbers. The neural network processing operator, through the neural network processor, calculates the currently prepared instructions and data, and triggers an interrupt as a sign of processing completion; The data node operator refers to the data node information that needs to be referenced during the processing of the neural network model, and therefore no specific operation is performed on it.

8. A firmware running system for a neural network processor according to claim 7, characterized in that, The firmware operating system also includes an intermediate layer output comparison unit; The intermediate layer output comparison unit is configured to compare the intermediate layer output of the neural network model with the correct intermediate layer output during the calculation of a complete neural network model, after the data input operator is executed or before the data output operator is executed. If the comparison is correct, it means that all operations up to the current data input or output operator have been executed correctly.

9. A firmware running system for a neural network processor according to claim 1, characterized in that, The firmware running system also includes a final output comparison unit; The final output comparison unit is configured to compare the final output of the neural network model with the final correct output after one or more complete calculations of the neural network model. If the comparison is correct, it means that the neural network model has run completely and correctly.

Images