A time cost calculation method, system and readable storage medium of a neural network processor

Abstract

This invention relates to a method, system, and readable storage medium for calculating the time cost of a neural network processor. Based on a behavioral-level simulator, it calculates the execution time of a transport unit and the execution time of a vector computation unit. The calculation of the transport unit's execution time includes: the transport unit's transport execution time, the transport unit's update time, and the time for the transport unit to issue a release-wait instruction. The execution time of the vector computation unit includes: the vector computation unit's computation time, the vector computation unit's update time, and the time for the vector computation unit to issue a release-wait instruction. The calculation of the transport unit's execution time and the calculation of the vector computation unit's execution time are performed in parallel. By utilizing a behavioral-level simulator during the chip design phase to obtain the same time cost as the execution time on a real chip, accurate performance data is obtained, providing a data foundation for chip architecture optimization.

Description

Technical Field

[0001] This invention relates to the field of simulators, and more specifically to a method, system, and readable storage medium for calculating the time cost of a neural network processor. Background Technology

[0002] The performance data of the neural network processor's vector units within the framework of the compiler is crucial for guiding performance tuning. Chip designers use this performance data to identify architectural flaws and optimize the compiler's framework for more efficient processing. However, performance data on the vector units of neural network processors acquired from finished chips is outdated and cannot guide architecture optimization during the chip design phase. Simulating the workflow of complex neural network processor vector units during the design phase, without access to finished chips, and obtaining accurate performance data, especially the time cost within that data, is a critical factor limiting the performance optimization of neural network processor architecture design. Summary of the Invention

[0003] To address the technical problem that existing technologies struggle to obtain accurate performance data for the complex workflow of vector units in neural network processors, this invention proposes a method, system, and readable storage medium for calculating the time cost of neural network processors. This enables the acquisition of accurate performance data during the chip design phase to guide chip architecture optimization.

[0004] This invention provides a method for calculating the time cost of a neural network processor. The method is based on a behavior-level simulator and calculates the execution time of a transport unit and the execution time of a vector calculation unit.

[0005] The execution time of the calculation and handling unit includes: the handling unit's handling time, the handling unit's update time, and the time when the handling unit issues a release waiting instruction;

[0006] The execution time of the vector calculation unit includes: the calculation time of the vector calculation unit, the update time of the vector calculation unit, and the time when the vector calculation unit issues the release waiting instruction;

[0007] The execution time of the calculation and transportation unit and the execution time of the calculation and vector calculation unit are executed in parallel.

[0008] Preferably, the time for the transport unit to perform transport is calculated based on the transport unit being in the transport state. The time cost calculation for the transport unit being in the transport state includes: pushing the number of cycles required for the transport unit to perform the current transport to the end of the cache used to store the transport completion time point, as the end time of the current transport.

[0009] Preferably, the update time of the transport unit is calculated based on the transport unit being in an update state, where the transport unit is in an update state as follows:

[0010] The transport unit has not started or completed transport, and the transport unit currently has no transport instruction configured, and the transport unit has completed at least one transport operation;

[0011] Alternatively, the transport unit receives a release waiting command from the vector calculation unit or from another transport unit in the opposite direction to the transport unit's transport data.

[0012] Preferably, the time cost calculation for the transport unit in the update state includes: updating the value of the header of the cache used to store the transport completion time to the register used to count the transport execution time, and deleting the value of the header of the cache used to store the transport completion time.

[0013] Preferably, the time for the transport unit to issue the release waiting instruction is calculated based on the transport unit being in the state of issuing the release waiting instruction. The time cost calculation for the transport unit being in the state of issuing the release waiting instruction includes: saving the actual number of cycles for the transport unit to issue the release waiting instruction to a cache used to store the data required for the handshake of the execution unit.

[0014] Preferably, the update time of the vector computing unit is calculated based on the vector computing unit being in an update state. The time cost calculation of the vector computing unit being in an update state includes: updating the data in the cache used to store the data required for the handshake between the vector computing unit and the transport unit to the register used to count the execution time.

[0015] Preferably, the time it takes for the vector computing unit to issue a release-wait instruction is calculated based on the vector computing unit being in a release-wait instruction state. The time cost calculation for the vector computing unit being in the release-wait instruction state includes:

[0016] When the pipeline parsing of the vector computing unit obtains that the current instruction is to issue a release-wait instruction, the value of the register used for statistical execution time is recorded at the current moment.

[0017] Preferably, the transport unit includes instructions for transporting data outside the chip to storage inside the chip, and instructions for transporting stored data inside the chip to outside the chip.

[0018] The present invention also provides a time cost calculation system for a neural network processor, comprising:

[0019] Behavioral simulator

[0020] The first time cost calculation unit is used to calculate the execution time of the material handling unit.

[0021] A second time cost calculation unit used to calculate the execution time of the vector calculation unit;

[0022] The first time cost calculation unit is configured to calculate the transportation time of the transportation unit, the update time of the transportation unit, and the time when the transportation unit issues a release waiting instruction;

[0023] The second time cost calculation unit is configured to perform the calculation time of the vector calculation unit, the update time of the vector calculation unit, and the time when the vector calculation unit issues the release waiting instruction;

[0024] The first time cost calculation unit and the second time cost calculation unit are executed in parallel.

[0025] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0026] Compared with the prior art, the present invention provides a method, system and readable storage medium for calculating the time cost of a neural network processor. By using a behavioral-level simulator, multiple instructions are executed in parallel during the chip design stage to calculate the execution time of instructions of different types. The time cost is obtained on the behavioral-level simulator, which is the same as the execution time of the corresponding instruction on the chip. This enables accurate acquisition of chip performance parameters during the chip design stage, providing a data foundation for chip designers to perform architecture optimization based on performance data, which is mainly based on time cost, to expose architectural defects during the design stage.

[0027] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0028] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0029] Figure 1 A flowchart of a method for calculating the time cost of a neural network processor provided in an embodiment of the present invention;

[0030] Figure 2 A flowchart illustrating the operation of a neural network processor vector unit provided in an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of the data transfer process provided in an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of the vector calculation process provided in an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the various embodiments of the present invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0034] The first embodiment of the present invention provides a method for calculating the time cost of a neural network processor. Please refer to [link to relevant documentation]. Figure 1 .

[0035] The instructions corresponding to the vector units inside the neural network processor are divided into three categories:

[0036] Vector calculation instructions, such as addition (vadd), subtraction (vsub), multiplication (vmul), shift (vasr / vlsr / vasl / vlsl), comparison (vcmp), swap (vmovv2v / vmovvacc2v, etc.) and memory access instructions (vld / vst);

[0037] Transport commands, such as edma_cfg, edma_ext_addr, edma_burst, etc.

[0038] Synchronization instructions, such as unblock and wait_on.

[0039] Based on the functionality and performance of the behavior-level simulator, vector calculation instructions and synchronization instructions are simulated at the cycle level. The execution time required on the behavior-level simulator is the same as the execution time on the chip, that is, the time obtained on the behavior-level simulator is the real time. For transport instructions, since only the parsing instructions are at the cycle level, the bus starts transporting once the instructions are parsed, so the behavior-level simulator is used for simulation.

[0040] See Figure 2The workflow of a neural network processor vector unit in one embodiment includes two data transfer units, eidma0 and eidma1, for transferring data from off-chip to on-chip storage, and two data transfer units, eodma0 and eodma1, for transferring data from on-chip storage to off-chip. Multiple instructions are executed in parallel through synchronization instructions. These instructions include at least: eidma transfer instructions for transferring data from off-chip to on-chip storage, vector calculation instructions, and eodma transfer instructions for transferring data from on-chip storage to off-chip. The transfer time for the eidma transfer instructions for transferring data from off-chip to on-chip storage and the eodma transfer instructions for transferring data from on-chip storage to off-chip storage are behavior-level transfer times, i.e., not the actual transfer time of the instructions on the chip; the vector calculation instructions have cycle-level transfer times, i.e., the actual time of the instructions on the chip.

[0041] The time cost simulation of the neural network vector processing unit is based on a behavioral simulator. During the chip design stage rather than the finished chip, the chip instructions are simulated by the behavioral simulator to obtain a number of cycles similar to the test data of the finished chip as the time cost. This provides a basis for chip designers to optimize the architecture based on the architectural defects exposed by the performance data, which is mainly based on time cost, during the design stage.

[0042] Specifically, the time cost of calculating the neural network vector processing unit is based on the execution time of the transport unit calculated by the behavior-level simulator, and the execution time of the vector calculation unit; the execution time of the transport unit includes: the transport unit's transport time, the transport unit's update time, and the time for the transport unit to issue a release-wait instruction; the execution time of the vector calculation unit includes: the vector calculation unit's calculation time, the vector calculation unit's update time, and the time for the vector calculation unit to issue a release-wait instruction; wherein, the execution time of calculating the transport unit and the execution time of calculating the vector calculation unit are executed in parallel.

[0043] Specifically, the execution time of the transport unit is calculated based on whether it is in the transport execution state, the update time of the transport unit is calculated based on whether it is in the update state, and the time for issuing the release-wait instruction is calculated based on whether it is in the release-wait instruction state. Similarly, the execution time of the vector calculation unit is calculated based on whether it is in the calculation execution state, the update time of the vector calculation unit is calculated based on whether it is in the update state, and the time for issuing the release-wait instruction is calculated based on whether it is in the release-wait instruction state. The execution order of the transport unit's transport execution, update, and release-wait instruction depends on the software programming method; as an optional embodiment, the execution order can be parallel. Likewise, the execution order of the vector calculation unit's calculation, update, and release-wait instruction depends on the software programming method; as an optional embodiment, the execution order can be parallel.

[0044] In some embodiments, a transfer unit is involved, including instructions for transferring data from outside the chip to a cache stored inside the chip, and instructions for transferring stored data from inside the chip to a cache outside the chip. The cycle counts of the two are calculated in the same way. A transfer instruction register and multiple caches are prepared. The time calculation process specifically includes: calculating the transfer time of the transfer unit, calculating the update time of the transfer unit, and calculating the time when the transfer unit issues a release-wait instruction.

[0045] See Figure 3 This is an example of a specific transport process. Parsing the transport instruction `eidma` requires 10 cycles. The `eidma` transport unit on the behavioral simulator starts transporting at time 11 and completes the transport in this cycle. Parsing the transport instruction `eidma` also requires 10 cycles, i.e., times 10 to 19 in reality. The `eidma` transport unit on the behavioral simulator starts transporting at time 21 and completes the transport in this cycle. On the finished chip, the transport process occurs twice during instruction execution. The first actual transport takes 150 cycles, i.e., from times 10 to 159 in reality, and the second actual transport takes 200 cycles, i.e., from times 160 to 359 in reality.

[0046] In practical neural network processors, to improve the execution efficiency of the vector units, vector calculation instructions and transport instructions are executed in parallel. Therefore, it is difficult to calculate the actual execution time of the vector units. Under the complex workflow of the vector units in the neural network processor described in the above embodiments, when the behavior-level simulator executes the assembly code, some instructions are executed at the cycle level and some at the behavior level, making it difficult to determine the actual execution time on the chip using the behavior-level simulator.

[0047] In the workflow of the vector unit of the neural network processor provided in the above embodiments, calculating the time for the transport unit to perform transport includes the following steps:

[0048] S1 prepares to move the instruction register and multiple caches, including:

[0049] The register eidma_cycle is used to track the execution time of the transport process;

[0050] The cache eidma_transfer_end_fifo used to store the time point when the transfer is completed;

[0051] The caches eidma_ub_vu_fifo and eodma_ub_vu_fifo are used to store the data required for the handshake between the transport unit and the vector calculation unit.

[0052] eidma_cycle_bk is used to store the number of implementation cycles when the handling unit completes handling.

[0053] All of the above caches follow the first-in, first-out (FIFO) principle.

[0054] S2 determines the state of the transport unit. If the transport unit is in the transport execution state, the transport execution time is calculated. If the transport unit is in the update state, the update time is calculated. If the transport unit is in the state of issuing a release waiting instruction, the time for issuing the release waiting instruction is calculated. The above calculations are performed in parallel based on the state of the transport unit.

[0055] A. Calculation of the transport time when the transport unit is in the transport execution state.

[0056] When the EIDMA transfer instruction is executing within the current transfer cycle, the transfer time of the EIDMA transfer instruction is calculated based on two sets of data: the amount of data to be transferred as configured in the EIDMA transfer instruction and the bus bandwidth. The specific calculation method for the transfer time does not affect the implementation of the technical effect in this embodiment. The transfer time is pushed to the end of the cache eidma_transfer_end_fifo, and at the same time, the data in the cache eidma_cycle_bk is recorded in the register eidma_cycle.

[0057] by Figure 3 The following is an example of the transfer process. Transfer 0 starts at time 10. Assuming transfer 0 requires 150 cycles, the time for transfer 0 is 160, and 160 is stored at the end of `eidma_transfer_end_fifo`. Transfer 1 starts at time 160. Assuming transfer 1 requires 200 cycles, the time for transfer 1 ends at time 360, and 360 is stored again at the end of `eidma_transfer_end_fifo` when the transfer ends.

[0058] B. Calculation of update time for the transport unit in the update state

[0059] The determination that the transfer unit is in the update state is as follows: the transfer unit has not started transfer or has completed transfer (i.e., eidma idle state), and the transfer unit currently has no transfer instruction configuration (e.g., eidma_cfg, eidma_ext_addr, eidma_burst, etc.), and the transfer unit has completed at least one transfer (i.e., the cache eidma_transfer_end_fifo is greater than 1);

[0060] Alternatively, the transport unit eidma receives a release-wait command from the vector calculation unit or the transport unit eodma.

[0061] When the transport unit is in the update state, the value of the header of the cache used to save the transport completion time is updated to the register used to count the transport execution time, and the value of the header of the cache used to save the transport completion time is deleted.

[0062] by Figure 3Taking the shown transfer process as an example, the configuration completion point of transfer unit eidma0 is 9, and the actual start point of transfer unit eidma0 is 10. Assume the transfer instruction requires 150 cycles. The configuration completion point of transfer unit eidma1 is 19, and the actual start point of transfer unit eidma1 is 160. Assume the transfer requires 200 cycles. After both transfer units eidma0 and eidma1 have completed their configuration through cycle-level simulation, the transfer unit satisfies the conditions of completing one transfer operation, the transfer is currently complete, and there are no transfer instructions being configured. That is, it meets the conditions for being in the update state. The value of eidma_transfer_end_fifo.front is updated to the register eidma_cycle used to count the transfer execution time, and the value of the cached eidma_transfer_end_fifo header used to store the transfer completion time is deleted. In the next cycle, if the transport unit meets the conditions of completing a transport operation and there is currently no transport instruction configured, i.e., it meets the conditions of being in the update state, it will update the eidma_transfer_end_fifo.front value to the eidma_cycle register used to count the transport execution time, and delete the value of the cache eidma_transfer_end_fifo header used to store the transport completion time.

[0063] C. Calculation of the time the transport unit is in the state of issuing a release waiting command.

[0064] When the transport unit is in the state of issuing a release waiting instruction, the actual number of cycles in which the transport unit issues the release waiting instruction is saved to the cache used to store the handshake data required by the execution unit.

[0065] by Figure 3 Taking the transport process shown as an example, when the transport unit eidma executes the instruction eidma.ub.vu or the instruction eidma.ub.eodma, the number of cycles corresponding to when the transport unit eidma issues the release waiting instruction is saved to the cache eidma_ub_vu_fifo, which is used to store the data required for the handshake between the transport unit and the vector calculation unit, or the cache eidma_ub_vu_fifo, which is used to store the data required for the handshake between the transport unit and the vector calculation unit, or the cache eidma_ub_eodma_fifo, which is used to store the data required for the handshake between the transport unit and the transport unit.

[0066] By performing different events in parallel for transport units under different conditions, and combining three parallel calculations for time cost with different caches corresponding to different conditions (i.e., cache eidma_transfer_end_fifo for storing the transport completion time; cache eidma_ub_vu_fifo and cache eodma_ub_vu_fifo for storing the data required for the handshake between the transport unit and the vector calculation unit; cache eidma_ub_eodma_fifo for storing the data required for the handshake between transport units; and eidma_cycle_bk for storing the number of implementation cycles when the transport unit completes transport), this embodiment achieves accurate acquisition of actual time cost similar to that of the finished chip based on a behavioral simulator.

[0067] See Figure 4 This is an example of a vector calculation process. The completion time of the eidma transport instruction is 160. The vector calculation unit starts executing calculation instructions from time 160. Assuming that the execution of calculation instructions takes 100 cycles, after the vector calculation unit completes the execution of calculation instructions, it issues a release-wait instruction vu.ub.eodma to the eodma transport module at time 260.

[0068] In the workflow of the vector unit of the neural network processor provided in the above embodiments, the calculation time for the vector calculation unit to perform calculations includes the following steps:

[0069] S1 prepares the instruction register and multiple caches for computation, including:

[0070] The register vu_cycle is used for statistical calculations of execution time;

[0071] The caches vu_ub_eidma_fifo and vu_ub_eodma_fifo are used to store the data required for the handshake between the vector calculation unit and the transport unit.

[0072] The real-time cycle register, vu_cycle_bk, is used to store the time when the vector calculation unit completes its calculation.

[0073] All of the above caches follow the first-in, first-out (FIFO) principle.

[0074] S2 determines the state of the vector computing unit. If the vector computing unit is in the update state, the update time of the vector computing unit is calculated. If the vector computing unit is in the state of issuing a release waiting instruction, the time of issuing the release waiting instruction of the vector computing unit is calculated. The above calculations are performed in parallel based on the state of the vector computing unit.

[0075] A. Calculation of the update time of the vector calculation unit in the update state.

[0076] If the vector calculation unit receives a release waiting instruction from the transport unit (including transport unit eidma or transport unit eodma) (see...) Figure 4 If the vector computation unit is in the update state (e.g., vu.wo.eidma or eodma.wo.vu), then the vector computation unit is considered to be in the update state at the current moment, and the update time calculation is performed. This time point is then updated to the register vu_cycle used for statistical calculation execution time.

[0077] B. The vector calculation unit in the state of issuing a release-wait command calculates the time it takes for the vector calculation unit to issue the release-wait command.

[0078] Since the computation instructions of the vector computing unit are simulated at the cycle level on the behavior-level simulator, i.e., consistent with the finished chip, when the vector computing pipeline parses the current instruction as a wait-release instruction vu.ub.eidma or vu.ub.eodma, this moment is recorded in the register vu_cycle used for statistical calculation execution time. Specifically, when the wait-release instruction is vu.ub.eidma, the cache vu_ub_eidma_fifo, which stores the data required for the handshake between the vector computing unit and the transport unit, is recorded in the register vu_cycle used for statistical calculation execution time; when the wait-release instruction is vu.ub.eodma, the cache vu_ub_eodma_fifo, which stores the data required for the handshake between the vector computing unit and the transport unit, is recorded in the register vu_cycle used for statistical calculation execution time.

[0079] After the behavior-level simulator completes execution, the execution time of the transport unit and the vector calculation unit is determined by the values ​​in three registers: vu_cycle (for calculating execution time), eidma_cycle (for calculating transport execution time), and eodma_cycle (for calculating transport execution time). This allows for the determination of the model's near-realistic execution time on the finished chip. Furthermore, the established caches include eidma_transfer_end_fifo (for storing the transport completion time), eidma_ub_vu_fifo and eodma_ub_vu_fifo (for storing data required for handshaking between the transport unit and the vector calculation unit), eidma_ub_eodma_fifo (for storing data required for handshaking between transport units), and vu_ub_eidma_fifo and vu_ub_eodma_fifo (for storing data required for handshaking between vector calculation units). Recording the start and end times of transport instructions and vector calculation instructions can also help architects optimize the architecture and improve software and hardware performance.

[0080] A second embodiment of the present invention provides a time cost calculation system for a neural network processor, comprising:

[0081] Behavioral simulator

[0082] The first time cost calculation unit is used to calculate the execution time of the material handling unit.

[0083] A second time cost calculation unit used to calculate the execution time of the vector calculation unit;

[0084] The first time cost calculation unit is configured to calculate the transportation time of the transportation unit, the update time of the transportation unit, and the time when the transportation unit issues a release waiting instruction;

[0085] The second time cost calculation unit is configured to perform the calculation time of the vector calculation unit, the update time of the vector calculation unit, and the time when the vector calculation unit issues the release waiting instruction;

[0086] The first time cost calculation unit and the second time cost calculation unit are executed in parallel.

[0087] The time for the transport unit to perform transport, the time for the transport unit to update, and the time for the transport unit to issue a release-wait instruction are calculated based on whether the transport unit is in the transport execution state, the transport unit update state, or the transport unit issues a release-wait instruction. The execution order of the transport unit performing transport, the transport unit updating, and the transport unit issuing the release-wait instruction depends on the software programming method. As an optional embodiment, the execution order can be parallel execution.

[0088] The execution order of the vector computing unit's calculation time, update time, and release-wait instruction issuance time depends on the software programming method. As an optional embodiment, the execution order can be parallel execution.

[0089] As an optional embodiment, the behavioral simulator includes two data transfer units, eidma0 and eidma1, for transferring data from external storage to internal storage, and two data transfer units, eodma0 and eodma1, for transferring data from internal storage to external storage. Multiple instructions are executed in parallel via synchronization instructions, which include at least: eidma transfer instructions for transferring data from external storage to internal storage, vector calculation instructions, and eodma transfer instructions for transferring data from internal storage to external storage.

[0090] In some embodiments, a data transfer unit is involved, including instructions for transferring off-chip data to an on-chip cache and instructions for transferring on-chip stored data to an off-chip cache, with the cycle counts of both calculated in the same way. Specifically, a first time cost calculation unit is configured to: prepare a data transfer instruction register and multiple caches, calculate the data transfer time performed by the data transfer unit, calculate the data transfer update time, and calculate the time when the data transfer unit issues a release-wait instruction.

[0091] Specifically, the first-time cost calculation unit includes: a storage unit, including a transport instruction register and multiple caches, specifically the transport instruction register and multiple caches including:

[0092] The register eidma_cycle is used to track the execution time of the transport process;

[0093] The cache eidma_transfer_end_fifo used to store the time point when the transfer is completed;

[0094] The caches eidma_ub_vu_fifo and eodma_ub_vu_fifo are used to store the data required for the handshake between the transport unit and the vector calculation unit.

[0095] eidma_cycle_bk is used to store the number of implementation cycles when the handling unit completes handling.

[0096] All of the above caches follow the first-in, first-out (FIFO) principle.

[0097] The first time cost calculation unit also includes a judgment module, which is used to determine the state of the transport unit. If the transport unit is in the state of performing transport, updating, or issuing a release waiting instruction, the time calculation module is triggered.

[0098] The first time cost calculation unit also includes a time calculation module, which is used to calculate the execution time of the transport instruction. Specifically, if the transport unit is in the transport execution state, the transport execution time is calculated; if the transport unit is in the update state, the update time is calculated; if the transport unit is in the state of issuing a release waiting instruction, the release waiting instruction issuance time is calculated. The execution order of the above calculations depends on the software programming method. As an optional embodiment, the execution order can be parallel execution.

[0099] The time calculation module is used to calculate the transport time when the transport unit is in the transport state.

[0100] When the EIDMA transfer instruction is executing within the current transfer cycle, the transfer time of the EIDMA transfer instruction is calculated based on two sets of data: the amount of data to be transferred as configured in the EIDMA transfer instruction and the bus bandwidth. The specific calculation method for the transfer time does not affect the implementation of the technical effect in this embodiment. The transfer time is pushed to the end of the cache eidma_transfer_end_fifo, and at the same time, the data in the cache eidma_cycle_bk is recorded in the register eidma_cycle.

[0101] The time calculation module is also used to calculate the execution update time when the transport unit is in an update state:

[0102] The determination that the transfer unit is in the update state is as follows: the transfer unit has not started transfer or has completed transfer (i.e., eidma idle state), and the transfer unit currently has no transfer instruction configuration (e.g., eidma_cfg, eidma_ext_addr, eidma_burst, etc.), and the transfer unit has completed at least one transfer (i.e., the cache eidma_transfer_end_fifo is greater than 1);

[0103] Alternatively, the transport unit eidma receives a release-wait command from the vector calculation unit or the transport unit eodma.

[0104] When the transport unit is in the update state, the value of the header of the cache used to save the transport completion time is updated to the register used to count the transport execution time, and the value of the header of the cache used to save the transport completion time is deleted.

[0105] The time calculation module is also used to calculate the time of issuing the release waiting command state when the handling unit is in the state of issuing the release waiting command:

[0106] When the transport unit is in the state of issuing a release waiting instruction, the actual number of cycles in which the transport unit issues the release waiting instruction is saved to the cache used to store the handshake data required by the execution unit.

[0107] As an optional embodiment, the second time cost calculation unit includes a calculation instruction register and multiple caches, specifically including:

[0108] The register vu_cycle is used for statistical calculations of execution time;

[0109] The caches vu_ub_eidma_fifo and vu_ub_eodma_fifo are used to store the data required for the handshake between the vector calculation unit and the transport unit.

[0110] The real-time cycle register, vu_cycle_bk, is used to store the time when the vector calculation unit completes its calculation.

[0111] All of the above caches follow the first-in, first-out (FIFO) principle.

[0112] The second time cost calculation unit also includes a vector calculation unit judgment module, which is used to judge the state of the vector calculation unit. If the vector calculation unit is in the update state or the state of issuing a release waiting instruction, the vector calculation unit calculation module is triggered.

[0113] The second time cost calculation unit also includes a vector calculation unit calculation module, used to calculate the vector calculation unit instruction execution time. The vector calculation unit instruction execution time includes: calculating the vector calculation unit update time if the vector calculation unit is in an update state, and calculating the vector calculation unit issue release waiting instruction time if the vector calculation unit is in a release waiting instruction state. The execution order of the above calculations depends on the software programming method. As an optional embodiment, the execution order can be parallel execution.

[0114] Specifically, the vector calculation unit calculation module is used to calculate the vector calculation unit update time when the vector calculation unit is in an update state:

[0115] If the vector calculation unit receives a release waiting instruction from the transport unit (including transport unit eidma or transport unit eodma) (see...) Figure 4 If the vector computation unit is in the update state (e.g., vu.wo.eidma or eodma.wo.vu), then the vector computation unit is considered to be in the update state at the current moment, and the update time calculation is performed. This time point is then updated to the register vu_cycle used for statistical calculation execution time.

[0116] Specifically, the vector computing unit calculation module is used to calculate the time when the vector computing unit issues a release-wait instruction when it is in a state of issuing a release-wait instruction:

[0117] Since the computation instructions of the vector computing unit are simulated at the cycle level on the behavior-level simulator, i.e., consistent with the finished chip, when the vector computing pipeline parses the current instruction as a wait-release instruction vu.ub.eidma or vu.ub.eodma, this moment is recorded in the register vu_cycle used for statistical calculation execution time. Specifically, when the wait-release instruction is vu.ub.eidma, the cache vu_ub_eidma_fifo, which stores the data required for the handshake between the vector computing unit and the transport unit, is recorded in the register vu_cycle used for statistical calculation execution time; when the wait-release instruction is vu.ub.eodm, the cache vu_ub_eodma_fifo, which stores the data required for the handshake between the vector computing unit and the transport unit, is recorded in the register vu_cycle used for statistical calculation execution time.

[0118] As an optional embodiment, the time cost calculation system of the neural network processor further includes: a synthesis unit, configured to determine the execution time of the transport unit and the vector calculation unit by means of the values ​​in three registers after the behavior-level simulator has completed execution, namely, register vu_cycle for calculating execution time, register eidma_cycle for calculating transport execution time, and register eodma_cycle for calculating transport execution time, thereby determining the similar real execution time of the model on the finished chip. In addition, the established caches include eidma_transfer_end_fifo for storing the moment when the transfer is completed, eidma_ub_vu_fifo and eodma_ub_vu_fifo for storing data required for the handshake between the transfer unit and the vector computing unit, eidma_ub_eodma_fifo for storing data required for the handshake between transfer units, and vu_ub_eidma_fifo and vu_ub_eodma_fifo for storing data required for the handshake between the vector computing unit and the transfer unit. The records of the start and end times of transfer instructions and vector computing instructions can also help architects optimize the architecture and improve software and hardware performance.

[0119] A third embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0120] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the methods described above.

[0122] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A method for calculating the time cost of a neural network processor, characterized in that, Based on a behavioral simulator, chip instructions are simulated during the chip design phase to optimize the architecture for architectural defects exposed by performance data that is mainly based on time cost during the design phase, and to calculate the execution time of the transport unit and the execution time of the vector calculation unit. The execution time of the calculation and handling unit includes: the handling unit's handling time, the handling unit's update time, and the time when the handling unit issues a release waiting instruction; The execution time of the vector calculation unit includes: the calculation time of the vector calculation unit, the update time of the vector calculation unit, and the time when the vector calculation unit issues the release waiting instruction; The execution time of the computational transport unit and the execution time of the vector computation unit are executed in parallel. The transport time of the eIDMA transport instruction that transports data from outside the chip to the storage inside the chip and the eODMA transport instruction that transports data stored inside the chip to outside the chip is the behavior-level transport time, not the actual transport time of the instruction on the chip; the vector computation instruction has a cycle-level transport time, which is the actual time of the instruction on the chip.

2. The method for calculating the time cost of a neural network processor according to claim 1, characterized in that, The time cost of the transport unit in the transport execution state is calculated based on the transport unit being in the transport execution state. The time cost calculation of the transport unit in the transport execution state includes: pushing the number of cycles required for the transport unit to perform the current transport to the end of the cache used to store the transport completion time point, as the end time of the current transport.

3. The method for calculating the time cost of a neural network processor according to claim 1, characterized in that, The update time of the transport unit is calculated based on the fact that the transport unit is in an update state. The state of the transport unit being in an update state is as follows: The transport unit has not started or completed transport, and the transport unit currently has no transport instruction configured, and the transport unit has completed at least one transport operation; Alternatively, the transport unit receives a release waiting command from the vector calculation unit or from another transport unit in the opposite direction to the transport unit's transport data.

4. The method for calculating the time cost of a neural network processor according to claim 3, characterized in that, The time cost calculation for the transport unit in the update state includes: updating the value of the header of the cache used to store the transport completion time to the register used to count the transport execution time, and deleting the value of the header of the cache used to store the transport completion time.

5. The method for calculating the time cost of a neural network processor according to claim 1, characterized in that, The calculation of the time it takes for the transport unit to issue a release waiting instruction is based on the state in which the transport unit is in the state of issuing a release waiting instruction. The time cost calculation for the transport unit being in the state of issuing a release waiting instruction includes: saving the actual number of cycles in which the transport unit issues a release waiting instruction to a cache used to store the data required for handshaking by the execution unit.

6. The method for calculating the time cost of a neural network processor according to claim 1, characterized in that, The update time of the vector computing unit is calculated based on the vector computing unit being in an update state. The time cost calculation of the vector computing unit being in an update state includes: updating the data in the cache used to store the data required for the handshake between the vector computing unit and the transport unit to the register used to count the execution time.

7. The method for calculating the time cost of a neural network processor according to claim 1, characterized in that, The calculation of the time cost of the vector computing unit issuing the release-wait instruction based on the vector computing unit being in the release-wait instruction state includes: When the pipeline parsing of the vector computing unit obtains that the current instruction is to issue a release-wait instruction, the value of the register used for statistical execution time is recorded at the current moment.

8. The method for calculating the time cost of a neural network processor according to any one of claims 1-7, characterized in that, The transfer unit includes instructions for transferring data from outside the chip to storage inside the chip, and instructions for transferring data stored inside the chip to outside the chip.

9. A time cost calculation system for a neural network processor, characterized in that, include: Behavioral simulator It is used to simulate chip instructions during the chip design phase, and to perform architecture tuning based on performance data exposed during the design phase, which is mainly based on time cost. The first time cost calculation unit is used to calculate the execution time of the material handling unit. A second time cost calculation unit used to calculate the execution time of the vector calculation unit; The first time cost calculation unit is configured to calculate the transportation time of the transportation unit, the update time of the transportation unit, and the time when the transportation unit issues a release waiting instruction; The second time cost calculation unit is configured to perform the calculation time of the vector calculation unit, the update time of the vector calculation unit, and the time when the vector calculation unit issues the release waiting instruction; The first time cost calculation unit and the second time cost calculation unit are executed in parallel. The eIDMA transport instruction, which moves data from outside the chip to the storage inside the chip, has a transport time at the behavior level, which is not the actual transport time of the instruction on the chip. The vector calculation instruction has a transport time at the cycle level, which is the actual time of the instruction on the chip.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images