A data processing method, device and terminal of a neural network simulator

Abstract

The application belongs to the technical field of data processing, and particularly relates to a data processing method and device of a neural network simulator and a terminal, wherein the method comprises the following steps: acquiring instruction data; carrying out data of a source end to a destination end in a transaction level carrying manner according to first carrying parameters carried by a first carrying instruction; carrying data of the destination end to a cache according to second carrying parameters carried by a second carrying instruction; if the cache is not empty, performing the granularity operation based on the data in the cache to obtain a period level data processing result; the period level accurate instruction operation and the transaction level fuzzy data carrying are realized, the instruction of the neural network simulator can be calculated according to the period level, the consistency and accuracy with the hardware are maintained, the period level dependence of the data carrying is optimized, and the complexity of the neural network simulator is reduced.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to a data processing method, device and terminal for a neural network simulator. Background Art

[0002] With the development of artificial intelligence and big data technologies, neural network simulators have demonstrated significant advantages in processor microarchitecture design, TVM tool chain development and promotion, and RTL development and verification.

[0003] However, as neural network processors process large amounts of data, multiple data dimensions, and complex and diverse calculation methods, the current data processing methods of neural network simulators can no longer meet usage requirements. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method, device, terminal and computer-readable storage medium for a neural network simulator, which can reduce the complexity of the neural network simulator, so that the neural network simulator can play a great role in the big data handling, high computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification and other aspects of the neural network processor.

[0005] A first aspect of an embodiment of the present application provides a data processing method for a neural network simulator, comprising:

[0006] Acquire instruction data; the instruction data includes a first transport instruction carrying a first transport parameter, a second transport instruction carrying a second transport parameter, and a granularity calculation instruction;

[0007] Transferring the data from the source end to the destination end using a transaction-level transfer method according to the first transfer parameter carried by the first transfer instruction;

[0008] Moving the data of the destination end to the cache with cycle-level accuracy according to the second moving parameter carried by the second moving instruction;

[0009] If the cache is not empty, granularity calculations are performed based on the data in the cache to obtain cycle-level data processing results.

[0010] Optionally, based on the data processing method of a neural network simulator provided in the first aspect above, in a first possible implementation of the present application, the above-mentioned transferring the data from the source end to the destination end using a transaction-level transfer method includes:

[0011] Based on the communication handshake between the source end and the destination end, the data of the source end is transferred to the destination end in a transaction-level manner.

[0012] Optionally, based on the data processing method of a neural network simulator provided in the first aspect above, in a second possible implementation of the present application, the second transport parameter includes an operation mode corresponding to the currently transported data;

[0013] The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter carried by the second transfer instruction includes:

[0014] Calculate the actual data volume corresponding to the data on the destination end required for the granularity calculation according to the operation mode corresponding to the currently transported data;

[0015] According to the actual data volume and the data volume transferred from the source end to the destination end, determining whether the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation;

[0016] If the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation, the data of the destination end is transferred to the cache at a cycle-level accuracy according to the second transfer parameter.

[0017] Optionally, based on the data processing method of a neural network simulator provided in the first aspect above, in a third possible implementation of the present application, the second transport parameter includes a cutting parameter and an operation mode corresponding to the currently transported data;

[0018] The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter carried by the second transfer instruction includes:

[0019] cutting the data of the destination end according to the cutting parameter in the second transport parameter with cycle-level accuracy to obtain cutting data;

[0020] Calculate the cutting data according to the calculation mode corresponding to the currently transported data to obtain the target data required for the granularity calculation;

[0021] The target data is moved to the cache.

[0022] Optionally, based on the data processing method of a neural network simulator provided in the first aspect above, and the first, second, and third possible implementations above, in a fourth possible implementation of the present application, when the target data required for the granularity calculation is partial data in the matrix data at the destination end, the second transport parameter includes the first position coordinates corresponding to each data in the partial data;

[0023] The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter includes:

[0024] The data corresponding to the first position coordinates in the matrix data of the destination end is moved to the cache.

[0025] Optionally, based on the data processing method of a neural network simulator provided in the first aspect above, and the first, second, and third possible implementations above, in a fifth possible implementation of the present application, when the target data required for the granularity calculation is a value of a pre-winograd transformation, the second transport parameter includes the second position coordinates of the data in the 4*4 data table required for the pre-winograd transformation;

[0026] The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter includes:

[0027] The 4*4 data table stored in the destination end is read according to the second position coordinates, and the value of the pre-winograd transformation is calculated based on the 4*4 data table, and the value of the pre-winograd transformation is moved to the cache.

[0028] Optionally, based on the data processing method of a neural network simulator provided in the first aspect above, and the first, second, and third possible implementations above, in a sixth possible implementation of the present application, before performing the granularity operation based on the data in the cache if the cache is not empty to obtain a cycle-level data processing result, the method includes:

[0029] Obtaining the value of the wrap flag of the cache, and the read address and write address of the cache;

[0030] Whether the cache is empty is determined according to whether the read address and the write address of the cache coincide with each other and the value of the wrap flag of the cache.

[0031] A second aspect of an embodiment of the present application provides a data processing device for a neural network simulator, comprising:

[0032] an acquisition unit, configured to acquire instruction data; the instruction data including a first transport instruction carrying a first transport parameter, a second transport instruction carrying a second transport parameter, and a granularity calculation instruction;

[0033] a first transfer unit, configured to transfer the data from the source end to the destination end in a transaction-level manner according to the first transfer parameter carried in the first transfer instruction;

[0034] a second transfer unit, configured to transfer the data of the destination end to the cache with cycle-level accuracy according to a second transfer parameter carried in the second transfer instruction;

[0035] A data processing unit is configured to, if the cache is not empty, perform granular operations based on the data in the cache to obtain cycle-level data processing results.

[0036] Optionally, based on the data processing device of the neural network simulator provided in the second aspect above, in a first possible implementation of the present application, the first transport unit is further configured to:

[0037] Based on the communication handshake between the source end and the destination end, the data of the source end is transferred to the destination end in a transaction-level manner.

[0038] Optionally, based on the data processing device of the neural network simulator provided in the second aspect above, in a second possible implementation of the present application, the second transport parameter includes an operation mode corresponding to the currently transported data;

[0039] The second transport unit is further used for:

[0040] Calculate the actual data volume corresponding to the data on the destination end required for the granularity calculation according to the operation mode corresponding to the currently transported data;

[0041] According to the actual data volume and the data volume transferred from the source end to the destination end, determining whether the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation;

[0042] If the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation, the data of the destination end is transferred to the cache at a cycle-level accuracy according to the second transfer parameter.

[0043] Optionally, based on the data processing device of a neural network simulator provided in the second aspect above, in a third possible implementation of the present application, the second transport parameter includes a cutting parameter and an operation mode corresponding to the currently transported data;

[0044] The second transport unit is further used for:

[0045] cutting the data of the destination end according to the cutting parameter in the second transport parameter with cycle-level accuracy to obtain cutting data;

[0046] Calculate the cutting data according to the calculation mode corresponding to the currently transported data to obtain the target data required for the granularity calculation;

[0047] The target data is moved to the cache.

[0048] Optionally, based on the data processing device of a neural network simulator provided in the second aspect above, and the first, second, and third possible implementations above, in a fourth possible implementation of the present application, when the target data required for the granularity calculation is partial data in the matrix data at the destination end, the second transport parameter includes the first position coordinates corresponding to each data in the partial data;

[0049] The second transport unit is further used for:

[0050] The data corresponding to the first position coordinates in the matrix data of the destination end is moved to the cache.

[0051] Optionally, based on the data processing device of a neural network simulator provided in the first aspect above, and the first, second, and third possible implementations above, in a fifth possible implementation of the present application, when the target data required for the granularity calculation is a value of a pre-winograd transformation, the second transport parameter includes the second position coordinates of the data in the 4*4 data table required for the pre-winograd transformation;

[0052] The second transport unit is further used for:

[0053] The 4*4 data table stored in the destination end is read according to the second position coordinates, and the value of the pre-winograd transformation is calculated based on the 4*4 data table, and the value of the pre-winograd transformation is moved to the cache.

[0054] Optionally, based on the data processing method of a neural network simulator provided in the second aspect above, and the first, second, and third possible implementations above, in a sixth possible implementation of the present application, the data processing unit is further configured to:

[0055] Before performing the granularity operation based on the data in the cache to obtain a cycle-level data processing result if the cache is not empty, obtaining a value of a wrap flag of the cache, and a read address and a write address of the cache;

[0056] Whether the cache is empty is determined according to whether the read address and the write address of the cache coincide with each other and the value of the wrap flag of the cache.

[0057] A third aspect of an embodiment of the present application provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0058] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0059] In an embodiment of the present application, the data from the source end is transferred to the destination end using a transaction-level transfer method, and the data from the destination end is transferred to a cache with cycle-level accuracy, and then the granularity operation is performed based on the data in the cache and the granularity operation instruction. The neural network simulator of the present application mixes the cycle-level and transaction-level simulator design methods, realizes cycle-level precise instruction operations and transaction-level fuzzy data transfer, and also enables the instructions of the neural network simulator to be calculated at the cycle level, maintaining consistency and accuracy with the hardware, and also optimizes the cycle-level dependence of data transfer, reducing the complexity of the neural network simulator, and enabling the neural network simulator to play a great role in the big data transfer, high computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc. of the neural network processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 This is a schematic diagram of the implementation flow of a data processing method of a neural network simulator provided in an embodiment of the present application;

[0062] Figure 2 This is a schematic diagram of data cutting provided by an embodiment of the present application;

[0063] Figure 3 This is a schematic diagram of a first specific implementation flow of step 103 of a data processing method for a neural network simulator provided in an embodiment of the present application;

[0064] Figure 4 This is a schematic diagram of a second specific implementation flow of step 103 of the data processing method of a neural network simulator provided in an embodiment of the present application;

[0065] Figure 5 Schematic diagram of matrix data handling provided by an embodiment of the present application;

[0066] Figure 6 This is a schematic diagram of determining whether a cache is empty, provided in an embodiment of the present application;

[0067] Figure 7Schematic diagram of the structure of the data processing device of the neural network simulator provided in an embodiment of the present application;

[0068] Figure 8 It is a schematic diagram of the structure of the terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0070] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0071] It should also be understood that the terms used in the description of the present application are only used to describe specific embodiments and are not intended to limit the present application.

[0072] A neural network simulator is a technical tool that provides a model or research prototype for an artificial neural network. Generally speaking, neural network simulators serve as a resource for researchers studying how neural networks work. Various types of data collection help simulators assess what's happening within artificial neural networks. To effectively demonstrate how neural networks operate to researchers, neural network simulators often include versatile visual interfaces that display data graphically. Many of these simulators have multiple windows, which can be labeled to easily identify data modules.

[0073] Traditional neural network simulators use either cycle-level or transaction-level modeling. However, simple cycle-level modeling significantly impacts the modeling complexity, runtime, and toolchain usage of neural network processors. Transaction-level modeling alone can only be used for approximate simulation and preliminary evaluation, failing to achieve true consistency with actual hardware. Neither of these two types of neural network simulators can meet the demands of neural network processors, which are increasingly processing large amounts of data, multiple data dimensions, and complex and diverse computational methods.

[0074] Based on this, the embodiments of the present application provide a data processing method, device and terminal for a neural network simulator, which mixes cycle-level and transaction-level simulator design methods, can achieve cycle-level precise instruction operations and transaction-level fuzzy data handling, maintain consistency and accuracy with the hardware, and can play a great role in the big data handling, high computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification and other aspects of neural network processors.

[0075] In order to illustrate the technical solution of the present application, specific embodiments are provided below.

[0076] like Figure 1 The following is a schematic diagram illustrating a flow chart for implementing a data processing method for a neural network simulator provided in an embodiment of the present application. This method is applied to a terminal and can be executed by a data processing device of the neural network simulator configured on the terminal. The terminal can be an intelligent terminal such as a computer or server. The data processing method for the neural network simulator can include steps 101 to 104, which are described in detail as follows:

[0077] Step 101: Obtain instruction data.

[0078] In the embodiment of the present application, the instruction data may include a first transport instruction carrying a first transport parameter, a second transport instruction carrying a second transport parameter, and a granularity operation instruction. Furthermore, the instruction data may be instruction data obtained from an instruction control flow by different modules in the neural network simulator.

[0079] For example, the above-mentioned first transport instruction is the instruction data obtained by the exdma module in the neural network simulator from the instruction control flow, the above-mentioned second transport instruction is the instruction data obtained by the xdma module in the neural network simulator from the instruction control flow, and the above-mentioned granularity operation instruction is the instruction data obtained by the granularity operation module cube in the neural network simulator from the instruction control flow.

[0080] The first transfer instruction is used to transfer data from the source end to the destination end in a transaction-level transfer manner, and the second transfer instruction is used to transfer data from the destination end to the cache in a cycle-level precision manner.

[0081] The above-mentioned source end and destination end can be different modules in the neural network simulator. For example, the above-mentioned source end and destination end can be exdma modules in the neural network simulator, such as eidma module and eodma, and the above-mentioned destination end can be xdma modules in the neural network simulator, such as idma module and odma.

[0082] In some embodiments of the present application, optionally, the first transfer parameter may include the amount of data transferred from the source end to the destination end each time, that is, the handshake granularity, for example, 1Kb or 5Kb, and the total amount of data currently required to be transferred.

[0083] The second transport parameter may include a cutting parameter and an operation mode corresponding to the current transport data.

[0084] The above operation modes may include winograd operation mode, matrix operation mode, padding mode, deconvlution mode and dilated mode.

[0085] The above-mentioned cutting parameters may include the cutting parameters H, W, and D for cutting the data in three directions: height (zeta), width (epsilon = ci), and depth (dense), as well as the dense number, zeta number, epsilon number, number of sliding windows in the kernel dense dimension, winograd_loop16, and the number of weight parameter reuses. Among them, when the operation mode corresponding to the current data being transported is the winograd operation mode, winograd_loop16 = 16, indicating 16 piexl loops; when the operation mode corresponding to the current data being transported is the non-winograd operation mode, winograd_loop16 = 1.

[0086] For example, Figure 2 As shown, when H=16, W=32, and D=8, the data at the destination end is cut to obtain data ci0, and by cutting along the ci direction, data ci0 to data ci7 can be obtained in sequence.

[0087] Furthermore, the data at the source end may be image data or parameter data, which is not limited in this application.

[0088] Step 102 : According to the first transfer parameter carried in the first transfer instruction, the data at the source end is transferred to the destination end in a transaction-level transfer manner.

[0089] In an embodiment of the present application, moving data from the source end to the destination end belongs to the data preparation process, which adopts transaction-level moving and is loosely coupled. It can be independent of the cycle-level moving process, that is, it is independent of the cycle-level moving process of moving the data from the destination end to the cache, so that the data from the destination end can be prepared in advance, reducing the waiting time when moving the data from the destination end to the cache.

[0090] Optionally, in some embodiments of the present application, in the above step 102, transferring the data from the source end to the destination end using a transaction-level transfer method may include: based on a communication handshake between the source end and the destination end, transferring the data from the source end to the destination end using a transaction-level transfer method.

[0091] In an embodiment of the present application, the communication handshake between the source end and the destination end means that before the data of the destination end is transferred to the cache destination end with cycle-level accuracy, the destination end needs to wait for the source end to write the data to the destination end according to the first transfer parameter, and after the source end writes the data to the destination end, it needs to notify the destination end that the data is ready.

[0092] Specifically, before the destination reads data from its own storage space dm, it sends a wait signal dest_wo_src to the source, waiting for the source to write the data to the destination's dm. After the source writes the data to the destination, it sends an enable signal src_ub_dest to the destination, which then reads the data from dm. After completing the data read, it sends an enable signal dest_ub_src to the source, notifying it that the data space has been released.

[0093] It should be noted that in some embodiments of the present application, before the destination end reads data from DM, it can continue to send a wait signal dest_wo_src to the source end, and the source end will accumulate the count and store the data in DM in advance. There is no need to wait for the destination end to finish reading the data in DM before continuing to send an enable signal dest_wo_src to the source end, which reduces the waiting time for data reading and improves the efficiency of data storage and reading.

[0094] Step 103 : The data at the destination end is transferred to the cache with cycle-level accuracy according to the second transfer parameter carried in the second transfer instruction.

[0095] In the embodiment of the present application, the cache refers to the storage space corresponding to the granularity calculation. The above-mentioned transfer of the data from the destination end to the cache belongs to the periodic level transfer and calculation.

[0096] Specifically, in some embodiments of the present application, Figure 3 As shown, in the above step 103, the data at the destination end is moved to the cache with cycle-level accuracy according to the second moving parameter carried by the second moving instruction, which may include the following steps 301 to 303.

[0097] Step 301 : Cut the data at the destination end with cycle-level accuracy according to the cutting parameters in the second transport parameters to obtain cut data.

[0098] As shown in step 101, the cutting parameters can include cutting parameters H, W, D for cutting data from three directions of height (zeta), width (epsilon = ci), and depth (dense), and the number of times of dense, the number of times of zeta, the number of times of epsilon, the number of times of kernel dense dimension sliding window, winograd_loop16, the number of times of weight parameter multiplexing, and the like.

[0099] In step 302, the cutting data is operated according to the operation mode corresponding to the current data to be carried, to obtain target data required by the granularity operation.

[0100] In the embodiment of the present application, the operation mode can include padding mode, deconvlution mode, and dilated mode.

[0101] In step 302, the cutting data is operated according to the operation mode corresponding to the current data to be carried, to obtain target data required by the granularity operation. The operation can include the following cases: if the operation mode corresponding to the current data to be carried is the deconvlution mode, the cutting data is calculated by inserting 0 to obtain the target data required by the granularity operation; if the operation mode corresponding to the current data to be carried is the dilated mode, the data after the address jump of the cutting data is calculated to obtain the target data required by the granularity operation.

[0102] In step 303, the target data is carried to the cache.

[0103] In the embodiment of the present application, after the target data required by the granularity operation is calculated in the process of carrying data, the target data is directly carried to the cache, without intermediate caching.

[0104] In step 104, if the cache is not empty, the granularity operation is performed based on the data in the cache to obtain a period-level data processing result.

[0105] In the embodiment of the present application, when the cache is not empty, it indicates that the data for the granularity operation has been prepared, and thus the granularity operation, i.e., the cube operation, can be directly performed based on the data in the cache. The calculation is a period-level accurate calculation, and thus the output of the period-level accurate result is realized, which can be used for RTL data calculation comparison.

[0106] In an embodiment of the present application, the data from the source end is transferred to the destination end using a transaction-level transfer method, and the data from the destination end is transferred to a cache with cycle-level accuracy, and then the granularity operation is performed based on the data in the cache and the granularity operation instruction. Thus, the neural network simulator of the present application mixes the cycle-level and transaction-level simulator design methods, realizes cycle-level precise instruction operations and transaction-level fuzzy data transfer, maintains consistency and accuracy with the hardware, and can play a great role in the big data transfer, high computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc. of the neural network processor.

[0107] Exemplarily, when the above-mentioned neural network simulator is a neural network simulator for simulating an artificial neural network model for face recognition, the data at the above-mentioned source end may be facial image data, and the first transport parameter carried by the above-mentioned first transport instruction may include the data volume of the facial image data. The first transport instruction is used to transport the facial image data at the source end to the destination end in a transaction-level transport manner. The second transport parameter carried by the above-mentioned second transport instruction may include cutting parameters H0, W0, and D0 for cutting the facial image data in three directions of data height (zeta), width (epsilon=ci), and depth (dense), which are used to cut the facial image data at the destination end with cycle-level accuracy to obtain cut data, so as to realize the transport of the facial image data at the destination end to the cache with cycle-level accuracy, and when the cache is not empty, perform granular operations based on the data in the cache, that is, cube operations, to finally obtain cycle-level face recognition results.

[0108] Among them, the above-mentioned cube operation refers to using a neural network algorithm to operate on the facial image data in the cache to obtain a face classification result. For example, the face recognition result corresponding to the above-mentioned facial image data is Zhang San's face or Li Si's face.

[0109] The above-mentioned neural network algorithms may include Layer Cubing algorithm (layer-by-layer algorithm), By-layer Spark Cubing algorithm, Fast (in-mem) Cubing algorithm, that is, "By Segment" or "By Split" algorithm and other neural network algorithms, which are not limited in this application.

[0110] This application transfers the facial image data from the source end to the destination end using a transaction-level transfer method, and transfers the facial image data from the destination end to a cache with cycle-level accuracy, and then performs granular operations based on the facial image data in the cache and the granular operation instructions. This allows the neural network simulator of this application to mix cycle-level and transaction-level simulator design methods, achieve cycle-level precise instruction operations and transaction-level fuzzy data transfer, and maintain consistency and accuracy with the hardware in the process of simulating artificial neural network models to achieve facial image recognition. It can play a great role in big data transfer, high computing power characteristics, architecture evaluation, instruction set tool chain development, RTL verification, etc. of neural network processors.

[0111] It should also be noted that the above-mentioned neural network simulator can also be a neural network simulator used to simulate the working process of artificial neural network models for other application scenarios. For example, the above-mentioned neural network simulator can also be a simulator used to simulate the working process of artificial neural network models for license plate recognition, obstacle recognition or animal classification. This application does not impose any restrictions on this.

[0112] It should also be noted that, in addition to image data, the data at the source end may also be different types of data such as voice data, and this application does not impose any restrictions on the data type of the data at the source end.

[0113] Optionally, when the data at the source end is voice data, the neural network simulator can be a neural network simulator for analyzing and processing voice data. For example, the neural network simulator can be a neural network simulator for classifying, denoising, and other processing of voice data.

[0114] Optionally, in some embodiments of the present application, in the above step 103, in the process of transferring the data of the destination end to the cache according to the cycle-level accuracy, the amount of data transferred from the source end to the destination end and the actual amount of data required for the granularity calculation can be synchronized first to determine whether there is data that can be transferred from the destination end, and then the data is transferred. Specifically, Figure 4 As shown, the above step 103 can also be implemented through the following steps 401 to 403.

[0115] Step 401 : Calculate the actual data volume corresponding to the data at the destination end required for the granularity operation according to the operation mode corresponding to the currently transported data.

[0116] In an embodiment of the present application, the operation mode corresponding to the currently transported data may include a winograd operation mode, a matrix operation mode, a padding mode, a deconvlution mode, and a dilated address jump mode.

[0117] Since the target data required by the granularity operation can be obtained according to the operation mode corresponding to the current data carrying, rather than the data in the destination directly, the data amount corresponding to the target data required by the granularity operation can be inconsistent with the real data amount corresponding to the data in the destination required by the granularity operation. For example, the data amount corresponding to the target data required by the granularity operation is greater than the real data amount corresponding to the data in the destination required by the granularity operation. And since the data amount of the target data is known and is a fixed data amount, the real data amount corresponding to the data in the destination required by the granularity operation can be deduced according to the operation mode corresponding to the current data carrying.

[0118] For example, when the operation mode corresponding to the current data carrying is the insert 0 mode, the real data amount corresponding to the data in the destination required by the granularity operation can be calculated according to the operation mode and the data amount of the target data.

[0119] Step 402: judging whether the data amount in the destination is greater than or equal to the real data amount required by the granularity operation according to the real data amount and the data amount of the data carried from the source to the destination.

[0120] In the embodiment of the present application, the calculated real data amount and the data amount of the data carried from the source are matched to determine whether the data amount of the data carried from the source to the destination reaches the real data amount required by the granularity operation. When it does not, the data carrying from the source to the destination is waited.

[0121] Step 403: if the data amount in the destination is greater than or equal to the real data amount required by the granularity operation, the data in the destination is carried to the cache according to the second carrying parameter with the accuracy of the cycle level.

[0122] When the data amount in the destination meets the real data amount required by the granularity operation, it indicates that the data amount in the destination has reached the real data amount required by the granularity operation, and thus the data carrying can be performed.

[0123] In the embodiment of the present application, since it is only necessary to synchronize the amount of data transferred from the source end to the destination end and the actual amount of data required for granular calculations before transferring the data of the destination end to the cache at the cycle-level accuracy, there is no need for synchronization of the intermediate process at the cycle-level. This allows the transaction-level transfer process of transferring the data from the source end to the destination end to be independent of the cycle-level transfer process of transferring the data from the destination end to the cache. The data of the destination end can be prepared in advance, reducing the waiting time when transferring the data of the destination end to the cache. In addition, in the process of transferring data, the embodiment of the present application calculates the data that needs to jump addresses or fill in modes such as padding, deconvlution, and dilated while transferring the real and valid DM data, thereby avoiding the access to the DDR cache and bandwidth in the middle of the data, thereby improving the efficiency of data transfer.

[0124] Optionally, in some embodiments of the present application, the data from the source end can be moved to the destination end by adopting a transaction-level moving method, and when the data from the destination end is moved to the cache with cycle-level accuracy, by abstracting the data sets that depend on operations in different modes, such as abstracting the data sets that depend on operations in the transpose operation mode and the winograd operation mode, the system complexity is reduced for the operation-level iteration of complex signals in the RTL cycle-level scenario, thereby reducing the complexity of cycle-level modeling and greatly improving the performance of the neural network simulator.

[0125] Specifically, in some embodiments of the present application, when the target data required for the above-mentioned granularity operation is part of the matrix data of the destination end, the above-mentioned second transport parameter may include the first position coordinates corresponding to each data in the part of the data; in the above-mentioned step 103, the data of the destination end is transported to the cache with periodic level accuracy according to the second transport parameter, which may include: transporting the data corresponding to the first position coordinates in the matrix data of the destination end to the cache.

[0126] For example, if the target data required for the granularity calculation is the first row of data after the matrix data at the destination end is transposed, then the above-mentioned second transfer parameter may include the first position coordinates corresponding to each data in this row of data in the matrix data at the destination end, and then select this row of data from the matrix data at the destination end according to the first position coordinates and transfer it to the cache.

[0127] Specifically, such as Figure 5 As shown, when the matrix data at the destination end is 16*16 matrix data, if the target data required for the granularity calculation is the first column of the matrix data, the 16 data contained in this column can be selected through the first position coordinate and moved to the cache.

[0128] It should be noted that, since the data handling method of traditional neural network simulators needs to be synchronized with the hardware, it is impossible to abstractly process the matrix data containing multiple rows and columns according to the operation mode corresponding to the currently handled data to obtain the data that the operation mode depends on, or it is necessary to perform complex operation iterations to obtain the data. For example, for the above-mentioned 16*16 matrix data, it is impossible to select the 16 data contained in this column through the first position coordinate. Therefore, there are problems of high data operation complexity and high system complexity.

[0129] This application inputs a data instruction loop parameter matrix and transposes special rows in the matrix. For example, by transposing the coordinates (x, y) of the data in the matrix, x=y, y=x is made, and the data required for granular calculations is obtained. There is no need to synchronize data with the hardware, and the data handling is decoupled, which effectively reduces the system complexity for the computational level iteration of complex signals in RTL cycle-level scenarios, reduces the complexity of cycle-level modeling, and greatly improves the performance of the neural network simulator.

[0130] Optionally, in some embodiments of the present application, when the target data required for the above-mentioned granularity calculation is the value of the pre-winograd transformation, the second transport parameter may include the second position coordinates of the data in the 4*4 data table required for the pre-winograd transformation.

[0131] In the above step 103, the data of the destination end is moved to the cache with cycle-level accuracy according to the second transfer parameter, which may include: reading the 4*4 data table stored at the destination end according to the second position coordinates, and calculating the value of the pre-winograd transformation based on the 4*4 data table, and moving the value of the pre-winograd transformation to the cache.

[0132] For example, as shown in Table 1 below, according to the coordinates of the first data d0-d2-d8+d10 in the 4*4 data table required for the Winograd pre-transformation, a 4*4 data table is obtained by shifting 3 coordinates downward and rightward, and the value of the Winograd pre-transformation is calculated based on the 4*4 data table.

[0133] Table 1:

[0134]

[0135] It should be noted that, since the data handling method of the traditional neural network simulator needs to be synchronized with the hardware, it can only read one data in the 4*4 data table at a time, and cannot read the entire 4*4 data table stored at the destination according to the second position coordinate, or it requires complex calculation iterations to obtain the data. Therefore, there are problems of high data calculation complexity and high system complexity.

[0136] This application obtains a 4*4 data table by shifting the coordinates of the first data d0-d2-d8+d10 in the 4*4 data table required by the Winograd pre-transformation downward and right by 3 coordinates. This does not require handshaking with the hardware cache, and can decouple data transportation, effectively reducing the system complexity for the computational-level iteration of complex signals in RTL cycle-level scenarios. While reducing the complexity of cycle-level modeling, it significantly improves the performance of the neural network simulator.

[0137] The neural network simulator instruction set of this application is calculated at the cycle level, maintaining consistency and accuracy with the hardware. Data transfer is carried out at the transaction level. Through the basic handshake granularity, it abstracts the data sets that depend on the calculation of scenarios such as transposition and winograd algorithm, padding, deconvlution, dilated, etc. in different modes, reduces the system complexity of the register transfer level, and iterates the complex signal operation level of the cycle-level scenario, reduces the complexity of cycle-level modeling, and greatly improves the system performance.

[0138] Optionally, in some implementations of the present application, before the above step 104, it may be checked whether the cache is empty.

[0139] Specifically, the cache can be determined to be empty by obtaining the value of the wrap flag in the cache and whether the read address and write address coincide. If the cache is empty, data is waited for to be written. If it is not empty, the granular operation is performed based on the data in the cache to obtain the cycle-level data processing result. The cache uses a first-in-first-out data reading and writing method, and the wrap flag value can be obtained through the cube operation module through read and write interaction with the destination end.

[0140] For example, when the destination writes data to the cache and the cube operation reads data from the cache, Figure 6 As shown in (a), when the wrap flag ring_flag = 0, it means that the cache read and write are in a non-wrapped state. If the read and write addresses are the same, it means that the cache is empty. Figure 6 As shown in (b), when the wrap flag ring_flag = 1, it indicates that the read and write of the cache has wrapped. In this case, the read and write addresses are the same, indicating that the cache is full.

[0141] In an embodiment of the present application, the value of the above-mentioned wrap flag and the read address and write address of the cache are used to determine whether the cache is empty, which simplifies the handshake process of the cycle-level cube operation, improves the operation efficiency, reduces the complexity of the neural network simulator, and makes the cycle-level cube operation and hardware settings independent of each other.

[0142] Optionally, in some embodiments of the present application, after performing the granularity operation based on the data in the cache and obtaining the periodic data processing result, the result can also be stored in the destination end and provided to other granularity operations.

[0143] In an embodiment of the present application, the data from the source end is transferred to the destination end in a transaction-level manner, and the data from the destination end is transferred to a cache with cycle-level accuracy, and then the granularity operation is performed based on the data in the cache and the granularity operation instruction. Thus, the neural network simulator of the present application mixes the cycle-level and transaction-level simulator design methods, realizes cycle-level precise instruction operations and transaction-level fuzzy data transfer, so that the instructions of the neural network simulator can be calculated at the cycle level, maintaining consistency and accuracy with the hardware, and optimizing the cycle-level dependency of data transfer, thereby reducing the complexity of the neural network simulator, and can play a great role in the aspects of big data transfer, large computing power characteristics, architecture evaluation, instruction set tool chain development, and RTL verification of neural network processors.

[0144] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders.

[0145] Figure 7 A structural schematic diagram of a data processing device 700 of a neural network simulator provided in an embodiment of the present application is shown, which includes an acquisition unit 701, a first transport unit 702, a second transport unit 703 and a data processing unit 704.

[0146] An acquisition unit 701 is configured to acquire instruction data; the instruction data includes a first transport instruction carrying a first transport parameter, a second transport instruction carrying a second transport parameter, and a granularity calculation instruction;

[0147] A first transfer unit 702 is configured to transfer the data from the source end to the destination end in a transaction-level manner according to the first transfer parameter carried in the first transfer instruction;

[0148] A second transfer unit 703 is configured to transfer the data of the destination end to the cache with cycle-level accuracy according to a second transfer parameter carried in the second transfer instruction;

[0149] The data processing unit 704 is configured to, if the cache is not empty, perform the granularity operation based on the data in the cache to obtain a cycle-level data processing result.

[0150] In some embodiments of the present application, the first transport unit 702 is further configured to:

[0151] Based on the communication handshake between the source end and the destination end, the data of the source end is transferred to the destination end in a transaction-level manner.

[0152] In some embodiments of the present application, the second transport parameter includes an operation mode corresponding to the data currently being transported; the second transport unit 703 is further configured to:

[0153] Calculate the actual data volume corresponding to the data on the destination end required for the granularity calculation according to the operation mode corresponding to the currently transported data;

[0154] According to the actual data volume and the data volume transferred from the source end to the destination end, determining whether the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation;

[0155] If the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation, the data of the destination end is transferred to the cache at a cycle-level accuracy according to the second transfer parameter.

[0156] In some embodiments of the present application, the second transport parameter includes a cutting parameter and an operation mode corresponding to the currently transported data;

[0157] The second transport unit 703 is further used for:

[0158] Cutting the data of the destination end according to the cutting parameter in the second transport parameter with cycle-level accuracy to obtain cutting data;

[0159] Calculate the cutting data according to the calculation mode corresponding to the currently transported data to obtain the target data required for the granularity calculation;

[0160] The target data is moved to the cache.

[0161] In some embodiments of the present application, when the target data required for the granularity calculation is partial data in the matrix data at the destination end, the second transport parameter includes the first position coordinates corresponding to each data in the partial data; and the second transport unit 703 is further configured to:

[0162] The data corresponding to the first position coordinates in the matrix data of the destination end is moved to the cache.

[0163] In some embodiments of the present application, when the target data required for the granularity calculation is a value of a pre-winograd transformation, the second transport parameter includes the second position coordinates of the data in the 4*4 data table required for the pre-winograd transformation; the second transport unit 703 is further configured to:

[0164] The 4*4 data table stored in the destination end is read according to the second position coordinate, and the value of the pre-winograd transformation is calculated based on the 4*4 data table, and the value of the pre-winograd transformation is moved to the cache.

[0165] In some embodiments of the present application, the data processing unit is further configured to:

[0166] Before performing the granularity operation based on the data in the cache to obtain a cycle-level data processing result if the cache is not empty, obtaining a value of a wrap flag of the cache, and a read address and a write address of the cache;

[0167] Whether the cache is empty is determined according to whether the read address and the write address of the cache coincide with each other and the value of the wrap flag of the cache.

[0168] It should be noted that for the convenience and simplicity of description, the specific working process of the data processing device 700 of the neural network simulator described above can refer to the above Figures 1 to 6 The corresponding process of the method described in will not be repeated here.

[0169] like Figure 8 As shown, the present application provides a terminal for implementing the data processing method of the neural network simulator. The terminal 8 may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a memory allocation program. When the processor 80 executes the computer program 82, the steps in the embodiments of the data processing method of the neural network simulator are implemented, such as Figure 1 Alternatively, when the processor 80 executes the computer program 82, the functions of the modules / units in the above device embodiments are realized, for example, Figure 7 Functions of units 701 to 704 are shown.

[0170] The computer program can be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal. For example, the computer program can be divided into an acquisition unit, a first carrying unit, a second carrying unit and a data processing unit, and the specific functions of each unit are as follows:

[0171] The acquisition unit is configured to acquire instruction data; the instruction data includes a first carrying instruction carrying a first carrying parameter, a second carrying instruction carrying a second carrying parameter and a granularity operation instruction;

[0172] The first carrying unit is configured to carry data of a source end to a destination end in a transaction level carrying manner according to the first carrying parameter carried by the first carrying instruction;

[0173] The second carrying unit is configured to carry data of the destination end to a cache according to the second carrying parameter carried by the second carrying instruction according to a cycle level precision;

[0174] The data processing unit is configured to execute the granularity operation based on the data in the cache to obtain a cycle level data processing result if the cache is not empty.

[0175] The terminal can be a computer, a server or other computing device. The terminal can include, but is not limited to, the processor 80, the memory 81. Those skilled in the art can understand that, Figure 8 The terminal is only an example and does not constitute a limitation on the terminal, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the terminal can also include an input / output device, a network access device, a bus, etc.

[0176] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0177] The memory 81 can be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 81 can also be an external storage device of the terminal, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like equipped on the terminal. Further, the memory 81 can include both an internal storage unit and an external storage device of the terminal. The memory 81 is used to store the computer program and other programs and data required by the terminal. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0178] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for convenient distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0179] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0180] Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or in combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0181] In the embodiments provided in this application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0183] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0184] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0185] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A data processing method for a neural network simulator, characterized in that: include: Acquire instruction data; the instruction data includes a first transport instruction carrying a first transport parameter, a second transport instruction carrying a second transport parameter, and a granularity calculation instruction; Transferring the data from the source end to the destination end using a transaction-level transfer method according to the first transfer parameter carried by the first transfer instruction; Moving the data of the destination end to the cache with cycle-level accuracy according to the second moving parameter carried by the second moving instruction; If the cache is not empty, granularity calculations are performed based on the data in the cache to obtain cycle-level data processing results.

2. The data processing method of the neural network simulator according to claim 1, wherein: The data from the source end is transferred to the destination end in a transaction-level manner, including: Based on the communication handshake between the source end and the destination end, the data of the source end is transferred to the destination end in a transaction-level manner.

3. The data processing method of the neural network simulator according to claim 1, wherein: The second transport parameter includes an operation mode corresponding to the currently transported data; The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter carried by the second transfer instruction includes: Calculating the actual data volume of the data at the destination end required for the data processing according to the operation mode corresponding to the currently transported data; According to the actual data volume and the data volume transferred from the source end to the destination end, determining whether the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation; If the data volume of the destination end is greater than or equal to the actual data volume required for the granularity calculation, the data of the destination end is transferred to the cache at a cycle-level accuracy according to the second transfer parameter.

4. The data processing method of the neural network simulator according to claim 1, wherein: The second transport parameter includes a cutting parameter and an operation mode corresponding to the currently transported data; The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter carried by the second transfer instruction includes: Cutting the data of the destination end according to the cutting parameter in the second transport parameter with cycle-level accuracy to obtain cutting data; Calculate the cutting data according to the calculation mode corresponding to the currently transported data to obtain the target data required for the granularity calculation; The target data is moved to the cache.

5. The data processing method of the neural network simulator according to any one of claims 1 to 4, characterized in that: When the target data required for the granularity calculation is partial data in the matrix data at the destination end, the second transport parameter includes the first position coordinates corresponding to each data in the partial data; The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter includes: The data corresponding to the first position coordinates in the matrix data of the destination end is moved to the cache.

6. The data processing method of a neural network simulator according to any one of claims 1 to 4, characterized in that: When the target data required for the granularity calculation is a value before Winograd transformation, the second transport parameter includes the second position coordinates of the data in the 4*4 data table required for Winograd transformation; The step of transferring the data of the destination end to the cache with cycle-level accuracy according to the second transfer parameter includes: The 4*4 data table stored in the destination end is read according to the second position coordinates, and the value of the pre-winograd transformation is calculated based on the 4*4 data table, and the value of the pre-winograd transformation is moved to the cache.

7. The data processing method of a neural network simulator according to any one of claims 1 to 4, characterized in that: Before performing granularity calculations based on the data in the cache to obtain cycle-level data processing results if the cache is not empty, the method includes: Obtaining the value of the wrap flag of the cache, and the read address and write address of the cache; Whether the cache is empty is determined according to whether the read address and the write address of the cache coincide with each other and the value of the wrap flag of the cache.

8. A data processing device for a neural network simulator, characterized in that: include: an acquisition unit, configured to acquire instruction data; the instruction data including a first transport instruction carrying a first transport parameter, a second transport instruction carrying a second transport parameter, and a granularity calculation instruction; a first transfer unit, configured to transfer the data from the source end to the destination end in a transaction-level manner according to the first transfer parameter carried in the first transfer instruction; a second transfer unit, configured to transfer the data of the destination end to the cache with cycle-level accuracy according to a second transfer parameter carried in the second transfer instruction; A data processing unit is configured to, if the cache is not empty, perform granularity operations based on the data in the cache to obtain cycle-level data processing results.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images