Neural network processor, system on chip, data processing method and storage medium

Abstract

Disclosed are a neural network processor, a system-on-chip, a data processing method and a storage medium, and relates to the technical field of systems-on-chip, the neural network processor comprising a first processor core, the first processor core comprising: a first buffer for caching a first input tensor corresponding to a first neural network layer in a neural network model; the first direct memory access controller is used for reading a second input tensor corresponding to the first neural network layer from the second buffer and writing the second input tensor into the operation array; and the operation array is used for reading the first input tensor from the first buffer and executing first operation based on the first input tensor and the second input tensor to obtain a first output tensor. According to the technical scheme, the second input tensor in the second buffer is read through the first direct memory access controller, frequent initiation of CPU instructions can be avoided, and the calculation efficiency when the neural network processor processes the neural network model is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of system on chip, and in particular to a neural network processor, a system on chip, a data processing method and a storage medium. Background Art

[0002] Generally, when a more complex neural network model is processed by a neural network processor of a system on a chip (SoC), and the processor core in the neural network processor processes more neural network layers, it is necessary to frequently initiate central processing unit CPU instructions to read weight parameters and / or data to be processed from a higher-level cache in the processor core or a memory outside the processor core to implement operations on the read weight parameters and / or data to be processed. This data reading method of frequently initiating CPU instructions will frequently occupy the CPU bus bandwidth, which not only increases the CPU bus bandwidth occupancy and system power consumption, but also reduces the computational efficiency of the neural network processor in processing the neural network model. Summary of the invention

[0003] When weight parameters and / or part of the data to be processed are read through the data reading method of CPU instructions, the CPU bus bandwidth will be frequently occupied, which not only increases the CPU bus bandwidth occupancy and system power consumption, but also reduces the computational efficiency of the neural network processor in processing the neural network model.

[0004] In order to solve the above technical problems, the present disclosure provides a neural network processor, the neural network processor includes a first processor core, the first processor core includes:

[0005] A first buffer, used to cache a first input tensor corresponding to a first neural network layer in the neural network model;

[0006] A first direct memory access controller, configured to read a second input tensor corresponding to the first neural network layer from a second buffer, and write the second input tensor into a computing array;

[0007] The operation array is used to read the first input tensor from the first buffer, and perform a first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain a first output tensor.

[0008] A second aspect of the present disclosure provides a system on chip, including an on-chip memory, a second direct memory access controller, and a neural network processor, the neural network processor including at least one processor core, the at least one processor core including a first processor core and a second processor core;

[0009] On-chip memory for caching the fourth input tensor;

[0010] a second direct memory access controller, configured to read a fourth input tensor from the on-chip memory and write the fourth input tensor into the first processor core and the second processor core;

[0011] a first processor core configured to perform a fifth operation corresponding to a fifth neural network layer in the neural network model based on the fourth input tensor;

[0012] The second processor core is used to respond to the sixth neural network layer and the fifth neural network layer in the neural network model reusing the fourth input tensor, and perform a sixth operation corresponding to the sixth neural network layer based on the fourth input tensor.

[0013] A third aspect of the present disclosure provides a data processing method, which is applied to a first processor core in a neural network processor, and the method includes:

[0014] caching a first input tensor corresponding to a first neural network layer in the neural network model by a first buffer in the first processor core;

[0015] Reading a second input tensor corresponding to the first neural network layer from a second buffer through a first direct memory access controller in the first processor core, and writing the second input tensor into a computing array in the first processor core;

[0016] The first input tensor is read from the first buffer through the operation array, and a first operation corresponding to the first neural network layer is performed based on the first input tensor and the second input tensor to obtain a first output tensor.

[0017] A fourth aspect of the present disclosure provides an electronic device, the electronic device comprising: a processor; a memory for storing instructions executable by the processor;

[0018] The processor is used to read executable instructions from the memory and execute the instructions to implement the data processing method of the third aspect.

[0019] The fifth aspect of the present disclosure provides a computer-readable storage medium, which stores a computer program for executing a data processing method according to the third aspect.

[0020] In the neural network processor provided by the embodiment of the present disclosure, since the first processor core includes a first direct memory access controller, the second input tensor stored in the second buffer can be directly read through the first direct memory access controller. Therefore, not only can the reading speed of the second input tensor be improved, but also there is no need to frequently initiate CPU instructions, which can reduce the CPU bus bandwidth usage and system power consumption. Furthermore, the operation array can relatively quickly perform the first operation corresponding to the first neural network layer on the first input tensor sent by the first buffer and the second input tensor written by the first direct memory access controller, thereby improving the computational efficiency of the neural network processor in processing the neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural diagram of a single processor core provided by an exemplary embodiment of the present disclosure.

[0022] Figure 2 It is a structural diagram of a neural network processor provided by an exemplary embodiment of the present disclosure.

[0023] Figure 3 It is a schematic diagram of the structure of another neural network processor provided by an exemplary embodiment of the present disclosure.

[0024] Figure 4 It is a structural diagram of another neural network processor provided by an exemplary embodiment of the present disclosure.

[0025] Figure 5 It is a structural diagram of another neural network processor provided by an exemplary embodiment of the present disclosure.

[0026] Fig. 6A It is a structural diagram of another neural network processor provided by an exemplary embodiment of the present disclosure.

[0027] Figure 6B It is a structural diagram of another neural network processor provided by an exemplary embodiment of the present disclosure.

[0028] Figure 7 It is a structural diagram of another neural network processor provided by an exemplary embodiment of the present disclosure.

[0029] Figure 8 It is a schematic diagram of the structure of a system on chip provided by an exemplary embodiment of the present disclosure.

[0030] Fig. 9 It is a schematic diagram of the structure of another system on chip provided by an exemplary embodiment of the present disclosure.

[0031] Fig.10It is a flowchart of a data processing method provided by an exemplary embodiment of the present disclosure.

[0032] Fig.11 It is a flowchart of another data processing method provided by an exemplary embodiment of the present disclosure.

[0033] Fig.12 It is a flowchart of another data processing method provided by an exemplary embodiment of the present disclosure.

[0034] Fig.13 It is a flowchart of another data processing method provided by an exemplary embodiment of the present disclosure.

[0035] Fig.14 It is a flowchart of another data processing method provided by an exemplary embodiment of the present disclosure.

[0036] Fig.15 It is a structural schematic diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0038] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0039] Application Overview

[0040] At present, at least one neural network layer group in a neural network model is mainly processed by the processor core in the neural network processor of the SoC to fully utilize the computing resources of the SoC, accelerate the reasoning process of the neural network model, and improve computing efficiency.

[0041] However, with the development of deep learning technology, the complexity of neural network models is getting higher and higher, and the neural network layer group in the neural network model includes more and more neural network layers, and thus the weight parameters corresponding to the neural network layers and / or the amount of data to be processed are also getting larger and larger. When the weight parameters and / or the amount of data to be processed corresponding to the neural network layer group are greater than the amount of data that can be stored in the first-level cache L1M in the processor core, part of the weight parameters and / or part of the data to be processed need to be stored in a higher-level cache in the processor core or in a memory outside the processor core.

[0042] The following embodiment takes the example of storing part of the weight parameters and / or part of the data to be processed in a higher-level second-level cache L2M in the processor core when the amount of data that can be stored in the first-level cache L1M in the processor core is smaller than the amount of all weight parameters and / or data to be processed corresponding to the neural network layer group.

[0043] Figure 1 FIG. 1 is a schematic diagram of a structure of a single processor core provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the processor core 10 may include a level 1 cache (L1M) 101 , a systolic array 102 , and a level 2 cache (L2M) 103 .

[0044] The L1M 101 and the systolic array 102 may communicate with each other via an internal bus in the processor core 10 , and the L1M 101 and the L2M may communicate with each other via a CPU bus.

[0045] L1M 101 is used to cache a portion of weight parameters and / or a portion of data to be processed corresponding to at least one neural network layer group to be processed by the processor core 10. L2M 103 is used to cache another portion of weight parameters and / or another portion of data to be processed corresponding to at least one neural network layer group to be processed by the processor core 10. The pulse array 102 is used to read the weight parameters and / or data to be processed in L1M 101 based on the internal bus between the pulse array 102 and L1M 101, and perform corresponding operations on the weight parameters and / or data to be processed corresponding to at least one neural network layer group.

[0046] refer to Figure 1As shown, it is taken as an example that at least one neural network layer group to be processed by the processor core 10 includes a first neural network layer group, a part of weight parameters corresponding to the first neural network layer group is stored in L1M 101, and input feature data corresponding to the first neural network layer group and another part of weight parameters corresponding to the first neural network layer group are stored in L2M 103. When the first neural network layer of multiple neural network layers in the first neural network layer group is processed by the processor core 10, the input feature data stored in L2M 103 can be read through CPU instructions, and the input feature data stored in L2M 103 can be written into L1M101 to perform corresponding operations on the input feature data. When other neural network layers in the multiple neural network layers are processed by the processor core 10, if the weight data corresponding to the other neural network layers is stored in L2M 103, it is also necessary to read the weight parameters corresponding to the other neural network layers in L2M 103 through CPU instructions to perform corresponding operations. That is, when processing the first neural network layer group through the processor core 10, CPU instructions need to be frequently initiated, so the CPU bus bandwidth will be frequently occupied, which not only increases the CPU bus bandwidth occupancy and system power consumption, but also reduces the computational efficiency of the neural network processor in processing the neural network model.

[0047] Figure 2 FIG. 1 is a schematic diagram of a neural network processor provided by an exemplary embodiment of the present disclosure. Figure 2 As shown, the neural network processor 20 may include multiple Figure 1 A processor core 10 is shown.

[0048] refer to Figure 2 It can be seen that when multiple processor cores 10 in the neural network processor 20 simultaneously process the neural network layer groups corresponding to each processor core 10, CPU instructions will be initiated more frequently, and the bus bandwidth will be occupied more frequently, and then the bandwidth occupancy and system power consumption will be extremely large, seriously affecting the computational efficiency of the neural network processor in processing the neural network model.

[0049] Based on the above technical problems, an embodiment of the present disclosure provides a neural network processor, which can establish an access path to a higher-level cache in the processor core or a memory outside the processor core through a direct memory access controller by setting a direct memory access controller in the processor core of the neural network processor. Thus, when the processor core processes the neural network layer group, it can quickly read the weight parameters or data to be processed stored in a higher-level cache in the processor core or a memory outside the processor core based on the direct memory access controller. Therefore, the present disclosure does not need to frequently initiate CPU instructions, can reduce bandwidth occupancy and system power consumption, and improves the computational efficiency of the neural network processor in processing neural network models.

[0050] Example Circuit

[0051] Figure 3 FIG. 1 is a schematic diagram of the structure of another neural network processor provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the neural network processor 30 includes a first processor core 31. The first processor core 31 includes a first cache 311, a first direct memory access controller 312 and an operation array 313.

[0052] Among them, the first buffer 311 is used to cache the first input tensor corresponding to the first neural network layer in the neural network model; the first direct memory access controller 312 is used to read the second input tensor corresponding to the first neural network layer from the second buffer, and write the second input tensor into the operation array 313; the operation array 313 is used to read the first input tensor from the first buffer 311, and perform the first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain the first output tensor.

[0053] Exemplarily, the neural network processor 30 may include one or more processor cores. The embodiment of the present disclosure does not limit the number of processor cores included in the neural network processor 30. The embodiment of the present disclosure takes the neural network processor 30 including N processor cores as an example for illustrative description, where N is an integer greater than or equal to 1. The first processor core 31 may be any processor core among the N processor cores.

[0054] Exemplarily, the access speed of the first cache 311 is greater than the access speed of the second cache. In some examples, the first cache 311 may be a first-level cache LIM in the first processor core 31. The second cache may be a second-level cache L2M in the first processor core 31, or may be an L2M or DDR outside the first processor core 31. The embodiment of the present disclosure does not limit the type of the second cache. The first processor core 31 may include the second cache, or may not include the second cache. When the second cache is the second-level cache L2M in the first processor core 31, the first processor core 31 includes the second cache; when the second cache is an L2M or DDR outside the first processor core 31, the first processor core 31 does not include the second cache. The following embodiment is exemplified by taking the second cache as a DDR outside the first processor core 31 as an example.

[0055] The first direct memory access controller 312 may be a remote direct memory access (RDMA) controller disposed between the second buffer and the operation array 313. The operation array 313 may be a circuit for performing operations corresponding to the neural network layer group, and the operation array may also be referred to as a systolic array. If the second input tensor is cached in the second buffer, the second buffer may be directly accessed through the first direct memory access controller 312 to read the second input tensor in the second buffer, and the second input tensor may be written into the operation array 313, so that the operation array 313 can perform the corresponding first operation based on the second input tensor.

[0056] The first neural network layer may be any neural network layer in one or more neural network layer groups processed by the first processor core 31. Exemplarily, the one or more neural network layer groups processed by the first processor core 31 may include multiple neural network layers coupled in series. In some examples, according to the order of the neural network layers in the multiple neural network layers, the multiple neural network layers may include a first neural network layer, multiple intermediate neural network layers, and a last neural network layer. The first neural network layer may be any neural network layer in the multiple neural network layers. The following embodiments are exemplified by taking the first neural network layer as the first neural network layer in the multiple neural network layers.

[0057] Exemplarily, if the first neural network layer is the first neural network layer in the neural network layer group, the data to be processed corresponding to the first neural network layer is the data to be processed corresponding to the neural network layer group. In some examples, the data to be processed may be feature data to be processed. For example, the data to be processed corresponding to the first neural network layer may be the first input feature data.

[0058] Take the case where the data to be processed corresponding to the first neural network layer is the first input feature data. In some examples, the first input tensor may be the first weight parameter corresponding to the first neural network layer, and the second input tensor may be the first input feature data. In other examples, the first input tensor may be the first input feature data, and the second input tensor may be the first weight parameter corresponding to the first neural network. The disclosed embodiments do not limit the data types of the first input tensor and the second input tensor. The following embodiments are exemplified by taking the first input tensor as the first weight parameter and the second input tensor as the first input feature data as an example.

[0059] Exemplarily, the first operation corresponding to the first neural network layer includes at least one of a linear transformation, a convolution operation, and a pooling operation. The disclosed embodiment does not limit the type of the first operation corresponding to the first neural network layer. The following is an exemplary description using the example that the first operation corresponding to the first neural network layer includes a linear operation. For example, the first operation corresponding to the first neural network layer may include a multiplication operation.

[0060] It is understandable that the SOC can write the weight data and the data to be processed corresponding to the neural network model into the DDR during the initialization phase, and then write the weight data in the DDR into the L1M of the corresponding processor core. If the capacity of the L1M of the processor core is less than the data volume of the weight parameters corresponding to the processor core, a part of the weight parameters corresponding to the processor core can be written into the L1M, and another part of the weight parameters corresponding to the processor core can continue to be stored in the DDR, or written into a higher-level cache (for example, L2M).

[0061] For example, the first input tensor is the first weight parameter, and the data amount of the first weight parameter is smaller than the capacity of the first buffer 311. The SOC may write the first weight parameter into the first buffer 311, so that the first input tensor is cached in the first buffer 311.

[0062] Exemplarily, the first direct memory access controller 312 reads the second input tensor corresponding to the first neural network layer from the second buffer and writes the second input tensor into the operation array 313, which may include: the first direct memory access controller 312 receives the first read address and the first write address sent by the operation array 313, reads the second input tensor corresponding to the first neural network from the second buffer based on the first read address, and writes the second input tensor into the corresponding storage location in the operation array 313 based on the first write address.

[0063] Exemplarily, the first read address may be an address in the second buffer for caching the second input tensor, and the first read address may be generated by the operation array 313. The first write address may be an address in the operation array 313 for caching the second input tensor, and the first write address may also be generated by the operation array 313. In some examples, the operation array 313 may include an address generator. The address generator may configure the first address of the address generator according to the configuration instruction. Furthermore, the address generator may calculate the first read address and the first write address based on the first address, and send the first read address and the first write address to the first direct memory access controller 312.

[0064] The operation array 313 reads the first input tensor from the first buffer 311, which may include: the operation array 313 generates a second read address and a second write address, sends the second read address and the second write address to the first buffer 311, the first buffer 311 reads the first input tensor based on the second read address, and writes the first input tensor into the corresponding storage location in the operation array 313 based on the second write address.

[0065] Exemplarily, the second read address may be an address in the first buffer 311 storing the first input tensor, and the second read address may be generated by the operation array 313. The second write address may be an address in the operation array 313 for storing the first input tensor, and the second write address may also be generated by the operation array 313. The implementation of the embodiment of the present disclosure for generating the second read address and the second write address by the operation array 313 will not be described in detail herein.

[0066] In some examples, the first cache 311 may include a plurality of first storage units and a first cache controller. The first cache controller may read the first input tensor from the plurality of first storage units based on the second read address, and send the first input tensor to a corresponding storage location in the operation array 313 based on the second write address.

[0067] For example, taking the first operation as a multiplication operation, the operation array 31 can perform a multiplication operation on the first input tensor and the second input tensor to obtain a first output tensor of the first neural network layer. The first output tensor of the first neural network layer can also be called the first output feature data.

[0068] In the neural network processor provided by the embodiment of the present disclosure, since the first processor core includes a first direct memory access controller, the second input tensor stored in the second buffer can be directly read through the first direct memory access controller, which is not affected by the CPU bus resource preemption and can avoid frequent CPU instruction initiation. Therefore, not only can the reading speed of the second input tensor be improved, but also there is no need to frequently initiate CPU instructions, which can reduce the CPU bus bandwidth occupation and system power consumption. In addition, the operation array can relatively quickly perform the first operation corresponding to the first neural network layer on the first input tensor sent by the first buffer and the second input tensor written by the first direct memory access controller, thereby improving the computational efficiency of the neural network processor in processing the neural network model.

[0069] In some other embodiments of the present disclosure, the first input tensor and the second input tensor corresponding to the first neural network layer can be stored in the first buffer 311, so that the operation array 313 can directly read the first input tensor and the second input tensor from the first buffer 311 to perform the first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain the first output tensor.

[0070] In some further embodiments of the present disclosure, the first input tensor and the second input tensor corresponding to the first neural network layer may be stored in a second cache, so that the operation array 313 may read the first input tensor and the second input tensor from the second cache based on the first direct memory access controller 312 to perform the first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain a first output tensor.

[0071] In some embodiments of the present disclosure, at least two neural network layers in the neural network layer group to be processed by the first processor core 31 may correspond to the same input tensor (e.g., weight parameter), that is, the at least two neural network layers may reuse the same input tensor. The number of the at least two neural network layers may be 2, 3, or 4, and the embodiments of the present disclosure do not limit the number of neural network layers in the at least two neural network layers. The following embodiments are exemplified by taking the at least two neural network layers including the second neural network layer and the third neural network layer as an example.

[0072] In some examples, the second neural network layer and the third neural network layer in the neural network layer group to be processed by the first processor core 31 reuse the same weight parameter, and the weight parameter is the third input tensor. Figure 3 As shown, the first direct memory access controller 312 is also used to read the third input tensor from the second buffer and write the third input tensor into the operation array 313 in response to the second neural network layer and the third neural network layer in the neural network model reusing the third input tensor; the operation array 313 is also used to execute the second operation corresponding to the second neural network layer based on the third input tensor, and execute the third operation corresponding to the third neural network layer.

[0073] Exemplarily, the second neural network layer may be any one of the multiple neural network layers that the first processor core 31 needs to process, and the second neural network layer may be the same as the first neural network layer or different from the first neural network layer. The disclosed embodiment does not limit the relationship between the second neural network layer and the first neural network layer. The following embodiment is exemplified by taking the second neural network layer being different from the first neural network layer as an example.

[0074] The third neural network layer can be any neural network layer different from the second neural network layer among the multiple neural network layers that the first processor core 31 needs to process. The third neural network layer can be the same as the first neural network layer or different from the first neural network layer. The disclosed embodiment does not limit the relationship between the third neural network layer and the first neural network layer. The following embodiment is illustrative by taking the example that the third neural network layer is different from the first neural network layer.

[0075] In some examples, the third neural network layer may be the next neural network layer of the second neural network layer, or may be a neural network layer after the second neural network layer and connected to the second neural network layer through other neural network layers. The following embodiments are exemplified by taking the third neural network layer as the next neural network layer of the second neural network layer as an example.

[0076] In some examples, the third input tensor may be a second weight parameter reused by the second neural network layer and the third neural network layer. In some examples, the third input tensor may be cached in the first buffer 311 or in the second buffer. The present disclosure does not limit the cache location of the third input tensor. The following embodiments are exemplified by taking the third input tensor cached in the second buffer as an example.

[0077] Exemplarily, the second operation corresponding to the second neural network layer and the third operation corresponding to the third neural network layer may be the same as or different from the first operation corresponding to the first neural network layer. The embodiments of the present disclosure do not limit the specific implementation of the second operation corresponding to the second neural network layer and the third operation corresponding to the third neural network layer.

[0078] Take the third input tensor cached in the second buffer as an example. Figure 3 As shown, since the second neural network layer and the third neural network layer reuse the third input tensor, when the operation array 313 processes the second neural network layer, and / or when the operation array 313 processes the second neural network layer, a third read address and a third write address will be generated, and the third read address and the third write address will be sent to the first direct memory access controller 312, so that the first direct memory access controller 312 can determine that the third neural network layer and the second neural network layer reuse the third input tensor when receiving the third read address and the third write address. The first direct memory access controller 312 can read the third input tensor based on the third read address and the third write address and write the third input tensor to the operation array 313.

[0079] In some examples, the third read address may correspond to an address in the second buffer storing the third input tensor. The third write address may correspond to an address in the operation array 313 for storing the third input tensor. Similar to the implementation of the first read address and / or the second read address, the third read address may also be generated by the operation array 313. The implementation of the third address generated by the operation array 313 in the embodiment of the present disclosure will not be repeated here.

[0080] Exemplarily, the first direct memory access controller 312 may read the second weight parameter from the second buffer based on the third read address, and write the second weight parameter to a corresponding storage location in the operation array 313 based on the third write address.

[0081] In some examples, the operation array 313 performs a second operation corresponding to the second neural network layer based on the second weight parameter, which may include: the operation array 313 performs a multiplication operation on the second input feature data and the second weight parameter based on the second input feature data corresponding to the second neural network layer to obtain second output feature data.

[0082] Take the third neural network layer as the next neural network layer of the second neural network layer as an example. After the operation array 313 performs the second operation corresponding to the second neural network layer based on the second weight parameter, the operation array 313 can then process the third neural network layer. And when the operation array 313 processes the third neural network layer, the operation array 313 can directly perform the third operation corresponding to the third neural network layer based on the second weight parameter under the third write address when generating the third read address and the third write address.

[0083] In some examples, operation array 313 directly performs a third operation corresponding to the third neural network layer based on the second weight parameter in operation array 313, which may include: operation array 313 directly multiplies the second weight parameter in operation array 313 and the second output feature data (corresponding to the third input feature data of the third neural network layer) to obtain the third output feature data.

[0084] Take the third neural network layer as an example, which is a neural network layer after the second neural network layer and connected to the third neural network through other neural network layers. When the operation array 313 processes the third neural network layer, the operation array 313 can generate and send a third read address and a third write address to the first direct memory access controller 312 again. Then, the first direct memory access controller 312 reads the second weight parameter based on the third read address and the third write address, and writes the second weight parameter into the operation array 313. Thus, the operation array 313 can perform the third operation corresponding to the third neural network layer based on the second weight parameter.

[0085] The neural network processor provided by the embodiment of the present disclosure can quickly read the third input tensor through the first direct memory access controller when the second neural network layer and the third neural network layer reuse the third input tensor and the third input tensor is cached in the second cache, so that the operation array can quickly execute the second operation corresponding to the second neural network layer based on the third input tensor, and quickly execute the third operation corresponding to the third neural network layer based on the third input tensor, thereby improving the computational efficiency of the operation array.

[0086] In other examples, at least two neural network layers in the neural network layer group processed by the first processor core 31 may reuse the same input feature data. For example, the fourth neural network layer in the neural network layer group processed by the first processor core 31 and the next neural network layer of the first neural network layer reuse the same input feature data, and the input feature data is the first output tensor.

[0087] Continue to refer Figure 3 As shown, the operation array 313 is also used to respond to the multiplexing control signal to instruct the fourth neural network layer in the neural network model to multiplex the first output tensor, write the first output tensor into the first buffer 311 based on the multiplexing control signal, and / or write the first output tensor into the second buffer through the first direct memory access controller 312;

[0088] The operation array 313 is also used to read the first output tensor from the first buffer 311 and / or read the first output tensor from the second buffer through the first direct memory access controller 312, and perform a fourth operation corresponding to the fourth neural network layer based on the first output tensor.

[0089] It is understandable that the input feature data and output feature data of each neural network layer can be determined in the design stage of the neural network model, so as to determine whether there is feature data reuse in the neural network layer group. In the case of determining that there is feature data reuse in the neural network layer group, a reuse control instruction can be generated to control the computing array in the corresponding processor core to perform feature reuse.

[0090] Exemplarily, the multiplexing control instruction may be a control instruction for controlling the saving and reading of multiplexing feature data. The multiplexing control instruction may include a multiplexing feature identifier and an address for saving the multiplexing feature. In some examples, if the multiplexing feature is a first output tensor, the multiplexing feature identifier may be an identifier of the first output tensor, and the address for saving the multiplexing feature may be a fourth write address for storing the first output tensor in the first buffer, or may be a fifth write address for storing the first output tensor in the second buffer.

[0091] Exemplarily, the fourth neural network layer may be a neural network layer that follows the first neural network layer in the neural network layer group processed by the first processor core 31 and is connected to the first neural network layer through other neural network layers. In some examples, the fourth neural network layer may be the same as the second neural network layer or the third neural network layer, or may be different from the second neural network layer and / or the third neural network layer. The embodiments of the present disclosure do not limit the relationship between the fourth neural network layer and the second neural network layer and the third neural network layer. The following embodiments are exemplified by taking the fourth neural network layer being different from the second neural network layer and the third neural network layer as an example.

[0092] The fourth operation corresponding to the fourth neural network layer may be the same as or different from the first operation corresponding to the first neural network layer. The present disclosure embodiment does not limit the specific implementation of the fourth operation corresponding to the fourth neural network layer.

[0093] For example, continue to refer to Figure 3 As shown, the operation array 313 can determine the reused first output tensor based on the reuse feature identifier in the reuse control instruction. Furthermore, after the operation array 313 performs the first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain the first output tensor, it obtains the address for storing the first output tensor in the reuse control instruction. And when the address for storing the first output tensor is the fourth write address in the first buffer 311, the first output tensor is written to the fourth write address in the first buffer; as early as when the address for storing the first output tensor is the fifth write address in the second buffer, the fifth write address and the first output tensor are sent to the first direct memory access controller 312, and the first direct memory access controller 312 stores the first output feature in the fifth write address in the second buffer in response to the fifth write address and the first output tensor. Among them, the fourth write address can be the address in the first buffer 311 for caching the first output tensor, and the fifth write address can be the address in the second buffer for caching the first output tensor.

[0094] Furthermore, when the first processor core 31 processes the fourth neural network layer, the operation array 313 may generate and send a fourth read address (corresponding to the fourth write address) and a sixth write address to the first buffer 311; the first buffer 311 may read the first output tensor based on the fourth read address, and write the first output tensor to the corresponding storage location in the operation array 313 based on the sixth write address. The fourth read address may be an address for caching the first output tensor in the first buffer 311, and the sixth write address may be an address for caching the first output tensor in the operation array 313.

[0095] In some examples, when the first processor core 31 processes the fourth neural network layer, the operation array 313 may generate and send a fifth read address (corresponding to the fifth write address) and a seventh write address to the first direct memory access controller 312, and the first direct memory access controller 312 reads the first output tensor from the second buffer based on the fifth read address, and writes the first output tensor to the corresponding storage location in the operation array 313 based on the seventh write address. The fifth read address may be an address for caching the first output tensor in the second buffer, and the seventh write address may be an address for caching the first output tensor in the operation array 313.

[0096] Since the generation method of the fourth read address, the sixth write address, the fifth read address, and the seventh write address by the operation array 313 is similar to the implementation method of the first read address and the first write address by the operation array 313, the embodiments of the present disclosure will not be repeated here.

[0097] Since the implementation manner in which the first buffer 311 reads the first output tensor based on the fourth read address and writes the first output tensor to the corresponding storage location in the operation array 313 based on the sixth write address is similar to the implementation manner in which the first buffer 311 reads the first input tensor based on the second read address and writes the first input tensor to the corresponding storage location in the operation array 313 based on the second write address, the embodiments of the present disclosure will not be repeated here.

[0098] The operation array 313 performs a fourth operation corresponding to the fourth neural network layer based on the first output tensor, which may include: the operation array 313 performs a multiplication operation on the first output tensor and the third weight parameter corresponding to the fourth neural network layer to obtain a fourth output feature corresponding to the fourth neural network layer.

[0099] In the neural network processor provided by the embodiment of the present disclosure, when the fourth neural network layer reuses the first output tensor, the operation array stores the first output tensor in the first buffer or the second buffer, and when the first processor core processes the fourth neural network layer, the first output tensor can be written into the operation array through the first buffer or the first direct memory access controller can be used to quickly read the first output tensor and write the first output tensor into the operation array. Therefore, the fourth operation corresponding to the fourth neural network layer can be performed relatively quickly based on the first output tensor, thereby improving the computational efficiency of the operation array.

[0100] In some embodiments of the present disclosure, at least two processor cores in a neural network processor may process a neural network layer with the same weight parameters, that is, the at least two processor cores may reuse the same weight parameters.

[0101] Exemplarily, the number of processor cores in the at least two processor cores may be any value greater than or equal to 2 and less than or equal to N. In some examples, the at least two processor cores may include 2 processor cores. In other examples, the at least two processor cores may include 3 processor cores. The embodiment of the present disclosure does not limit the number of processor cores in the at least two processor cores. The embodiment of the present disclosure is exemplified by taking the at least two processor cores including a first processor core and a second processor core as an example.

[0102] Take the example of the first processor core and the second processor core reusing the same weight parameters. In some examples, at least one neural network layer processed by the first processor core and at least one neural network layer processed by the second processor core may reuse the same weight parameters. For example, a neural network layer processed by the first processor core and a neural network layer processed by the second processor core reuse the same weight parameters. For another example, two neural network layers processed by the first processor core and one neural network layer processed by the second processor core reuse the same weight parameters. The embodiments of the present disclosure are exemplified by taking the example of a neural network layer processed by the first processor core and a neural network layer processed by the second processor core reusing the same weight parameters.

[0103] When the fifth neural network layer processed by the first processor core and the sixth neural network layer processed by the second processor core reuse the same weight parameter, and the weight parameter is the fourth input tensor, the fourth input tensor needs to be cached in the first cache or the second cache corresponding to the first processor core, and the first cache or the second cache corresponding to the second processor core, respectively. In this way, the area overhead of the first cache or the second cache will be increased.

[0104] Based on the above technical problems, an embodiment of the present disclosure provides a neural network processor, which caches the fourth input tensor reused by the fifth neural network layer processed by the first processor core and the sixth neural network layer processed by the second processor core on an on-chip memory outside the neural network processor, and then sets a second direct memory access controller between the on-chip memory and the neural network processor. When the first processor core processes the fifth neural network layer, the fourth input tensor on the on-chip memory is read through the second direct memory access controller to enable the first processor core to process the fifth neural network layer. When the second processor core processes the sixth neural network layer, the fourth input tensor on the on-chip memory is read through the second direct memory access controller to enable the second processor core to process the sixth neural network layer.

[0105] like Figure 4 As shown in the above Figure 3 Based on the illustrated embodiment, the neural network processor 30 also includes a second processor core 32.

[0106] The first processor core 31 is used to read a fourth input tensor corresponding to a fifth neural network layer in the neural network model from the on-chip memory through the second direct memory access controller, and perform a fifth operation corresponding to the fifth neural network layer based on the fourth input tensor;

[0107] The second processor core 32 is used to respond to the sixth neural network layer and the fifth neural network layer in the neural network model multiplexing the fourth input tensor, read the fourth input tensor from the on-chip memory through the second direct memory access controller, and execute the sixth operation corresponding to the sixth neural network layer based on the fourth input tensor.

[0108] Exemplarily, the fifth neural network layer may be a neural network layer whose input tensor includes the fourth input tensor among any of the multiple neural network layers processed by the first processor core 31. In some examples, the fifth neural network layer may be the same as any of the first neural network layer, the second neural network layer, the third neural network layer, and the fourth neural network layer, or may be different from the first neural network layer, the second neural network layer, the third neural network layer, and the fourth neural network layer. The embodiment of the present disclosure does not limit the relationship between the fifth neural network layer and the first neural network layer, the second neural network layer, the third neural network layer, and the fourth neural network layer. The embodiment of the present disclosure takes the example that the fifth neural network layer is different from the first neural network layer, the second neural network layer, the third neural network layer, and the fourth neural network layer for exemplary description.

[0109] Exemplarily, the sixth neural network layer may be a neural network layer whose input tensor includes the fourth input tensor among any of the plurality of neural network layers processed by the second processor core 32. Similar to the implementation of the fifth neural network layer, the embodiments of the present disclosure will not be described in detail herein.

[0110] Correspondingly, the fifth operation corresponding to the fifth neural network layer and the sixth operation corresponding to the sixth neural network layer may be the same as any of the first operation corresponding to the first neural network layer, the second operation corresponding to the second neural network layer, the third operation corresponding to the third neural network layer, and the fourth operation corresponding to the fourth neural network layer, or may be different from the first operation corresponding to the first neural network layer, the second operation corresponding to the second neural network layer, the third operation corresponding to the third neural network layer, and the fourth operation corresponding to the fourth neural network layer. The disclosed embodiment does not limit the types of the fifth operation corresponding to the fifth neural network layer and the sixth operation corresponding to the sixth neural network layer.

[0111] Exemplarily, the fourth input tensor may be a fourth weight parameter reused by the fifth neural network layer and the sixth neural network layer. In some examples, the fourth weight parameter may be the same as any of the first weight parameter, the second weight parameter, and the third weight parameter, or may be different from the first weight parameter, the second weight parameter, and the third weight parameter. The present disclosure embodiment does not limit the specific implementation of the fourth weight parameter. The present disclosure embodiment takes the example of being different from the first weight parameter, the second weight parameter, and the third weight parameter as an example for exemplary description.

[0112] The on-chip memory may be a DDR outside the neural network processor on the SOC (i.e., outside the first processor core 31 and the second processor core 32). The on-chip memory may be used to store the fourth input tensor multiplexed by the first processor core 31 and the second processor core 32. In some examples, the on-chip memory may be used to store the fourth weight parameter.

[0113] The second direct memory access controller may be an RDMA coupled between the on-chip memory and the neural network processor.

[0114] In some examples, the first processor core 31 may generate and send a sixth read address and an eighth write address to the second direct memory access controller when processing the fifth neural network layer, and the second direct memory access controller may read the fourth weight parameter from the on-chip memory based on the sixth read address, and write the fourth weight parameter to a corresponding storage location in the first processor core 31 based on the eighth write address. The sixth read address may be an address in the on-chip memory where the fourth input tensor is stored. The eighth write address may be an address in the first processor core 31 for caching the fourth input tensor.

[0115] In some examples, the eighth write address may be an address in the second buffer of the first processor core 31. Therefore, after writing the fourth input tensor into the second buffer, it is also necessary to read the fourth input tensor in the second buffer through the first direct memory access controller 312 and write the fourth input tensor into the operation array 313.

[0116] The implementation of writing the fourth input tensor into the second buffer and reading the fourth input tensor in the second buffer through the first direct memory access controller 312, and writing the fourth input tensor into the operation array 313 can be seen in the above Figure 3 The relevant description of the illustrated embodiment will not be repeated here.

[0117] Since the implementation method of executing the fifth operation corresponding to the fifth neural network layer based on the fourth input tensor is similar to the implementation method of executing the first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor, executing the second operation corresponding to the second neural network layer based on the third input tensor..., the embodiments of the present disclosure will not be repeated here.

[0118] In some examples, if the read address generated by the first processor core 31 when processing the fifth neural network layer is the same as the read address generated by the second processor core 32 when processing the sixth neural network layer, and is the sixth read address, it is determined that the sixth neural network layer and the fifth neural network layer reuse the fourth input tensor.

[0119] Exemplarily, when processing the sixth neural network layer, the second processor core 32 may generate and send a sixth read address and a ninth write address to the second direct memory access controller, and the second direct memory access controller reads the fourth weight parameter from the on-chip memory based on the sixth read address, and writes the fourth weight parameter to the corresponding storage location in the second processor core 32 based on the ninth write address. The ninth write address is the address in the second processor core 32 for caching the fourth input tensor.

[0120] In some examples, the ninth write address may be an address in the second cache of the second processor core 32. Similar to the first processor core 31, after the fourth input tensor is written into the second cache of the second processor core 32, the fourth input tensor also needs to be written into the operation array in the second processor core 32 through the first direct memory access controller in the second processor core 32.

[0121] The implementation of writing the fourth input tensor into the second buffer of the second processor core 32 and writing the fourth input tensor into the operation array of the second processor core 32 through the first direct memory access controller in the second processor core 32 can be seen in the above Figure 3 The relevant description of the illustrated embodiment will not be repeated here.

[0122] Since the implementation method of executing the sixth operation corresponding to the sixth neural network layer based on the fourth input tensor is similar to the implementation method of executing the first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor, executing the second operation corresponding to the second neural network layer based on the third input tensor..., the embodiments of the present disclosure will not be repeated here.

[0123] The neural network processor provided by the embodiment of the present disclosure can quickly read the fourth input tensor corresponding to the fifth neural network layer from the on-chip memory through the second direct memory access controller when the first processor core processes the fifth neural network layer, so as to perform the fifth operation corresponding to the fifth neural network layer based on the fourth input tensor, thereby accelerating the calculation speed of the first processor core. And when the second processor core processes the sixth neural network layer, it can quickly read the fourth input tensor corresponding to the sixth neural network layer from the on-chip memory through the second direct memory access controller, so as to perform the sixth operation corresponding to the sixth neural network layer based on the fourth input tensor, thereby accelerating the calculation speed of the second processor core. At the same time, since the fourth input tensor only needs to be stored in the on-chip memory, it is not necessary to store the first processor core and the second processor core separately, which can reduce the area overhead of the cache.

[0124] like Figure 5 As shown in the above Figure 3Based on the illustrated embodiment, the operation array 313 may include a controller 3131 , a third buffer 3132 , and an operation circuit 3133 .

[0125] The controller 3131 is used to read the first input tensor from the first buffer 311 and write the first input tensor into the third buffer; and generate an operation control signal;

[0126] A first direct memory access controller 312, configured to read the second input tensor from the second buffer and write the second input tensor into the third buffer 3132;

[0127] A third buffer 3132, used for caching the first input tensor and the second input tensor;

[0128] The operation circuit 3133 is used to read the first input tensor and the second input tensor from the third buffer in response to the operation control signal, perform a first operation based on the first input tensor and the second input tensor, and obtain a first output tensor.

[0129] For example, the controller 3131 may be a hardware logic circuit specially designed for the specific requirements of the systolic array, and the controller 3131 may be used to manage and coordinate the data flow and the operations of the processing units in the entire computing array.

[0130] The third buffer 3132 can be used to buffer data to be processed by the operation array 313 , and the operation array 313 can only read the data buffered in the third buffer 3132 .

[0131] The computing circuit 3133 is a basic unit in the systolic array responsible for performing actual computing tasks. In some examples, the computing circuit 3133 may include multiple processing elements (PEs) arranged according to certain rules, and the PEs are interconnected to work together to complete complex computing tasks.

[0132] In some examples, the second write address may be an address in the third buffer 3132 of the operation array 313 for caching the first input tensor. The controller 3131 may obtain the second read address and the second write address, and send the second read address and the second write address to the first buffer 311. The first buffer 311 reads the first input tensor based on the second read address, and writes the first input tensor to a corresponding position in the third buffer 3132 based on the second write address. The second write address is an address in the third buffer 3132 of the operation array 313.

[0133] In some examples, the first write address may be an address for caching the second input tensor in the third buffer 3132 of the operation array 313. The first direct memory access controller 312 may write the second input tensor to a corresponding position in the third buffer 3132 based on the first write address.

[0134] Exemplarily, the operation control signal may be a signal for controlling the operation circuit 3133 to perform the first operation. In some examples, the operation control signal may be used to control the start time and end time of the operation, and what type of operation the operation circuit 3133 performs.

[0135] The neural network processor provided by the embodiment of the present disclosure reads the first input tensor through the controller and writes the first input tensor into the third buffer, reads the second input tensor from the second buffer through the first direct memory access controller, and writes the second input tensor into the third buffer. In this way, the operation circuit can perform the first operation corresponding to the operation control signal on the first input tensor and the second input tensor, ensuring that each PE in the operation circuit can perform the calculation task according to the predetermined order and rules, thereby realizing efficient parallel calculation.

[0136] In other embodiments, Fig. 6A As shown in the above Figure 5 Based on the illustrated embodiment, the operation array 313 may further include an address generator 3134. The address generator 3134 may be coupled to the controller 3131. The address generator 3134 may generate a first read address, a first write address, a second read address, and a second write address, and send the first read address, the first write address, the second read address, and the second write address to the controller 3134. The first read address and the first write address are sent to the first direct memory access controller 312 through the controller 3134, and the second read address and the second write address are sent to the first buffer 311.

[0137] In some other embodiments, Figure 6B As shown in the above Figure 5 Based on the illustrated embodiment, the operation array 313 may further include an address generator 3134. The address generator 3134 may be coupled to the first buffer 311 and the first direct memory access controller 312. The address generator 3134 may generate a first read address, a first write address, a second read address, and a second write address, and send the first read address and the first write address to the first direct memory access controller 312, and send the second read address and the second write address to the first buffer 311.

[0138] Thus, the first direct memory access controller 312 can read the second input tensor from the second buffer based on the first read address, and write the second input tensor to the third buffer 3132 in the operation array 313 based on the first write address; the first buffer 311 can read the first input tensor from the first buffer 311 based on the second read address, and write the first input tensor to the third buffer 3132 in the operation array 313 based on the second write address.

[0139] In some examples of the present disclosure, Figure 7 As shown in the above Figure 5 Based on the illustrated embodiment, the operation array 313 further includes a selector 3135 .

[0140] The controller 3131 is specifically used to generate a selection control signal; the selector 3135 is used to select to output the first input tensor in the first buffer 311 to the third buffer or to output the second input tensor in the first direct memory access controller 312 to the third buffer 3132 in response to the selection control signal.

[0141] Exemplarily, the selector 3135 may be a two-to-one selector, and includes a first input terminal, a second input terminal, a control terminal, and an output terminal. The first input terminal may be coupled to the first buffer 311; the second input terminal may be coupled to the first direct memory access controller 312; the control terminal may be coupled to the controller 3131; and the output terminal may be coupled to the third buffer 3132.

[0142] Exemplarily, the selection control signal may be a signal for controlling the selector 3135 to form a first data transmission path between the first buffer 311 and the third buffer 3132 or a second data transmission path between the first direct memory access controller 312 and the third buffer 3132. In some examples, the selection control signal includes a first voltage signal. In other examples, the selection control signal includes a second voltage signal.

[0143] Exemplarily, the first voltage signal and the second voltage signal may be signals of opposite levels. In some examples, the first voltage signal may be a signal corresponding to a logic high level "1", and the second voltage signal may be a signal corresponding to a logic low level "0". In other examples, the first voltage signal may be a signal corresponding to a logic low level "1", and the second voltage signal may be a signal corresponding to a logic high level "0". The embodiments of the present disclosure do not limit the implementation method of the first voltage signal and the second voltage signal. The embodiments of the present disclosure are exemplified by taking the first voltage signal as a signal corresponding to a logic high level "1" and the second voltage signal as a signal corresponding to a logic low level "0".

[0144] Taking the selection control signal as the first voltage signal as an example, in some examples, the controller 3131 may generate the first voltage signal when sending the second read address and the second write address to the first buffer 311 .

[0145] Taking the selection control signal as the second voltage signal as an example, in some examples, the controller 3131 may generate the second voltage signal when sending the first read address and the first write address to the first direct memory access controller 312 .

[0146] Exemplarily, the selector 3125 may form a first data transmission path between the first buffer 311 and the third buffer 3132 in response to the selection control signal being a first voltage signal, so that the first buffer 311 can send the first input tensor to the third buffer 3132 based on the first data transmission path after reading the first input tensor; the selector 3125 may form a second data transmission path between the first direct memory access controller 312 and the third buffer 3132 in response to the selection control signal being a second voltage signal, so that the first direct memory access controller 312 can send the second input tensor to the third buffer 3132 based on the second data transmission path after reading the second input tensor.

[0147] The neural network processor provided by the embodiment of the present disclosure sets a selector in the operation array, and controls the selector through a controller to select and form a first data transmission path between the first buffer and the third buffer or a second data transmission path between the first direct memory access controller and the third buffer. In this way, the first buffer can write the first input tensor to the third buffer based on the first data transmission path, and the first direct memory access controller can write the second input tensor to the third buffer based on the second transmission path, thereby realizing the selection and switching of the first input tensor and the second input tensor to be written into the third buffer.

[0148] In some embodiments of the present disclosure, in order to save the area overhead of the buffer, the compressed input tensor can be stored in the first buffer 311 or the second buffer, and after the operation array 313 reads the compressed input tensor, the operation circuit 3133 decompresses the compressed input tensor before processing the input tensor to implement the corresponding operation on the input tensor. For example, the first input tensor cached in the first buffer 311 can be the compressed first input tensor (compressed data), and / or the second input tensor cached in the second buffer can be the compressed second input tensor. The embodiment of the present disclosure is exemplified by taking the first input tensor cached in the first buffer 311 as the compressed first input tensor and the second input tensor cached in the second buffer as the uncompressed second input tensor as an example.

[0149] In some examples, the first input tensor may be compressed in advance to obtain a compressed first input tensor, and the compressed first input tensor may be written into the DDR (second buffer) when the SOC is powered on, and then the compressed first input tensor may be written from the DDR into the first buffer 311, so that the compressed first input tensor is cached in the first buffer 311.

[0150] In other examples, the second input tensor may be compressed in advance to obtain a compressed second input tensor, and the compressed second input tensor may be written to the DDR (second buffer) when the SOC is powered on, so that the compressed second input tensor is cached in the second buffer.

[0151] In some embodiments of the present disclosure, the first input tensor cached in the first buffer 311 is a compressed first input tensor, and the second input tensor cached in the second buffer is an uncompressed second input tensor. After the first buffer 311 writes the first input tensor cached in the first buffer 311 to the third buffer 3132. Figure 5 As shown, the controller 3131 is also used to read the first input tensor from the third buffer 3132, and in response to the first input tensor being compressed data, decompress the first input tensor to obtain the decompressed first input tensor, and write the decompressed first input tensor into the third buffer 3132; the third buffer 3132 is also used to cache the decompressed first input tensor and the second input tensor; the operation circuit 3133 is used to respond to the operation control signal, read the decompressed first input tensor and the second input tensor from the third buffer 3132, perform a first operation based on the decompressed first input tensor and the second input tensor, and obtain a first output tensor.

[0152] Exemplarily, the controller 3131 may read the first input tensor from the third buffer based on the read address corresponding to the second write address, and when it is determined that the read first input tensor is compressed data, call a decompression module (which may be within the operation array 313 or outside the operation array 313, and the present disclosed embodiment is not limited to this) to decompress the first input tensor to obtain the decompressed first input tensor, and write the decompressed first input tensor into the third buffer 3132 based on the second write address.

[0153] Exemplarily, the operation circuit 3133 responds to the operation control signal, reads the first input tensor and the second input tensor after decompression from the third buffer 3132, performs the first operation based on the first input tensor and the second input tensor after decompression, and obtains the first output tensor. The implementation method is similar to the implementation method of reading the first input tensor and the second input tensor from the third buffer in response to the operation control signal, performing the first operation based on the first input tensor and the second input tensor, and obtaining the first output tensor, and the embodiments of the present disclosure are not repeated here.

[0154] The neural network processor provided by the embodiment of the present disclosure caches the compressed data corresponding to the first input tensor in the first buffer, and when the first buffer writes the compressed data into the third buffer, the controller decompresses the compressed data, so that the first input tensor after decompression is cached in the third buffer, and the first operation is implemented based on the first input tensor after decompression. Since the first input tensor cached in the first buffer is compressed data, the area overhead of the first buffer can be reduced.

[0155] In some other embodiments of the present disclosure, when the second neural network layer and the third neural network layer reuse the third input tensor, in order to save the area overhead of the second buffer, the third input tensor can be compressed in advance to obtain the compressed third input tensor. And when the SOC is powered on, the compressed third input tensor is written to the second buffer. Then, when the operation array 313 processes the second neural network layer or the third neural network layer, the compressed third input tensor can be read through the first direct memory access controller 312, and the compressed third processing tensor can be written to the third buffer 3132 in the operation array 313.

[0156] Thus, the controller 3131 is also used to read the third input tensor from the third buffer 3132, and in response to the third input tensor being compressed data, decompress the third input tensor to obtain the decompressed third input tensor, and write the decompressed third input tensor into the third buffer 3132; the third buffer 3132 is also used to cache the decompressed third input tensor. Taking the second neural network layer as an example, the operation circuit 3133 is also used to respond to the operation control signal corresponding to the second neural network layer, read the decompressed third input tensor from the third buffer 3132, and perform the second operation based on the decompressed third input tensor.

[0157] In some other embodiments of the present disclosure, when the fourth neural network layer reuses the first output tensor, in order to save the area overhead of the first buffer or the second buffer, the controller 3131 may first call a compression module (which may be inside the operation array 313 or outside the operation array 313, and the embodiments of the present disclosure are not limited to this) to compress the first output tensor features to obtain the compressed first output tensor, and then write the compressed first output tensor into the first buffer 311 or the second buffer.

[0158] Thus, the controller 3131 is further used to read the first output tensor from the third buffer 3132, and in response to the first output tensor being compressed data, decompress the first output tensor to obtain the first output tensor after decompression, and write the first output tensor after decompression into the third buffer 3132; the third buffer 3132 is also used to cache the first output tensor after decompression. The operation circuit 3133 is also used to respond to the operation control signal corresponding to the fourth neural network layer, read the first output tensor after decompression from the third buffer 3132, and perform the fourth operation based on the first output tensor after decompression.

[0159] In some other embodiments of the present disclosure, when the fifth neural network layer processed by the first processor core 31 and the sixth neural network layer processed by the second processor core 32 reuse the fourth input tensor, in order to save the area overhead of the on-chip memory, the fourth input tensor may be compressed in advance to obtain the compressed fourth input tensor. When the SOC is powered on, the compressed fourth input tensor is written into the on-chip memory.

[0160] Next, when processing the fifth neural network layer, the first processor core 31 can read the compressed fourth input tensor from the on-chip memory through the second direct memory access controller, and write the compressed fourth input tensor into the second buffer in the first processor core 31. Next, the compressed fourth input tensor is read from the second buffer in the first processor core 31 through the first direct memory access controller 312 in the first processor core 31, and the compressed fourth input tensor is written into the third buffer 3132 in the first processor core 31.

[0161] At the same time, when the second processor core 32 processes the sixth neural network layer, the compressed fourth input tensor can be read from the on-chip memory through the second direct memory access controller, and the compressed fourth processing tensor can be written into the second buffer in the second processor core 32. Then, the compressed fourth input tensor is read from the second buffer in the second processor core 32 through the first direct memory access controller in the second processor core 32, and the compressed fourth processing tensor is written into the third buffer of the second processor core 32.

[0162] Thus, the controller 3131 in the first processor core 31 is further used to read the fourth input tensor from the third buffer 3132, and in response to the fourth input tensor being compressed data, decompress the fourth input tensor to obtain the fourth input tensor after decompression, and write the fourth input tensor after decompression into the third buffer 3132; the third buffer 3132 is further used to cache the fourth input tensor after decompression. The operation circuit 3133 is further used to respond to the operation control signal corresponding to the fifth neural network layer, read the fourth input tensor after decompression from the third buffer 3132, and perform the fifth operation based on the fourth input tensor after decompression.

[0163] Since the functions of the controller in the second processor core, the third cache in the second processor core, and the operating circuit in the second processor core are similar to those of the controller in the first processor core, the third cache in the first processor core, and the operating circuit in the first processor core, the embodiments of the present disclosure are not described in detail herein.

[0164] Based on the above embodiments, an embodiment of the present disclosure provides a system on chip. Figure 8 FIG. 1 is a schematic diagram of a system on a chip structure provided by an exemplary embodiment of the present disclosure. Figure 8 As shown, the system on chip 80 may include an on-chip memory 801, a second direct memory access controller 802, and a neural network processor 803, the neural network processor 803 includes at least one processor core, and the at least one processor core includes a first processor core 8031 ​​and a second processor core 8032;

[0165] On-chip memory 801, used for caching the fourth input tensor;

[0166] A second direct memory access controller 802 is used to read a fourth input tensor from the on-chip memory 801 and write the fourth input tensor into the first processor core 8031 ​​and the second processor core 8032;

[0167] The first processor core 8031 ​​is configured to perform a fifth operation corresponding to a fifth neural network layer in the neural network model based on the fourth input tensor;

[0168] The second processor core 8032 is used to respond to the sixth neural network layer and the fifth neural network layer in the neural network model reusing the fourth input tensor, and perform a sixth operation corresponding to the sixth neural network layer based on the fourth input tensor.

[0169] like Fig. 9 As shown, in Figure 8 On the basis of the embodiment shown in the embodiment shown, the first processor core 8031 ​​includes: a second cache 901, a first direct memory access controller 902 and an operation array 903;

[0170] A second direct memory access controller 802, configured to write the fourth input tensor into the second buffer 901;

[0171] A second buffer 901, used for caching a fourth input tensor;

[0172] A first direct memory access controller 902 is used to read the fourth input tensor from the second buffer 901 and write the fourth input tensor into the operation array 903;

[0173] The operation array 903 is used to perform a fifth operation based on the fourth input tensor.

[0174] Since the structure of the second processor core 8032 is the same as that of the first processor core 8031 ​​, the structure of the second processor core 8032 is not described in detail in the embodiment of the present disclosure.

[0175] Based on the above embodiments, the present disclosure provides a data processing method, which is applied to Figure 3 In the first processor core 31 in the neural network processor 30 shown. Fig.10 1001 to 1003.

[0176] Step 1001, cache a first input tensor corresponding to a first neural network layer in a neural network model through a first cache in a first processor core.

[0177] Step 1002, read the second input tensor corresponding to the first neural network layer from the second cache through the first direct memory access controller in the first processor core, and write the second input tensor into the operation array.

[0178] Step 1003, read the first input tensor from the first buffer through the operation array, and perform a first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain a first output tensor.

[0179] like Fig.11 As shown, in Fig.10 Based on the illustrated embodiment, the data processing method further includes the following steps 1004 and 1005 .

[0180] Step 1004, in response to the second neural network layer and the third neural network layer in the neural network model multiplexing the third input tensor, read the third input tensor from the second buffer through the first direct memory access controller, and write the third input tensor into the operation array;

[0181] Step 1005, performing a second operation corresponding to the second neural network layer based on the third input tensor through the operation array, and performing a third operation corresponding to the third neural network layer.

[0182] like Fig.12 As shown in the above Fig.10 Based on the illustrated embodiment, the data method further includes the following steps 1006 and 1007 .

[0183] Step 1006, instructing the fourth neural network layer in the neural network model to reuse the first output tensor in response to the multiplexing control instruction through the operation array, writing the first output tensor to the first buffer based on the multiplexing control signal, and / or writing the first output tensor to the second buffer through the first direct memory access controller.

[0184] Step 1007, read the first output tensor from the first cache through the operation array, and / or, read the first output tensor from the second cache through the first direct memory access controller; perform a fourth operation corresponding to the fourth neural network layer based on the first output tensor.

[0185] like Fig.13 As shown in the above Fig.10 Based on the illustrated embodiment, the data method further includes the following steps 1008 and 1009 .

[0186] Step 1008, read the fourth input tensor corresponding to the fifth neural network layer in the neural network model from the on-chip memory through the second direct memory access controller in the first processor core, and perform the fifth operation corresponding to the fifth neural network layer based on the fourth input tensor.

[0187] Step 1009, in response to the sixth neural network layer and the fifth neural network layer in the neural network model multiplexing the fourth input tensor through the second processor core, read the fourth input tensor from the on-chip memory through the second direct memory access controller, and execute the sixth operation corresponding to the sixth neural network layer based on the fourth input tensor.

[0188] like Fig.14 As shown in the above Fig.10 Based on the illustrated embodiment, step 1002 may include the following step 1401 , and step 1003 may include the following steps 1402 to 1404 .

[0189] Step 1401: read a second input tensor from a second buffer through a first direct memory access controller, and write the second input tensor into a third buffer.

[0190] Step 1402, read the first input tensor from the first buffer through the controller, and write the first input tensor into the third buffer; and generate an operation control signal.

[0191] Step 1403: Cache the first input tensor and the second input tensor through a third buffer.

[0192] Step 1404: In response to the operation control signal, the operation circuit reads the first input tensor and the second input tensor from the third buffer, performs a first operation based on the first input tensor and the second input tensor, and obtains a first output tensor.

[0193] In some embodiments of the present disclosure, the data processing method further includes: generating a selection control signal by a controller. Writing the first input tensor to the third buffer may include: selecting, by a selector, to output the first input tensor in the first buffer to the third buffer in response to the selection control signal. Writing the second input tensor by the first direct memory access controller to the third buffer may include: selecting, by a selector, to output the second input tensor in the first direct memory access controller to the third buffer in response to the selection control signal.

[0194] In some embodiments of the present disclosure, after the first buffer writes the first input tensor into the third buffer, the data processing method further includes: reading the first input tensor from the third buffer through the controller, decompressing the first input tensor in response to the first input tensor being compressed data to obtain the first input tensor after decompression, and writing the first input tensor after decompression into the third buffer. Correspondingly, step 1404 specifically includes: reading the first input tensor and the second input tensor after decompression from the third buffer through the operation circuit in response to the operation control signal, performing the first operation based on the first input tensor and the second input tensor after decompression to obtain the first output tensor.

[0195] Exemplary Electronic Devices

[0196] Fig.15 is a schematic diagram of a structure of an electronic device provided by an exemplary embodiment of the present disclosure, such as Fig.15 As shown, the electronic device 150 includes one or more processors 1501 and a memory 1502 .

[0197] The processor 1501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 150 to perform desired functions.

[0198] The memory 1502 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1501 may execute the program instructions to implement the data processing methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0199] In one example, the electronic device 150 may further include: an input device 1503 and an output device 1504 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0200] Of course, to simplify, Fig.15 Only some of the components related to the present disclosure in the electronic device 150 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 150 may also include any other appropriate components.

[0201] Exemplary computer program products and computer-readable storage media

[0202] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the functional safety testing method of an integrated circuit according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0203] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0204] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the functional safety testing method of an integrated circuit according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0205] Computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0206] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0207] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A neural network processor, comprising a first processor core, wherein the first processor core comprises: A first buffer, used to cache a first input tensor corresponding to a first neural network layer in the neural network model; A first direct memory access controller, configured to read a second input tensor corresponding to the first neural network layer from a second buffer, and write the second input tensor into a computing array; The operation array is used to read the first input tensor from the first buffer, and perform a first operation corresponding to the first neural network layer based on the first input tensor and the second input tensor to obtain a first output tensor.

2. The neural network processor according to claim 1, wherein: The first direct memory access controller is further configured to read the third input tensor from the second buffer and write the third input tensor into the operation array in response to the second neural network layer and the third neural network layer in the neural network model multiplexing the third input tensor; The operation array is also used to perform a second operation corresponding to the second neural network layer based on the third input tensor, and to perform a third operation corresponding to the third neural network layer.

3. The neural network processor according to claim 1, wherein: The operation array is further used to instruct the fourth neural network layer in the neural network model to reuse the first output tensor in response to the multiplexing control instruction, write the first output tensor into the first buffer based on the multiplexing control signal, and / or write the first output tensor into the second buffer through the first direct memory access controller; The operation array is further configured to read the first output tensor from the first buffer, and / or read the first output tensor from the second buffer via the first direct memory access controller; Perform a fourth operation corresponding to the fourth neural network layer based on the first output tensor.

4. The neural network processor of claim 1 , further comprising a second processor core; The first processor core is used to read a fourth input tensor corresponding to a fifth neural network layer in the neural network model from the on-chip memory through a second direct memory access controller, and perform a fifth operation corresponding to the fifth neural network layer based on the fourth input tensor; The second processor core is used to respond to the sixth neural network layer in the neural network model multiplexing the fourth input tensor with the fifth neural network layer, read the fourth input tensor from the on-chip memory through the second direct memory access controller, and execute a sixth operation corresponding to the sixth neural network layer based on the fourth input tensor.

5. The neural network processor according to any one of claims 1 to 4, wherein: The operation array includes a controller, a third buffer and an operation circuit; The controller is configured to read the first input tensor from the first buffer and write the first input tensor into the third buffer; and generate an operation control signal; The first direct memory access controller is used to read the second input tensor from the second buffer and write the second input tensor into the third buffer; The third buffer is used to cache the first input tensor and the second input tensor; The operation circuit is used to read the first input tensor and the second input tensor from the third buffer in response to the operation control signal, perform the first operation based on the first input tensor and the second input tensor, and obtain the first output tensor.

6. The neural network processor according to claim 5, wherein: The operation array also includes a selector; The controller is further used to generate a selection control signal; The selector is used to select, in response to the selection control signal, to output the first input tensor in the first buffer to the third buffer or to output the second input tensor in the first direct memory access controller to the third buffer.

7. The neural network processor according to claim 5, wherein: The controller is further configured to read the first input tensor from the third buffer, decompress the first input tensor in response to the first input tensor being compressed data to obtain a decompressed first input tensor, and write the decompressed first input tensor into the third buffer; The third buffer is further used to cache the first input tensor and the second input tensor after the decompression process; The operation circuit is used to respond to the operation control signal, read the first input tensor and the second input tensor after decompression from the third buffer, perform the first operation based on the first input tensor and the second input tensor after decompression, and obtain the first output tensor.

8. The neural network processor according to any one of claims 1 to 4, wherein: The first processor core includes the second cache.

9. A system on chip, comprising: An on-chip memory, a second direct memory access controller, and a neural network processor, wherein the neural network processor includes at least one processor core, wherein the at least one processor core includes a first processor core and a second processor core; The on-chip memory is used to cache the fourth input tensor; The second direct memory access controller is configured to read the fourth input tensor from the on-chip memory and write the fourth input tensor into the first processor core and the second processor core; The first processor core is used to perform a fifth operation corresponding to a fifth neural network layer in the neural network model based on the fourth input tensor; The second processor core is used to execute a sixth operation corresponding to the sixth neural network layer based on the fourth input tensor in response to the sixth neural network layer and the fifth neural network layer in the neural network model reusing the fourth input tensor.

10. The system on chip according to claim 9, wherein: The first processor core includes a second cache, a first direct memory access controller and an operation array; The second direct memory access controller is configured to write the fourth input tensor into the second buffer; The second buffer is used to cache the fourth input tensor; The first direct memory access controller is used to read the fourth input tensor from the second buffer and write the fourth input tensor into the operation array; The operation array is used to perform the fifth operation based on the fourth input tensor.

11. A data processing method, applied to a first processor core in a neural network processor, the method comprising: caching a first input tensor corresponding to a first neural network layer in a neural network model by a first buffer in the first processor core; Reading a second input tensor corresponding to the first neural network layer from a second buffer through a first direct memory access controller in the first processor core, and writing the second input tensor into a computing array in the first processor core; The first input tensor is read from the first buffer through the operation array, and a first operation corresponding to the first neural network layer is performed based on the first input tensor and the second input tensor to obtain a first output tensor.

12. An electronic device comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the data processing method described in claim 11.

13. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the data processing method according to claim 11.

Images