Single-core processing core and control method thereof, many-core communication architecture, chip

Abstract

The application discloses a single-core processing core and a control method thereof, a many-core communication architecture and a chip. Each single-core processing core has an on-chip network interface and a memory access interface for external data transmission, and inter-core data transmission is performed through the on-chip network interface. The single-core processing core has a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus. The queue transmission unit is used for realizing hardware queue expansion and acceleration between front and back. The weight gradient transmission unit is used for providing transmission acceleration for frequent reading and writing of weight gradient DRAM memory of a deep neural network. The weight gradient is stored in the DRAM memory to accelerate gradient reading and writing access of the weight gradient calculation. Based on this, the embodiment of the application can improve the transmission performance and energy efficiency of inter-core communication and reduce the DNN training data transmission delay.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the chip design technical field, in particular to a single-core processing core and a control method thereof, a many-core communication architecture and a chip. BACKGROUND

[0002] In the related art, the current mainstream DNN (Deep Neural Networks) training platform adopts a GPU (Graphics Processing Unit) chip, and the advantages of the GPU mainly lie in the acceleration of calculation and high parallelism, and the GPU has strong versatility in operators and communication; however, the GPU is based on a shared memory transmission architecture, and inter-core communication is realized based on a multi-level cache and a cache consistency protocol, and the transmission architecture cannot be well adapted to the characteristics of DNN training data flow and model static deployment, and there is still a large optimization space for transmission delay and energy efficiency. SUMMARY

[0003] The embodiment of the present application provides a single-core processing core and a control method thereof, a many-core communication architecture and a chip, which can improve the transmission performance and energy efficiency of inter-core communication and reduce the DNN training data transmission delay.

[0004] In a first aspect, an embodiment of the present application provides a many-core communication architecture, the single-core processing core has an on-chip network interface and a memory access interface for external data transmission, the single-core processing core internally includes a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus, the control unit is responsible for the activation of computing instructions, memory transmission instructions and network transmission instructions, the computing instructions are pre-programmed and stored in the computing engine, the memory transmission instructions are pre-programmed and stored in the DMA unit, the memory transmission instructions include DMA write instructions and DMA read instructions, the network transmission instructions are pre-programmed and stored in the NI unit, the network transmission instructions include NI send instructions and NI receive instructions, the control unit is electrically connected with the computing engine, the NI unit, the DMA unit, the queue transmission unit through the control bus respectively, the data exchange unit is electrically connected with the parameter RAM, the NI unit, the DMA unit, the queue transmission unit, the weight gradient transmission unit through the data exchange bus respectively, the queue transmission unit is used to realize the hardware queue expansion and acceleration between the front and back, the weight gradient transmission unit is used to provide transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory, the weight gradient is stored in the DRAM memory, and the control unit triggers the queue transmission unit to generate a queue transmission instruction through a control signal, and the computing engine triggers the weight gradient transmission unit to generate a weight gradient transmission instruction through a control signal.

[0005] In some embodiments, the NI unit has the on-chip network interface, the NI unit sends data of the parameter RAM in the source single-core processing core to the destination single-core processing core according to the NI send instruction, and writes data received from the destination single-core processing core into the parameter RAM in the source single-core processing core according to the NI receive instruction, and the NI send instruction and the NI receive instruction are associated by the same tag number.

[0006] In some embodiments, the DMA unit is used to realize data transmission between the parameter RAM and the DRAM memory, the DMA write instruction and the DMA read instruction contain data type coding, source address information and target address information corresponding to the parameter RAM, the source address information of the DMA write instruction and the target address information of the DMA read instruction are used to indicate the internal offset of the parameter RAM, and the data exchange unit accesses the parameter RAM according to the data type coding, the source address information and the target address information.

[0007] In some embodiments, the queue transfer unit provides a first-in-first-out hardware queue mechanism to accelerate data transfer between the front and back directions before and after training, and provides a configurable queue depth in the DRAM memory.

[0008] In some embodiments, the computing engine forms an interlocking fit with the weight gradient transfer unit.

[0009] In some embodiments, the data exchange unit is used to provide data exchange between the parameter RAM and the core, and the data exchange unit uses ID addressing, unified network and memory access data scheduling in the single-core processing core.

[0010] In a second aspect, embodiments of the present application provide a many-core communication architecture, having at least two single-core processing cores as described in the first aspect, the on-chip network interface is used for data transmission between any two single-core processing cores, the on-chip network interface is electrically connected to the on-chip router, the on-chip router is used for data forwarding between any two single-core processing cores, the single-core processing core exclusively occupies a block of dynamic random access memory (DRAM) space, and the memory logic is isolated between any two single-core processing cores.

[0011] In a third aspect, embodiments of the present application provide a chip comprising the many-core communication architecture as described in the second aspect.

[0012] In a fourth aspect, embodiments of the present application provide a control method of a single-core processing core, applied to the single-core processing core as described in the first aspect, the method comprising:

[0013] In the case of determining that the execution of the previous instruction group is completed, the control unit activates the NI receiving instruction corresponding to the NI unit and the DMA reading instruction corresponding to the DMA unit, and starts data input transmission to the parameter RAM;

[0014] The control unit sends a control signal to the queue transfer unit, so that the queue transfer unit generates a queue transfer instruction according to the control signal, and starts queue data input transmission to the parameter RAM according to the queue transfer instruction;

[0015] In the case of checking that the input data is completed, the control unit activates the computing engine to start calculation;

[0016] In the case of determining that the calculation of the computing engine is completed, the control unit activates the NI sending instruction corresponding to the NI unit and the DMA writing instruction corresponding to the DMA unit, wherein the NI sending instruction contains the queue data between the front and back directions.

[0017] In the case of checking that the data output is completed, the control unit determines that the execution of the current instruction group is completed.

[0018] In some embodiments, the method further comprises:

[0019] The queue transfer unit receives a queue message from the NI unit;

[0020] Locate the hardware queue ID according to the data type encoding in the message header of the queue message;

[0021] Maintain the tail pointer and head pointer corresponding to the hardware queue ID;

[0022] When the input data associated with the calculation instruction needs to use the queue data, the control unit generates a queue_load instruction and distributes it to the queue transfer unit;

[0023] The queue transfer unit reads the data block according to the head pointer and transfers it to the parameter RAM.

[0024] In some embodiments, the method further comprises:

[0025] When the calculation engine completes the calculation of a single weight gradient element, it notifies the weight gradient transfer unit to start by setting the lock signal to 1;

[0026] After the weight gradient transfer unit completes the current element store and the next weight gradient element load, it notifies the calculation engine to start the next weight gradient element calculation by clearing the lock signal;

[0027] Provide the address of the weight gradient in the DRAM;

[0028] Execute a DMA write instruction to transfer the weight gradient element from the SRAM to the DRAM;

[0029] Execute a DMA read instruction to transfer the weight gradient element from the DRAM to the SRAM.

[0030] The single-core processing core and the control method thereof, the many-core communication architecture and the chip are provided according to the embodiments of the present application. The single-core processing core of one embodiment has an on-chip network interface and a memory access interface for external data transmission. The single-core processing core internally includes a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus. The control unit is responsible for the activation of computing instructions, memory transmission instructions and network transmission instructions. The computing instructions are pre-arranged and stored in the computing engine. The memory transmission instructions are pre-arranged and stored in the DMA unit. The memory transmission instructions include DMA write instructions and DMA read instructions. The network transmission instructions are pre-arranged and stored in the NI unit. The network transmission instructions include NI sending instructions and NI receiving instructions. The control unit is electrically connected with the computing engine, the NI unit, the DMA unit and the queue transmission unit through the control bus. The data exchange unit is electrically connected with the parameter RAM, the NI unit, the DMA unit, the queue transmission unit and the weight gradient transmission unit through the data exchange bus. The queue transmission unit is used to realize the hardware queue expansion and acceleration between the front and back. The weight gradient transmission unit is used to provide transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory. The weight gradient is stored in the DRAM memory. The control unit triggers the queue transmission unit to generate a queue transmission instruction through a control signal. The computing engine triggers the weight gradient transmission unit to generate a weight gradient transmission instruction through a control signal. The many-core communication architecture of one embodiment has at least two single-core processing cores described above. The single-core processing core has an on-chip network interface and a memory access interface for external data transmission. The on-chip network interface is used for data transmission between any two single-core processing cores. The on-chip network interface is electrically connected with an on-chip router. The on-chip router is used for data forwarding between any two single-core processing cores. The single-core processing core exclusively occupies a dynamic random access memory (DRAM) space. The memory of any two single-core processing cores is logically isolated. The chip of one embodiment includes the many-core communication architecture described above.The control method of the single-core processing core of one embodiment is applied to the single-core processing core described above, and the method comprises the following steps: in the case that it is determined that the execution of the previous instruction group is completed, the control unit activates the NI receiving instruction corresponding to the NI unit and the DMA reading instruction corresponding to the DMA unit, and starts the data input transmission to the parameter RAM; the control unit sends a control signal to the queue transmission unit, so that the queue transmission unit generates a queue transmission instruction according to the control signal, and starts the queue data input transmission to the parameter RAM according to the queue transmission instruction; in the case that it is checked that the input data is completed, the control unit activates the calculation engine to start the calculation; in the case that it is determined that the calculation of the calculation engine is completed, the control unit activates the NI sending instruction corresponding to the NI unit and the DMA writing instruction corresponding to the DMA unit, wherein the NI sending instruction contains the queue data between the front and back directions; in the case that it is checked that the data output is completed, the control unit determines that the execution of the current instruction group is completed. Based on this, each single-core processing core has an on-chip network interface and a memory access interface for external data transmission, which supports two sets of data transmission modes, namely on-chip network access and memory access, and the inter-core data transmission is performed through the on-chip network interface. The single-core processing core has a calculation engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus inside, and the single-core processing core provides programmable transmission instructions, the transmission instructions include memory transmission instructions and network transmission instructions, the memory transmission instructions are pre-programmed and stored in the DMA unit, and the network transmission instructions are pre-programmed and stored in the NI unit, which is suitable for various models and deployment schemes, wherein the queue transmission instruction and the weight gradient transmission instruction are generated in real time according to the control signal of the control unit, and do not need to be pre-programmed by software. The control unit activates the calculation instruction of the calculation engine, the calculation engine performs corresponding calculation, and notifies the control unit after the calculation is completed, so as to accelerate the connection between transmission and calculation, realize the quick switching of processing, transmission and calculation. The queue transmission unit and the weight gradient transmission unit inside the single-core processing core are special hardware transmission units for DNN training, the queue transmission unit is used to realize the hardware queue expansion and acceleration between the front and back directions, and the weight gradient transmission unit is used to provide transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory, the weight gradient is stored in the DRAM memory, so as to accelerate the gradient reading and writing access of the weight gradient calculation. The special hardware transmission unit for DNN training can offload the control and management overhead of software, and reduce the DNN training data transmission delay. Therefore, the embodiment of the present application can improve the transmission performance and energy efficiency of inter-core communication, and reduce the DNN training data transmission delay.

[0031] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structures particularly pointed out in the description and claims of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings are included to provide a further understanding of the technical scheme of the present application, constitute a part of the specification and are used together with the embodiments of the present application to explain the technical scheme of the present application, and do not constitute a limitation on the technical scheme of the present application.

[0033] Figure 1 is a structural schematic diagram of a many-core communication architecture provided by an embodiment of the present application;

[0034] Figure 2 is a structural schematic diagram of a single-core processing core provided by an embodiment of the present application;

[0035] Figure 3A is a control method flowchart of a single-core processing core provided by an embodiment of the present application;

[0036] Figure 3B is a control flowchart of a control unit provided by an embodiment of the present application;

[0037] Figure 4A is a control method flowchart of a single-core processing core provided by another embodiment of the present application;

[0038] Figure 4B is a control flowchart of a queue transmission unit provided by an embodiment of the present application;

[0039] Figure 5A is a control method flowchart of a single-core processing core provided by another embodiment of the present application;

[0040] Figure 5B is a control flowchart of a weight gradient transmission unit provided by an embodiment of the present application;

[0041] Figure 6 is a schematic diagram of cooperation of a calculation instruction and a transmission instruction to realize different model state machines provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0043] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple) is more than two, greater than, less than, more than, etc. is not included in the number, above, below, within, etc. is understood to include the number. If there is a description of "first", "second" and the like, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the order of the indicated technical features.

[0044] A deep neural network (DNN) is a multi-layer network computing model that simulates the working of the human nervous system and has been widely used in automatic language translation and image recognition. The training of DNN is usually completed through supervised learning, including two main steps of forward propagation and back propagation. Forward propagation is the process of transmitting input data from the input layer to the output layer, and back propagation is the process of transmitting errors from the output layer to the input layer, adjusting the weights and offsets between neurons in each layer according to the errors, so that the output of the network can correctly reflect the actual situation of the input data. Back propagation can be divided into two sub-steps, including error propagation and weight gradient.

[0045] For the research and development of DNN training acceleration, the three steps of FP, EP and WG are mainly implemented. The neural network model is continuously developing, and the operator and data connection relationship is constantly changing, but the functions of the three units of FP, EP and WG and the connection relationship between the units have not changed in essence in the training field. Therefore, the chip design for training acceleration can form a set of reusable and continued architecture around these three units, and the reuse degree of transmission is relatively higher.

[0046] Taking a convolutional neural network (CNN) as an example, FP mainly handles the convolution calculation of activation function A and weight W and proceeds layer by layer in the forward direction, EP mainly handles the convolution calculation of error E and weight W and proceeds layer by layer in the reverse direction, and WG mainly handles the convolution of activation function A (provided in the forward direction) and error E and proceeds synchronously with the reverse EP processing. With the increase of the number of DNN layers, the regularization method BN (based on batch samples) in the neural network is introduced. BN increases the data transmission from forward FP to reverse EP, including mean / variance and other intermediate data. BN increases the learning update of scaling parameters γ and β, which can be regarded as an extension of weight W, and the gradient calculation of γ and β needs to be increased in WG. The introduction of residual changes the connection relationship between layers, but the layer-to-layer correspondence and data connection between forward and backward have not changed. The spiking neural network (SNN) mainly changes the activation function A to the pulse S form, and introduces the operator of time dimension, which increases the arrangement dimension T of the model.

[0047] There are two main ways to improve model training deployment performance: data parallelism and model parallelism. Data parallelism requires each node to contain all the parameters and weights of the model, and different nodes run different sample data at the same time. Finally, the accumulated results of each node WG are updated to the weight W. Data parallelism requires high memory capacity for a single node, so it is more suitable for cluster deployment solutions, i.e., a single node corresponds to a system composed of multiple chip interconnections. Model parallelism divides the model into multiple regions, and the regions are in pipeline parallelism. A single region is suitable for chip-level or on-chip single-core deployment. Model parallelism requires sample data to flow between cores. Since the data flow does not need to turn back between cores, it is more suitable for chip network (NOC) and many-core architecture. GPGPU can also implement model parallelism, based on shared memory architecture, unified memory addressing, and suitable for address-continuous memory access.

[0048] One challenge of chip design supporting model parallelism is that the pipeline requires a large amount of memory support. Since neural network training requires data caching between forward and backward, forward to backward needs to be pushed layer by layer, the time delay increases linearly with the number of DNN network layers, and the required cache capacity also linearly increases. Although there are other ways to reduce cache requirements, such as reducing data precision, alternating forward and backward flow, and breakpoint recalculation, for chip design, even with a NOC architecture, it is still necessary to design a high-bandwidth channel between the many-core and DRAM memory and an efficient transmission method to save the time cost of training.

[0049] In related technologies, the current mainstream DNN (Deep Neural Networks) training platform uses a GPU (Graphics Processing Unit) chip. The advantages of GPU mainly lie in the acceleration of calculation and high parallelism, and it has strong versatility in terms of operators and communication. However, the transmission architecture of GPU is based on shared memory, and inter-core communication is achieved based on multi-level cache and cache consistency protocol, which cannot well adapt to the characteristics of DNN training data flow and model static deployment. There is still a lot of optimization space for transmission delay and energy efficiency.

[0050] To solve the above technical problems, the embodiment of the present application provides a single core and a control method thereof, a many-core communication architecture and a chip. The single core processing core of one embodiment has an on-chip network interface and a memory access interface for external data transmission. The single core processing core internally includes a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus. The control unit is responsible for the activation of computing instructions, memory transmission instructions and network transmission instructions. The computing instructions are pre-arranged and stored in the computing engine. The memory transmission instructions are pre-arranged and stored in the DMA unit. The memory transmission instructions include DMA write instructions and DMA read instructions. The network transmission instructions are pre-arranged and stored in the NI unit. The network transmission instructions include NI send instructions and NI receive instructions. The control unit is electrically connected with the computing engine, the NI unit, the DMA unit and the queue transmission unit through the control bus. The data exchange unit is electrically connected with the parameter RAM, the NI unit, the DMA unit, the queue transmission unit and the weight gradient transmission unit through the data exchange bus. The queue transmission unit is used to realize the hardware queue expansion and acceleration between the front and back. The weight gradient transmission unit is used to provide transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory. The weight gradient is stored in the DRAM memory. The control unit triggers the queue transmission unit to generate a queue transmission instruction through a control signal. The computing engine triggers the weight gradient transmission unit to generate a weight gradient transmission instruction through a control signal. The many-core communication architecture of one embodiment has at least two single core processing cores described above. The single core processing core has an on-chip network interface and a memory access interface for external data transmission. The on-chip network interface is used for data transmission between any two single core processing cores. The on-chip network interface is electrically connected with an on-chip router. The on-chip router is used for data forwarding between any two single core processing cores. The single core processing core exclusively occupies a dynamic random access memory (DRAM) space. The memory of any two single core processing cores is logically isolated. The chip of one embodiment includes the many-core communication architecture described above.The control method of the single-core processing core of one of the embodiments is applied to the single-core processing core described above, and the method comprises the following steps: in the case of determining that the execution of the previous instruction group is completed, the control unit activates the NI receiving instruction corresponding to the NI unit and the DMA reading instruction corresponding to the DMA unit, and starts the data input transmission to the parameter RAM; the control unit sends a control signal to the queue transmission unit, so that the queue transmission unit generates a queue transmission instruction according to the control signal, and starts the queue data input transmission to the parameter RAM according to the queue transmission instruction; in the case of checking that the input data is completed, the control unit activates the calculation engine to start calculation; in the case of determining that the calculation of the calculation engine is completed, the control unit activates the NI sending instruction corresponding to the NI unit and the DMA writing instruction corresponding to the DMA unit, wherein the NI sending instruction contains the queue data between the front and back directions; in the case of checking that the data output is completed, the control unit determines that the execution of the current instruction group is completed. Based on this, each single-core processing core has an on-chip network interface and a memory access interface for external data transmission, which supports two sets of data transmission modes, namely on-chip network access and memory access, and the inter-core data transmission is through the on-chip network interface. The single-core processing core has a calculation engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus inside, and the single-core processing core provides programmable transmission instructions, the transmission instructions include memory transmission instructions and network transmission instructions, the memory transmission instructions are pre-programmed and stored in the DMA unit, and the network transmission instructions are pre-programmed and stored in the NI unit, which is suitable for various models and deployment schemes, wherein the queue transmission instruction and the weight gradient transmission instruction are generated in real time according to the control signal of the control unit, and do not need to be pre-programmed by software. The control unit activates the calculation instruction of the calculation engine, and the calculation engine corresponding to the calculation instruction notifies the control unit after the completion, so as to accelerate the transmission and calculation, realize the quick switching of processing and transmission, and realize the quick switching of processing and transmission. The queue transmission unit and the weight gradient transmission unit inside the single-core processing core are special hardware transmission units for DNN training, the queue transmission unit is used for realizing the hardware queue expansion and acceleration between the front and back directions, and the weight gradient transmission unit is used for providing transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory, the weight gradient is stored in the DRAM memory, so as to accelerate the gradient reading and writing access of the weight gradient calculation. The special hardware transmission unit for DNN training can unload the control and management overhead of software, and reduce the DNN training data transmission delay. Therefore, the embodiment of the application can improve the transmission performance and energy efficiency of inter-core communication, and reduce the DNN training data transmission delay.

[0051] As Figure 1 and Figure 2 shown, Figure 1 is a many-core communication architecture structure schematic diagram provided by one embodiment of the application, Figure 2is a single-core internal structure schematic diagram provided by an embodiment of the present application. Each single-core processing core has an on-chip network interface and a memory access interface for external data transmission, which supports two sets of data transmission methods, namely on-chip network access and memory access, and inter-core data transmission goes through the on-chip network interface. Each single-core processing core is mainly composed of a computing engine 201, a control unit 202, a data exchange unit 203, a parameter RAM (Random Access Memory) 204, an NI unit 205, a DMA (Di rect Memory Access) unit 206, a queue transmission unit 207, a WG (Weight Grad ients) transmission unit 208, a data exchange bus 209 and a control bus 210. The control unit 202 is electrically connected with the computing engine 201, the NI unit 205, the DMA unit 206 and the queue transmission unit 207 through the control bus 210, and the data exchange unit 203 is electrically connected with the parameter RAM 204, the NI unit 205, the DMA unit 206, the queue transmission unit 207 and the WG transmission unit 208 through the data exchange bus 209. The in-core instructions are distributed in each unit, for example, the computing instructions are pre-arranged in the computing engine 201, the memory transmission instructions are pre-arranged in the DMA unit 206, and the network transmission instructions are pre-arranged in the NI unit 205, which is suitable for various models and deployment schemes. The queue transmission instructions and the weight gradient transmission instructions are generated in real time according to the control signal of the control unit 202, without software pre-arrangement. It should be noted that the queue transmission unit and the weight gradient transmission unit are special hardware transmission units for DNN training, the queue transmission unit is used to realize the hardware queue expansion and acceleration between the forward and backward directions, and the weight gradient transmission unit is used to provide transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory, and the weight gradient is stored in the DRAM memory to accelerate the gradient reading and writing access of the weight gradient calculation. The special hardware transmission unit for DNN training can offload the control and management overhead of software and reduce the DNN training data transmission delay. Therefore, the embodiment of the present application can improve the transmission performance and energy efficiency of inter-core communication and reduce the DNN training data transmission delay.

[0052] It can be understood that the control unit 202 is responsible for the activation of the calculation instruction, the memory transmission instruction and the network transmission instruction, and controls the overall execution sequence. The instructions are pre-arranged in each unit in the core, for example, the calculation instruction is pre-arranged in the calculation engine 201, the memory transmission instruction is pre-arranged in the DMA unit 206, and the network transmission instruction is pre-arranged in the NI unit 205. The control unit 202 activates a group of related calculation and transmission instructions round by round based on the model cutting and the task deployed to the single core. The group is executed in series, and the effect is similar to the memory barrier in the processor. The control within the group is in the order of input data->calculation->output data.

[0053] It can be understood that the calculation engine 201 receives the calculation instruction activation information from the control unit 202, starts the corresponding calculation, and notifies the control unit 202 after completion, so as to accelerate the transmission and calculation, and realize the quick switching of processing and transmission and calculation. The calculation instruction contains the operator type, the internal offset of the RAM input and output parameter, the parameter size and the like.

[0054] It can be understood that the parameter RAM 204 provides the input data buffer required by the calculation and the result output buffer.

[0055] It can be understood that the NI unit 205 has an on-chip network interface, and the network transmission instruction includes an NI sending instruction and an NI receiving instruction. The NI unit sends the data of the parameter RAM in the source single-core processing core to the destination single-core processing core according to the NI sending instruction, and writes the data received from the destination single-core processing core into the parameter RAM in the source single-core processing core according to the NI receiving instruction. The pair of NI sending instruction and NI receiving instruction are associated through the same tag number.

[0056] It can be understood that the NI unit 205 provides interface processing of network transmission. The NI sends the data transmission from the parameter RAM to other cores, and the NI receives the data received from other cores to write into the corresponding parameter RAM. The NI instruction provides routing information, indicating that the network reaches the destination core. The NI instruction provides the parameter type and the parameter size, indicating the docking of the NI and the parameter RAM. The NI sending instruction is shown in the following table TABLE1, supporting unicast and multicast.

[0057] TABLE1

[0058] Domain name Description tag Pair with receive mode Send unicast src_addr RAM internal offset address dst_ID Destination ID dcode Data type code size Transfer length

[0059] The NI receiving instruction is shown in the following table TABLE2, supporting the network receiving 1-to-1 and multi-to-1 aggregation communication model.

[0060] TABLE2

[0061] Domain name Description tag Pair with send mode Receive independent, aggregate dst_addr RAM internal offset address src_ID Source ID dcode Data type code size Transfer length

[0062] NI instruction is passive, after receiving network message, it needs to transmit data to destination according to NI instruction. NI sending and receiving are associated through same TAG label number, and network message header carries TAG information.

[0063] It can be understood that the DMA unit 206 is used to realize data transmission between the parameter RAM and the DRAM memory, and the memory transmission instruction includes a DMA write instruction and a DMA read instruction, the DMA write instruction and the DMA read instruction contain data type coding, source address information and target address information of the corresponding parameter RAM, and the source address information and the target address information are used to indicate internal offsets of the parameter RAM, and the data exchange unit accesses the parameter RAM according to the data type coding, the source address information and the target address information.

[0064] It can be understood that the DMA unit 206 realizes data transmission between the general parameter RAM and the DRAM memory. The DMA write transmits data from the parameter RAM to the DRAM. The DMA read transmits data from the DRAM to the corresponding parameter RAM. In order to realize fusion transmission of the DMA and the NI in the core, the instruction of the DMA is specially designed, and specific reference can be made to the following table TABLE 3.

[0065] TABLE 3

[0066] Domain name Description mode DMA read or write src_addr Source address, DMA read as RAM internal offset address dst_addr Destination address, DMA write as RAM internal offset address dcode Data type code size Transfer length

[0067] The DMA instruction provides dcode data type coding corresponding to the specific parameter RAM. The DMA write src_addr and the DMA read dst_addr indicate internal offsets of the parameter RAM. Therefore, the data exchange unit 203 can access the parameter RAM according to the data type dcode and the RAM internal offset.

[0068] It can be understood that the queue transmission unit 207 realizes hardware queue expansion and acceleration between the front and the back. The queue transmission unit 207 provides a hardware queue mechanism to accelerate data transmission between the front and the back, and provides a configurable queue depth in the DRAM. The data is managed as a whole according to the queue FlF0 (Firstlnput First Output, first-in first-out), and the data block size and the internal data format are pre-set. When the back core receives a queue type data coding from the network interface NI, the queue transmission unit 207 automatically generates a queue_store instruction, and stores data in the queue buffer according to a queue tail pointer. When the input data associated with the calculation instruction needs to use the queue data, the control unit 202 generates a queue_load instruction and distributes it to the queue transmission unit 207, and reads the data block according to a queue head pointer and transmits it to the parameter RAM.

[0069] It can be understood that the WG transmission unit 208 is a transmission acceleration unit provided for the frequent read and write of the WG in the DRAM memory. In the neural network model DNN, the weight W storage occupies the largest proportion, the gradient involved in the WG needs to be processed in batches, occupies the same cache capacity as W, and needs to be frequently accessed in memory read and write. The WG transmission unit 208 supports storing the gradient in the DRAM and accelerating the gradient read and write access of the WG calculation.

[0070] It can be understood that the computing engine 201 and the WG transmission unit 208 form interlocking cooperation, the current WG calculation result is written back to the memory, and the next round of WG data is transmitted from the memory to the parameter RAM, and both are completed. The WG calculation is started. The WG transmission unit does not need software dynamic intervention, and is automatically triggered by the interlocking control signal, realizing low-latency WG transmission.

[0071] It can be understood that the data exchange unit 203 provides data exchange between the parameter RAM and the core. A single core presents a continuous address space to the outside, and the internal parameter RAM addressing is completed through the data type dcode. That is, the data exchange unit 203 uses ID addressing in the core, and unifies the data scheduling of network and memory access. In the core, the ID+RAM internal offset address mode is used for unified addressing.

[0072] It can be understood that the data exchange bus 209 carries the unified message transmission format in the core based on ID addressing.

[0073] It can be understood that the control bus 210 carries the trigger signal and instruction activation information between the control unit 202 and the computing engine 201, the NI unit 205, the DMA unit 206, and the queue transmission unit 207.

[0074] Based on this, the embodiment of the application can improve the transmission performance and energy efficiency of inter-core communication, and reduce the DNN training data transmission delay.

[0075] The embodiment of the application also provides a many-core communication architecture, as shown in Figure 1 The many-core communication architecture has at least two single-core processing cores as described above. Each single-core processing core has an on-chip network interface and a memory access interface for external data transmission. Inter-core data transmission is through the on-chip network interface. The on-chip network interface is used for data transmission between any two single-core processing cores. The on-chip network interface is electrically connected to the on-chip router router. The on-chip router router is used for data forwarding between any two single-core processing cores. The single-core processing core exclusively occupies a block of dynamic random access memory DRAM space. The memory of any two single-core processing cores is logically isolated, which is easy for system deployment and expansion. The many-core communication architecture is expanded through the on-chip network. The DRAM memory is mainly used for storage capacity expansion of the core.

[0076] In an embodiment, since the many-core communication architecture adopts at least two single-core processing cores described above, the many-core communication architecture can achieve the same technical effects as the single-core processing core. Each single-core processing core of the many-core communication architecture has a network-on-chip interface and a memory access interface for external data transmission, which supports two sets of data transmission modes, namely network-on-chip access and memory access, and the inter-core data transmission goes through the network-on-chip interface. The single-core processing core has a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus, and a control bus inside, and the single-core processing core provides programmable transmission instructions, including memory transmission instructions and network transmission instructions. The memory transmission instructions are pre-programmed and stored in the DMA unit, and the network transmission instructions are pre-programmed and stored in the NI unit, which is suitable for various models and deployment schemes. Among them, the queue transmission instruction and the weight gradient transmission instruction are generated in real time according to the control signal of the control unit, without the need for software pre-programming. The control unit activates the computing instruction of the computing engine, and the computing engine corresponding to the computing instruction notifies the control unit after the computing is completed, so as to accelerate the transmission and computing connection and realize the rapid switching of processing and transmission. The queue transmission unit and the weight gradient transmission unit inside the single-core processing core are special hardware transmission units for DNN training. The queue transmission unit is used to realize the hardware queue expansion and acceleration between forward and backward, and the weight gradient transmission unit is used to provide transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory. The weight gradient is stored in the DRAM memory to accelerate the gradient read and write access of the weight gradient calculation. The special hardware transmission unit for DNN training can offload the control and management overhead of software and reduce the DNN training data transmission delay. It should be pointed out that, due to the characteristics of DNN training acceleration, NOC fusion, and shared memory, the many-core communication architecture of the present application has higher energy efficiency, reusability and inheritance advantages, and is more suitable for supporting coarse-grained computing instructions. The coarse-grained computing instruction has many types of parameters and large amount of data, which can take advantage of the parallel and continuous efficient characteristics of transmission. Therefore, the many-core communication architecture of the present application can improve the transmission performance and energy efficiency of inter-core communication and reduce the DNN training data transmission delay.

[0077] The chip has the many-core communication architecture, each single-core processing core of the many-core communication architecture has an on-chip network interface and a memory access interface for external data transmission, supports two sets of data transmission modes, namely on-chip network access and memory access, and the inter-core data transmission is performed through the on-chip network interface. The single-core processing core has a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transmission unit, a weight gradient transmission unit, a data exchange bus and a control bus inside, and the single-core processing core provides programmable transmission instructions, the transmission instructions include memory transmission instructions and network transmission instructions, the memory transmission instructions are pre-programmed and stored in the DMA unit, and the network transmission instructions are pre-programmed and stored in the NI unit, and the single-core processing core is suitable for various models and deployment schemes. The queue transmission instruction and the weight gradient transmission instruction are generated in real time according to the control signal of the control unit, and do not need to be pre-programmed by software. The control unit activates the computing instruction of the computing engine, the computing engine performs corresponding computing, and after the computing is completed, the control unit is informed to accelerate the transmission and the computing, and realize the fast switching of processing and transmission. The queue transmission unit and the weight gradient transmission unit inside the single-core processing core are special hardware transmission units for DNN training, the queue transmission unit is used for realizing the hardware queue expansion and acceleration between forward and backward, and the weight gradient transmission unit is used for providing transmission acceleration for the frequent reading and writing of the weight gradient of the deep neural network in the DRAM memory. The weight gradient is stored in the DRAM memory to accelerate the gradient reading and writing access of the weight gradient calculation. The special hardware transmission unit for DNN training can offload the control and management overhead of software and reduce the DNN training data transmission delay. It should be pointed out that, the chip has the characteristics of DNN training acceleration, NOC fusion and shared memory, the many-core communication architecture has higher energy efficiency, reuse and inheritance advantages, and is more suitable for supporting coarse-grained computing instructions. The coarse-grained computing instruction has many types of parameters and large data, and can play the parallel and continuous efficient characteristics of transmission. Therefore, the chip can improve the transmission performance and energy efficiency of inter-core communication and reduce the DNN training data transmission delay.

[0078] As Figure 3A shown, the embodiment of the application further provides a control method of a single-core processing core, and the control method is applied to the single-core processing core.

[0079] The control method includes but is not limited to the following steps:

[0080] In step S11, in the case where it is determined that the execution of the previous instruction group is completed, the control unit activates the NI receiving instruction corresponding to the NI unit and the DMA reading instruction corresponding to the DMA unit, and starts the data input transmission to the parameter RAM;

[0081] Step S12, the control unit sends a control signal to the queue transmission unit to make the queue transmission unit generate a queue transmission instruction according to the control signal, and start the queue data to the parameter RAM input transmission according to the queue transmission instruction;

[0082] Step S13, in the case of checking the input data completion, the control unit activates the calculation engine to start the calculation;

[0083] Step S14, in the case of determining the calculation engine calculation completion, the control unit activates the NI unit corresponding to the NI sending instruction and the DMA unit corresponding to the DMA write instruction, wherein the NI sending instruction contains the queue data between the front and back;

[0084] Step S15, in the case of checking the data output completion, the control unit determines that the current round of instruction group execution ends.

[0085] It can be understood that, Figure 3B For the control flow of the control unit, the control flow is based on the continuous circulation of the instruction group. The execution order in the instruction group guarantees data input transmission -> calculation -> data output transmission. Multiple transmission instructions are input concurrently, and multiple transmission instructions are output concurrently. The application realizes the acceleration of transmission and calculation by performing transmission instruction completion check, processing transmission and calculation fast switching, realizing the connection of acceleration transmission and calculation, so as to improve the transmission performance and energy efficiency. It should be noted that the instructions generated by the hardware itself are not in the flow of the control unit, such as the operation of the WG. The specific control flow of the control unit is as follows:

[0086] S301: Check the current round of instruction group start condition, which requires the execution of the last round of instruction group to be completed.

[0087] S302: Identify the calculation related NI receiving and DMA read instructions, generate activation information to the NI unit and the DMA unit, and each transmission unit executes the instructions of the activated area to start the data to the parameter RAM input transmission.

[0088] S303: Identify the calculation related queue transmission instruction, trigger the queue_load instruction to the queue transmission unit to start the queue data to the parameter RAM input transmission.

[0089] S304: Check whether the activated NI receiving, DMA read, queue_load instructions are executed, and wait in a loop.

[0090] S305: Activate the calculation related instructions, and the calculation engine starts the calculation.

[0091] S306: Wait for the calculation completion indication.

[0092] S307: Identify the NI send and DMA write instructions related to the computation, generate activation information to the NI unit and DMA unit, each transmission unit executes the instructions of the activated area, and starts the data output transmission from the parameter RAM. The NI send instruction contains the queue data transmission between the front and back, which is automatically converted into a queue_store instruction after the NI receives the destination core.

[0093] S308: Check whether the activated NI send and DMA write instructions are executed, and wait in a loop. After completion, set the end flag of this round of execution.

[0094] In an embodiment, as shown in the figure, the control method further includes but is not limited to the following steps: Figure 4A

[0095] Step S21: The queue transmission unit receives the queue message from the NI unit;

[0096] Step S22: According to the data type code in the message header of the queue message, locate the hardware queue ID;

[0097] Step S23: Maintain the tail pointer and head pointer corresponding to the hardware queue ID;

[0098] Step S24: When the input data associated with the computation instruction needs to use the queue data, the control unit generates a queue_load instruction and distributes it to the queue transmission unit;

[0099] Step S25: The queue transmission unit reads the data block according to the head pointer and transmits it to the parameter RAM.

[0100] It can be understood that, as shown in the figure, Figure 4B Figure 4B is the control flow of the queue transmission unit, which is mainly implemented by hardware. The hardware maintains the queue FIFO (first in first out) between the front and back cores, and stores or reads one queue element at a time. The specific control flow of the queue transmission unit is as follows:

[0101] S401: Receive the message from the NI unit, from other front cores, and the transmission type in the message header is the front-back queue. According to the data type in the message header, locate the hardware queue ID. The hardware generates a queue_store instruction, and the DMA address of the instruction is calculated and generated by the hardware based on the queue ID, tail pointer wptr, and element size. The data sequence between the DNN training front and back is generally first in first out. If there is a need for sequence adjustment between elements, the hardware can support regular mapping of element DMA addresses in queue_store.

[0102] ​​S402: Maintain tail pointer wptr for each queue ID, wptr = wptr + 1 after element is enqueued. Queue is full when queue level = wptr - rptr, near full triggers back pressure to NI.

[0103] S403: Implement DMA write to queue buffer according to queue_store instruction.

[0104] S404: Queue can be mapped to SRAM or DRAM, depending on buffer size requirement.

[0105] S405: When input data associated with computation instruction needs to use queue data, control unit generates queue_load instruction to S405, then reads data block according to queue head pointer and transfers to parameter RAM.

[0106] S406: Maintain head pointer rptr for each queue ID, rptr = rptr + 1 after element is dequeued. queue_load can be executed when wptr ≠ rptr, which means there is valid element in queue.

[0107] In an embodiment, as shown in Figure 5A the control method further includes but is not limited to the following steps:

[0108] Step S31, when the computation engine finishes single weight gradient element computation, informs the weight gradient transfer unit to start by setting the lock signal to 1;

[0109] Step S32, after the weight gradient transfer unit finishes current element store and next weight gradient element load, informs the computation engine to start next weight gradient element computation by clearing the lock signal;

[0110] Step S33, provides the address of the weight gradient in DRAM;

[0111] Step S34, executes DMA write instruction to transfer the weight gradient element from SRAM to DRAM;

[0112] Step S35, executes DMA read instruction to transfer the weight gradient element from DRAM to SRAM.

[0113] It can be understood that, as shown in Figure 5B Figure 5B ​The control flow for the WG transfer unit is described below, and this flow is primarily implemented in hardware. WGs are stored in DRAM. The WG calculation unit loads one element at a time, calculates it, and then writes the element back to its DRAM location. Because the WG elements are arranged regularly and consecutively in DRAM, WG calculation and WG transfer can be iterated multiple times, with seamless calculation and transfer processing implemented in hardware, requiring no software intervention. The specific control flow for the WG transfer unit 208 is as follows:

[0114] S501: After the calculation engine completes the calculation of a single WG element, it notifies the WG transfer 208 to start by setting the lock signal to 1; after the WG transfer completes the storage of the current element and the loading of the next element, it notifies the calculation engine to start the calculation of the next element by clearing the lock signal to zero.

[0115] S502: Provides the base address of WG in DRAM.

[0116] S503: Executes the DMA write instruction generated by S501, transferring elements from SRAM to DRAM.

[0117] S504: Executes the DMA read instruction generated by S501, transferring elements from DRAM to SRAM.

[0118] like Figure 6 As shown, Figure 6 This example demonstrates how computation and transmission instructions work together to implement different model state machines. Each single-core processing core can pre-program a task state machine, including both computation and transmission actions. Because the architecture supports coarse-grained computation and transmission instruction combinations, flexible DNN training models can be implemented through pre-programming of instructions.

[0119] For example, Figure 6 The state decomposition of core 2 is as follows:

[0120] 1) Receive S0 from core 1 from NI;

[0121] 2) Calculate the intermediate result sum2 by performing convolution / matrix multiplication;

[0122] 3) Receive sum3 from core 3 of NI and perform vector calculation of sum2^=sum3+sum2;

[0123] 4) Vector calculation generates s2;

[0124] 5) DMA output s2;

[0125] 6) Transfer between front and back queues.

[0126] Based on this, the embodiments of the present invention have at least the following beneficial effects:

[0127] 1) Adapt to multiple models and deployment schemes. Each core supports two sets of data transmission methods, memory access and NOC access. Provide transmission instruction programming, two ways in the core through ID+RAM internal offset address method unified addressing.

[0128] 2) Accelerate transmission and calculation convergence. Perform transmission instruction completion check, handle transmission and calculation fast switching.

[0129] 3) Transmission expansion and acceleration between forward and backward. Provide hardware queue mechanism to accelerate data transmission between forward and backward, and provide configurable queue depth in DRAM.

[0130] 4) WG calculation data transmission acceleration. The weight W storage in neural network model occupies the largest proportion, the gradient involved in WG needs batch processing, which occupies the same cache capacity as W and needs frequent memory read-write access. Support storing gradient in DRAM and accelerating WG calculation gradient read-write access.

[0131] The above is a specific description of the preferred embodiment of the present application, but the present application is not limited to the above-mentioned embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application. These equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A single-core processing core, characterized in that, The single-core processing core has an on-chip network interface and memory access interface for external data transmission. Internally, the single-core processing core includes a computing engine, a control unit, a data exchange unit, a parameter RAM, an NI unit, a DMA unit, a queue transfer unit, a weight gradient transfer unit, a data exchange bus, and a control bus. The control unit is responsible for activating computing instructions, memory transfer instructions, and network transfer instructions. The computing instructions are pre-programmed and stored in the computing engine. The memory transfer instructions are pre-programmed and stored in the DMA unit, including DMA write and DMA read instructions. The network transfer instructions are pre-programmed and stored in the NI unit, including NI send and NI receive instructions. The control unit communicates with the NI unit via... The control bus is electrically connected to the computing engine, the NI unit, the DMA unit, and the queue transfer unit, respectively. The data exchange unit is electrically connected to the parameter RAM, the NI unit, the DMA unit, the queue transfer unit, and the weight gradient transfer unit, respectively, through the data exchange bus. The queue transfer unit is used to realize hardware queue expansion and acceleration between forward and backward directions. The weight gradient transfer unit is used to provide transmission acceleration for frequent read and write of DRAM memory for the weight gradients of deep neural networks. The weight gradients are stored in the DRAM memory. The control unit triggers the queue transfer unit to generate queue transfer instructions through control signals. The computing engine triggers the weight gradient transfer unit to generate weight gradient transfer instructions through control signals.

2. The single-core processing core according to claim 1, characterized in that, The NI unit has the on-chip network interface. The NI unit sends data from the parameter RAM in the source single-core processing core to the destination single-core processing core according to the NI send instruction, and writes data received from the destination single-core processing core to the parameter RAM in the source single-core processing core according to the NI receive instruction. The paired NI send instructions and NI receive instructions are associated with the same tag number.

3. The single-core processing core according to claim 1, characterized in that, The DMA unit is used to implement data transfer between the parameter RAM and the DRAM memory. The DMA write instruction and the DMA read instruction contain the data type encoding, source address information and target address information corresponding to the parameter RAM. The source address information of the DMA write instruction and the target address information of the DMA read instruction are used to indicate the internal offset of the parameter RAM. The data exchange unit accesses the parameter RAM according to the data type encoding, the source address information and the target address information.

4. The single-core processing core according to claim 1, characterized in that, The queue transmission unit provides a first-in-first-out hardware queue mechanism to accelerate data transmission between training forward and backward, and provides a configurable queue depth in the DRAM memory.

5. The single-core processing core according to claim 1, characterized in that, The computing engine and the weight gradient transfer unit are interlocked.

6. The single-core processing core according to claim 1, characterized in that, The data exchange unit is used to provide the exchange of parameters between RAM and external data. The data exchange unit uses ID addressing within the single-core processing core to unify data scheduling for network and memory access.

7. A many-core communication architecture, characterized in that, The device has at least two single-core processing cores as described in any one of claims 1 to 6, the on-chip network interface is used for data transmission between any two of the single-core processing cores, the on-chip network interface is electrically connected to an on-chip router, the on-chip router is used for data forwarding between any two of the single-core processing cores, each single-core processing core occupies a dedicated dynamic random access memory (DRAM) space, and the memory of any two single-core processing cores is logically isolated.

8. A chip, characterized in that, It includes the many-core communication architecture as described in claim 7.

9. A control method for a single-core processing kernel, characterized in that, Applied to a single-core processing core as described in any one of claims 1 to 6, the method comprises: Upon confirming that the previous instruction group has been executed, the control unit activates the NI receive instruction corresponding to the NI unit and the DMA read instruction corresponding to the DMA unit, and initiates data input transmission to the parameter RAM. The control unit sends a control signal to the queue transmission unit, so that the queue transmission unit generates a queue transmission instruction according to the control signal, and starts the transmission of queue data to the parameter RAM according to the queue transmission instruction; Once the input data check is complete, the control unit activates the computing engine to start the calculation. When it is determined that the computing engine has completed the calculation, the control unit activates the NI send instruction corresponding to the NI unit and the DMA write instruction corresponding to the DMA unit, wherein the NI send instruction contains queue data between forward and backward directions; Once the data output is complete, the control unit determines that the current round of instruction execution has ended.

10. The control method for a single-core processing core according to claim 9, characterized in that, The method further includes: The queue transmission unit receives queue messages from the NI unit; The hardware queue ID is located based on the data type encoding in the header of the queue message; Maintain the tail pointer and head pointer of the corresponding hardware queue ID; When the input data associated with the calculation instruction requires queue data, the control unit generates a queue_load instruction and distributes it to the queue transmission unit; The queue transmission unit reads data blocks according to the head pointer and transmits them to the parameter RAM.

11. The control method for a single-core processing core according to claim 9, characterized in that, The method further includes: When the computing engine completes the calculation of a single weight gradient element, it notifies the weight gradient transmission unit to start by setting the lock signal to 1. After the weight gradient transmission unit has completed transmitting the current element store and the next weight gradient element load, it notifies the calculation engine to start the calculation of the next weight gradient element by clearing the lock signal; Provides the address of the weight gradient in DRAM; Execute the DMA write instruction to transfer the weight gradient elements from SRAM to DRAM; Execute a DMA read instruction to transfer the weight gradient elements from DRAM to SRAM.

Images