Register block for time division multiplexing, neural network processor and operation method

Abstract

The invention relates to the field of artificial intelligence and the technical field of chips, in particular to a register block for time division multiplexing, a neural network processor and an operation method. Each register pair comprises a first register and a second register; the first register is connected with the second register; the first register is used for performing data exchange with the corresponding second register in response to overturning of the characterization signal; and the second register is used for responding to the representation signal to overturn and performing data exchange with the corresponding first register. According to the register block, rapid data switching and configuration updating can be achieved through the first register and the shadow register, namely the second register, corresponding to the first register, so that multiple tasks or algorithms are supported to alternately run on the same hardware resource, the utilization efficiency of the hardware resource is remarkably improved, and the register block is particularly suitable for a time division multiplexing scene; such as multi-task processing in NPU.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and the field of chips, and in particular to a register group for time division multiplexing, a neural network processor and a running method. BACKGROUND

[0002] In the field of artificial intelligence (AI) and the field of chips, a neural network processing unit (NPU) has become the core hardware supporting AI tasks due to its efficient neural network computing capability. In practical applications, it is generally necessary to run multiple tasks on the same NPU chip at the same time. To meet this demand under limited hardware resources, time division multiplexing technology is widely used in NPU design.

[0003] A specific time division multiplexing scheme is that multiple tasks share the same set of hardware resources of an NPU, and through a time allocation mechanism, the hardware resources are used to perform different tasks in different time periods, thereby supporting multiple tasks with a single hardware resource.

[0004] However, the existing time division multiplexing scheme of the NPU has an efficiency bottleneck: when different tasks are frequently switched, configuration parameters need to be frequently written into registers, which will cause the configuration parameters to take too long and occupy the effective working time of the hardware resources such as registers, resulting in a significant reduction in the overall running efficiency of the NPU chip. SUMMARY

[0005] Embodiments of the present application provide a register group for time division multiplexing, a neural network processor and a running method to solve the problem of low NPU running efficiency in a time division multiplexing scenario.

[0006] In a first aspect, the embodiments of the present application provide a register group for time division multiplexing, comprising a plurality of register pairs; each register pair comprises a first register and a second register; the first register is connected to the second register; the first register is configured to exchange data with the second register corresponding thereto in response to a flip of a representation signal; wherein the representation signal is a control signal for indicating the register to exchange data, which is triggered to flip by a time slice switching; the second register is configured to exchange data with the first register corresponding thereto in response to a flip of the representation signal.

[0007] In one possible implementation, the first register comprises at least a first data input end and a first data output end; the first register is further configured to receive and store data through the first data input end, and output the stored data through the first data output end.

[0008] In a second aspect, the embodiments of the present application provide a neural network processor for time division multiplexing, comprising at least a scheduler and an execution unit; the scheduler is connected to the execution unit; the execution unit comprises an execution unit controller, a plurality of groups of read direct memory access (RDMA) register groups, at least one group of core register groups, and at least one group of write direct memory access (WDMA) register groups; the execution unit controller is connected to the RDMA register groups, the core register groups, and the WDMA register groups; the RDMA register groups, the core register groups, and the WDMA register groups are the register groups for time division multiplexing in the first aspect and each of the implementation manners thereof; the scheduler is configured to flip a representation signal and input the flipped representation signal to the execution unit controller; the representation signal is a control signal for indicating the register to exchange data, which is triggered to flip by time slice switching; the execution unit controller is configured to receive the representation signal and input the representation signal to the RDMA register groups, the core register groups, and the WDMA register groups, so that a first register inside the RDMA register groups, the core register groups, and the WDMA register groups responds to the representation signal to flip and exchanges data with a second register corresponding thereto, and a second register inside the RDMA register groups, the core register groups, and the WDMA register groups responds to the representation signal to flip and exchanges data with the first register corresponding thereto.

[0009] In a possible implementation manner, the execution unit further comprises an execution unit logic circuit; the execution unit logic circuit is connected to the execution unit controller; the execution unit logic circuit comprises a plurality of RDMA modules, at least one core module, and at least one WDMA module; each RDMA module is connected to a group of RDMA register groups; the core module is connected to the core register groups; and the WDMA module is connected to the WDMA register groups.

[0010] In a possible implementation manner, the first register and the second register in the RDMA register groups are configured to store RDMA configuration data, and the first register and the second register store the same or different RDMA configuration data; the first register and the second register in the core register groups are configured to store core configuration data, and the first register and the second register store the same or different core configuration data; and the first register and the second register in the WDMA register groups are configured to store WDMA configuration data, and the first register and the second register store the same or different WDMA configuration data.

[0011] In a possible implementation, each rdma module is coupled with the first data output end of the corresponding first register, and the rdma module is configured to obtain the rdma configuration data stored in the corresponding first register through the first data output end of the corresponding first register; the core module is coupled with the first data output end of the corresponding first register, and the core module is configured to obtain the core configuration data stored in the corresponding first register through the first data output end of the corresponding first register; and the wdma module is coupled with the first data output end of the corresponding first register, and the wdma module is configured to obtain the wdma configuration data stored in the corresponding first register through the first data output end of the corresponding first register.

[0012] In a possible implementation, the scheduler is further configured to generate a register configuration instruction, and input the register configuration instruction to the execution unit controller, the register configuration instruction carrying the rdma configuration data, the core configuration data and / or the wdma configuration data; and the execution unit controller is further configured to modify the configuration data stored in the first register in the rdma register group, the core register group and / or the wdma register group through the first data input end in response to the register configuration instruction.

[0013] In a third aspect, an embodiment of the present application provides a neural network processor running method for time division multiplexing, applied to the neural network processor for time division multiplexing in the foregoing second aspect and each implementation manner thereof; the method comprises the following steps: a scheduler flips a representation signal, and inputs the flipped representation signal to an execution unit controller; wherein the representation signal is a control signal for instructing a register to exchange data, and is flipped in response to time slice switching; and the execution unit controller receives the representation signal and inputs the representation signal to an rdma register group, a core register group and a wdma register group, so that a first register inside the rdma register group, the core register group and the wdma register group exchanges data with a corresponding second register in response to the representation signal, and a second register inside the rdma register group, the core register group and the wdma register group exchanges data with a corresponding first register in response to the representation signal.

[0014] In a possible implementation, the method further includes: the scheduler constructing a first virtual neural network processor (NPU) and a second virtual NPU; the first virtual NPU and the second virtual NPU are alternately run through time slice switching; and the step of the scheduler flipping the representation signal includes: the scheduler flipping the representation signal in response to time slice switching; the time slice switching refers to switching from a time slice of the first virtual NPU to a time slice of the second virtual NPU, or the time slice switching refers to switching from a time slice of the second virtual NPU to a time slice of the first virtual NPU; when the representation signal is 0, the first register is configured to store configuration data of the execution unit for the first virtual NPU, and the second register is configured to store configuration data of the execution unit for the second virtual NPU; when the representation signal is flipped from 0 to 1, the first register exchanges data with the second register corresponding to the first register, the first register is switched from storing the configuration data of the execution unit for the first virtual NPU to storing the configuration data of the execution unit for the second virtual NPU, and the second register is switched from storing the configuration data of the execution unit for the second virtual NPU to storing the configuration data of the execution unit for the first virtual NPU; when the representation signal is 1, the first register is configured to store configuration data of the execution unit for the second virtual NPU, and the second register is configured to store configuration data of the execution unit for the first virtual NPU; when the representation signal is flipped from 1 to 0, the first register exchanges data with the second register corresponding to the first register, the first register is switched from storing the configuration data of the execution unit for the second virtual NPU to storing the configuration data of the execution unit for the first virtual NPU, and the second register is switched from storing the configuration data of the execution unit for the first virtual NPU to storing the configuration data of the execution unit for the second virtual NPU.

[0015] In a possible implementation, the method further includes: the scheduler generating a register configuration instruction, and inputting the register configuration instruction to the execution unit controller; the register configuration instruction carries rdma configuration data, core configuration data, and / or wdma configuration data; and the execution unit controller modifies configuration data stored in the first register in the rdma register group, the core register group, and / or the wdma register group through the first data input end in response to the register configuration instruction.

[0016] From the above, the embodiment of the application provides a register group for time division multiplexing, a neural network processor and a running method. The register group comprises a plurality of register pairs; each register pair comprises a first register and a second register; the first register is connected with the second register; the first register is configured to perform data exchange with the second register corresponding to the first register in response to a flip of a representation signal; the representation signal is a control signal for indicating the register to perform data exchange, and the flip of the representation signal is triggered by time slice switching; and the second register is configured to perform data exchange with the first register corresponding to the second register in response to the flip of the representation signal. The register group can realize fast data switching and configuration updating through the first register and the second register corresponding to the first register, thereby supporting a plurality of tasks or algorithms to alternately run on the same hardware resource, and significantly improving the utilization efficiency of the hardware resource, and is particularly suitable for a time division multiplexing scenario, such as multi-task processing in an NPU. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A structural schematic diagram of the register group for time division multiplexing provided by the embodiment of the application is shown in the figure.

[0018] Figure 2 A structural schematic diagram of the first register provided by the embodiment of the application is shown in the figure.

[0019] Figure 3 A structural schematic diagram of the neural network processor for time division multiplexing provided by the embodiment of the application is shown in the figure.

[0020] Figure 4 A structural schematic diagram of the execution unit provided by the embodiment of the application is shown in the figure.

[0021] Figure 5 A structural schematic diagram of the execution unit controller provided by the embodiment of the application is shown in the figure.

[0022] Figure 6 A flow schematic diagram of the neural network processor running method for time division multiplexing provided by the embodiment of the application is shown in the figure.

[0023] In the figure, 100 represents a register pair; 101 represents a first register; 102 represents a second register; 1011 represents a first data input end; 1012 represents a first data output end; 201 represents a scheduler; 2011 represents an instruction loader; 2012 represents an instruction decoder; 202 represents an execution unit; 2021 represents an execution unit controller; 2022 represents an execution unit logic circuit; 2022a represents an rdma module; 2022b represents a core module; 2022c represents a wdma module; 2023 represents an rdma register group; 2024 represents a core register group; 2025 represents a wdma register group; 203 represents a double data rate synchronous dynamic random access memory (DDR); and 204 represents an on-chip memory (OCM). DETAILED DESCRIPTION

[0024] In order to enable persons skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the scope of protection of the present application.

[0025] In order to solve the problem of low running efficiency of NPU chip in time division multiplexing, the present application at least provides a register group for time division multiplexing, which can include multiple pairs of register pairs. Two associated registers in each pair of register pairs can cache different configuration parameters. Then, data exchange can be performed between the two associated registers when a job is switched. It is worth noting that the data exchange at the hardware level of the register can shorten the path of data configuration, and can reduce the number of data configuration instructions, which can achieve the purpose of significantly reducing the parameter configuration time. Further, on the basis of the register group for time division multiplexing, the present application can also provide a neural network processor NPU for time division multiplexing and a corresponding running method thereof, which can significantly improve the running efficiency of the NPU chip.

[0026] Figure 1 The structure diagram of the register group for time division multiplexing provided by the embodiments of the present application is shown.

[0027] As shown in Figure 1 The present application provides a register group for time division multiplexing, which can include multiple pairs of register pairs 100; each pair of register pairs 100 includes a first register 101 and a second register 102; the first register 101 is connected with the second register 102. In this way, data exchange can be performed between the first register 101 and the second register 102. The second register 102 can be referred to as a shadow register of the first register 101, and the first register 101 is a non-shadow register.

[0028] Figure 1The "Reg0" and "Reg0_shadow" are a pair of register pairs 100, the "Reg0" is a first register 101, and the "Reg0_shadow" is a second register 102, and the "Reg0_shadow" is a shadow register of the "Reg0". The "Reg1" and "Reg1_shadow" are a pair of register pairs 100, the "Reg1" is a first register 101, and the "Reg1_shadow" is a second register 102, and the "Reg1_shadow" is a shadow register of the "Reg1". The "Regn" and "Regn_shadow" are a pair of register pairs 100, the "Regn" is a first register 101, and the "Regn_shadow" is a second register 102, and the "Regn_shadow" is a shadow register of the "Regn".

[0029] Figure 1 In the figure, n pairs of register pairs 100 are shown. In actual applications, the number of register pairs 100 included in a register group can be set based on actual needs, and embodiments of the present application do not make specific limitations thereto.

[0030] Further, the first register 101 is configured to perform data exchange with the second register 102 corresponding thereto in response to the characteristic signal being flipped, wherein the characteristic signal is a control signal for indicating the register to perform data exchange, which is triggered to flip by time slice (time slice switching) switching. The second register 102 is configured to perform data exchange with the first register 101 corresponding thereto in response to the characteristic signal being flipped.

[0031] The data exchange refers to that the first register 101 and the second register 102 perform bidirectional transmission of the data stored by themselves in response to the trigger condition that the characteristic signal is flipped, that is, the first register 101 transmits the data currently stored by itself to the second register 102, and the second register 102 transmits the data currently stored by itself to the first register 101, so as to finally realize the exchange of the storage data of the two, and provide a data basis for supporting the time division multiplexing function of subsequent switching work configuration.

[0032] It can be understood that the first register 101 can include an enable end or a control end to receive the characteristic signal.

[0033] It is worth noting that when the register group is applied to the NPU, the characteristic signal can be an "npu_duty" signal generated by the NPU.

[0034] In some implementations, the embodiment of the application can be designed by hardware, so that the first register 101 interacts with an external bus (for example, an AXI bus axibus), and the second register 102 is not directly connected to the external bus, and is only used for data exchange with the corresponding first register 101. This architecture makes the second register 102 functionally a shadow register of the first register 101, and the content of the second register 102 is always a data mirror of the first register 101 before the last data exchange, so as to realize instantaneous switching of data.

[0035] It can be understood that the working principle of the register is to stably store binary data through an internal flip-flop circuit. The core of the application is that two registers can complete direct exchange of all stored data in a short time through a hardware circuit. This mechanism avoids the long waiting for reloading configuration data from the external bus during task switching, thereby fundamentally solving the problem of chip efficiency reduction caused by frequent configuration.

[0036] Figure 2 The structure diagram of the first register provided by the embodiment of the application is shown.

[0037] As shown in Figure 2 , further, the first register 101 at least includes a first data input port 1011 and a first data output port 1012; the first register is further configured to receive data through the first data input port and store the data, and output the stored data through the first data output port.

[0038] It should be noted that the first data input port 1011 is a port for writing or updating the data stored in the first register 101 from an external data source, that is, a configuration port. Correspondingly, the first data output port 1012 is a port for other hardware function modules to read the current data stored in the first register 101 in real time, that is, an access port. Through the first data input port 1011 and the first data output port 1012, the first register 101 can efficiently complete the write and read operations of data, thereby supporting complex hardware functions and multi-task processing.

[0039] It can be understood that the first data input port 1011 and the first data output port 1012 are only two of the many ports of the first register 101, and the first register 101 can further include a clock port and a reset port and the like control ports. The embodiment of the application does not make specific limitation on this.

[0040] From the above, the embodiment of the present application provides a register group for time division multiplexing, comprising a plurality of register pairs 100; each register pair 100 comprises a first register 101 and a second register 102; the first register 101 is connected with the second register 102; the first register 101 is configured to perform data exchange with the second register 102 corresponding thereto in response to a flip of a characteristic signal, wherein the characteristic signal is a control signal for indicating the register to perform data exchange, which is triggered to flip by time slice switching; the second register 102 is configured to perform data exchange with the first register 101 corresponding thereto in response to a flip of the characteristic signal. The register group can realize fast data switching and configuration updating through the first register 101 and the second register 102 corresponding thereto, so as to support multiple tasks or algorithms to alternately run on the same hardware resource without interfering with each other. This design significantly improves the utilization efficiency of hardware resources, reduces the problem of chip efficiency decline caused by frequent configuration, and is particularly suitable for time division multiplexing scenarios, such as multi-task processing in NPU.

[0041] Figure 3 The structure schematic diagram of the neural network processor for time division multiplexing provided by the embodiment of the present application is provided.

[0042] As shown in Figure 3 The neural network processor for time division multiplexing provided by the embodiment of the present application can comprise a dispatcher 201 and an execution unit (EU) 202; the dispatcher 201 is connected with the execution unit 202. The dispatcher 201 further comprises an instruction loader 2011 and an instruction decoder 2012. The execution unit 202 further comprises an execution unit controller 2021 and an execution unit logic circuit 2022.

[0043] Further, the NPU for time division multiplexing provided by the embodiment of the present application adopts a two-level hardware control structure: the first-level control logic is executed by the dispatcher 201, and the second-level control logic is executed by the execution unit 202.

[0044] In the first level control logic, the instruction loader 2011 is configured to load instructions from external storage, which can be a double data rate synchronous dynamic random access memory (DDR) 203. The instruction decoder 2012 is configured to decode instructions from on chip memory (OCM) 204 or loaded by the instruction loader 2011. Then, while the instructions of the current task are running, the instruction loader 2011 can load the instructions of the next task into the OCM 204, so the time of loading instructions can be hidden, and the occupancy of the OCM 204 can be greatly reduced. Based on this, the embodiment of the present application can reduce the design size of the OCM 204, and thus reduce the cost. The instruction loader 2011 can be composed of a read direct memory access (RDMA) unit and a write direct memory access (WDMA) unit. The RDMA unit is responsible for reading instructions from the DDR 203, and the WDMA unit is responsible for writing the read instructions into the OCM 204.

[0045] In the second level control logic, the execution unit logic circuit 2022 is connected with the execution unit controller 2021, and the execution unit controller 2021 directly manages the task start and configuration of each functional sub-module in the execution unit logic circuit 2022. In the embodiment of the present application, the scheduler 201 can be connected with the execution unit 202 through an output register configuration signal (regs_cfg) to transmit configuration information to the execution unit controller 2021 in the execution unit 202, so that the execution unit controller 2021 controls the execution unit logic circuit 2022 to complete the corresponding neural network operation task based on the configuration information.

[0046] As can be seen, the present application can realize efficient task scheduling, instruction management and hardware resource configuration through the above two-level hardware control structure.

[0047] It should be further pointed out that the NPU for time division multiplexing can also include other hardware structures, such as a bus (Bus), an interface module (Interface Module), etc., which are not specifically limited in the embodiment of the present application.

[0048] Figure 4 The structure schematic diagram of the execution unit provided in the embodiment of the present application is shown.

[0049] Among them, Figure 4 The top layer (EU top) architecture of the execution unit 202 is exemplarily shown.

[0050] Figure 5A structure diagram of an execution unit controller provided by an embodiment of the present application is shown.

[0051] As shown in Figure 4 and Figure 5 , the execution unit 202 further includes a plurality of groups of read direct memory access (RDMA) register groups (rdma_regs) 2023, at least one group of core register groups (core_regs) 2024, and at least one group of write direct memory access (WDMA) register groups (wdma_regs) 2025. Further, the execution unit controller 2021 is connected with the rdma register groups 2023, the core register groups 2024, and the wdma register groups 2025.

[0052] It is emphasized that the rdma register groups 2023, the core register groups 2024, and the wdma register groups 2025 are the register groups for time division multiplexing in the foregoing embodiments.

[0053] Further, continuing to refer to Figure 4 , the execution unit logic circuit 2022 can include a plurality of rdma modules 2022a, at least one core module 2022b, and at least one wdma module 2022c. Continuing to refer to Figure 5 , each of the rdma modules 2022a is connected with a group of rdma register groups 2023; the core module 2022b is connected with the core register groups 2024; and the wdma module 2022c is connected with the wdma register groups 2025.

[0054] Further, the first register 101 and the second register 102 in the rdma register groups 2023 are used to store rdma configuration data, and the first register 101 and the corresponding second register 102 store the same or different rdma configuration data; the first register 101 and the second register 102 in the core register groups 2024 are used to store core configuration data, and the first register 101 and the corresponding second register 102 store the same or different core configuration data; and the first register 101 and the second register 102 in the wdma register groups 2025 are used to store wdma configuration data, and the first register 101 and the corresponding second register 102 store the same or different wdma configuration data.

[0055] Further, the scheduler 201 is configured to: flip a characteristic signal, and input the flipped characteristic signal to the execution unit controller 2021, wherein the characteristic signal is a control signal for instructing the register to exchange data, and is flipped by the time slice switching.

[0056] In the embodiments of the present application, the characteristic signal is, for example, an "npu_duty" signal, or the characteristic signal can be referred to as a "regs set from dispatcher" signal.

[0057] The execution unit controller 2021 is configured to receive the characteristic signal and input the characteristic signal to the rdma register group 2023, the core register group 2024 and the wdma register group 2025, so that the first register 101 inside the rdma register group 2023, the core register group 2024 and the wdma register group 2025 responds to the characteristic signal to flip and exchange data with the corresponding second register 102, and the second register 102 inside the rdma register group 2023, the core register group 2024 and the wdma register group 2025 responds to the characteristic signal to flip and exchange data with the corresponding first register 101.

[0058] It can be understood that, through the characteristic signal, the execution unit controller 2021 can make all non-shadow registers in the rdma register group 2023, the core register group 2024 and the wdma register group 2025 exchange data with the shadow registers. It should be emphasized that the execution unit controller 2021 can only send the characteristic signal to the port of the first register 101, at this time the first register 101 can respond to the characteristic signal to trigger the "data exchange operation", and the second register 102 does not need to additionally receive the characteristic signal, and only needs to start the "data exchange operation" under the trigger of the first register 101, so as to indirectly respond to the characteristic signal. In this way, the wiring design of the second register 102 can be simplified.

[0059] It should be noted that, continuing to refer to Figure 5 , the dispatcher 201 can specifically input the characteristic signal "regs set from dispatcher" to the microcode register mapping module (mcode regmap) in the execution unit controller 2021, and then the microcode register mapping module can generate a "regs signal" and transmit the "regs signal" to the first register 101 of the rdma register group 2023, the core register group 2024 and the wdma register group 2025 in the control body (ctrlbody). Further, the data stored in the second register 102 of the rdma register group 2023, the core register group 2024 and the wdma register group 2025 can be determined through the "data exchange" operation, that is, the data stored in the second register 102 is derived from the first register 101, and is the configuration data stored in the first register 101 before the "data exchange operation".

[0060] It should be noted that the configuration parameters between two adjacent tasks are partially the same or different, for example, between different tasks of the same neural network algorithm, only a small part of the configuration parameters are changed. When switching tasks, compared with reconfiguring all configuration parameters through an external bus, the embodiment of the present application can use the "shadow register" (i.e. the second register 102) as a data buffer, through the data input end of the first register 101, write the changed configuration parameters into the second register 102, and the unchanged parameters continue to remain in it. This mechanism greatly reduces the amount of data that needs to be transmitted through the external bus, effectively reduces the bus load and power consumption, and significantly reduces the number of mcode, thereby reducing the occupation of the on-chip memory 204 by the mcode, and achieving the purpose of reducing the capacity of the on-chip memory 204.

[0061] It should be noted that the rdma module 2022a, the core module 2022b and the wdma module 2022c in the execution unit logic circuit 2022 are controlled by independent control channels, and the control signal transmission between them does not interfere with each other; at the same time, these three modules each correspond to an independent register mapping (regmap) module. This independent channel control provides a hardware basis for time division multiplexing configuration switching based on the register group of each module.

[0062] In some implementations, continuing to refer to Figure 4 In the top-level architecture of the execution unit 202, the execution unit controller 2021 can receive external control and data through the axi bus. In addition, the execution unit controller 2021 can send control flow signals to the rdma module 2022a (in "control flow *n" in the middle, it means sending control flow signals to n rdma modules 2022a respectively), and the rdma module 2022a can return status signals idle to the execution unit controller 2021 (in "idle *n" in the middle, it means that n rdma modules 2022a return status signals to the execution unit controller 2021 respectively). Figure 4 Figure 4 Figure 4 ​​After the data is obtained from the outside, the data is transmitted to the core module 2022b in the form of a data stream, and the core module 2022b can perform neural network operations such as stream data movement (SDMA), custom operation (TENG), or convolution (conv). The data stream after the operation is transmitted to the wdma module 2022c, and the wdma module 2022c outputs the data to the outside through the axi bus. At the same time, the execution unit controller 2021 can also interact with the core module 2022b and the wdma module 2022c through the control flow signal and the idle signal to coordinate the work of the core module 2022b and the wdma module 2022c, and finally complete the process of “data reading → calculation → data writing”.

[0063] In some implementations, continuing to refer to Figure 5 In the bottom layer architecture of the execution unit 202, the instructions can be sent to the execution unit controller 2021 for analysis through the OCM read path. Specifically, the demux can distribute the instructions to the rdma ch0 to rdma chN-1 channels to prepare for subsequent data reading paths. After entering the control body inside the execution unit controller 2021, the decoder and the execution subunit (decoder & exu) decode the instructions and generate control signals (mcode set to eu modules). Then, the control signals are sent to the rdma register group 2023, the core register group 2024, and / or the wdma register group 2025 to configure the operations of the rdma module 2022a, the core module 2022b, and / or the wdma module 2022c.

[0064] In some implementations, the first register 101 and the second register 102 in the rdma register group 2023, the core register group 2024, and the wdma register group 2025 can be 1 kilobit (1Kilobit) capacity general purpose registers (GPRs1kb).

[0065] Further, it should be noted that each rdma module 2022a is coupled to the first data output end 1012 of the first register 101 in the corresponding rdma register group 2023, and the rdma module 2022a is configured to obtain the rdma configuration data stored in the first register 101 through the first data output end 1012 of the first register 101.

[0066] The core module 2022b is coupled with the first data output end 1012 of the first register 101 in the core register group 2024 corresponding to the core module 2022b, and the core module 2022b is configured to acquire the core configuration data stored in the first register 101 through the first data output end 1012 of the first register 101 in the core register group 2024 corresponding to the core module 2022b.

[0067] The wdma module 2022c is coupled with the first data output end 1012 of the first register 101 in the wdma register group 2025 corresponding to the wdma module 2022c, and the wdma module 2022c is configured to acquire the wdma configuration data stored in the first register 101 through the first data output end 1012 of the first register 101 in the wdma register group 2025 corresponding to the wdma module 2022c.

[0068] Based on the above, it can be known that the non-shadow register (the first register 101) in the register group is directly interacted with the rdma module 2022a, the core module 2022b and the wdma module 2022c, that is, only the non-shadow register (or the functional register) outputs to the rdma module 2022a, the core module 2022b and the wdma module 2022c for use by the rdma module 2022a, the core module 2022b and the wdma module 2022c. The shadow register (the second register) does not interact with the rdma module 2022a, the core module 2022b and the wdma module 2022c.

[0069] Further, it should be noted that the scheduler 201 is further configured to generate a register configuration instruction and input the register configuration instruction to the execution unit controller 2021, and the register configuration instruction carries the rdma configuration data, the core configuration data and / or the wdma configuration data.

[0070] It should be noted that in various application scenarios, such as arrival of a new computing task, dynamic adjustment of current task parameters, or pre-configuration for the next task, the scheduler 201 will generate a corresponding register configuration instruction.

[0071] The execution unit controller 2021 is further configured to modify the configuration data stored in the first register 101 in the rdma register group 2023, the core register group 2024 and / or the wdma register group 2025 through the first data input end 1011 in response to the register configuration instruction.

[0072] Exemplarily, the rdma register group 2023 specifically includes an rdma register group a, an rdma register group b, and an rdma register group c, the core register group 2024 specifically can be a core register group d, and the wdma register group 2025 specifically can be a wdma register group e. Then, the scheduler 201 can modify the values in part or all of the registers through the register configuration instruction. For example, when only the values of the core register group d need to be modified, the scheduler 201 can generate a corresponding register configuration instruction, which is routed to the core register group d by the execution unit controller 2021, and then the value updating of the core register group d is accurately completed, so that the high flexibility and granularity control of the configuration updating are realized.

[0073] From the above, the embodiment of the application provides a neural network processor for time division multiplexing, which comprises a scheduler 201 and an execution unit 202; the scheduler 201 is connected with the execution unit 202; the execution unit 202 comprises an execution unit controller 2021, an rdma register group 2023, at least one core register group 2024, and at least one wdma register group 2025; the execution unit controller 2021 is connected with the rdma register group 2023, the core register group 2024, and the wdma register group 2025; the scheduler 201 is used for flipping a representation signal and inputting the flipped representation signal to the execution unit controller 2021, wherein the representation signal is a control signal for indicating the data exchange of the registers, which is triggered to flip by the time slice switching; the execution unit controller 2021 is used for receiving the representation signal and inputting the representation signal to the rdma register group 2023, the core register group 2024, and the wdma register group 2025, so that a first register 101 inside the rdma register group 2023, the core register group 2024, and / or the wdma register group 2025 responds to the representation signal to flip, performs data exchange with a corresponding second register 102, and the second register 102 inside the rdma register group 2023, the core register group 2024, and / or the wdma register group 2025 responds to the representation signal to flip, performs data exchange with the corresponding first register 101. The neural network processor realizes the instantaneous switching of the hardware configuration parameters by setting the exchangeable shadow registers in the rdma register group 2023, the core register group 2024, and the wdma register group 2025, effectively eliminates the configuration reloading overhead during the task switching, and significantly improves the running efficiency and hardware resource utilization rate of the NPU in the time division multiplexing mode.

[0074] Figure 6 The flowchart of the running method of the neural network processor for time division multiplexing provided by the embodiment of the application is shown.

[0075] As Figure 6 shown, the embodiment of the present application provides a neural network processor running method for time division multiplexing. The method can be applied to the neural network processor for time division multiplexing in the foregoing embodiments. The method can include the following steps S100-S200:

[0076] S100: The scheduler 201 flips the representation signal and inputs the flipped representation signal to the execution unit controller 2021, wherein the representation signal is a control signal for indicating the register to exchange data, which is triggered to flip by time slice switching.

[0077] In the embodiment of the present application, the timing of the scheduler 201 flipping the representation signal includes but is not limited to task switching, virtual NPU switching, etc. The virtual NPU refers to a plurality of logical NPU instances created on the same physical NPU hardware through time slice rotation. Each virtual NPU has an independent configuration context but shares the same physical computing resources. The flipping of the representation signal as a hardware-level synchronization event marks the switching point of the execution context between different virtual NPUs. The specific steps will be described in detail below, and will not be repeated here.

[0078] It should be further pointed out that the time slice mechanism is a scheduling method for realizing the time-sharing sharing of a single physical computing resource (such as CPU, NPU) by time allocation. Specifically, the execution time of the processor can be divided into a series of extremely small, fixed-length or variable time periods, each of which is called a "time slice". Then, multiple tasks (or virtual NPUs) are allowed to use these time slices in turn, thereby simulating the effect of "multiple tasks running simultaneously".

[0079] S200: The execution unit controller 2021 receives the representation signal and inputs the representation signal to the rdma register group 2023, the core register group 2024, and the wdma register group 2025, so that the first register 101 inside the rdma register group 2023, the core register group 2024, and the wdma register group 2025 responds to the flipping of the representation signal to exchange data with the corresponding second register 102, and the second register 102 inside the rdma register group 2023, the core register group 2024, and the wdma register group 2025 responds to the flipping of the representation signal to exchange data with the corresponding first register 101.

[0080] In this way, the neural network processor can quickly switch between different tasks (here, different tasks can refer to different tasks of the same algorithm), without the need to reload all configuration parameters, thereby reducing the configuration time and improving the overall running efficiency of the neural network processor.

[0081] It is worth noting that before step S100, the method for running the time division multiplexing neural network processor provided in the embodiments of the present application can further include the following step S300.

[0082] S300: The scheduler 201 constructs a first virtual neural network processor NPU and a second virtual NPU; wherein the first virtual NPU and the second virtual NPU are alternately run based on time slice switching.

[0083] Wherein, the first virtual NPU can be virtual NPU0, and the second virtual NPU can be virtual NPU1.

[0084] It is worth noting that the time slice allocation is not strictly in the fixed order of “first virtual NPU→second virtual NPU→first virtual NPU”, but is dynamically adjusted according to the task demand. For example, two time slices can be continuously allocated to the first virtual NPU to execute its continuous tasks, and then switched to the time slice of the second virtual NPU to adapt to the task continuity requirement of different virtual NPUs.

[0085] It can be understood that time slice switching means “task switching”.

[0086] Further, the steps of step S100 can specifically include the following step S101: the scheduler 201 flips the representation signal in response to time slice switching; wherein the time slice switching refers to switching from the time slice of the first virtual NPU to the time slice of the second virtual NPU, or the time slice switching refers to switching from the time slice of the second virtual NPU to the time slice of the first virtual NPU.

[0087] Regarding the correspondence between the representation signal and the register data, it is as follows:

[0088] The representation signal is used to indicate which virtual npu uses the hardware resources (including but not limited to the execution unit 202). When the representation signal is 0, the first register 101 is used to store the configuration data of the execution unit for the first virtual NPU, and the second register 102 is used to store the configuration data of the execution unit for the second virtual NPU. In other words, when the representation signal is 0, the first register 101 acts as a “non-Shadow register” for storing the configuration data of the execution unit 202 for the first virtual NPU currently authorized to use the hardware resources; the second register 102 acts as a “Shadow register” for storing the configuration data of the execution unit 202 for the second virtual NPU last authorized (i.e. the core role of the Shadow register is to temporarily store the hardware configuration parameters of the last virtual NPU).

[0089] It should be noted that the data stored in the first register 101 and the second register 102 can be the configuration data of the virtual NPU to the execution unit 202, for example:

[0090] For the first register 101 and the second register 102 in the rdma register group 2023, the data stored therein can include the data read address (such as the starting address of the external memory) of the rdma module 2022a, the single transmission length, the data cache strategy, the channel priority, etc. For the first register 101 and the second register 102 in the core register group 2024, the data stored therein can include the operation type (such as convolution) of the core module 2022b, the operation precision (such as FP16 / INT8), the input / output data format, the intermediate result temporary storage address, etc. For the first register 101 and the second register 102 in the wdma register group 2025, the data stored therein can include the data write target address of the wdma module 2022c, the verification mode (such as CRC verification enabled), the transmission completion interrupt trigger condition, etc. These data directly determine the working mode of each module in the execution unit 202, and are the core configuration basis for the virtual NPU to exclusively use the physical hardware resources.

[0091] It can be understood that the data stored in the first register 101 and the second register 102 can be partially the same or different.

[0092] When the representation signal is flipped from 0 to 1, the first register 101 and the corresponding second register 102 exchange data, the first register 101 switches from storing the configuration data of the first virtual NPU to the execution unit 202 to storing the configuration data of the second virtual NPU to the execution unit 202, and the second register 102 switches from storing the configuration data of the second virtual NPU to the execution unit 202 to storing the configuration data of the first virtual NPU to the execution unit 202. In other words, when the representation signal is flipped from 0 to 1 (triggering the time slice to switch to the second virtual NPU), the first register 101 and the second register 102 will automatically complete data exchange: the first register 101 switches from storing the configuration data of the first virtual NPU to storing the configuration data of the second virtual NPU, and the second register 102 switches from storing the configuration data of the second virtual NPU to storing the configuration data of the first virtual NPU.

[0093] Further, when the representation signal is 1, the first register 101 is used to store the configuration data of the second virtual NPU pair execution unit 202, and the second register 102 is used to store the configuration data of the first virtual NPU pair execution unit 202. In other words, when the representation signal is 1, the first register 101 acts as a non-shadow register to store the configuration data of the currently active second virtual NPU, and the second register 102 acts as a shadow register to store the configuration data of the last active first virtual NPU.

[0094] When the representation signal is flipped from 1 to 0, the first register 101 and its corresponding second register 102 exchange data, the first register 101 switches from storing the configuration data of the second virtual NPU pair execution unit 202 to storing the configuration data of the first virtual NPU pair execution unit 202, and the second register 102 switches from storing the configuration data of the first virtual NPU pair execution unit 202 to storing the configuration data of the second virtual NPU pair execution unit 202. In other words, when the representation signal is flipped from 1 to 0 (time slice switching back to the first virtual NPU), the first register 101 and the second register 102 again automatically exchange data: the first register 101 re-stores the configuration data of the first virtual NPU, and the second register 102 stores the configuration data of the second virtual NPU.

[0095] Through the above mechanism, whenever the virtual NPU authorized to use the hardware resource is switched, the shadow register (second register 102) and the non-shadow register (first register 101) will automatically complete data exchange through hardware logic, which not only ensures that the shadow register always stores the configuration parameters of the last virtual NPU, but also enables the non-shadow register to quickly load the configuration data of the current virtual NPU, realizing seamless switching between virtual NPUs and efficient reuse of hardware resources.

[0096] It should be further pointed out that the number of virtual NPUs is not limited to two, and can be constructed based on actual tasks. Correspondingly, the number of shadow registers corresponding to the non-shadow register is not limited to one, and the shadow register can be horizontally expanded based on the number of virtual NPUs.

[0097] In some implementations, the sending of the representation signal can be divided into two logics: one is that the representation signal is continuously sent in normal state, and its state is only triggered to flip at time slice switching (corresponding to the task switching of the first virtual NPU and the second virtual NPU) to trigger the data exchange of the first register 101 and the corresponding second register 102 through state change; the other is that the representation signal is not continuously sent, and only after the flip of the time slice switching trigger signal, the flipped signal is sent to the register group, which is also used to trigger the data exchange between the above registers.

[0098] Further, the method for running the time-division multiplexing neural network processor provided by the embodiments of the present application can further include steps S401-S402.

[0099] S401: The scheduler 201 generates a register configuration instruction and inputs the register configuration instruction to the execution unit controller 2021, the register configuration instruction carrying rdma configuration data, core configuration data and / or wdma configuration data.

[0100] S402: The execution unit controller 2021 responds to the register configuration instruction and modifies the configuration data stored in the first register 101 in the rdma register group, the core register group and / or the wdma register group through the first data input end 1011.

[0101] In this way, the data stored in the first register 101 can be updated. Then, when the execution unit 202 runs a new task, since the configuration parameters can be preloaded to the first register, it is not necessary to modify all the function registers, but only to reconfigure the parameters that need to be modified for this task, greatly reducing the configuration time and improving the model running efficiency.

[0102] In some implementations, the method for running the time-division multiplexing neural network processor provided by the embodiments of the present application can further include steps S501, S502 and S503.

[0103] S501: The rdma module 2022a acquires the rdma configuration data stored in its corresponding first register 101 through the first data output end 1012 of the first register 101 thereof;

[0104] S502: The core module 2022b acquires the core configuration data stored in its corresponding first register 101 through the first data output end 1012 of the first register 101 thereof;

[0105] S503: The wdma module 2022c acquires the wdma configuration data stored in its corresponding first register 101 through the first data output end 1012 of the first register 101 thereof.

[0106] It can be understood that the above steps can occur during the running of the rdma module 2022a, the core module 2022b and the wdma module 2022c.

[0107] It should be noted that the above process is only part of the core links of the register configuration management of the scheduler 201 and the execution unit 202. In actual operation, the cooperative control of the two covers more dimensions: for example, the scheduler 201 needs to perform dynamic task scheduling according to the task priority, allocate resources based on hardware load, and monitor the running state of each module in the execution unit 202 (such as operation progress, abnormal interruption signal) in real time; the execution unit 202 needs to respond to the timing control instructions of the scheduler 201, coordinate the data flow of the internal RDMA / Core / WDMA modules, and the like, which are not limited in the embodiments of the present application.

[0108] From the above, the embodiments of the present application provide a register group for time division multiplexing, a neural network processor and a running method. Specifically, the embodiments of the present application realize multiple advantages through the hardware-level register data exchange and virtual NPU switching mechanism:

[0109] Each individual register group includes a register pair, the register pair includes a shadow register and a non-shadow register, the shadow register can save configuration parameters of the execution unit 202 which are not authorized to be executed by the virtual npu. When switching the authorized virtual npu, the values in the shadow register and the non-shadow register are exchanged, so that the configuration parameters used by different virtual npus can be preserved, and during the running process, all configuration parameters do not have to be modified, the configuration time is shortened, and the chip running efficiency is improved. When switching the authorized virtual npu, the values in the shadow register and the non-shadow register are automatically exchanged by hardware, and the software does not have to perform other operations, the application mode is simple, and the chip running efficiency is improved;

[0110] The mechanism also reduces the number of microcodes (mcode), and does not need to realize the configuration logic of virtual NPU switching through a large number of microcode instructions, thereby reducing the occupation demand of microcodes on the on-chip memory 204, and reducing the capacity design of the on-chip memory 204.

[0111] In addition, the rdma module 2022a, the core module 2022b and the wdma module 2022c in the execution unit 202 are respectively controlled by independent channels, and are respectively provided with a separate register mapping (regmap) module, each regmap module is composed of a shadow register and a non-shadow register. This modular design not only ensures the independence of the operation of each module, but also realizes the effective preservation of different virtual NPU configuration parameters through the separate storage of the register group, thereby providing a basic support for fast switching and efficient multiplexing.

[0112] In specific implementation, the present application further provides a computer storage medium, wherein the computer storage medium can store a program, and the program can include some or all steps in the embodiments of the method for running the time division multiplexing neural network processor provided by the present application when executed. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0113] It is easy to understand that, based on the several embodiments provided by the present application, the skilled in the art can combine, split, recombine, etc. the embodiments of the present application to obtain other embodiments, and these embodiments do not exceed the protection scope of the present application.

[0114] The above specific embodiments have further detailed the purposes, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above is only the specific implementation of the embodiments of the present application, and is not used to limit the protection scope of the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A register bank for time division multiplexing, characterized by The register group comprises a plurality of register pairs; each register pair comprises a first register and a second register; the first register is connected with the second register; The first register is configured to perform data exchange with the second register corresponding thereto in response to a flip of a representation signal; the representation signal is a control signal for indicating register data exchange triggered by time slice switching; The second register is configured to perform data exchange with the first register corresponding thereto in response to a flip of the representation signal.

2. The register set for time division multiplexing of claim 1, wherein, The first register comprises at least a first data input end and a first data output end; the first register is further configured to receive data through the first data input end and store the data, and output the stored data through the first data output end.

3. A neural network processor for time division multiplexing, characterized by, The neural network processor comprises at least a scheduler and an execution unit; the scheduler is connected with the execution unit; The execution unit comprises an execution unit controller, a plurality of groups of read direct memory access (RDMA) register groups, at least one group of core register groups, and at least one group of write direct memory access (WDMA) register groups; the execution unit controller is connected with the RDMA register groups, the core register groups, and the WDMA register groups; the RDMA register groups, the core register groups, and the WDMA register groups are the register groups for time division multiplexing according to any one of claims 1-2; The scheduler is configured to flip a representation signal and input the flipped representation signal to the execution unit controller; the representation signal is a control signal for indicating register data exchange triggered by time slice switching; The execution unit controller is configured to receive the representation signal and input the representation signal to the RDMA register groups, the core register groups, and the WDMA register groups, so that the first registers inside the RDMA register groups, the core register groups, and the WDMA register groups perform data exchange with the second registers corresponding thereto in response to a flip of a representation signal, and the second registers inside the RDMA register groups, the core register groups, and the WDMA register groups perform data exchange with the first registers corresponding thereto in response to a flip of the representation signal.

4. The neural network processor for time division multiplexing of claim 3, wherein, The execution unit further comprises an execution unit logic circuit; the execution unit logic circuit is connected with the execution unit controller; The execution unit logic circuit comprises a plurality of RDMA modules, at least one core module, and at least one WDMA module; Each of the RDMA modules is connected with a group of the RDMA register groups; The core module is connected with the core register groups; The WDMA module is connected with the WDMA register groups.

5. The neural network processor for time division multiplexing of claim 4, wherein, The first register and the second register in the rdma register group are used to store rdma configuration data, and the first register and the corresponding second register store the same or different rdma configuration data; The first register and the second register in the core register group are used to store core configuration data, and the first register and the corresponding second register store the same or different core configuration data; The first register and the second register in the wdma register group are used to store wdma configuration data, and the first register and the corresponding second register store the same or different wdma configuration data.

6. The neural network processor for time division multiplexing of claim 5, wherein, Each of the rdma modules is coupled to the first data output end of the corresponding first register, and the rdma module is configured to obtain the rdma configuration data stored in the corresponding first register through the first data output end of the corresponding first register. The core module is coupled to the first data output end of the corresponding first register, and the core module is configured to obtain the core configuration data stored in the corresponding first register through the first data output end of the corresponding first register. The wdma module is coupled to the first data output end of the corresponding first register, and the wdma module is configured to obtain the wdma configuration data stored in the corresponding first register through the first data output end of the corresponding first register.

7. The neural network processor for time division multiplexing according to claim 5 or 6, wherein, The scheduler is further configured to generate a register configuration instruction and input the register configuration instruction to the execution unit controller, wherein the register configuration instruction carries rdma configuration data, core configuration data and / or wdma configuration data. The execution unit controller is further configured to modify the configuration data stored in the first register of the rdma register group, the core register group and / or the wdma register group through the first data input end in response to the register configuration instruction.

8. A method for running a time-division-multiplexed neural network processor, the method comprising: The method is applied to the neural network processor for time division multiplexing according to any one of claims 3-7; the method comprises: The scheduler flips the representation signal and inputs the flipped representation signal to the execution unit controller; wherein the representation signal is a control signal for indicating register data exchange, which is triggered to flip by time slice switching; The execution unit controller receives the representation signal and inputs the representation signal to the rdma register group, the core register group and the wdma register group, so that the first register inside the rdma register group, the core register group and the wdma register group responds to the representation signal to flip and exchanges data with the corresponding second register, and the second register inside the rdma register group, the core register group and the wdma register group responds to the representation signal to flip and exchanges data with the corresponding first register.

9. The method of claim 8, wherein, The method further comprises: The scheduler constructs a first virtual neural network processor (NPU) and a second virtual NPU; wherein the first virtual NPU and the second virtual NPU are alternately run through time slice switching; The step of flipping the representation signal by the scheduler comprises: the scheduler flips the representation signal in response to the time slice switching; wherein the time slice switching refers to switching from the time slice of the first virtual NPU to the time slice of the second virtual NPU, or the time slice switching refers to switching from the time slice of the second virtual NPU to the time slice of the first virtual NPU; When the representation signal is 0, the first register is used to store the configuration data of the first virtual NPU for the execution unit, and the second register is used to store the configuration data of the second virtual NPU for the execution unit; When the representation signal is flipped from 0 to 1, the first register exchanges data with the second register corresponding thereto, the first register is switched from storing the configuration data of the first virtual NPU for the execution unit to storing the configuration data of the second virtual NPU for the execution unit, and the second register is switched from storing the configuration data of the second virtual NPU for the execution unit to storing the configuration data of the first virtual NPU for the execution unit; When the representation signal is 1, the first register is used to store the configuration data of the second virtual NPU for the execution unit, and the second register is used to store the configuration data of the first virtual NPU for the execution unit; When the representation signal is flipped from 1 to 0, the first register exchanges data with the second register corresponding thereto, the first register is switched from storing the configuration data of the second virtual NPU for the execution unit to storing the configuration data of the first virtual NPU for the execution unit, and the second register is switched from storing the configuration data of the first virtual NPU for the execution unit to storing the configuration data of the second virtual NPU for the execution unit.

10. The method of claim 8, wherein, The method further comprises: The scheduler generates a register configuration instruction and inputs the register configuration instruction to the execution unit controller, wherein the register configuration instruction carries rdma configuration data, core configuration data and / or wdma configuration data; The execution unit controller modifies the configuration data stored in the first register of the rdma register group, the core register group and / or the wdma register group through the first data input end in response to the register configuration instruction.

Images