Register groups for time division multiplexing, neural network processor

Abstract

The application relates to the fields of artificial intelligence and chip technology, in particular to a register group for time division multiplexing and a neural network processor, the register group comprising a plurality of register pairs; each register pair comprises a first register and a second register; the first register is connected with the second register; the first register is used for performing data exchange with the corresponding second register in response to the flip of a representation signal; and the second register is used for performing data exchange with the corresponding first register in response to the flip of the representation signal. The register group can realize fast data switching and configuration updating through the first register and the corresponding shadow register (the second register), thereby supporting the alternate running of multiple tasks or algorithms on the same hardware resource, significantly improving the utilization efficiency of the hardware resource, and being particularly suitable for time division multiplexing scenarios, such as the multi-task processing in an NPU.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and chips, and in particular to a register set and neural network processor for time-division multiplexing. Background Technology

[0002] In the fields of Artificial Intelligence (AI) and chips, Neural Network Processing Units (NPUs) have become core hardware supporting AI tasks due to their efficient neural network computing capabilities. In practical applications, multiple tasks typically need to run simultaneously on the same NPU chip. To meet this requirement with limited hardware resources, time-division multiplexing technology is widely used in NPU design.

[0003] The specific time-division multiplexing scheme is as follows: multiple tasks reuse the same set of hardware resources of the NPU. Through a time allocation mechanism, the hardware resources execute different tasks at different times, thereby enabling a single hardware resource to support multiple tasks.

[0004] However, the existing time-division multiplexing scheme of NPU has an efficiency bottleneck: when switching between different tasks frequently, the configuration parameters need to be rewritten to the register frequently. This operation will cause the configuration parameters to take too long, occupying the effective working time of hardware resources such as registers, resulting in a significant reduction in the overall operating efficiency of the NPU chip. Utility Model Content

[0005] This application provides a register group and a neural network processor for time-division multiplexing to solve the problem of low NPU operating efficiency in time-division multiplexing scenarios.

[0006] In a first aspect, embodiments of this application provide a register group for time-division multiplexing, including multiple pairs of registers; each pair of registers includes a first register and a second register; the first register and the second register are connected; the first register is used to: exchange data with its corresponding second register in response to a flip of a characterization signal; wherein, the characterization signal is a control signal that is triggered by a time slice switching and is used to instruct the registers to exchange data; the second register is used to: exchange data with its corresponding first register in response to a flip of the characterization signal.

[0007] In one possible implementation, the first register includes at least a first data input terminal and a first data output terminal; the first register is also used to: receive and store data through the first data input terminal, and output the stored data through the first data output terminal.

[0008] Secondly, embodiments of this application provide a neural network processor for time-division multiplexing. The neural network processor includes at least: a scheduler and an execution unit; the scheduler is connected to the execution unit; the execution unit includes an execution unit controller, multiple sets of read direct memory access (RDM) register sets, at least one set of core register sets, and at least one set of write direct memory access (WDMA) register sets; the execution unit controller is connected to the RDM register sets, core register sets, and WDMA register sets, and the RDM register sets, core register sets, and WDMA register sets are all time-division multiplexing register sets as described in the first aspect and its various implementations; the scheduler is used to: toggle the representation signal and to flip the signal... The characterization signal is then input to the execution unit controller; wherein, the characterization signal is a control signal triggered by time slice switching and used to instruct registers to exchange data; the execution unit controller is used to: receive the characterization signal and input the characterization signal to the rdma register group, core register group and wdma register group, so that the first register inside the rdma register group, core register group and wdma register group to flip in response to the characterization signal and exchange data with its corresponding second register, and to make the second register inside the rdma register group, core register group and wdma register group to flip in response to the characterization signal and exchange data with its corresponding first register.

[0009] In one possible implementation, the execution unit further includes execution unit logic circuitry; the execution unit logic circuitry is connected to the execution unit controller; the execution unit logic circuitry includes multiple rdma modules, at least one core module, and at least one wdma module; wherein each rdma module is connected to a set of rdma registers; and the core module is connected to the core register set; and the wdma module is connected to the wdma register set.

[0010] In one possible implementation, the first and second registers in the rdma register group are used to store rdma configuration data, and the portion of the rdma configuration data stored in the first register and its corresponding second register may be the same or different; the first and second registers in the core register group are used to store core configuration data, and the portion of the core configuration data stored in the first register and its corresponding second register may be the same or different; the first and second registers in the wdma register group are used to store wdma configuration data, and the portion of the wdma configuration data stored in the first register and its corresponding second register may be the same or different.

[0011] In one possible implementation, each rdma module is coupled to the first data output terminal of its corresponding first register. The rdma module is used to: obtain the rdma configuration data stored in its corresponding first register through the first data output terminal of its corresponding first register; the core module is coupled to the first data output terminal of its corresponding first register. The core module is used to: obtain the core configuration data stored in its corresponding first register through the first data output terminal of its corresponding first register; and the wdma module is coupled to the first data output terminal of its corresponding first register. The wdma module is used to: obtain the wdma configuration data stored in its corresponding first register through the first data output terminal of its corresponding first register.

[0012] In one possible implementation, the scheduler is further configured to: generate register configuration instructions and input the register configuration instructions to the execution unit controller, the register configuration instructions carrying rdma configuration data, core configuration data and / or wdma configuration data; the execution unit controller is further configured to: in response to the register configuration instructions, modify the configuration data stored in the first register in the rdma register set, core register set and / or wdma register set through the first data input terminal.

[0013] Thirdly, embodiments of this application provide a method for operating a time-division multiplexed neural network processor, applied to the time-division multiplexed neural network processor in the aforementioned second aspect and its various implementations; the method includes: a scheduler toggling a representation signal and inputting the toggled representation signal to an execution unit controller; wherein the representation signal is a control signal triggered by time slice switching and used to instruct registers to exchange data; the execution unit controller receives the representation signal and inputs the representation signal to an rdma register group, a core register group, and a wdma register group, so that a first register inside the rdma register group, the core register group, and the wdma register group toggles in response to the representation signal and exchanges data with its corresponding second register, and so that a second register inside the rdma register group, the core register group, and the wdma register group toggles in response to the representation signal and exchanges data with its corresponding first register.

[0014] In one possible implementation, the method further includes: a scheduler constructing a first virtual neural network processor (NPU) and a second virtual NPU; wherein the first virtual NPU and the second virtual NPU operate alternately by switching time slices; and the step of the scheduler flipping the representation signal includes: the scheduler flipping the representation signal in response to the time slice switching; wherein the time slice switching refers to switching from the time slice of the first virtual NPU to the time slice of the second virtual NPU, or, the time slice switching refers to switching from the time slice of the second virtual NPU to the time slice of the first virtual NPU; when the representation signal is 0, a first register is used to store configuration data of the first virtual NPU for the execution unit, and a second register is used to store configuration data of the second virtual NPU for the execution unit; when the representation signal flips from 0 to 1, the first register and its corresponding second register exchange data. The register switches from storing configuration data of the first virtual NPU for the execution unit to storing configuration data of the second virtual NPU for the execution unit, and the second register switches from storing configuration data of the second virtual NPU for the execution unit to storing configuration data of the first virtual NPU for the execution unit. When the indicator signal is 1, the first register is used to store configuration data of the second virtual NPU for the execution unit, and the second register is used to store configuration data of the first virtual NPU for the execution unit. When the indicator signal flips from 1 to 0, the first register and its corresponding second register exchange data, and the first register switches from storing configuration data of the second virtual NPU for the execution unit to storing configuration data of the first virtual NPU for the execution unit, and the second register switches from storing configuration data of the first virtual NPU for the execution unit to storing configuration data of the second virtual NPU for the execution unit.

[0015] In one possible implementation, the method further includes: a scheduler generating a register configuration instruction and inputting the register configuration instruction to an execution unit controller, the register configuration instruction carrying rdma configuration data, core configuration data and / or wdma configuration data; and the execution unit controller, in response to the register configuration instruction, modifying the configuration data stored in the first register in the rdma register group, core register group and / or wdma register group through a first data input terminal.

[0016] As described above, this application provides a register group, neural network processor, and operating method for time-division multiplexing. The register group includes multiple pairs of registers; each pair includes a first register and a second register; the first register and the second register are connected; the first register is used to exchange data with its corresponding second register in response to a flipping of a representation signal; wherein the representation signal is a control signal triggered by time slice switching and used to instruct the registers to exchange data; the second register is used to exchange data with its corresponding first register in response to a flipping of the representation signal. This register group, through the first register and its corresponding shadow register (i.e., the second register), enables rapid data switching and configuration updates, thereby supporting the alternating operation of multiple tasks or algorithms on the same hardware resource, significantly improving the utilization efficiency of hardware resources. It is particularly suitable for time-division multiplexing scenarios, such as multi-task processing in an NPU. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a register group for time-division multiplexing provided in an embodiment of this application;

[0018] Figure 2 A schematic diagram of the structure of the first register provided in an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of the structure of a neural network processor for time-division multiplexing provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the structure of the execution unit provided in the embodiments of this application;

[0021] Figure 5 This is a schematic diagram of the structure of the execution unit controller provided in the embodiments of this application;

[0022] Figure 6 This is a flowchart illustrating a method for operating a neural network processor for time-division multiplexing, as provided in an embodiment of this application.

[0023] Among them, 100 - register pair; 101 - first register; 102 - second register; 1011 - first data input terminal; 1012 - first data output terminal; 201 - scheduler; 2011 - instruction loader; 2012 - instruction decoder; 202 - execution unit; 2021 - execution unit controller; 2022 - execution unit logic circuit; 2022a - rdma module; 2022b - core module; 2022c - wdma module; 2023 - rdma register group; 2024 - core register group; 2025 - wdma register group; 203 - double data rate synchronous dynamic random access memory (DDR); 204 - on-chip memory (OCM). Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.

[0025] To address the low operating efficiency issue in time-division multiplexing (TDM) NPU chips, this application proposes at least one register group for TDM. This register group can include multiple pairs of registers, where two associated registers (reg) in each pair can cache different configuration parameters. During job switching, data can be exchanged between the two associated registers. It is worth noting that this hardware-level data exchange can shorten the data configuration path and reduce the number of data configuration instructions, significantly reducing parameter configuration time. Furthermore, based on the register group for TDM, this application also provides a TDM neural network processor (NPU) and its corresponding operating method, which can significantly improve the operating efficiency of the NPU chip.

[0026] Figure 1 This is a schematic diagram of the structure of a register group for time-division multiplexing provided in an embodiment of this application.

[0027] like Figure 1 As shown, this application embodiment provides a register group for time-division multiplexing, which may include multiple pairs of registers 100; each pair of registers 100 includes a first register 101 and a second register 102; the first register 101 and the second register 102 are connected. In this way, the first register 101 and the second register 102 can exchange data. The second register 102 can be referred to as the shadow register of the first register 101, and the first register 101 is a non-shadow register.

[0028] Figure 1In this context, "Reg0" and "Reg0_shadow" are register pairs (100), where "Reg0" is the first register (101), "Reg0_shadow" is the second register (102), and "Reg0_shadow" is the shadow register of "Reg0". Similarly, "Reg1" and "Reg1_shadow" are register pairs (100), where "Reg1" is the first register (101), "Reg1_shadow" is the second register (102), and "Reg1_shadow" is the shadow register of "Reg1". Likewise, "Regn" and "Regn_shadow" are register pairs (100), where "Regn" is the first register (101), "Regn_shadow" is the second register (102), and "Regn_shadow" is the shadow register of "Regn".

[0029] Figure 1 The diagram shows n pairs of registers 100. In practical applications, the number of register pairs 100 included in a register group can be set according to actual needs. This application does not impose a specific limitation on this.

[0030] Furthermore, the first register 101 is used to exchange data with its corresponding second register 102 in response to the flipping of the characterization signal, wherein the characterization signal is a control signal triggered by time slice switching and used to instruct the registers to exchange data. The second register 102 is used to exchange data with its corresponding first register 101 in response to the flipping of the characterization signal.

[0031] Data exchange refers to the bidirectional transmission of the data stored in the first register 101 and the second register 102 in response to the trigger condition of the characterization signal flipping. That is, the first register 101 transmits its currently stored data to the second register 102, and the second register 102 transmits its currently stored data to the first register 101, thus realizing the exchange of the stored data between the two and providing a data foundation for subsequent switching configuration and supporting time-division multiplexing functions.

[0032] It is understood that the first register 101 may include an enable terminal or a control terminal to receive characterization signals.

[0033] It is worth noting that when applying the register set to the NPU, the characterization signal can be the "npu_duty" signal generated by the NPU.

[0034] In some implementations, embodiments of this application can be designed in hardware such that the first register 101 interacts with an external bus (e.g., the AXI bus), while the second register 102 is not directly connected to the external bus and is only used to exchange data with its corresponding first register 101. This architecture makes the second register 102 a shadow register of the first register 101 in function, and its content is always a mirror image of the data of the first register 101 before the most recent data exchange, thereby realizing instantaneous data switching.

[0035] Understandably, registers work by stably storing binary data through internal flip-flop circuits. The core of this application lies in enabling two registers to directly exchange all stored data in a short time via hardware circuitry. This mechanism avoids the lengthy wait of reloading configuration data from the external bus during task switching, thus fundamentally solving the problem of chip efficiency degradation caused by frequent configuration.

[0036] Figure 2 A schematic diagram of the structure of the first register provided in an embodiment of this application.

[0037] like Figure 2 As shown, the first register 101 further includes at least a first data input port 1011 and a first data output port 1012; the first register is also used to: receive and store data through the first data input port, and output the stored data through the first data output port.

[0038] It is worth noting that the first data input terminal 1011 is a port used to write or update the data stored inside the first register 101 from an external data source, i.e., a configuration port. Correspondingly, the first data output terminal 1012 is a port for other hardware functional modules to read the data currently stored in the first register 101 in real time, i.e., an access port. Through the first data input terminal 1011 and the first data output terminal 1012, the first register 101 can efficiently complete data writing and reading operations, thereby supporting complex hardware functions and multi-tasking.

[0039] It is understood that the first data input terminal 1011 and the first data output terminal 1012 are only two of the many ports of the first register 101. The first register 101 may also include control ports such as a clock port and a reset port. This application embodiment does not specifically limit this.

[0040] As described above, this application provides a register group for time-division multiplexing, including multiple pairs of registers 100. Each pair of registers 100 includes a first register 101 and a second register 102. The first register 101 and the second register 102 are connected. The first register 101 is used to exchange data with its corresponding second register 102 in response to a flipping of a characterization signal. The characterization signal is a control signal triggered by a time slice switch and used to instruct the registers to exchange data. The second register 102 is used to exchange data with its corresponding first register 101 in response to a flipping of the characterization signal. This register group, through the first register 101 and its corresponding shadow register, the second register 102, enables rapid data switching and configuration updates, thereby supporting multiple tasks or algorithms to run alternately on the same hardware resource without interfering with each other. This design significantly improves the utilization efficiency of hardware resources and reduces the chip efficiency degradation caused by frequent configuration. It is particularly suitable for time-division multiplexing scenarios, such as multitasking in an NPU.

[0041] Figure 3 This is a schematic diagram of the structure of a neural network processor for time-division multiplexing provided in an embodiment of this application.

[0042] like Figure 3 As shown in the figure, this application provides a neural network processor for time-division multiplexing. The time-division multiplexing NPU may include a scheduler 201 and an execution unit (EU) 202; the scheduler 201 is connected to the execution unit 202. The scheduler 201 further includes an instruction loader 2011 and an instruction decoder 2012. The execution unit 202 further includes an execution unit controller 2021 and execution unit logic circuitry 2022.

[0043] Furthermore, the NPU for time-division multiplexing provided in this application embodiment adopts a two-level hardware control structure: the first-level control logic is executed by the scheduler 201, and the second-level control logic is executed by the execution unit 202.

[0044] In the first-level control logic, the instruction loader 2011 loads instructions from external storage, such as Double Data Rate SDRAM (DDR) 203. The instruction decoder 2012 decodes instructions from on-chip memory (OCM) 204 or loaded by the instruction loader 2011. Thus, while the instructions for the current task are running, the instruction loader 2011 can load the instructions for the next task into the OCM 204. Therefore, the instruction loading time can be hidden, significantly reducing the occupancy of the OCM 204. Based on this, the design size of the OCM 204 can be reduced in this embodiment, thereby lowering the cost. The instruction loader 2011 can consist of a Read Direct Memory Access (rdma) unit and a Write Direct Memory Access (wdma) unit. The rdma unit is responsible for reading instructions from the DDR 203, and the wdma unit is responsible for writing the read instructions into the OCM 204.

[0045] In the second-level control logic, the execution unit logic circuit 2022 is connected to the execution unit controller 2021. The execution unit controller 2021 directly manages the task startup and configuration of each functional sub-module in the execution unit logic circuit 2022. In this embodiment, the scheduler 201 can connect to the execution unit 202 through the output register configuration signal (regs_cfg) to transmit configuration information to the execution unit controller 2021 within the execution unit 202, enabling the execution unit controller 2021 to control the execution unit logic circuit 2022 to complete the corresponding neural network operation task based on the configuration information.

[0046] As can be seen, this application can achieve efficient task scheduling, instruction management and hardware resource allocation through the above two-level hardware control structure.

[0047] It should also be noted that the NPU used for time-division multiplexing may also include other hardware structures, such as a bus, an interface module, etc., which are not specifically limited in this application embodiment.

[0048] Figure 4 This is a schematic diagram of the structure of the execution unit provided in the embodiments of this application.

[0049] in, Figure 4 The top-level architecture of execution unit 202 is shown as an example.

[0050] Figure 5This is a schematic diagram of the structure of the execution unit controller provided in an embodiment of this application.

[0051] like Figure 4 and Figure 5 As shown, the execution unit 202 also includes multiple sets of read direct memory access rdma register sets (rdma_regs) 2023, at least one set of core register sets (core_regs) 2024, and at least one set of write direct memory access wdma register sets (wdma_regs) 2025. Furthermore, the execution unit controller 2021 is connected to the rdma register set 2023, the core register set 2024, and the wdma register set 2025.

[0052] It should be emphasized that rdma register group 2023, core register group 2024 and wdma register group 2025 are all register groups used for time division multiplexing in the aforementioned embodiments.

[0053] For further details, please refer to [link / reference]. Figure 4 The execution unit logic circuit 2022 may include multiple RDMA modules 2022a, at least one core module 2022b, and at least one WDMA module 2022c. See also... Figure 5 Each rdma module 2022a is connected to a set of rdma registers 2023; and the core module 2022b is connected to the core register set 2024; and the wdma module 2022c is connected to the wdma register set 2025.

[0054] Furthermore, in rdma register group 2023, the first register 101 and the second register 102 are used to store rdma configuration data, and the rdma configuration data stored in the first register 101 and its corresponding second register 102 may be the same or different; in core register group 2024, the first register 101 and the second register 102 are used to store core configuration data, and the core configuration data stored in the first register 101 and its corresponding second register 102 may be the same or different; in wdma register group 2025, the first register 101 and the second register 102 are used to store wdma configuration data, and the wdma configuration data stored in the first register 101 and its corresponding second register 102 may be the same or different.

[0055] Furthermore, the scheduler 201 is used to: flip the characterization signal and input the flipped characterization signal to the execution unit controller 2021, wherein the characterization signal is a control signal that is triggered by time slice switching and is used to instruct the register to exchange data.

[0056] In this embodiment of the application, the characterization signal is, for example, the "npu_duty" signal, or the characterization signal may be called the "regs set from dispatcher" signal.

[0057] The execution unit controller 2021 is configured to: receive a characterization signal and input the characterization signal to the rdma register group 2023, the core register group 2024 and the wdma register group 2025, so that the first register 101 inside the rdma register group 2023, the core register group 2024 and the wdma register group 2025 flips in response to the characterization signal and exchanges data with its corresponding second register 102; and to cause the second register 102 inside the rdma register group 2023, the core register group 2024 and the wdma register group 2025 to flip in response to the characterization signal and exchange data with its corresponding first register 101.

[0058] Understandably, through the representation signal, the execution unit controller 2021 enables all non-shadow registers in the rdma register group 2023, core register group 2024, and wdma register group 2025 to exchange data with the shadow registers. It is important to emphasize that the execution unit controller 2021 can send only the representation signal to the port of the first register 101. In this case, the first register 101 can respond to the representation signal and trigger a "data exchange operation," while the second register 102 does not need to receive an additional representation signal; it only needs to initiate the "data exchange operation" upon triggering by the first register 101, thus achieving the purpose of indirectly responding to the representation signal. This simplifies the wiring design of the second register 102.

[0059] It should be noted that, please continue to refer to [link / reference needed]. Figure 5 Specifically, the scheduler 201 can input the characterization signal "regs set from dispatcher" to the microcode register mapping module (mcode regmap) within the execution unit controller 2021. This microcode register mapping module then generates a "regs signal" and transmits it to the first register 101 of the rdma register group 2023, core register group 2024, and wdma register group 2025 in the control body (ctrlbody). Furthermore, the data stored in the second register 102 of the rdma register group 2023, core register group 2024, and wdma register group 2025 can be determined through a "data exchange" operation. That is, the data stored in the second register 102 originates from the first register 101 and is the configuration data stored in the first register 101 before the "data exchange operation."

[0060] It should be further noted that the configuration parameters of two adjacent tasks may be partially the same or different. For example, between different tasks segmented by the same neural network algorithm, often only a small portion of the configuration parameters change. During task switching, instead of reconfiguring all configuration parameters via the external bus, this embodiment can utilize a "shadow register" (i.e., the second register 102) as a data buffer. Changed configuration parameters are written to the second register 102 through the data input terminal of the first register 101, while unchanged parameters remain there. This mechanism significantly reduces the amount of data that needs to be transmitted via the external bus, effectively reducing bus load and power consumption. It also significantly reduces the number of microcodes (mcode), thereby reducing the mcode's footprint on the on-chip memory 204 and achieving the goal of reducing the capacity of the on-chip memory 204.

[0061] It should be further noted that the rdma module 2022a, core module 2022b, and wdma module 2022c in the execution unit logic circuit 2022 are controlled by independent control channels, and the control signal transmission between them does not interfere with each other. At the same time, each of these three modules corresponds to an independent register mapping (regmap) module. This independent channel control provides a hardware foundation for each module to implement time-division multiplexing configuration switching based on its own register group.

[0062] For some implementation methods, please refer to [link / reference]. Figure 4 In the top-level architecture of execution unit 202, execution unit controller 2021 can receive external control and data via the AXI bus. Furthermore, execution unit controller 2021 can send control flow signals via RDMA module 2022a. Figure 4 In this context, "control flow * n" indicates sending control flow signals to n RDMA modules 2022a respectively, and the RDMA module 2022a can return a status signal idle to the execution unit controller 2021. Figure 4 In this context, "idle*n" indicates that n RDMA modules 2022a respectively return status signals to the execution unit controller 2021. The RDMA modules 2022a are connected via the AXI bus (…). Figure 4In the diagram, "axi bis*n" indicates that n RDMA modules 2022a communicate with the Axi bus respectively. After acquiring data from the outside, they are transmitted to the core module 2022b in the form of a data stream. The core module 2022b can then perform neural network operations such as Streaming Data Shift (SDMA), Customized Encoder Programming (TENG), or Convolution (conv). The completed data stream is then transmitted to the WDMA module 2022c, which outputs the data externally via the Axi bus. Simultaneously, the execution unit controller 2021 can also interact with the core module 2022b and the WDMA module 2022c through control flow signals and idle signals to coordinate their operation, ultimately completing the process of "data reading → computation → data writing".

[0063] For some implementation methods, please refer to [link / reference]. Figure 5 In the underlying architecture of execution unit 202, instructions can be sent to execution unit controller 2021 for parsing via the OCM read path. Specifically, the demultiplexer demux can distribute instructions to multiple channels from rdma ch0 to rdma chN-1, preparing for subsequent data read paths. After entering the control body (ctrl body) inside execution unit controller 2021, the decoder and execution subunit (decoder & exu) decode the instructions and generate control signals (mcode set to eu modules). Then, the control signals are sent to rdma register group 2023, core register group 2024, and / or wdma register group 2025 respectively to configure the operation of rdma module 2022a, core module 2022b, and / or wdma module 2022c.

[0064] In some implementations, the first register 101 and the second register 102 in the rdma register group 2023, the core register group 2024 and the wdma register group 2025 can be general purpose registers with a capacity of 1 kilobit, that is, the specification of the first register 101 and the second register 102 can be GPRs1kb.

[0065] Furthermore, it should be noted that each rdma module 2022a is coupled to the first data output terminal 1012 of the first register 101 in its corresponding rdma register group 2023. The rdma module 2022a is used to obtain the rdma configuration data stored in the first register 101 through the first data output terminal 1012 of the first register 101 in its corresponding rdma register group 2023.

[0066] The core module 2022b is coupled to the first data output terminal 1012 of the first register 101 in its corresponding core register group 2024. The core module 2022b is used to obtain the core configuration data stored in the first register 101 through the first data output terminal 1012 of the first register 101 in its corresponding core register group 2024.

[0067] The WDMA module 2022c is coupled to the first data output terminal 1012 of the first register 101 in its corresponding WDMA register group 2025. The WDMA module 2022c is used to obtain the WDMA configuration data stored in the first register 101 through the first data output terminal 1012 of the first register 101 in its corresponding WDMA register group 2025.

[0068] Based on the above, it can be seen that the non-shadow registers (first register 101) in the register group are the registers that directly interact with the RDMA module 2022a, core module 2022b, and WDMA module 2022c. That is, only the non-shadow registers (or function registers) output to the RDMA module 2022a, core module 2022b, and WDMA module 2022c for their use. The shadow registers (second registers) do not interact with the RDMA module 2022a, core module 2022b, and WDMA module 2022c.

[0069] Furthermore, it should be noted that the scheduler 201 is also used to: generate register configuration instructions and input the register configuration instructions to the execution unit controller 2021. The register configuration instructions carry rdma configuration data, core configuration data and / or wdma configuration data.

[0070] It is worth noting that in various application scenarios, such as when a new computing task arrives, the parameters of the current task need to be dynamically adjusted, or when pre-configuration is required for the next task, the scheduler 201 will generate the corresponding register configuration instructions.

[0071] The execution unit controller 2021 is also configured to modify the configuration data stored in the first register 101 in the rdma register group 2023, the core register group 2024 and / or the wdma register group 2025 via the first data input terminal 1011 in response to a register configuration instruction.

[0072] For example, RDMA register group 2023 specifically includes RDMA register group a, RDMA register group b, and RDMA register group c; core register group 2024 can specifically be core register group d; and WDMA register group 2025 can specifically be WDMA register group e. Then, scheduler 201 can modify the values ​​in some or all registers through register configuration instructions. For example, when only the value of core register group d needs to be modified, scheduler 201 can generate a corresponding register configuration instruction. This instruction will be routed by execution unit controller 2021 to core register group d, thereby precisely updating its value and achieving highly flexible and granular control of configuration updates.

[0073] As can be seen from the above, this application provides a neural network processor for time-division multiplexing. The neural network processor includes a scheduler 201 and an execution unit 202. The scheduler 201 is connected to the execution unit 202. The execution unit 202 includes an execution unit controller 2021, an RDMA register group 2023, at least one core register group 2024, and at least one WDMA register group 2025. The execution unit controller 2021 is connected to the RDMA register group 2023, the core register group 2024, and the WDMA register group 2025. The scheduler 201 is used to: toggle a representation signal and input the toggled representation signal to the execution unit controller 2021, wherein the representation signal is toggled due to time slice switching. The control signal is used to instruct the registers to exchange data. The execution unit controller 2021 is used to: receive the characterization signal and input the characterization signal to the rdma register group 2023, core register group 2024 and wdma register group 2025, so that the first register 101 inside the rdma register group 2023, core register group 2024 and / or wdma register group 2025 flips in response to the characterization signal and exchanges data with its corresponding second register 102, and to cause the second register 102 inside the rdma register group 2023, core register group 2024 and wdma register group 2025 to flip in response to the characterization signal and exchange data with its corresponding first register 101. This neural network processor realizes instantaneous switching of hardware configuration parameters by setting exchangeable shadow registers in the rdma register group 2023, core register group 2024 and wdma register group 2025, effectively eliminating the configuration overload overhead during task switching, and significantly improving the operating efficiency and hardware resource utilization of the NPU in time-division multiplexing mode.

[0074] Figure 6 This is a flowchart illustrating a method for operating a neural network processor for time-division multiplexing, as provided in an embodiment of this application.

[0075] like Figure 6 As shown, this application provides a method for running a time-division multiplexed neural network processor, which can be applied to the time-division multiplexed neural network processor in the foregoing embodiments. The method may include the following steps S100-S200:

[0076] S100: Scheduler 201 flips the characterization signal and inputs the flipped characterization signal to execution unit controller 2021. The characterization signal is a control signal that is triggered by time slice switching and is used to indicate the register to exchange data.

[0077] In this embodiment, the timing of the scheduler 201 toggling the representation signal includes, but is not limited to, task switching and virtual NPU switching. A virtual NPU refers to multiple logical NPU instances created on the same physical NPU hardware using a time-slice rotation method. Each virtual NPU has an independent configuration context but shares the same physical computing resources. The toggling of this representation signal, as a hardware-level synchronization event, marks the switching point of the execution context between different virtual NPUs. Specific steps will be detailed below and will not be repeated here.

[0078] It should also be noted that the time-slicing mechanism is a scheduling method that uses time allocation to allow multiple tasks to share a single physical computing resource (such as CPU or NPU) in a time-sharing manner. Specifically, the processor's execution time can be divided into a series of extremely small time intervals of fixed or variable length, each interval being called a "time slice." Multiple tasks (or virtual NPUs) then take turns using these time slices, thereby simulating the effect of "multiple tasks running simultaneously."

[0079] S200: The execution unit controller 2021 receives the characterization signal and inputs the characterization signal to the rdma register group 2023, the core register group 2024 and the wdma register group 2025, so that the first register 101 inside the rdma register group 2023, the core register group 2024 and the wdma register group 2025 flips in response to the characterization signal and exchanges data with its corresponding second register 102, and causes the second register 102 inside the rdma register group 2023, the core register group 2024 and the wdma register group 2025 to flip in response to the characterization signal and exchange data with its corresponding first register 101.

[0080] In this way, the neural network processor can quickly switch between different tasks (different tasks can refer to different tasks divided by the same algorithm) without reloading all configuration parameters, thereby reducing configuration time and improving the overall operating efficiency of the neural network processor.

[0081] It is worth noting that, prior to step S100, the neural network processor running method for time-division multiplexing provided in this application embodiment may further include the following step S300.

[0082] S300: Scheduler 201 constructs a first virtual neural network processor (NPU) and a second virtual NPU; wherein the first virtual NPU and the second virtual NPU operate alternately based on time slice switching.

[0083] The first virtual NPU can be virtual NPU0, and the second virtual NPU can be virtual NPU1.

[0084] It's worth noting that time slice allocation is not strictly in a fixed order of "first virtual NPU → second virtual NPU → first virtual NPU," but is dynamically adjusted according to task requirements. For example, two consecutive time slices can be allocated to the first virtual NPU to execute its continuous tasks, and then the time slice can be switched to the second virtual NPU to adapt to the task continuity requirements of different virtual NPUs.

[0085] Understandably, time slice switching implies "task switching".

[0086] Furthermore, step S100 may specifically include the following step S101: the scheduler 201 toggles the representation signal in response to time slice switching; wherein, time slice switching refers to switching from the time slice of the first virtual NPU to the time slice of the second virtual NPU, or, time slice switching refers to switching from the time slice of the second virtual NPU to the time slice of the first virtual NPU.

[0087] The correspondence between the characterization signal and the register data is as follows:

[0088] The characterization signal indicates which virtual NPU is using hardware resources (including but not limited to execution unit 202). When the characterization signal is 0, the first register 101 stores the configuration data of the first virtual NPU for the execution unit, and the second register 102 stores the configuration data of the second virtual NPU for the execution unit. In other words, when the characterization signal is 0, the first register 101 acts as a "non-Shadow register," storing the configuration data of the first virtual NPU currently authorized to use hardware resources for the execution unit 202; the second register 102 acts as a "Shadow register," storing the configuration data of the second virtual NPU previously authorized to use the execution unit 202 (i.e., the core function of the Shadow register is to temporarily store the hardware configuration parameters of the previous virtual NPU).

[0089] It should be noted here that the data actually stored in the first register 101 and the second register 102 can be the configuration data of the virtual NPU for the execution unit 202, for example:

[0090] For the first register 101 and the second register 102 in the rdma register group 2023, the stored data may include the data read address of the rdma module 2022a (such as the starting address of external memory), single transmission length, data caching strategy, channel priority, and other configurations. For the first register 101 and the second register 102 in the core register group 2024, the operation type (such as convolution), operation precision (such as FP16 / INT8), input and output data format, intermediate result temporary storage address, and other configurations of the core module 2022b can be stored. For the first register 101 and the second register 102 in the wdma register group 2025, the data write target address of the wdma module 2022c, the verification method (such as CRC verification enable), transmission completion interrupt triggering condition, and other configurations can be stored. These data directly determine the working mode of each module in the execution unit 202 and are the core configuration basis for the virtual NPU to exclusively use physical hardware resources.

[0091] It is understandable that the data stored in the first register 101 and the second register 102 may be partially the same or different.

[0092] When the representation signal flips from 0 to 1, the first register 101 and its corresponding second register 102 exchange data. The first register 101 switches from storing configuration data of the first virtual NPU for the execution unit 202 to storing configuration data of the second virtual NPU for the execution unit 202, and the second register 102 switches from storing configuration data of the second virtual NPU for the execution unit 202 to storing configuration data of the first virtual NPU for the execution unit 202. In other words, when the representation signal flips from 0 to 1 (triggering a time slice switch to the second virtual NPU), the first register 101 and the second register 102 automatically complete the data exchange: the first register 101 switches from storing configuration data of the first virtual NPU to storing configuration data of the second virtual NPU, and the second register 102 switches from storing configuration data of the second virtual NPU to storing configuration data of the first virtual NPU.

[0093] Furthermore, when the representation signal is 1, the first register 101 is used to store the configuration data of the second virtual NPU for the execution unit 202, and the second register 102 is used to store the configuration data of the first virtual NPU for the execution unit 202. In other words, when the representation signal is 1, the first register 101 acts as a non-shadow register, storing the configuration data of the currently active second virtual NPU; the second register 102 acts as a shadow register, storing the configuration data of the previously active first virtual NPU.

[0094] When the representation signal flips from 1 to 0, the first register 101 and its corresponding second register 102 exchange data. The first register 101 switches from storing the configuration data of the second virtual NPU for the execution unit 202 to storing the configuration data of the first virtual NPU for the execution unit 202, and the second register 102 switches from storing the configuration data of the first virtual NPU for the execution unit 202 to storing the configuration data of the second virtual NPU for the execution unit 202. In other words, when the representation signal flips from 1 to 0 (the time slice switches back to the first virtual NPU), the first register 101 and the second register 102 automatically exchange data again: the first register 101 re-stores the configuration data of the first virtual NPU, and the second register 102 stores the configuration data of the second virtual NPU.

[0095] Through the above mechanism, whenever a virtual NPU authorized to use hardware resources switches, the Shadow register (second register 102) and the non-Shadow register (first register 101) will automatically exchange data through hardware logic. This ensures that the Shadow register always stores the configuration parameters of the previous virtual NPU, and enables the non-Shadow register to quickly load the configuration data of the current virtual NPU, thereby achieving seamless switching between virtual NPUs and efficient reuse of hardware resources.

[0096] It should also be noted that the number of virtual NPUs is not limited to two; it can be configured based on the actual task. Correspondingly, the number of shadow registers corresponding to non-shadow registers is not limited to one; the shadow registers can be horizontally scaled based on the number of virtual NPUs.

[0097] In some implementations, the transmission of the characterization signal can be divided into two logics: one is that the characterization signal is continuously transmitted under normal conditions, and its state is only triggered to flip when the time slice is switched (corresponding to the task switching between the first virtual NPU and the second virtual NPU), so as to trigger the data exchange between the first register 101 and the corresponding second register 102 through the state change; the other is that the characterization signal is not continuously transmitted, and the flipped signal is only sent to the register group after the time slice is switched to trigger the signal to flip, which is also used to trigger the data exchange between the above registers.

[0098] Furthermore, the neural network processor operation method for time-division multiplexing provided in the embodiments of this application may also include the following steps S401-S402.

[0099] S401: Scheduler 201 generates register configuration instructions and inputs the register configuration instructions to execution unit controller 2021. The register configuration instructions carry rdma configuration data, core configuration data and / or wdma configuration data.

[0100] S402: In response to a register configuration instruction, the execution unit controller 2021 modifies the configuration data stored in the first register 101 in the rdma register group, core register group and / or wdma register group through the first data input terminal 1011.

[0101] This allows the data stored in the first register 101 to be updated. Therefore, when the execution unit 202 runs a new task, since the configuration parameters can be preloaded into the first register, it is not necessary to modify all function registers; only the parameters required for this task need to be reconfigured, greatly reducing configuration time and improving model running efficiency.

[0102] In some implementations, the neural network processor running method for time-division multiplexing provided in this application embodiment may further include the following steps S501, S502 and S503.

[0103] S501: The rdma module 2022a obtains the rdma configuration data stored in its corresponding first register 101 through the first data output terminal 1012 of its corresponding first register 101;

[0104] S502: The core module 2022b obtains the core configuration data stored in its corresponding first register 101 through the first data output terminal 1012 of its corresponding first register 101;

[0105] S503: The WDMA module 2022c obtains the WDMA configuration data stored in its corresponding first register 101 through the first data output terminal 1012 of its corresponding first register 101.

[0106] It is understandable that the above steps can occur during the operation of rdma module 2022a, core module 2022b, and wdma module 2022c.

[0107] It should also be noted that the above process only represents some of the core aspects of register configuration management between the scheduler 201 and the execution unit 202. In actual operation, the collaborative control between the two also covers more dimensions: for example, the scheduler 201 needs to perform dynamic task scheduling based on task priority, allocate resources based on hardware load, and monitor the running status of each module in the execution unit 202 in real time (such as computation progress and abnormal interrupt signals); the execution unit 202 needs to respond to the timing control instructions of the scheduler 201 and coordinate the data flow of the internal RDMA / Core / WDMA modules, etc. This application embodiment does not specifically limit these aspects.

[0108] As can be seen from the above, the embodiments of this application provide a register group, a neural network processor, and an operating method for time-division multiplexing. Specifically, the embodiments of this application achieve multiple advantages through hardware-level register data exchange and virtual NPU switching mechanisms:

[0109] Each individual register group includes register pairs, which consist of shadow registers and non-shadow registers. The shadow registers can store configuration parameters for unauthorized virtual NPUs executing unit 202. When switching between authorized virtual NPUs, the values ​​in the shadow and non-shadow registers are swapped, thus preserving the configuration parameters used by different virtual NPUs. During operation, it is unnecessary to modify all configuration parameters, shortening configuration time and improving chip operating efficiency. The automatic swapping of shadow and non-shadow register values ​​during the switching between authorized virtual NPUs requires no further software intervention, simplifying the application and further improving chip operating efficiency.

[0110] This mechanism also reduces the number of microcodes, eliminating the need for a large number of microcode instructions to implement the configuration logic for virtual NPU switching, thereby reducing the microcode's requirement for on-chip memory 204 and reducing the capacity design of on-chip memory 204.

[0111] Furthermore, the rdma module 2022a, core module 2022b, and wdma module 2022c in execution unit 202 are each controlled by an independent channel and equipped with a separate register mapping (regmap) module. Each regmap module consists of shadow registers and non-shadow registers. This modular design ensures the independence of each module's operation and effectively preserves different virtual NPU configuration parameters through the separate storage of register groups, providing a fundamental support for rapid switching and efficient reuse.

[0112] In a specific implementation, this utility model also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, it may include some or all of the steps of the various embodiments of the neural network processor running method for time-division multiplexing provided by this utility model. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0113] It is readily understood that, based on the several embodiments provided in this application, those skilled in the art can combine, split, or reorganize the embodiments of this application to obtain other embodiments, none of which exceed the protection scope of this application.

[0114] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A register bank for time-division multiplexing, characterized in that, It includes multiple pairs of registers; each pair of registers includes a first register and a second register; the first register and the second register are connected; The first register is used to: exchange data with the corresponding second register in response to the flipping of the characterization signal; wherein the characterization signal is a control signal that is triggered by a time slice switching and is used to instruct the register to exchange data; The second register is used to exchange data with the first register corresponding to it in response to the flipping of the characterization signal.

2. The register group for time-division multiplexing according to claim 1, characterized in that, The first register includes at least a first data input terminal and a first data output terminal; the first register is also used to: receive and store data through the first data input terminal, and output the stored data through the first data output terminal.

3. A neural network processor for time-division multiplexing, characterized in that, The neural network processor includes at least: a scheduler and an execution unit; the scheduler is connected to the execution unit. The execution unit includes an execution unit controller, multiple sets of read direct memory access (RDMA) register sets, at least one set of core register sets, and at least one set of write direct memory access (WDMA) register sets; the execution unit controller is connected to the RDMA register sets, the core register sets, and the WDMA register sets, and the RDMA register sets, the core register sets, and the WDMA register sets are all time-division multiplexing register sets as described in any one of claims 1-2; The scheduler is used to: flip the characterization signal and input the flipped characterization signal to the execution unit controller; wherein the characterization signal is a control signal triggered by time slice switching and used to instruct registers to exchange data; The execution unit controller is configured to: receive the characterization signal and input the characterization signal to the rdma register group, the core register group, and the wdma register group, so that the first register inside the rdma register group, the core register group, and the wdma register group flips in response to the characterization signal and exchanges data with its corresponding second register; and to cause the second register inside the rdma register group, the core register group, and the wdma register group to flip in response to the characterization signal and exchange data with its corresponding first register.

4. The neural network processor for time-division multiplexing according to claim 3, characterized in that, The execution unit further includes an execution unit logic circuit; the execution unit logic circuit is connected to the execution unit controller. The execution unit logic circuit includes multiple rdma modules, at least one core module, and at least one wdma module; Each of the rdma modules is connected to a set of rdma register groups; Furthermore, the core module is connected to the core register group; Furthermore, the WDMA module is connected to the WDMA register group.

5. The neural network processor for time-division multiplexing according to claim 4, characterized in that, The first register and the second register in the rdma register group are used to store rdma configuration data, and the rdma configuration data stored in the first register and its corresponding second register may be the same or different. The first register and the second register in the core register group are used to store core configuration data, and the core configuration data stored in the first register and its corresponding second register may be the same or different. The first register and the second register in the WDMA register group are used to store WDMA configuration data, and the WDMA configuration data stored in the first register and its corresponding second register may be the same or different.

6. The neural network processor for time-division multiplexing according to claim 5, characterized in that, Each rdma module is coupled to the first data output terminal of its corresponding first register. The rdma module is used to: obtain the rdma configuration data stored in its corresponding first register through the first data output terminal of its corresponding first register. The core module is coupled to the first data output terminal of its corresponding first register. The core module is used to: obtain the core configuration data stored in its corresponding first register through the first data output terminal of its corresponding first register. The WDMA module is coupled to the first data output terminal of its corresponding first register. The WDMA module is used to: obtain the WDMA configuration data stored in its corresponding first register through the first data output terminal of its corresponding first register.

7. The neural network processor for time-division multiplexing according to claim 5 or 6, characterized in that, The scheduler is also configured to: generate register configuration instructions and input the register configuration instructions to the execution unit controller, wherein the register configuration instructions carry rdma configuration data, core configuration data and / or wdma configuration data; The execution unit controller is further configured to: in response to the register configuration instruction, modify the configuration data stored in the first register of the rdma register group, the core register group, and / or the wdma register group via the first data input terminal.

Images