Voltage and frequency dynamic adjustment circuit and neural network processor

Abstract

This disclosure provides a voltage and frequency dynamic adjustment circuit and a neural network processor, applied to a neural network processor that performs power management through a power manager and a frequency manager, and includes an operational layer and a control layer. The circuit includes a first power control module, used to generate a power mode execution request based on power mode configuration parameters, perform power mode operations on the control layer according to the power mode execution request, output a voltage adjustment signal of the control layer to the power manager, and output a frequency adjustment signal containing at least one clock frequency to the frequency manager; a second power control module, used to perform corresponding power mode operations on the operational layer according to the power mode execution request. Therefore, this disclosure can dynamically adjust the voltage and frequency of the power supply according to the actual load of the processor to achieve a balance between low power consumption and high performance.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit design technology, specifically to a voltage-frequency dynamic adjustment circuit and a neural network processor. Background Technology

[0002] With the rapid development of artificial intelligence technology, the power consumption control problem of neural network processors, as the core devices for performing deep learning tasks, has become increasingly prominent.

[0003] Current neural network accelerators, in pursuit of higher performance, typically lack low-power design, resulting in wasted energy and failing to meet the low-power requirements of edge devices or portable mobile devices. Furthermore, high energy consumption generates significant heat, necessitating robust cooling systems to ensure stable operation. This not only increases hardware costs and system complexity but also potentially occupies considerable space. Moreover, in many scenarios, neural network accelerators are only used intermittently or partially, leading to additional energy consumption that, while not generating practical value, severely reduces the chip's battery life.

[0004] Therefore, there is an urgent need for a circuit structure that can dynamically adjust voltage and frequency according to the processor load in order to achieve a balance between low power consumption and high performance. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide a voltage frequency dynamic adjustment circuit and a neural network processor to at least partially solve the above-mentioned problems.

[0006] According to a first aspect of the present disclosure, a voltage-frequency dynamic adjustment circuit is provided, which is applied to a neural network processor that performs power supply voltage and frequency adjustment through a power manager and a frequency manager. The neural network processor includes an operation layer and a control layer, and the control layer controls the logical operation operations of the operation layer. The voltage-frequency dynamic adjustment circuit includes: a first power consumption control module connected to the control layer and configured to: generate a power consumption mode execution request according to power consumption mode configuration parameters; and perform a corresponding power consumption mode operation on the control layer according to the power consumption mode execution request to output a voltage adjustment signal of the control layer to the power manager and a frequency adjustment signal containing at least one clock frequency to the frequency manager; and a second power consumption control module connected to the first power consumption control module and the operation layer and configured to: perform a corresponding power consumption mode operation on the operation layer according to the power consumption mode execution request.

[0007] According to a second aspect of the present disclosure, a neural network processor is provided, which includes a voltage-frequency dynamic adjustment circuit as described in the first aspect, for dynamically adjusting the voltage and frequency of the power supply of the neural network processor according to different power consumption operation modes of the neural network processor.

[0008] In summary, the voltage and frequency dynamic adjustment schemes provided in this disclosure can dynamically adjust the power supply voltage and frequency according to the actual load of the neural network processor, avoid high energy consumption of the chip under low load, effectively reduce the overall power consumption of the processor, improve the chip's battery life, and reduce power consumption costs. Attached Figure Description

[0009] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0010] Figure 1 This is a structural block diagram of a voltage frequency dynamic adjustment circuit, which is an exemplary embodiment of the present disclosure.

[0011] Figure 2 This is a structural block diagram of a voltage frequency dynamic adjustment circuit, which is another exemplary embodiment of this disclosure.

[0012] Figure 3 A detailed circuit diagram of a voltage frequency dynamic adjustment circuit, which is an exemplary embodiment of this disclosure. Detailed Implementation

[0013] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0014] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0015] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative and not restrictive. Those skilled in the art should understand that, unless explicitly stated in the context, they should be understood as "one or more". The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0017] Figure 1 A schematic diagram of the voltage-frequency dynamic adjustment circuit 200 according to an exemplary embodiment of the present disclosure is shown. The voltage-frequency dynamic adjustment circuit 200 of this embodiment can provide low-power mode management for the neural network processor 100. The neural network processor 100, for example, is a neural network accelerator (AI accelerator), which can perform power supply voltage adjustment management through the power manager 102 and clock frequency adjustment management through the frequency manager 104.

[0018] In this embodiment, the neural network processor 100 adopts a "control-computation" separate design architecture, which mainly includes a control layer 112 and a computation layer 114, so that the logical operation of the computation layer 114 can be controlled by the control layer 112.

[0019] The neural network processor 100 can be applied to various electronic terminals (e.g., mobile phones, edge devices, etc.) and supports scenarios such as artificial intelligence (AI) inference and multi-tasking.

[0020] For example, in Figure 3 In the example shown, the neural network processor 100 is NPU_SS (Neural Network Acceleration Subsystem), the control layer 112 is NPU_SS_PWR_WRAP layer, the computation layer 114 is IP_NPU layer, the power manager 102 is PMIC Ctrl, and the frequency manager 104 is PLL (Phase-Locked Loop).

[0021] The voltage and frequency dynamic adjustment circuit 200 mainly includes a first power consumption control module 210 and a second power consumption control module 220.

[0022] First power consumption control module 210 (e.g., Figure 3 The npuss_pctrl) is connected to the control layer 112 and is configured to generate a power mode execution request based on the power mode configuration parameters, perform low power mode operation on the control layer 112 based on the power mode execution request, output a voltage adjustment signal of the control layer 112 to the power manager 102, and output a frequency adjustment signal containing at least one clock frequency to the frequency manager 104.

[0023] In some embodiments, the power mode configuration parameters of the neural network processor 100 can be determined in the following ways:

[0024] Based on the actual load of the neural network processor 100 (e.g., performing a task), the power consumption mode of the neural network processor 100 is determined, and the corresponding power consumption mode configuration parameters are determined based on the power consumption mode of the neural network processor 100.

[0025] In this embodiment, the power consumption modes of the neural network processor 100 may include a normal operating mode (ON mode), at least one low power mode (including but not limited to: ICG mode / MEMORY SLP mode / MEMORY DSLP mode / MEMORYSD mode), and a shutdown mode (OFF mode).

[0026] In this configuration, ON indicates that the neural network processor 100 (NPU) needs to operate normally, while ICG indicates that the NPU has a small amount of idle time but may execute operations at any time. When the NPU's idle time increases, it can be set to MEMORY SLP / DSLP / SD mode. If the NPU clearly has no tasks to execute, it can be set to OFF state. Therefore, this embodiment sets different power consumption modes according to the actual load of the neural network processor 100, thereby optimizing the power consumption of the neural network processor 100.

[0027] Second power consumption control module 220 (e.g., Figure 3 The npu_pctrl) connects the first power control module 210 and the computing layer 114, and is configured to perform power mode operations on the computing layer 114 according to the power mode execution request.

[0028] In some embodiments, the second power consumption control module 220 may be integrated into the control layer 112.

[0029] In this embodiment, the second power consumption control module 220 and the first power consumption control module 210 are two relatively independent power consumption control threads.

[0030] Figure 2 This is a structural diagram of a voltage frequency dynamic adjustment circuit 200 according to another embodiment of the present disclosure, further showing... Figure 1The specific architecture diagram of the first power consumption control module 210 and the second power consumption control module 220.

[0031] The first power consumption control module 210 may further include a request unit 2102, a power consumption channel expansion unit 2104, a first power consumption control unit 2106, a first power consumption control generator 2108, a signal switching unit 2110, and a frequency adjustment unit 2112.

[0032] Request unit 2102 (e.g.) Figure 3 The R2P unit is configured to generate corresponding power mode execution requests based on the power mode configuration parameters of the neural network processor 100.

[0033] In some embodiments, different power consumption modes of the neural network processor (including but not limited to normal operation mode, multiple low power consumption modes, and shutdown mode) can be determined according to the actual load of the neural network processor, and the corresponding power consumption mode configuration parameters can be determined accordingly.

[0034] In some embodiments, the requesting unit 2102 can be configured via bus 2002 (e.g., Figure 3 The X2P bus can be used to obtain external power mode configuration parameters.

[0035] Power channel extension unit 2104 (e.g.) Figure 3 The NPUSS_LPE_P, connecting request unit 2102, first power control unit 2106 and second power control module 220, is configured to distribute power mode execution requests from request unit 2102 to first power control unit 2106 and second power control module 220 (second power control unit 2202).

[0036] First power consumption control unit 2106 (e.g., Figure 3 The NPUSS_PCU is connected between the power channel extension unit 2104 and the first power control generator 2108. It is configured to send a corresponding power mode switching request to the neural network processor 100 according to the power mode execution request broadcast by the power channel extension unit 2104, and send a power mode execution request to the first power control generator 2108 in response to the request enable response of the neural network processor 100 (i.e., the power mode switching request is passed).

[0037] For example, the first power consumption control unit 2106 may include an asynchronous timing synchronizer, a low-power mode priority arbitration module, and a low-power mode state machine (not shown). The asynchronous timing synchronizer is used to synchronize all inputs and outputs; the low-power mode priority arbitration module is used to determine the mode priority of multiple inputs; the low-power mode state machine can execute a power consumption mode transition based on the input information and send a power consumption mode transition request to the device. If the request is approved, a P-Channel request is sent to the first power consumption control generator 2108. In this way, the neural network processor 100 (NPU) can be controlled by the first power consumption control unit 2106 to be in different power consumption modes (e.g., ON / ICG / MEMORY SLP / MEMORY DSLP / MEMORY SD / OFF modes), thereby achieving power consumption optimization in different modes and reducing device power consumption.

[0038] First power consumption control generator 2108 (e.g., Figure 3 The NPUSS_BPC connects the first power consumption control unit 2106, the control layer 112, and the signal transfer unit 2110 (e.g., NPUSS_BPC). Figure 3 The NPUSS_PCA is configured to perform corresponding power mode operations on each component in the control layer 112 according to the power mode execution request of the first power control unit 2106, and output the voltage adjustment signal of the control layer 112 to the power manager 102 via the signal transfer unit 2110.

[0039] In some embodiments, the first power consumption control generator 2108 may perform power-on / off operations and / or voltage adjustment operations on each component in the control layer 112 according to the power consumption mode execution request, and in response to the execution response results of each component in the control layer 112, send the voltage adjustment signal of the control layer 112 to the power manager 102 through the signal transfer unit 2110.

[0040] In this embodiment, the control layer 112 includes at least one clock control unit 2114a, 2114b (e.g. Figure 3 The NPU_AAB_CCU), and the different clock control units 2114a and 2114b have different clock frequencies.

[0041] Frequency adjustment unit 2112 (e.g.) Figure 3 The NPUSS_LPE_C can be connected to the first power consumption control unit 2106 and each clock control unit 2114a, 2114b in the control layer 112 to execute requests according to the power consumption mode of the first power consumption control unit 2106, control each clock control unit 2114a, 2114b to perform corresponding clock frequency adjustment operations, and send each frequency adjustment signal of each clock control unit 2114a, 2114b to the frequency manager 104.

[0042] In addition, each clock control unit 2114a, 2114b (e.g. Figure 3 The NPU_AAB_CCU can also adjust its operation according to the clock frequency, generating clock start / stop signals for purposes such as control. Figure 3 The operating clock of the bus conversion bridge chip.

[0043] The second power consumption control module 220 includes a second power consumption control unit 2202 and a second power consumption control generator 2204. The computation layer 114 of the neural network processor 100 carries at least one computation core 1142 (e.g., Figure 3 CALC_CORE / CTRL_CORE).

[0044] Second power consumption control unit 2202 (e.g., Figure 3 The NPU_PR_PCU is connected to the power channel extension unit 2104 of the first power control module 210 and is configured to send state switching requests to each component in the computing layer 114 according to the power mode execution request broadcast by the power channel extension unit 2104, and in response to the request enable response (i.e. request pass response) of each component in the computing layer 114, output the power mode execution request to the second power control generator 2204.

[0045] Second power consumption control generator 2204 (e.g., Figure 3 The NPU_PR_BPC is connected to the second power control unit 2202 and each computing core 1142, and is configured to perform voltage adjustment operations on each computing core 1142 according to the power mode execution request of the second power control unit 2202.

[0046] It should be noted that, in this embodiment, since each computing core 1142 does not support the OFF mode, the second power consumption control generator 2204 does not need to output the voltage adjustment signal of each computing core 1142 to the power manager 102 according to the voltage adjustment operation response of each computing core 1142.

[0047] In this embodiment, the computation layer 114 further includes a memory 1144 (e.g., Figure 3 The IP_MEM connects the configuration bus 1002 of each computing core 1142 and the neural network processor 100.

[0048] The memory 1144 is used to prioritize providing data storage services to each processing core 1142. When the memory 1144 is not used by the processing cores 1142, it can also provide data storage services to the configuration bus 1002 of the neural network processor 100. Specifically, this disclosure designs two independent power control modules 210 and 220. The memory 1144 in the processing layer 114 is controlled by the first power control module 210, while each processing core 1142 in the processing layer 114 is controlled by the second power control module 220. Since the first power control module 210 and the second power control module 220 are two independent control threads, even when the processing cores 1142 in the processing layer 114 are not running, the memory 1144 remains powered on, thus providing data storage services to the configuration bus 1002 of the neural network processor 100. This allows the memory 1144 of the processing layer 114 to be reused by external units, reducing resource consumption costs.

[0049] It should be noted that, in addition to providing data storage services for the configuration bus 1002 of the neural network processor 100, the memory 1144 can also provide data storage services for other components in the neural network processor 100. Those skilled in the art can make design adjustments according to actual usage needs, and this disclosure does not impose any restrictions on this.

[0050] Figure 3 An example is shown Figure 1 and Figure 2 The circuit architecture diagram of the embodiment shown.

[0051] As shown in the figure, this circuit architecture mainly includes the neural network accelerator subsystem (NPU_SS), X2P bus, configuration register (Sysreg), phase-locked loop (PLL), power management integrated circuit controller (PMIC Ctrl), and power management integrated circuit (PMIC).

[0052] The Power Management Integrated Circuit (PMIC) is primarily responsible for the overall system power supply (VDD), control (CTRL), and configuration (CFG). The Phase-Locked Loop (PLL) generates a clock signal (clk) to provide a synchronous "beat" for the system, ensuring timing coordination among modules. The configuration register (Sysreg) stores the PLL's parameter configuration to dynamically set its clock frequency. The PMIC controller (Ctrl) is controlled by the configuration register (Sysreg), generating corresponding control signals that are transmitted to the PMIC to power its control chip.

[0053] The Neural Network Accelerator Subsystem (NPU_SS) includes two power control modules: npuss_pctrl and npu_pctrl.

[0054] The AXI (HWCG) in npuss_pctrl can obtain power mode configuration parameters from Sysreg through the configuration bus and send them to the R2P unit via the X2P bus. The R2P unit then generates a corresponding power mode execution request and broadcasts the power mode execution request to the NPUSS_PCU unit and the NPU_PR_PCU unit of the npu_pctrl module through the NPUSS_LPE_P unit.

[0055] The NPUSS_PCU unit can perform corresponding power mode switching on each component in the NPU_SS_PWR_WRAP layer according to the power mode execution request, and perform corresponding power mode operations (e.g., power-on / off operations and / or voltage adjustment operations) on each component in the NPU_SS_PWR_WRAP layer through NPUSS_BPC, and then send the corresponding voltage adjustment signal to the PMIC Ctrl unit through the NPUSS_PCA unit.

[0056] Simultaneously, the NPUSS_PCU unit also sends a power mode execution request to the NPUSS_P2C unit, and after the NPUSS_P2C unit performs protocol conversion, it sends the request to the NPUSS_LPE_C unit. The NPUSS_LPE_C unit can send the corresponding clock adjustment signal to the NPU_AAB_CCU so that the NPU_AAB_CCU can perform clock frequency adjustment operation, and send the corresponding clock start / stop signal to the bus conversion bridge chip, and send the corresponding clock frequency adjustment result to the PLL.

[0057] The NPU_PR_PCU unit in the Npu_pctrl module can perform corresponding power mode switching for each computing core (CALC_CORE / CTRL_CORE) in the IP_NPU layer according to the power mode execution request, and perform corresponding power mode operations (e.g., voltage adjustment operations) for each computing core (CALC_CORE / CTRL_CORE) in the IP_NPU layer through NPU_PR_BPC. In this embodiment, since each computing core does not support a shutdown mode, it is not necessary to further send corresponding voltage adjustment signals to PMIC Ctrl based on the power mode switching execution results of each computing core, that is, it is not necessary to set up a signal transmission channel between NPU_PR_BPC and PMIC Ctrl.

[0058] In summary, the voltage frequency dynamic adjustment circuits of the various embodiments of this disclosure, through the coordinated operation of two sets of power consumption control modules corresponding to the control layer 112 and the operation layer 114, flexibly adjust the voltage level according to different power consumption control modes, thereby meeting the usage requirements of different application scenarios, effectively reducing system power consumption, improving energy utilization efficiency, and ensuring the performance stability of the processor.

[0059] Furthermore, the voltage frequency dynamic adjustment circuits in the various embodiments of this disclosure achieve clock frequency adjustment in a software-controllable manner, which not only increases flexibility and expands application scenarios, but also facilitates the optimization of a more efficient and reasonable calculation mapping method through hardware and software collaboration, thereby reducing chip power consumption.

[0060] Furthermore, by using two sets of power control modules corresponding to the control layer 112 and the computing layer 114, the memory in the computing layer can be reused by external components, which can reduce resource consumption costs, expand application scenarios, and realize hierarchical control between different components.

[0061] Another embodiment of this disclosure provides a neural network processor 100, which includes the voltage and frequency dynamic adjustment circuit 200 described in the above embodiments, for dynamically adjusting the voltage and frequency of the power supply of the neural network processor 100 according to different power consumption operation modes of the neural network processor 100.

[0062] Not all modules in the above system architecture diagrams are necessary; some modules can be omitted as needed. The system structures described in the above embodiments can be physical or logical structures. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0063] In the above embodiments, the hardware modules can be implemented mechanically or electrically. For example, a hardware module may include permanent, dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operations. The hardware module may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operations. The specific implementation method (mechanical, dedicated, permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0064] The above embodiments are only used to illustrate the embodiments of this disclosure, and are not intended to limit the embodiments of this disclosure. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this disclosure. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this disclosure, and the patent protection scope of the embodiments of this disclosure should be defined by the claims.

Claims

1. A voltage-frequency dynamic adjustment circuit, wherein, A neural network processor, applied to perform power supply voltage and frequency adjustment via a power manager and a frequency manager, includes an operational layer and a control layer, wherein the control layer controls the logical operations of the operational layer. The voltage and frequency dynamic adjustment circuit includes: The first power consumption control module, which is connected to the control layer, is configured as follows: Based on the power consumption mode configuration parameters, a power consumption mode execution request is generated. Based on the power consumption mode execution request, the corresponding power consumption mode operation is performed on the control layer, thereby outputting the voltage adjustment signal of the control layer to the power manager and outputting the frequency adjustment signal containing at least one clock frequency to the frequency manager. The second power consumption control module, which connects the first power consumption control module and the computing layer, is configured as follows: According to the power mode execution request, the corresponding power mode operation is performed on the computing layer.

2. The circuit according to claim 1, wherein, The control layer includes at least one clock control unit, and the first power consumption control module includes: The request unit is configured to generate corresponding power mode execution requests based on the power mode configuration parameters of the neural network processor. A first power consumption control generator, which is connected to the control layer and the request unit, is configured to execute a request according to the power consumption mode of the request unit, perform a corresponding power consumption mode operation on the control layer, and output a voltage adjustment signal of the control layer to the power manager; The frequency adjustment unit, which connects the control layer and the request unit, is configured to execute a request according to the power consumption mode of the request unit, control each clock control unit to perform a corresponding clock frequency adjustment operation, and send each clock control unit's frequency adjustment signal to the frequency manager.

3. The circuit according to claim 2, wherein, The first power consumption control module further includes: A signal switching unit is connected between the first power consumption control generator and the power manager; The first power consumption control generator is configured to perform power-on / off operations and / or voltage adjustment operations on each component in the control layer according to the power consumption mode execution request, and to send a voltage adjustment signal to the power manager through the signal transfer unit in response to the execution response results of each component.

4. The circuit according to claim 2, wherein, The first power consumption control module includes a power consumption channel expansion unit and a first power consumption control unit; The power consumption channel extension unit connects the request unit, the first power consumption control unit, and the second power consumption control module, and is configured to distribute the power consumption mode execution request of the request unit to the first power consumption control unit and the second power consumption control module. The first power consumption control unit is connected to the first power consumption control generator and is configured to send a power consumption mode switching request to the control layer according to the power consumption mode execution request, and send the power consumption mode execution request to the first power consumption control generator in response to the request enable response of the control layer.

5. The circuit according to claim 2, wherein, Each clock control unit is also configured to generate a clock start / stop signal based on the clock frequency adjustment operation.

6. The circuit according to claim 4, wherein, The first power consumption control module includes: A protocol conversion unit, connected between the first power consumption control unit and the frequency adjustment unit, is configured to convert the power consumption mode execution request into a data protocol format recognizable by the frequency adjustment unit.

7. The circuit according to claim 4, wherein, The computation layer includes at least one computation core; The second power consumption control module includes: The second power consumption control unit, which is connected to the power consumption channel extension unit, is configured to send a state switching request to each component in the computing layer according to the power consumption mode execution request, and output the power consumption mode execution request in response to the request enable response of each component. A second power consumption control generator, connected to the second power consumption control unit and the at least one computing core, is configured to execute a request based on the power consumption mode and perform voltage adjustment operations on each computing core.

8. The circuit according to claim 7, wherein, The computation layer includes: A memory that connects the configuration bus of each processing core and the neural network processor; The memory is configured to prioritize providing data storage services to each computing core, and to provide data storage services to the configuration bus of the neural network processor when the memory is not used by the computing cores.

9. The circuit according to claim 7, wherein, The power mode configuration parameters of the neural network processor are determined in the following manner: Based on the actual load of the neural network processor, determine the power consumption mode of the neural network processor, and based on the power consumption mode of the neural network processor, determine the corresponding power consumption mode configuration parameters. The power consumption modes of the neural network processor include normal operation mode, multiple low-power modes, and shutdown mode.

10. The circuit according to claim 1, wherein, The neural network processor includes a neural network accelerator subsystem.

11. A neural network processor, comprising a voltage-frequency dynamic adjustment circuit as described in any one of claims 1 to 9, for dynamically adjusting the voltage and frequency of the power supply of the neural network processor according to different power consumption operation modes of the neural network processor.

Images