Edge-computing-oriented low-power-consumption neural network coprocessor chip architecture

Abstract

The invention provides an edge-computing-oriented low-power-consumption neural network coprocessor chip architecture, which comprises a core computing layer, a data processing layer and a power consumption and security management layer, and is characterized in that the core computing layer comprises an event-driven computing engine, a storage and computing integrated array and a dynamic reconfigurable internet. Compared with the prior art, the system has the following beneficial effects that by adding the core calculation layer, the data processing layer and the power consumption and safety management layer, the core calculation layer realizes non-zero feature selective activation through the data change threshold comparison circuit, the invalid calculation power consumption is reduced, the data carrying energy consumption is eliminated, and the task scheduler is matched to realize the dynamic path configuration of the calculation unit; the data processing layer matches a data stream with a calculation rhythm, realizes 30%-70% compression ratio of weight data and dynamically switches a calculation mode through a configuration register, and the power consumption and safety management layer dynamically adjusts threshold voltage according to a load, realizes optimal balance of power consumption and performance and outputs 32 / 16 / 8-bit flexible precision data, and can realize low-power-consumption operation.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of integrated circuit design, and particularly relates to a low-power neural network coprocessor chip architecture for edge computing. BACKGROUND

[0002] Edge computing refers to an open platform that fuses network, computing, storage and application core capabilities on the network edge close to objects or data sources, and provides edge intelligent services in proximity to meet the key needs of industry digitization in terms of agile connection, real-time business, data optimization, application intelligence, security and privacy protection. In modern technology, the number of terminal users is growing in a rapid increasing mode, and edge computing greatly relieves the load of cloud terminal and other end processing devices, and improves the response to terminal users. With the popularization of artificial intelligence edge devices, the deployment of neural network models in terminal devices faces the dual challenges of power consumption and computing power. The von Neumann bottleneck of separation of storage and computing in the traditional processor architecture leads to a large amount of energy waste, and the existing coprocessor has defects in sparse data processing, dynamic reconfiguration capability and security mechanism, and it is difficult to meet the needs of low power consumption, high performance and data security at the same time.

[0003] In view of the above, we hope to propose a new structure to solve the above technical problems. SUMMARY

[0004] In view of the defects of the prior art, the application aims to provide a low-power neural network coprocessor chip architecture for edge computing, which solves the problems proposed in the background art.

[0005] The application is implemented by the following technical scheme: a low-power neural network coprocessor chip architecture for edge computing, comprising a core computing layer, a data processing layer and a power consumption and security management layer, characterized in that the core computing layer comprises an event-driven computing engine for dynamically activating a processing unit according to the sparsity of input data, a storage-computing integrated array for performing weight-feature map multiplication and accumulation operation, and a dynamically reconfigurable interconnection network. The data processing layer comprises an input buffer, a weight compression pipeline and a CNN / RNN / attention unit, and the power consumption and security management layer comprises a near-threshold power supply system, a security island (TEE) and an output coupler, and the near-threshold power supply system comprises a voltage adjustment module and a timing error detection circuit.

[0006] As a preferred embodiment, the event-driven engine includes a data change detector and the trigger threshold is software configurable, the memory-compute integrated array is composed of SRAM macro units embedded with multiply-add circuits, the multiply-adder is embedded in the SRAM bit line, and the "read-compute-store" is completed in a single cycle, the dynamic reconfigurable interconnection network is based on a Crossbar switching structure + scheduler and supports dynamic configuration of data paths for CNN / RNN / Transformer computing units according to task requirements, and the dynamic path configuration of the computing unit is realized through the dynamic reconfigurable interconnection network cooperating with the task scheduler, and flexible mapping of different network architectures such as CNN / RNN is supported.

[0007] As a preferred embodiment, the memory-compute integrated array adopts a 64x64 or larger scale SRAM integrated unit with a multiply-adder, and the peak computing power density is ≥600TOPS / mm 2 The event-driven engine includes a data change threshold comparison circuit, and only generates an activation signal for a feature value with a change amplitude exceeding a preset threshold.

[0008] As a preferred embodiment, the event-driven engine realizes selective activation of non-zero features through a data change threshold comparison circuit to reduce invalid computing power consumption, and the memory-compute integrated array integrates 64x64 SRAM storage units and 1T1M multiply-adder arrays to support in-place computation of weights and feature maps and eliminate data transfer energy consumption. The dynamic reconfigurable interconnection network is based on a 128x128 Crossbar switching structure, cooperates with a task scheduler to realize dynamic path configuration of computing units, supports flexible mapping of different network architectures such as CNN / RNN / Transformer, and the core of the dynamic reconfigurable interconnection network is to realize dynamic connection of computing units through a crossbar (Crossbar) and provide a hardware descriptor language (HDL) template to facilitate users to map custom operators to physical resources, use the SRAM macro unit and multiply-add circuit integrated structure of the memory-compute integrated array to perform weight-feature map multiplication and accumulation operations, and realize efficient computation of memory-compute integration.

[0009] As a preferred embodiment, the event-driven engine inputs data change signals and outputs activation instructions, the memory-compute integrated array inputs activation instructions / weights and outputs intermediate computing structures, and the dynamic reconfigurable interconnection network inputs computing task instructions and outputs path configuration signals.

[0010] As a preferred embodiment, the input buffer is a FIFO queue with a depth of 2048, equipped with a data alignment circuit to match the data stream and the computing rhythm, and the weight compression pipeline includes three processing units (sparsification detection → Huffman encoding → run-length encoding) to realize a compression ratio of 30%-70% for weight data. The CNN / RNN / attention unit integrates 32 sets of CNN convolution kernels (3x3 / 5x5), RNN neuron units, and attention mechanism modules, dynamically switches the calculation mode through a configuration register, inputs a buffer with a data alignment circuit to match the data flow and the calculation rhythm, and realizes a weight data compression ratio of 30%-70% through a weight compression pipeline, and the CNN / RNN / attention unit dynamically switches the calculation mode through a configuration register.

[0011] As a preferred embodiment, the input buffer (FIFO) inputs raw data and outputs buffered data, the weight compression pipeline inputs original weights and outputs compressed weights, and the CNN / RNN / attention unit inputs intermediate feature maps and outputs high-level features.

[0012] As a preferred embodiment, the near-threshold power supply system adopts a 0.5V near-threshold voltage power supply, cooperates with an adaptive bulk bias (ABB) circuit, dynamically adjusts the threshold voltage according to the load, realizes the optimal balance of power consumption-performance, and further includes a bulk bias feedback loop that can compensate for process, temperature, and voltage (PVT) fluctuations in real time, ensures the normal operation of the chip at low voltage, and improves the yield of mass production. The security island module integrates a national SM4 encryption algorithm hardware accelerator that supports real-time encryption of weight parameters and intermediate results with an encryption throughput of 2Gbps. The output aggregator includes 8 parallel result splicing units that support pooling, fully connected, and other post-processing operations, and outputs 32 / 16 / 8-bit flexible precision data.

[0013] As a preferred embodiment, the near-threshold power supply system inputs a power consumption control signal and outputs a power supply voltage, the security island (TEE) inputs data to be encrypted and outputs encrypted data, and the output coupler inputs intermediate calculation structures and outputs final inference results.

[0014] As a preferred embodiment, the event-driven engine realizes sparse activation calculation to reduce invalid operations, the storage-computing integrated array realizes in-place calculation to reduce storage access energy consumption, the dynamic reconfigurable interconnection network realizes flexible mapping of different neural network architectures, the near-threshold power supply and adaptive bulk bias realize power consumption optimization, and the security island module guarantees data security.

[0015] By adopting the technical scheme, the application has the beneficial effects that: by increasing the core computing layer, the data processing layer and the power consumption and security management layer, the event-driven engine realizes selective activation of non-zero characteristics through the data change threshold comparison circuit, reduces invalid computing power consumption, then eliminates data carrying energy consumption through the memory-computing integrated array, realizes dynamic path configuration of the computing unit through the dynamic reconfigurable interconnection network cooperating with the task scheduler, supports flexible mapping of different network architectures such as CNN / RNN, the input buffer is equipped with a data alignment circuit to realize matching of data flow and computing rhythm, the weight compression pipeline realizes a compression ratio of 30%-70% of weight data, and the CNN / RNN / attention unit dynamically switches the computing mode through the configuration register, the near-threshold voltage (NTV) adjustment circuit dynamically adjusts the threshold voltage according to the load, realizes optimal balance of power consumption-performance, the security island module (TEE) encrypts the throughput and outputs 32 / 16 / 8-bit flexible precision data through the output coupler full connection and other post-processing operations, so that low-power operation can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 It is an overall structure schematic diagram of the low-power neural network co-processor chip architecture for edge computing.

[0018] Figure 2 It is a schematic diagram of the memory-computing integrated array circuit in the low-power neural network co-processor chip architecture for edge computing.

[0019] Figure 3 It is a schematic diagram of the event-driven engine timing logic in the low-power neural network co-processor chip architecture for edge computing.

[0020] Figure 4 It is a schematic diagram of the near-threshold voltage (NTV) adjustment circuit in the low-power neural network co-processor chip architecture for edge computing.

[0021] Figure 5 It is a schematic diagram of the dynamic reconfigurable interconnection network in the low-power neural network co-processor chip architecture for edge computing.

[0022] Figure 6 It is a schematic diagram of the dynamic reconfigurable architecture in the low-power neural network co-processor chip architecture for edge computing. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0024] Referring to Figures 1 to 6 , the present application provides a technical solution: a low-power neural network coprocessor chip architecture for edge computing, comprising: a core computing layer, a data processing layer, and a power consumption and security management layer, the core computing layer comprising an event-driven computing engine for dynamically activating a processing unit according to input data sparsity, a memory-computing integrated array for performing weight-feature map multiplication accumulation operation, and a dynamically reconfigurable interconnection network; The data processing layer comprises an input buffer (FIFO), a weight compression pipeline, and a CNN / RNN / attention unit. The power consumption and security management layer comprises a near-threshold power supply system, a security island (TEE), and an output coupler. The near-threshold power supply system comprises a voltage regulation module and a timing error detection circuit.

[0025] Referring to Figures 1-3 and Figures 5-6 , embodiment 1: The event-driven engine includes a data change detector and the trigger threshold is software configurable. The memory-computing integrated array is composed of SRAM macro units embedded with multiply-add circuits. The multiplier is embedded in the SRAM bit line, and the "read-compute-store" is completed in a single cycle. The dynamically reconfigurable interconnection network is based on the Crossbar switching structure + scheduler and supports the dynamic configuration of data paths of CNN / RNN / Transformer computing units according to task requirements. The dynamically reconfigurable interconnection network cooperates with the task scheduler to realize dynamic path configuration of the computing unit, and supports flexible mapping of different network architectures such as CNN / RNN.

[0026] The memory-computing integrated array adopts a 64x64 or larger scale SRAM and multiply-adder integrated unit, and the peak computing power density is ≥600TOPS / mm 2 The event-driven engine includes a data change threshold comparison circuit, and only generates an activation signal for feature values whose change amplitude exceeds a preset threshold.

[0027] The event-driven engine realizes selective activation of non-zero features through the data change threshold comparison circuit, thereby reducing invalid computing power consumption. The memory-computing integrated array integrates 64x64 SRAM storage units and 1T1M multiply-adder arrays, supports in-place computation of weights and feature maps, and eliminates data transfer energy consumption. The dynamic reconfigurable interconnection network is based on a 128x128 Crossbar switching structure, and cooperates with a task scheduler to realize dynamic path configuration of a computing unit, supports flexible mapping of different network architectures such as CNN / RNN / Transformer, and the core of the dynamic reconfigurable interconnection network is to realize dynamic connection of the computing unit through a crossbar, and to provide a hardware descriptor language (HDL) template to facilitate users to map a custom operator to physical resources, to use an SRAM macro unit and a multiply-add circuit integrated structure of a compute-in-memory array to perform weight-feature map multiplication and accumulation operation, and to realize efficient computation of compute-in-memory.

[0028] The event-driven engine inputs a data change signal and outputs an activation instruction, the compute-in-memory array inputs the activation instruction / weight and outputs an intermediate calculation structure, and the dynamic reconfigurable interconnection network inputs a calculation task instruction and outputs a path configuration signal.

[0029] Specifically, in the edge computing use, the processor chip first receives data through the core computing layer, the event-driven engine realizes selective activation of non-zero features through a data change threshold comparison circuit (monitors input data sparsity through the event-driven computing engine, dynamically activates processing units, and realizes dynamic calculation activation based on data sparsity), reduces invalid calculation power consumption, and the input data flows through a differential comparator to detect the change amount of consecutive frames, if the change amount is greater than the threshold, the pulse generator outputs an activation pulse, the activation pulse controls clock gating, and only the data change area is powered, then the SRAM macro unit and the multiply-add circuit integrated structure of the compute-in-memory array are used to perform weight-feature map multiplication and accumulation operation, to realize efficient computation of compute-in-memory, to eliminate data transfer energy consumption through the compute-in-memory array, to realize dynamic path configuration of the computing unit through the dynamic reconfigurable interconnection network cooperating with the task scheduler, and to support flexible mapping of different network architectures such as CNN / RNN, to identify the input data in the core layer, to realize selective activation of different processing units through different thresholds, to calculate different actual processing data values, and to realize the effect of low-power operation.

[0030] Please refer to Figure 1 , Figure 4 and Figure 6 , embodiment 2: The input buffer depth is a FIFO queue of 2048, equipped with a data alignment circuit to realize matching of data flow and calculation rhythm, the weight compression pipeline includes three processing units (sparsification detection→Huffman coding→run-length coding) to realize a compression ratio of 30%-70% of weight data; The CNN / RNN / attention unit integrates 32 groups of CNN convolution kernels (3×3 / 5×5), RNN neuron units and attention mechanism modules. The calculation mode is dynamically switched through the configuration register. The input cache is equipped with a data alignment circuit to match the data flow with the calculation rhythm. The weight compression pipeline achieves a compression ratio of 30%-70% for the weight data, and the calculation mode is dynamically switched through the configuration register of the CNN / RNN / attention unit.

[0031] The input buffer (FIFO) inputs the original data and outputs the buffered data, the weight compression pipeline inputs the original weights and outputs the compressed weights, and the CNN / RNN / attention unit inputs the intermediate feature map and outputs the high-level features.

[0032] The near-threshold power supply system uses a 0.5V near-threshold voltage and, in conjunction with an adaptive body bias (ABB) circuit, dynamically adjusts the threshold voltage based on the load to achieve an optimal balance between power consumption and performance. The near-threshold power supply system also includes a body bias feedback loop that compensates for process, temperature, and voltage (PVT) fluctuations in real time, ensuring normal chip operation at low voltages and improving mass production yield. The security island module integrates the national secret SM4 encryption algorithm hardware accelerator, supporting real-time encryption of weight parameters and intermediate results with an encryption throughput of 2Gbps. The output aggregator contains an 8-way parallel result splicing unit, supports post-processing operations such as pooling and full connection, and outputs 32 / 16 / 8-bit flexible precision data.

[0033] The near-threshold power supply system inputs the power consumption control signal and outputs the power supply voltage, the security island (TEE) inputs the data to be encrypted and outputs the encrypted data, and the output coupler inputs the intermediate computing structure and outputs the final inference result.

[0034] The event-driven engine implements sparse activation computing and reduces invalid operations. The integrated storage and computing array realizes in-situ data computing and reduces storage access energy consumption. The dynamically reconfigurable interconnection network realizes flexible mapping of different neural network architectures. The combination of near-threshold power supply and adaptive body bias realizes power consumption optimization. At the same time, data security is guaranteed through the safety island module.

[0035] Based on the first embodiment described above, the core layer processes input data and inputs the processed signal to the data processing layer. The input buffer is equipped with a data alignment circuit to achieve matching of data flow and computing rhythm. The weight compression pipeline achieves a compression ratio of 30%-70% for weight data through three-stage processing units (sparse detection → Huffman coding → run-length coding). The CNN / RNN / attention unit dynamically switches the computing mode through the configuration register, and the three modules of the data processing layer operate together. Secondly, the near-threshold power supply system adjusts the threshold voltage according to the load, realizes the optimal balance of power consumption-performance, realizes the lowest threshold voltage mode under the corresponding load performance operation, and outputs 32 / 16 / 8-bit flexible precision data after the encryption throughput of the secure island module (TEE) and the full connection of the output coupler and other post-processing operations, and the near-threshold power supply system adopts 0.5V near-threshold voltage power supply, cooperates with the adaptive bulk bias (ABB) circuit, adjusts the threshold voltage according to the load, realizes the optimal balance of power consumption-performance, and dynamically adjusts the bulk bias voltage according to the timing error in the work, compensates the performance fluctuation under NTV, so that low-power operation can be realized.

[0036] The above merely describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An edge-computing oriented low-power neural network co-processor chip architecture, comprising: The core computing layer, the data processing layer and the power consumption and security management layer are characterized in that the core computing layer comprises an event-driven computing engine for dynamically activating a processing unit according to input data sparsity, an in-memory computing array for performing weight-feature map multiplication and accumulation operation and a dynamically reconfigurable interconnection network; The data processing layer comprises an input buffer (FIFO), a weight compression pipeline and a CNN / RNN / attention unit, and the power consumption and security management layer comprises a near-threshold power supply system, a security island (TEE) and an output coupler, and the near-threshold power supply system comprises a voltage adjustment module and a timing error detection circuit.

2. The edge computing oriented low-power neural network co-processor chip architecture of claim 1, wherein: The event-driven engine includes a data change detector and a trigger threshold that is software configurable, the in-memory computing array is composed of SRAM macro units embedded with multiply-accumulate circuits, the multiplier is embedded in the SRAM bit line, and the "read-compute-store" is completed in a single cycle, and the dynamically reconfigurable interconnection network is based on a Crossbar switching structure + scheduler and supports dynamic configuration of data paths of CNN / RNN / Transformer computing units according to task requirements.

3. The edge computing oriented low-power neural network co-processor chip architecture of claim 1, wherein: The storage-computing integrated array adopts a 64*64 or more scale SRAM and integrated unit of multiplier-accumulator, and the peak computing power density is greater than or equal to 600 TOPS / mm 2 The event-driven engine includes a data change threshold comparison circuit, and only generates an activation signal for a characteristic value with a change amplitude exceeding a preset threshold.

4. The edge computing oriented low-power neural network co-processor chip architecture of claim 1, wherein: The event-driven engine realizes selective activation of non-zero features through a data change threshold comparison circuit, thereby reducing invalid computation power consumption, the in-memory computing array integrates 64x64 SRAM storage units and 1T1M multiplier arrays, supports in-place computation of weights and feature maps and eliminates data transfer energy consumption; The dynamically reconfigurable interconnection network is based on a 128x128 Crossbar switching structure, cooperates with a task scheduler to realize dynamic path configuration of computing units, supports flexible mapping of different network architectures such as CNN / RNN / Transformer, and the dynamically reconfigurable interconnection network core realizes dynamic connection of computing units through a crossbar (Crossbar) and provides a hardware descriptor language (HDL) template to facilitate users to map custom operators to physical resources.

5. The edge computing oriented low-power neural network co-processor chip architecture of claim 4, wherein: The event-driven engine inputs data change signals and outputs activation instructions, the in-memory computing array inputs activation instructions / weights and outputs intermediate computing structures, and the dynamically reconfigurable interconnection network inputs computing task instructions and outputs path configuration signals.

6. The edge computing oriented low-power neural network co-processor chip architecture of claim 1, wherein: The input buffer (FIFO) is a FIFO queue with a depth of 2048, equipped with a data alignment circuit to match the data flow and the computing rhythm, and the weight compression pipeline comprises three processing units (sparsity detection → Huffman coding → run-length coding) to realize a compression ratio of 30%-70% for weight data. The CNN / RNN / attention unit integrates 32 groups of CNN convolution kernels (3x3 / 5x5), RNN neuron units and attention mechanism modules, and dynamically switches the computing mode through configuration registers.

7. The edge computing oriented low-power neural network co-processor chip architecture of claim 6, wherein: The input buffer (FIFO) inputs raw data and outputs buffered data, the weight compression pipeline inputs raw weights and outputs compressed weights, and the CNN / RNN / attention unit inputs intermediate feature maps and outputs high-level features.

8. The edge computing oriented low-power neural network co-processor chip architecture of claim 1, wherein: The near-threshold power supply system adopts 0.5V near-threshold voltage power supply, cooperates with an adaptive bulk bias (ABB) circuit, dynamically adjusts the threshold voltage according to the load, realizes the optimal balance of power consumption-performance, further comprises a bulk bias feedback loop, can compensate the fluctuation of process, temperature and voltage (PVT) in real time, guarantees the normal work of the chip under low voltage, and improves the yield of mass production, the secure island (TEE) module integrates a national secret SM4 encryption algorithm hardware accelerator, supports real-time encryption of weight parameters and intermediate results, and the encryption throughput reaches 2Gbps; The output aggregator comprises 8 parallel result splicing units, supports post-processing operations such as pooling and full connection, and outputs 32 / 16 / 8-bit flexible precision data.

9. The edge computing oriented low-power neural network co-processor chip architecture of claim 8, wherein: The near-threshold power supply system inputs a power consumption control signal and outputs a power supply voltage, the secure island (TEE) inputs data to be encrypted and outputs encrypted data, and the output coupler inputs an intermediate calculation structure and outputs a final inference result.

10. The edge computing oriented low-power neural network co-processor chip architecture of claim 1, wherein: The event-driven engine realizes sparse activation calculation and reduces invalid operation, the memory-compute integrated array realizes in-place calculation of data and reduces storage access energy consumption, the dynamic reconfigurable interconnection network realizes flexible mapping of different neural network architectures, the near-threshold power supply is combined with adaptive bulk bias to realize power consumption optimization, and data security is guaranteed through the secure island module.

Images