Storage and calculation hardware accelerator, electronic equipment and data processing method

By designing a computing hardware accelerator in the artificial intelligence hardware acceleration chip, using register instruction processing unit and unified cache unit, the problem of traditional hardware accelerators in the processing of deep learning models is solved, and more efficient computing is achieved and lower power consumption.

CN119003139AInactive Publication Date: 2024-11-22NANJING INST OF INTELLIGENT TECH INST OF MICROELECTRONICS OF THE CHINESE ACAD OF

Patent Information

Application Number
CN202411489571.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-24
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing artificial intelligence hardware acceleration chips are restricted by the 'storage wall' and 'power wall' when processing deep learning models. The traditional system instruction set and data interaction methods are not suitable for the integrated computing model of storage and computing, resulting in low processing efficiency and power consumption. high.

Method used

A memory hardware accelerator is designed, including a register instruction processing unit, an operation execution unit and a unified cache unit. The register instruction processing unit receives external control instructions and generates operation instructions. The operation execution unit performs operations and caches results, and manages the unified cache unit. The loop or output of the calculation results reduces the interaction between the hardware accelerator and external data.

Benefits of technology

This has achieved improved computing efficiency and reduced power consumption. Through in-memory computing and unified cache design, data interaction complexity and power consumption in multiple rounds of computing are reduced, and the compatibility and response speed of neural network algorithms are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003139A_ABST
    Figure CN119003139A_ABST
Patent Text Reader

Abstract

The invention discloses a storage and calculation hardware accelerator, electronic equipment and a data processing method in the field of accelerators. The storage and calculation hardware accelerator comprises a register instruction processing unit; the register instruction processing unit is used for decoding an external master control input control instruction and issuing the decoded instruction; the operation execution unit is in one-way connection with the register instruction processing unit and is used for receiving the decoded instruction and performing operation on operation data; the unified cache unit is connected with the operation execution unit and is used for judging whether the data output by the operation execution unit is subjected to loop operation or interrupt operation or not; according to the core computing array, the in-memory computing technology is adopted, the unified cache unit and the register instruction processing unit are matched for instruction scheduling, the weight of the operation execution unit can be only loaded once from the outside in one round of computing, the interaction frequency of a hardware accelerator and external data is reduced, and the computing efficiency is improved. The problem of'storage wall 'commonly existing in a traditional hardware accelerator is solved, power consumption can be greatly reduced, and operation efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of accelerator technology, and in particular to a storage and computing hardware accelerator, an electronic device, and a data processing method. Background Art

[0002] With the resurgence of AI applications based on deep learning technology, dedicated hardware acceleration chips have been widely used. At present, the development of AI hardware acceleration chips is constrained by the "storage wall" and "power consumption wall". These two constraints are due to the architecture design of the hardware accelerator and the manufacturing process during the production process. For example, each layer of neural network operations in the deep learning model faces a large number of storage calls. Each read of the weight value is equivalent to a multiplication and accumulation operation. More than 90% of the computing resources are consumed in the process of data transfer, and the processing efficiency is low. The storage and computing integrated chip based on the non-von Neumann architecture has shown unique advantages in the field of processing convolutional neural network algorithms, especially suitable for such data access-intensive application scenarios. Its introduction has provided effective support for the industrialization development of AI chips.

[0003] New hardware accelerators based on integrated storage and computing technology provide ideas for solving the above problems. However, due to its new in-memory computing mode, the data writing method and data flow scheduling method of the integrated storage and computing unit are completely different from the traditional architecture. The traditional system instruction set architecture and data interaction method are not suitable for the in-memory computing mode, and cannot give full play to the advantages of the "memory" of the in-memory computing unit. Therefore, it is necessary to design a new hardware architecture and operation instruction set. Summary of the invention

[0004] The purpose of this application is to provide a storage and computing hardware accelerator, an electronic device, and a data processing method. Compared with traditional hardware acceleration units, the interaction between the hardware accelerator and external data is reduced, the power consumption is reduced, and the computing efficiency is improved.

[0005] In order to solve the above technical problems, this application is implemented by adopting the following technical solutions: In a first aspect, the present application discloses a storage computing hardware accelerator, which includes: A register instruction processing unit, used for receiving control instructions input by an external master control, decoding the control instructions to generate operation instructions; An operation execution unit, used for receiving the operation instruction and operation data, and performing operation on the operation data according to the operation instruction; The unified cache unit is used to cache the operation results of the operation execution unit, and input the operation results as operation data for the next round of operation to the operation execution unit for iterative operation, or output the operation results as the final operation result.

[0006] In a further solution of the present application, the register instruction processing unit includes: An instruction interception module, which is connected to an external master control via a configuration bus and is used to intercept and store control instructions input by the external master control; An instruction decoding module, used for obtaining control instructions input by an external master control from the instruction interception module and decoding and generating operation instructions; The instruction distribution module is used to obtain the operation instructions generated by decoding from the instruction decoding module and distribute them to the operation execution unit.

[0007] In a further solution of the present application, the operation execution unit includes: A cache module, used to receive external computing data and organize and temporarily store the computing data; A multiplication-accumulation module, used for performing multiplication-accumulation on the sorted and temporarily stored operation data, and performing addition operation on the multiplication-accumulation calculation results to obtain first data; A data path network module, used for performing convolution processing on the first data to obtain second data; A data activation module, used for activating the second data to obtain third data; A pooling operation module, used for performing pooling processing on the second data to obtain fourth data; The data path network module is also configured to perform full connection processing on the third data and the fourth data to obtain a calculation result.

[0008] In a further embodiment, the cache module includes: A weight data cache module, used to connect to external operation data and sort and temporarily store weight data in the operation data to obtain sixth data; The characteristic data cache module is used to connect to external operation data and sort and temporarily store the characteristic data in the operation data to obtain seventh data.

[0009] In a further embodiment, the multiplication-accumulation module comprises: an in-memory calculation module, used for performing multiplication and accumulation calculation processing on the sixth data and the seventh data to obtain the eighth data; The addition tree module is used to perform addition operation on the eighth data to obtain the first data.

[0010] In a further solution of the present application, the unified cache unit includes: A unified cache module, used for caching the operation results of the operation execution unit; The arbitration module is used to arbitrate and judge the calculation results after cache processing: if the calculation result can be used as the input data for the next round of calculation, the cached calculation result is directly input into the feature data cache module for the next round of calculation; if the calculation result is the final calculation result, the calculation result is output.

[0011] In a second aspect, an electronic device includes: a device body and the above-mentioned storage and computing hardware accelerator, wherein the storage and computing hardware accelerator is installed in the device body.

[0012] In a third aspect, a data processing method based on the above storage and computing hardware accelerator includes: In response to receiving a control instruction sent by an external master control, decoding the control instruction to generate an operation instruction; Performing operation on the received operation data according to the operation instruction to obtain the operation result; The operation result is cached, and the operation result is used as operation data for the next round of operation to perform iterative operation, or the operation result is output as the final operation result.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The core computing array of this application adopts in-memory computing technology, and cooperates with the unified cache unit and register instruction processing unit instruction scheduling, so that the weight of the operation execution unit is loaded from the outside only once in one round of calculation, which reduces the interaction frequency between the hardware accelerator and external data, improves the "storage wall" problem that is common in traditional hardware accelerators, and can greatly reduce power consumption and improve computing efficiency.

[0014] In addition, the data path network module is connected with the multiplication and accumulation module, the data activation module, the pooling operation module and the unified cache module to specify the flow direction of data in the four modules in one round of calculation. For example, in the application, the convolution, activation and pooling operation modules are integrated inside the accelerator, and the convolution operation module can be reused as a fully connected operation module. The data flow scheduling of the internal data path network module configured by the register can realize any combination of multiple modules, thereby improving the compatibility of the neural network algorithm.

[0015] In addition, the design of the internal unified cache unit allows the intermediate results of multiple rounds of calculations to be temporarily stored inside the accelerator without having to be output, reducing the complexity of data transfer between multiple rounds of operations, improving response speed and reducing power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the overall architecture of the storage and computing hardware accelerator provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present application and its application or use.

[0018] Embodiment 1

[0019] Observation attachment Figure 1 In this embodiment, a storage and calculation hardware accelerator is disclosed, including a register instruction processing unit, an operation execution unit and a unified cache unit; the register instruction processing unit is used to receive control instructions input by an external master control, decode the control instructions to generate operation instructions, and issue the operation instructions; the register instruction processing unit is unidirectionally connected to the operation execution unit, and the connection with the operation execution unit is used to receive operation data externally and to receive the operation instructions issued by the register instruction processing unit internally and to operate on the operation data; the unified cache unit is interconnected with the operation execution unit to determine whether the data output by the operation execution unit is subjected to cyclic operation or interrupt operation.

[0020] In this embodiment, the register instruction processing unit includes an instruction interception module, an instruction decoding module and an instruction distribution module; The instruction interception module is configured with a bus to connect to an external master control, and is configured to intercept and store control instructions input by the external master control. The instruction decoding module is configured to obtain control instructions input by the external master control from the instruction interception module and decode them to generate operation instructions; the instruction decoding module inputs the control instructions to the instruction distribution module, which distributes them to the operation execution unit.

[0021] In practice, the hardware accelerator system is in standby state after being powered on, waiting for the external master control to send control instructions. The control command sent by the master control is sent to the register instruction processing unit of the hardware accelerator through the configuration bus. The instruction interception module in the register instruction processing unit is responsible for removing the redundant header and tail information in the instructions transmitted by the configuration bus and preliminarily registering the valid instructions; the instruction decoding module is responsible for decoding the intercepted instructions according to the instruction rules, clarifying the functional content and target functional module corresponding to each instruction; the instruction distribution module is responsible for distributing the identified instructions to each functional module. When the instructions are sent to each functional module, a round of configuration is completed.

[0022] In this embodiment, the operation execution unit includes a weight data cache module, a feature data cache module, an in-memory calculation module, an addition tree module, a data path network module, a data activation module, and a pooling operation module; The weight data cache module connects the operation data through the first data bus and organizes and temporarily stores the weight data in the operation data to obtain the sixth data, and the feature data cache module connects the operation data through the second data bus and organizes and temporarily stores the feature data in the operation data to obtain the seventh data.

[0023] The in-memory calculation module receives the sixth data and the seventh data to perform multiplication and accumulation calculation processing to obtain the eighth data; and uses the addition tree module to perform addition operation processing on the eighth data to obtain the first data.

[0024] The data path network module performs convolution processing on the first data to obtain second data; the data activation module performs activation processing on the second data to obtain third data, and returns the third data to the data path network module; the pooling operation module performs pooling processing on the second data to obtain fourth data, and returns the fourth data to the data path network module; finally, the data path network module performs full connection processing on the third data and the fourth data to obtain an operation result, and inputs the operation result into the unified cache unit.

[0025] In this regard, the present application configures different functions for the in-memory computing modules, so that they can be used as convolutional layers or fully connected layers in the neural network algorithm to achieve reuse of computing resources; by configuring whether each module is enabled or not, functional modules that are not needed in a single round of neural network operations can be put to sleep to save power consumption; by configuring the size of each module, power consumption in calculations can also be reduced; by configuring the data transmission path, flexible order matching of the four operators of convolution, activation, pooling, and full connection in the neural network algorithm can be achieved, thereby achieving support for a wider range of neural network algorithms.

[0026] In this embodiment, the unified cache unit includes a unified cache module and an arbitration module; the unified cache module caches the calculation results, and the arbitration module arbitrates and judges the cached calculation results. The data that meets the requirements directly interrupts the calculation and outputs the results through the third data bus. When the requirements are not met, the results are input to the calculation execution unit for cyclic calculation until the results meet the requirements. That is, the output results of this round of calculation can be used as the input data of the next round of calculation, and the data transmission is completed directly in the accelerator, which reduces the data transfer in and out, reduces power consumption, and improves efficiency.

[0027] Workflow; First, a round of configuration is performed on the register instruction processing unit (described in detail above). The weight data and feature data involved in the calculation are transmitted to the weight data cache module and the feature data cache module via the corresponding data bus. The two cache modules can organize the input data timing and temporarily store the data. After the data is cached, it is sent to the in-memory calculation module for multiplication and accumulation calculation and the addition tree module for addition operation in turn, and then the data is transmitted to the data path network module.

[0028] The data path network module is connected to the multiplication and accumulation module (including the in-memory calculation module and the addition tree module), the data activation module, the pooling operation module and the unified cache module. After the function is configured through the configuration register, the flow direction of the data in the four modules in one round of calculation can be specified, thereby realizing the support of multiple operators in the neural network algorithm. The neural network algorithm generally includes convolutional layers, activation layers, pooling layers and fully connected layers. During the operation process, multiple operations are usually required to get the final result. For example, the classic LeNet-5 has a total of seven layers, from front to back, they are convolutional layer C1 - pooling layer S2 - convolutional layer C3 - pooling layer S4 - fully connected layer C5 - fully connected layer F6 - fully connected layer Output. The seven layers are connected in sequence, and the calculation output data of the previous layer is the input data of the next layer. When a similar algorithm runs on this hardware accelerator, for example, in the calculation of the first three layers: convolution layer C1 - pooling layer S2 - convolution layer C3, two convolution operations are used, and the input data of convolution layer C3 is the result of the convolution layer C1 and pooling layer S2. At this time, the result of S2 is cached in the unified cache module, and the arbitrator determines the output direction and outputs the cached data to the feature data cache module as the input data of the convolution layer C3; and when the operation reaches the seventh layer, that is, the fully connected layer Output, the result of the operation cached in the unified cache module should be output to the outside of the hardware accelerator as the final result. At this time, the arbitrator determines to output the data to the third data bus.

[0029] That is, after the data calculation is completed, it is temporarily stored in the unified cache module, and the arbitration module decides whether to output it in the end or use it as input to participate in the next round of calculation. When a round of calculation is completed, the interrupt signal is briefly pulled high to notify the upper-level master control that the calculation is completed and the next round of register configuration and data transmission can be carried out.

[0030] Embodiment 2

[0031] This embodiment discloses an electronic device, which includes a device body and the storage and computing hardware accelerator described in the first embodiment, and the storage and computing hardware accelerator is installed in the device body.

[0032] Embodiment 3

[0033] This embodiment discloses a data processing method implemented based on the storage and computing hardware accelerator in the above-mentioned embodiment 1, which includes the following steps: Waiting for the upper master control to send control instructions, decoding the control instructions input by the external master control through the register instruction processing unit, generating operation instructions and sending them down; The operation execution unit receives the operation instruction sent by the register instruction processing unit, and performs operation on the operation data according to the operation instruction; The operation result of the operation execution unit is received through the unified cache unit, and it is determined whether to perform a loop operation or an interrupt operation according to the operation result.

[0034] In actual applications, the in-memory convolution operation execution unit is the core, and the in-memory computing module, addition tree module, data activation module, pooling operation module, data path network module, unified cache module, arbitration module and feature data cache module in the accelerator can all be configured. By configuring the in-memory computing module with different functions, it can be used as a convolutional layer or fully connected layer in the neural network algorithm to achieve the reuse of computing resources; by configuring each module to be enabled or not, the functional modules that are not needed in a single round of neural network operation can be put into sleep mode to save power consumption. The accelerator in this application realizes efficient and high-freedom calling of each operation sub-module in the operation execution unit through a unique register instruction processing unit and data path network module design suitable for in-memory computing, thereby achieving the purpose of supporting multiple neural network operators in hardware with a hardware accelerator with an in-memory convolution operation execution unit as the core.

[0035] In the application, the accelerator integrates convolution, activation, and pooling operation modules, and the convolution operation module can be reused as a fully connected operation module. Through the data flow scheduling of the internal data path network module configured by registers, any combination of multiple modules can be realized, thereby improving the compatibility of the neural network algorithm.

[0036] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0037] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood by specific circumstances.

Claims

1. A storage and computing hardware accelerator, characterized in that: include: A register instruction processing unit, used for receiving control instructions input by an external master control, decoding the control instructions to generate operation instructions; An operation execution unit, used for receiving the operation instruction and operation data, and performing operation on the operation data according to the operation instruction; The unified cache unit is used to cache the operation results of the operation execution unit, and input the operation results as operation data for the next round of operation to the operation execution unit for iterative operation, or output the operation results as the final operation result.

2. The storage and computing hardware accelerator according to claim 1, characterized in that: The register instruction processing unit comprises: An instruction interception module, which is connected to an external master control via a configuration bus and is used to intercept and store control instructions input by the external master control; An instruction decoding module, used for obtaining control instructions input by an external master control from the instruction interception module and decoding and generating operation instructions; The instruction distribution module is used to obtain the operation instructions generated by decoding from the instruction decoding module and distribute them to the operation execution unit.

3. The storage and computing hardware accelerator according to claim 1, characterized in that: The operation execution unit comprises: A cache module, used to receive external computing data and organize and temporarily store the computing data; A multiplication-accumulation module, used for performing multiplication-accumulation on the sorted and temporarily stored operation data, and performing addition operation on the multiplication-accumulation calculation results to obtain first data; A data path network module, used for performing convolution processing on the first data to obtain second data; A data activation module, used for activating the second data to obtain third data; A pooling operation module, used for performing pooling processing on the second data to obtain fourth data; The data path network module is also used to perform full connection processing on the third data and the fourth data to obtain a calculation result.

4. The storage and computing hardware accelerator according to claim 3, characterized in that: The cache module comprises: A weight data cache module, used to connect to external operation data and sort and temporarily store weight data in the operation data to obtain sixth data; The characteristic data cache module is used to connect to external operation data and sort and temporarily store the characteristic data in the operation data to obtain seventh data.

5. The storage and computing hardware accelerator according to claim 4, characterized in that: The multiplication-accumulation module comprises: an in-memory calculation module, used for performing multiplication and accumulation calculation processing on the sixth data and the seventh data to obtain the eighth data; The addition tree module is used to perform addition operation on the eighth data to obtain the first data.

6. The storage and computing hardware accelerator according to claim 1, characterized in that: The unified cache unit comprises: A unified cache module, used for caching the operation results of the operation execution unit; The arbitration module is used to arbitrate and judge the calculation results after cache processing: if the calculation result can be used as the input data for the next round of calculation, the cached calculation result is directly input into the feature data cache module for the next round of calculation; if the calculation result is the final calculation result, the calculation result is output.

7. An electronic device, characterized in that: include: A device body and the storage-computing hardware accelerator as described in any one of claims 1 to 6, wherein the storage-computing hardware accelerator is installed in the device body.

8. A data processing method based on the storage and computing hardware accelerator according to any one of claims 1 to 6, characterized in that: include: In response to receiving a control instruction sent by an external master control, decoding the control instruction to generate an operation instruction; Performing operation on the received operation data according to the operation instruction to obtain the operation result; The operation result is cached, and the operation result is used as operation data for the next round of operation to perform iterative operation, or the operation result is output as the final operation result.

Citation Information

Patent Citations

  • Convolutional neural network hardware acceleration device, convolution calculation method, and storage medium

    CN108197705A

  • Multi-core storage processor architecture

    CN115456155A

  • Storage and calculation integrated chip and calculation method and device thereof

    CN116185498A

  • Neural network inference system and method

    CN117787364A

Cited By

  • In-memory computing device and method and electronic equipment

    CN119808846A