Sparse-aware scheduler, hybrid acceleration architecture, intelligent edge chips and devices

Abstract

The application discloses a sparse-aware scheduler, a hybrid acceleration architecture, an intelligent edge chip and equipment, wherein the sparse-aware scheduler comprises a sparse information extractor, which is used for associating a network layer in a neural network with sparse feature information of the network layer; a delay estimation unit, which is used for evaluating a delay of the network layer running on different accelerators according to the sparse feature information of each network layer; a load distributor, which is used for determining a configuration scheme of the accelerators according to a forward propagation delay in a first accelerator, a backward propagation delay in a second accelerator, a backward propagation delay in the first accelerator and a forward propagation delay in the second accelerator, so as to obtain a smaller delay; and a forward detection device, which is used for determining whether outputs of the first accelerator and the second accelerator are directly sent to input buffer areas of the other party according to an output result of the load distributor. The application has a lower delay time and a higher resource utilization rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuits, and in particular to a sparse perception scheduler, a hybrid acceleration architecture, an intelligent edge chip and a device. BACKGROUND

[0002] Edge computing is a distributed computing that migrates computing tasks and data storage from data centers or cloud environments to the edge of the network, that is, closer to the data source. In edge computing, data does not need to be directly uploaded to the cloud or centralized data processing system, not only reducing the delay of data transmission and improving the response speed of services, but also reducing the computing and storage burden of the cloud, improving the privacy and security of data, and reducing the risk of data leakage. In addition, edge computing can also save network bandwidth, as it reduces the transmission of large amounts of data in the network, especially suitable for applications that handle large amounts of data, such as Internet of Things (IoT) devices, autonomous driving, etc. More importantly, edge computing can support real-time or near-real-time data processing, which is of great value to applications that require fast decision-making, such as autonomous driving, industrial automation, etc.

[0003] Intelligent edge chips are microprocessors designed specifically for edge computing, with powerful computing capabilities packaged in a small hardware device to meet the needs of edge computing for low latency, high efficiency, and real-time processing. With the development of artificial intelligence and machine learning, some intelligent edge chips have begun to integrate specialized AI accelerators, such as neural network processors (NPUs), which can perform deep learning model inference on device side, greatly improving processing efficiency and response speed, while also protecting data privacy. As the core of edge devices, intelligent edge chips have stronger processing capabilities and can perform artificial intelligence calculations. Neural networks have deeply penetrated into various aspects of people's lives, and as people's understanding and application of neural networks continue to deepen, current neural network models have become increasingly complex, which undoubtedly increases the demand for hardware performance.

[0004] As neural network models increase in layers, nodes, and complexity of connections, the amount of data they need to process also increases, which requires hardware devices to have more powerful computing capabilities and larger memory capacities, thus the growing number of neural network model parameters and the limited hardware resources of edge devices are increasingly at odds. In addition, to train these complex neural network models, hardware devices also need to support efficient parallel computing and high-speed data transmission to ensure efficient running speed and low latency response time when processing large amounts of data and complex calculations. Therefore, there is a demand for intelligent edge chips with low latency and high resource utilization. SUMMARY

[0005] To solve the above technical problems, the application provides a sparse-aware scheduler, a hybrid acceleration architecture, an intelligent edge chip and an edge device, which have lower delay time and higher resource utilization.

[0006] The application provides a sparse-aware scheduler, which is suitable for an intelligent edge chip containing multiple accelerators and comprises a sparse information extractor, a delay estimation unit, a load distributor and a forward detection detector.

[0007] The sparse information extractor is used to associate a network layer in a neural network with sparse feature information of the network layer.

[0008] The delay estimation unit is used to estimate the delay of each network layer when the network layer runs on different accelerators according to the sparse feature information of the network layer.

[0009] The load distributor is used to determine a configuration scheme of the accelerators according to the forward propagation delay in the first accelerator, the backward propagation delay in the second accelerator, the backward propagation delay in the first accelerator and the forward propagation delay in the second accelerator, so as to obtain smaller delay.

[0010] The forward detection detector is used to determine whether the output of the first accelerator and the output of the second accelerator are directly sent to the input buffer area of the opposite side according to the output result of the load distributor.

[0011] The application further provides a hybrid acceleration architecture, which comprises multiple accelerators and the above sparse-aware scheduler.

[0012] The application further provides an intelligent edge chip, which comprises the above hybrid acceleration architecture.

[0013] The application further provides an edge device, which comprises the above intelligent edge chip.

[0014] In the embodiment of the application, the IPA unit and the ACC (accelerator) form a hybrid acceleration together to realize parallel computing and utilize the pipeline acceleration function of the CPU; meanwhile, the sparse-aware scheduler can estimate the delay based on the sparse level and arrange the FP and BP calculation to different acceleration units. In the scheduler of the application, the inter-layer delay is estimated according to the change of the sparse proportion between different layers, and the tasks are dynamically distributed to the two accelerators at the layer granularity, so that better performance and higher resource utilization are realized. BRIEF DESCRIPTION OF DRAWINGS

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram of a hybrid acceleration architecture provided in an embodiment of the present invention;

[0017] Figure 2 for Figure 1 A schematic diagram of the sparse sensing scheduler in the diagram;

[0018] Figure 3 for Figure 1 A diagram illustrating the unified memory mapping and read / write modes of a unified memory system;

[0019] Figure 4 A schematic diagram comparing the timeline of the traditional architecture, the sparse training timeline without scheduling, and the sparse training timeline with scheduling of the present invention is presented.

[0020] Figure 5 This diagram illustrates the comparison of time gaps and hardware utilization with and without scheduling. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] As mentioned before, deep neural networks (DNNs) are widely used in various fields, and with the continuous progress of the network, the amount of parameters and the amount of calculation have been increasing. Edge devices, such as Internet of Things (IoTs), are also equipped with the ability to run deep learning neural networks. These deep learning neural networks, which are specifically designed for edge applications, are called TinyML (Tiny Machine Learning). TinyML collects personal data collected by edge devices and runs neural networks locally on the device, avoiding time-consuming and power-consuming cloud data transmission, and enabling efficient and low-latency computing. In order to achieve the deployment of neural networks on the edge, it is necessary to solve the contradiction between the increasing amount of model parameters and the limited hardware resources of edge devices from the software and hardware levels. In addition to inference, the training of edge devices is becoming increasingly important due to its unique advantages. Compared with inference, the training of edge devices brings the possibility of personalization of neural networks. Local training can avoid the transmission of data to the cloud, protect personal privacy data, and customize the neural network deployed on the edge device according to user behavior. By collecting user data to fine-tune the neural network, the prediction accuracy of the personalized network can be improved by continuously fitting user habits. However, when training, the limited hardware resources of edge devices will be more strained. The training process includes forward propagation (FP) and backward propagation (BP). Backward propagation includes the calculation of weight gradients and intermediate activation layer gradients, so additional storage space is needed to store the results of the intermediate activation layer and the gradients, which accounts for 54% of the storage of backward propagation. In addition, there is a transpose operation on the matrix in the backward propagation, which will cause irregular access patterns to the storage medium.

[0023] In order to alleviate the demand for large storage, many techniques have been introduced into the training process, and sparse training is one of the solutions, which reduces the storage demand by skipping some bias and weight updates during backward propagation, while the accuracy loss is almost negligible. However, when the sparsity ratio changes, the number of operations required by the network exhibits a wide range of variation, resulting in unpredictable latency for a complete training iteration in practical applications, and a large difference in sparse ratio between forward and backward propagation can also lead to low utilization of hardware accelerators. Therefore, the present application proposes a sparse-aware scheduler in a hybrid acceleration architecture that can schedule neural network loads according to the real-time perceived sparsity ratio, not only reducing the latency time, but also improving the utilization of hardware resources.

[0024] In order to make the technical solutions of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A mixed acceleration architecture schematic diagram is provided for an embodiment of the present application, which is built based on an open-source lightweight TinyML platform, and it contains a 5-stage pipeline RISC-V CPU and peripheral IO. In order to better support sparse training of simultaneous forward and backward propagation, two acceleration units are designed in the architecture, as shown in Figure 1 The architecture in the embodiment includes two accelerators and a sparse-aware scheduler, and the two accelerators are used to support sparse training of simultaneous forward and backward propagation. The first accelerator is an in-pipeline accelerator (IPA) for performing vector multiply-accumulate, and the second accelerator is a dedicated multiply-accumulate array (ACC). The ACC receives control signals from the CPU through the bus interface of the CPU MEM (memory) stage, and the green dashed line in the figure represents the data flow on the system bus. The in-pipeline accelerator is configured in the memory access pipeline stage of the central processing unit (CPU) in the intelligent edge chip. The in-pipeline accelerator (in-pipeline acceleration) designs the accelerator in the pipeline of the CPU, which achieves considerable performance improvement with relatively small area overhead, which is crucial for resource-constrained TinyML devices. The in-pipeline accelerator implements a multiply-accumulate unit and can perform vector multiply-accumulate. The present application places the IPA in the memory access pipeline stage of the CPU, which can access the data cache and the ACC more quickly. The IPA will spend multiple cycles to complete the propagation of the specified layer according to the workload. Input, weight and output buffers are implemented using FIFO. For each calculation period, each FIFO will provide a set of data to the vector computation unit to complete the vector multiplication operation. The vector computation unit is composed of 4 multipliers and an adder tree.

[0026] In order to adapt to more sparse strategies and implement simultaneous forward and backward propagation, the present application designs a sparse-aware scheduler in the execution stage of the CPU, which dynamically schedules forward propagation and backward propagation according to real-time sparse information, and allocates the workload to the IPA and the ACC. The sparse-aware scheduler will balance the computation as much as possible to improve hardware utilization and bandwidth. If you want to switch the workload in the IPA and the ACC, the traditional method is to output the IPA / ACC to the respective output buffer, and then read each other. In order to avoid the overhead of such data transfer, the present application allows direct data transfer between the IPA and the ACC, similar to data forwarding. The sparse-aware scheduler will send a data forwarding signal, which means that the outputs of the two accelerators will be directly sent to each other. Figure 1The blue dashed line in the figure indicates the flow direction of the workload initialization and data pre-feeding signal, and the sparse updater in the write-back stage receives information submitted at the end of the running of the two accelerators, after which the sparse ratio information stored in the scheduler is updated. The sparse-aware scheduler designed in the present application is described in detail below.

[0027] Figure 2 For Figure 1 The structure diagram of the sparse-aware scheduler in the present application is shown in FIG. 1, which comprises a sparse information extractor, a delay estimation unit, a workload distributor and a pre-feed detector. Figure 2 The sparse-aware scheduler in the present application comprises a sparse information extractor, a delay estimation unit, a workload distributor and a pre-feed detector, wherein the sparse information extractor is configured to associate the network layer in the neural network with the sparse feature information of the network layer, wherein the sparse feature information comprises a sparse ratio, a sparse mode and the dimension of the weight matrix; the delay estimation unit is configured to evaluate the delay of each network layer running in different accelerators according to the sparse feature information of the network layer, to obtain the forward propagation delay in the first accelerator, the backward propagation delay in the second accelerator, the backward propagation delay in the first accelerator and the forward propagation delay in the second accelerator; the workload distributor is configured to determine the configuration scheme of the accelerator according to the forward propagation delay in the first accelerator, the backward propagation delay in the second accelerator, the backward propagation delay in the first accelerator and the forward propagation delay in the second accelerator, to obtain a smaller delay; and the pre-feed detector is configured to determine whether the output of the first accelerator and the output of the second accelerator are directly sent to the input buffer area of the opposite side according to the output result of the workload distributor.

[0028] In a specific embodiment, the workload distributor can be configured to compare the forward propagation delay in the first accelerator and the backward propagation delay in the second accelerator, to obtain a larger first delay; compare the backward propagation delay in the first accelerator and the forward propagation delay in the second accelerator, to obtain a larger second delay; and compare the first delay and the second delay, to select the configuration scheme of the accelerator corresponding to the smaller delay.

[0029] In order to balance the forward propagation (FP) and backward propagation (BP) calculations performed simultaneously, the present application designs a sparse-aware scheduler in the execution (EX) stage. Figure 2 The micro-architecture of this scheduler is shown in FIG. 2, which takes 6 cycles to complete. It consists of four parts: a sparse information extractor (sparse feature extractor), a delay estimation unit, a workload distributor and a pre-feed detector. The scheduler takes the layer ID (i.e. the layer number in the network) as input and generates data pre-feeding signals to the system bus and workload initiation signals to the MEM / WB registers as output. These signals will be sent to the IPA or ACC in the memory (MEM) stage to trigger their execution.

[0030] Sparse Feature Extractor (SFE): In SFE, a model table is used to associate network layer IDs with their Sparsity Ratio, Sparse Pattern, and Shape (i.e. dimension of weight matrix). The model table receives the layer IDs of forward propagation (FP) and backward propagation (BP) in SCHE instructions (a kind of computation instruction) and returns their sparsity to forward and backward information buffers, respectively. The extracted information will be used later to estimate the latency.

[0031] Latency Estimation Unit (LEU): The runtime latency of each sparse layer is evaluated by the following equation:

[0032] Estimated Layer Latency = a x b x Shape x Sparsity Ratio

[0033] Where a represents the Sparse Pattern (partial weights, full weights, bias only, etc.) coefficient, which can be programmed before model deployment. Another coefficient b is used to represent the different computation performance of IPA and ACC. LEU reads the forward and backward information buffers to get the Sparse Pattern, Shape, and Sparsity Ratio. Shape and Sparsity Ratio are loaded directly to the input buffer, while Sparse Pattern controls the specific selection. After all inputs are ready, the estimated latency is sent to a latency list. The accuracy of latency estimation is not required, as the scheduler only needs to estimate which propagation is faster.

[0034] Workload Scheduler (WD): A latency list stores four latency values in order: forward propagation latency in IPA (FP-IPA ), backward propagation latency in ACC (BP-ACC ), forward propagation latency in ACC (FP-ACC ), and backward propagation latency in IPA (BP-IPA ). WD reads this list and compares them to determine the final scheduling scheme (FSS) according to the following equation:

[0035] FSS = min(max(FP-IPA, BP-ACC ), max(FP-ACC, BP-IPA ))

[0036] The scheduler can dispatch FP to IPA and BP to ACC, or dispatch FP to ACC and BP to IPA. The min in the equation selects the scheduling combination with smaller latency. Then, the comparison result is sent to the subsequent FD unit.

[0037] Forwarding Detector (FD): To overlap data movement in IPA / ACC switch, the present application designs a FD unit. The final scheduling result is stored in a 1-bit register called next workload indicator (NWI), where 1'b0 means forward propagation (FP) in IPA and backward propagation (BP) in ACC, and 1'b1 means BP in IPA and FP in ACC. In addition to NWI, another 1-bit register, called current workload indicator (CWI), is introduced in this embodiment to record the state of the current acceleration unit. An XOR gate is applied to generate the data forwarding signal. If the forwarding signal is positive, IPA and ACC will feed their outputs to the input buffer of the other, in preparation for the next layer computation.

[0038] The embodiment of the present application forms a hybrid acceleration by IPA (In-Pipeline Acceleration) unit and ACC (Accelerator) to realize parallel computation and utilize the pipeline acceleration function of the CPU; at the same time, the sparse-aware scheduler can estimate the delay based on the sparsity level and arrange FP and BP computation to different acceleration units. In the scheduler of the present application, the inter-layer delay is estimated according to the sparsity ratio change between different layers, and the tasks are dynamically allocated to the two accelerators at layer granularity, so that better performance and higher resource utilization can be achieved.

[0039] To support low-cost transpose data fetching of neural network matrix, the hybrid acceleration architecture provided by the embodiment of the present application is further configured with a unified memory system for realizing regular weight reading in forward propagation and transpose weight reading in backward propagation. Figure 3 To Figure 1 The unified memory mapping and read-write mode of the unified memory system are combined with Figure 1 and Figure 3 In the embodiment of the present application, the unified memory system includes a weight memory bank (weight Bank), an intermediate activation layer memory bank (intermediate layer Bank), a gradient memory bank (gradient Bank) and a scalar memory bank (scalar Bank). The bank-associated memory structure can realize forward regular weight reading in forward propagation and transpose weight reading in backward propagation. In the unified memory system, four memory banks are divided: weight, intermediate activation layer, gradient and scalar memory. The highest two bits of the address are used as the index to distinguish different banks, so as to reduce the address encoding and decoding overhead. The address decoder sends the address to the correct bank through a multiplexer according to the highest two bits of the address, and the multiplexer outputs the correct data output according to the data address.

[0040] In the unified memory system, the application specially configures the storage bank for training to adapt to various matrix access modes, and the memory system unifies the regular read mode and the transposed read mode. A storage bank includes 16 memory arrays, each of which has a read-write port. Without loss of generality, using a matrix to map to n memory arrays is used for illustration, and each array stores n data. The mapping strategy is described as follows:

[0041] Mat[i][j]→Arr[(i+j)mod n][j],i,j∈[0,n-1]

[0042] For simplicity, the (j+1)th data of the (i+1)th array is referred to as Arr[i][j], and the (i+1)th row and (j+1)th column of the matrix are referred to as Mat[i][j]. In order to read the (k+1)th row or column of the matrix, each memory array reads one data according to the mapping strategy. Then the data array enters the rearrangement unit and performs k times of left circular shift. As for writing, the data array first enters the rearrangement unit and performs right circular shift, and then is written to the corresponding address.

[0043] Figure 3 An example of a matrix mapping to a memory array with multiple data acquisition modes in a unified memory system is shown. The blue arrow indicates the column read / write data stream, i.e., the transposed read / write, and the purple arrow indicates the row read / write data stream, i.e., the regular read / write. With such a circularly stored memory bank, the transposed data stream of matrix read / write is realized, which is stored in the weight bank, the intermediate activation layer bank and the gradient bank, respectively. During forward propagation (FP), only the regular data loading / storage mode is required. Since transposed loading and storage are required during back propagation (BP), both the weight bank and the intermediate activation layer bank adopt the circular storage format. By configuring the unified memory system in a hybrid acceleration architecture, the embodiment of the application realizes the low-cost data acquisition supporting the transposition of neural network matrices.

[0044] In the above embodiment, in order to effectively utilize the customized pipeline unit, the application also develops several customized instructions through RISC-V extension to control the newly developed hardware, as shown in Table 1. The customized instructions follow the RISC-V R type format, i.e., 7bit funct7, 5bit rs2, 5bit rs1, 3bit funct3, 5bit rd, 7bit Opcode. Once Opcode==0x0B&&funct3==0x7, the customized instruction is identified. The customized instruction includes three types: data transfer, calculation and update control.

[0045] Table 1 is a RISC-V extension instruction customized for simultaneous forward and backward transmission

[0046]

[0047]

[0048] SCHE generates forward signals to determine whether the MV is to send data to itself or to each other. The workload initiation signal generated by the SCHE informs the LD,ST to access the corresponding memory bank, and other instructions are also listed in Table 1.

[0049] The application also provides an intelligent edge chip comprising the hybrid acceleration architecture described above, which can be used in edge devices such as autonomous driving devices, automation control devices, etc.

[0050] Figure 4 The timelines of the traditional architecture, the sparse training timeline without scheduling, and the sparse training timeline with scheduling of the application are compared, and the schematic diagrams are shown in Figure 4 It can be seen that the traditional TinyML architecture may involve an ML acceleration unit in a CPU or low-power processor chip, and in order to implement on-device training, the CPU or accelerator needs to sequentially execute FP (forward propagation) and BP (back propagation), and cannot be parallel, as shown in (a) of Figure 4 When considering dynamic sparsity, the execution time of each training iteration varies, i.e., the delay is inversely proportional to the model sparsity, resulting in unpredictable training delay, which is fatal for scenarios that require stable delay. In contrast, by adding an IPA (In-Pipeline Acceleration) unit, the pipeline acceleration function of the CPU can be utilized to achieve parallel computation with the ACC (accelerator). However, because there is no workload scheduling, for example, FP is always executed in IPA, while BP is executed in ACC, due to the execution imbalance, there will be idle time in the execution process, resulting in limited performance improvement. As shown in (b) of Figure 4 The parallel acceleration of the i-th FP in IPA and the (i-1)-th BP in ACC is always overlapped. The task dependency forces the i-th BP to not start execution until the i-th FP is completed. Therefore, the scheduling between IPA and ACC is crucial to achieve high utilization and performance improvement. In order to improve the utilization and throughput, in the application, the scheduler estimates the delay based on the sparsity level, and arranges the FP and BP computation to different acceleration units, and the inter-layer delay is estimated according to the sparsity ratio change between different layers.

[0051] The operation of one sparse training iteration is described as follows: if the TinyML model has n layers, the scheduler is responsible for assigning the jth layer of the ith FP and the (n-j)th layer of the (i-1)th BP to the IPA and the ACC. Using the aforementioned equation, the scheduler evaluates two possible allocation combinations. By comparing the latency of these two combinations, one combination is selected for allocation and the IPA and the ACC are simultaneously initiated to execute their workloads. If both workloads complete execution at the same time, the resulting performance gap is zero. In most cases, the first to complete will wait until both the FP and the BP are completed. The preceding steps are repeated, and the scheduler assigns the (j+1)th layer of the ith FP and the (n-j-1)th layer of the (i-1)th BP to the IPA and the ACC.

[0052] The sparse information is updated at the end of one training iteration by the adopted dynamic sparse scheme. After the n assignments of FP layers and BP layers, the hyperparameters of the sparse information in the IPA and the ACC are sent to the sparse information updater. Then, the sparse information updater feeds back the latest sparse information to the scheduler. Finally, the weights are updated and the next training iteration is started. Figure 4 Fig. (c) demonstrates the scheduling mode timeline and the code snippet based on custom RISC-V instructions (see Table 1) of the present application. Since tasks are dynamically assigned to the two accelerators at layer granularity, better performance and higher resource utilization can be achieved.

[0053] Figure 5 For time gap comparison with / without scheduling and hardware rate utilization comparison, as shown in Figure 5 The sparse-aware scheduler can effectively reduce the time gap in the training iteration. Figure 5 Fig. (a) demonstrates the normalized time gap in training. Compared with no scheduling, the sparse-aware scheduler reduces the time gap by 3.6 times, 2.4 times, and 4.8 times on average in one training iteration of three models (MNV2, Proxy, and MCUNet). With the help of the scheduler, the ACC utilization on MCUNet (static sparse scheme) is increased from 46.30% to almost full utilization of 98.96% Figure 5 Fig. (b)).

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the foregoing embodiments of the present application have been described in detail, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A sparse sensing scheduler, suitable for intelligent edge chips containing multiple accelerators, characterized in that, include: Sparse information extractor, delay estimation unit, load distributor, and forward detector. The sparse information extractor is used to associate network layers in a neural network with the sparse feature information of those network layers. The latency estimation unit is used to evaluate the latency of each network layer when it runs on different accelerators based on the sparse feature information of each network layer. The load balancer is used to compare the forward propagation delay in the first accelerator and the backward propagation delay in the second accelerator to obtain the larger first delay; compare the backward propagation delay in the first accelerator and the forward propagation delay in the second accelerator to obtain the larger second delay; and then compare the first delay and the second delay to select the accelerator configuration scheme corresponding to the smaller delay. The forward detector is used to determine, based on the output of the load divider, whether the output of the first accelerator and the output of the second accelerator are directly sent to each other's input buffer.

2. The scheduler according to claim 1, characterized in that, The sparse feature information includes: sparsity ratio, sparsity pattern, and the dimension of the weight matrix.

3. A hybrid acceleration architecture, characterized in that, include: Multiple accelerators and a sparse sensing scheduler as described in any one of claims 1-2, The multiple accelerators are used to support sparse training with simultaneous forward and backward propagation.

4. The architecture according to claim 3, characterized in that, The plurality of accelerators consists of two units: the first accelerator is a pipelined tightly coupled acceleration unit used to perform vector multiplication and accumulation, and the second accelerator is a multiply-accelerate array.

5. The architecture according to claim 4, characterized in that, The pipelined tight-coupled acceleration unit is configured in the memory access pipeline stage of the central processing unit in the smart edge chip.

6. The architecture according to any one of claims 3-5, characterized in that, Also includes: A unified memory system is used to implement regular read weights during forward propagation and transposed read weights during backward propagation.

7. The architecture according to claim 6, characterized in that, The unified memory system includes: a weight memory library, an intermediate activation layer memory library, and a gradient memory library.

8. A smart edge chip, characterized in that, Including the hybrid acceleration architecture as described in any one of claims 3-7.

9. An edge device, characterized in that, Including the smart edge chip as described in claim 8.

Images