An attention region based dcnn accelerator

Abstract

An attention region-based DCNN accelerator, by periodically updating the attention region, has high recognition accuracy and low computational complexity; multiple computing cores are designed on the hardware, and sub-pictures are dynamically allocated to cope with different computing scenarios; fine-grained precision quantization is performed at the convolution kernel level, and a convolution kernel / output channel rearrangement method is proposed to support the calculation of convolution kernels with different precisions; real-time precision quantization is performed on the convolution window set, which can perform zero-rate detection on the output feature map in real time, and adaptive PEs and PE arrays are designed; the OS data flow system with the "two-dimensional data sliding" function can maximize the retention of data in the PE, thereby reusing Weight and Activation data in time and space, reducing data transmission bandwidth and additional power consumption caused by repeated data transmission, and supporting online convolution window set real-time zero-rate detection.

Description

Technical Field

[0001] This invention relates to the field of object detection, and more specifically to a DCNN accelerator based on attention regions. Background Technology

[0002] In previous one-stage object detection methods, there were no hardware accelerators optimized for video streams. However, unlike image detection, video object detection exhibits strong content correlation between adjacent frames. This means it's possible to predict the location of the target in the next frame and identify which regions in the field of view are not valuable. Furthermore, previous works often used accelerators with reconfigurable computational precision to quantize the precision requirements of the DCNN model offline, with the quantization granularity limited to the layer level. This approach has several drawbacks: first, layer-level precision quantization cannot distinguish the importance of different convolutional kernels within the same layer, resulting in a coarse-grained approach; second, offline precision quantization cannot dynamically quantize different regions (convolutional window sets) of the feature maps at each layer during model execution. These drawbacks limit these accelerators: first, they may perform unnecessary high-precision calculations, increasing computational resource overhead; second, they lack the flexibility to dynamically adjust computational precision for different video streams.

[0003] In terms of hardware, the parallel capabilities of GPUs are mainly reflected in the use of a large number of stream processors to perform parallel processing of multiple images, making GPUs suitable for training batches of images. However, in real-time video target recognition tasks, video frames arrive sequentially, making the computation time of a single image extremely sensitive. Accelerators must process video frames sequentially. This is especially true in scenarios with extremely high frame rate requirements (60fps, 120fps), such as in autonomous driving where early identification and prediction of dangerous targets is crucial for driving safety; millisecond-level early identification and prediction can shorten braking distances by several meters or even tens of meters. Many papers have pointed out the limitations of GPUs in this regard, namely their inability to fully utilize their batch image parallel processing capabilities. Therefore, previous designs often improved real-time video target recognition capabilities by increasing the processing performance of a single core, such as designing multiple piles to enhance convolutional kernel-level parallelism. Furthermore, drones used for ground target recognition often possess high-resolution aerial video, which places even higher demands on real-time processing performance. Summary of the Invention

[0004] To address the problems existing in the aforementioned background technologies, this paper proposes an attention-region-based DCNN accelerator that prunes the computational load in two main ways: first, by reducing the number of image regions involved in the computation, thus reducing the computational load in areas where there may be no target; and second, by performing fine-grained quantization of computational precision to reduce unnecessary high-precision calculations. Furthermore, the accelerator leverages the advantages of multi-GPU parallel processing. In summary, by combining multiple improvements, this paper achieves high-efficiency video stream target detection and dynamic precision reconfiguration computation.

[0005] A DCNN accelerator based on attention regions is proposed, which is based on a multi-core architecture, where each core processes one sub-image; when the sub-image is large, it is decomposed into two parts and processed by two cores; data communication is performed between the cores.

[0006] When the accelerator performs target recognition, the core parallelism is first sub-image parallelism. After the sub-images are recombined, the complete feature map is broadcast to all nodes. Different nodes obtain different convolution kernels, and the parallelism between nodes is transformed into convolution kernel-level parallelism.

[0007] For the video stream to be identified, complete frames are detected and identified at fixed intervals, and the accelerator updates the attention region based on the detection results; at other times, the accelerator only detects the attention region; the attention region includes a region that is slightly larger than the length and width of the target bounding box to be identified and the edge region of the image.

[0008] In the accelerator, fine-grained precision quantization is performed on the weight data and activation data of the sub-images, including convolution kernel granularity analysis of the weight data based on zero-rate proportion markers to calculate precision and sorting output, and convolution window set granularity quantization of the activation data based on zero-rate threshold markers to calculate precision and sorting output.

[0009] The beneficial effects achieved by this invention are as follows:

[0010] 1) An attention mechanism was proposed. Most of the time, the accelerator only detects images within the attention region. By periodically updating the attention region, both high recognition accuracy and significantly reduced computational load can be achieved. On the hardware side, multiple computing cores are designed within the chip. The accelerator dynamically allocates sub-images to these cores to handle two computational scenarios: "computing attention regions of different sizes" and "computing the entire image."

[0011] 2) Perform offline fine-grained precision quantization at the convolution kernel level on the DCNN model, and propose a method of convolution kernel / output channel rearrangement to support the accelerator's computation of convolution kernels with different precision.

[0012] 3) An online zero-rate detector is proposed to perform real-time precision quantization on each partition of the FeatureMap, i.e., the set of convolutional windows, which can detect zero rates in the output feature map in real time. PE and PEArray adapted to this method are also designed.

[0013] 4) The designed 8 PEArrays, together with WeightRegFiles, RightRegFiles, and BottomRegFiles, form an OS data flow system with "two-dimensional data sliding" function. This system can retain data within the PE to the greatest extent possible, thereby reusing Weight and Activation data in time and space, thus reducing data transmission bandwidth and additional power consumption caused by repeated data transmission. At the same time, it can well support real-time zero-rate detection of online convolutional window sets. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of parallel processing of image attention regions in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the Node structure in an embodiment of the present invention.

[0016] Figure 3 This is a schematic diagram illustrating the operation of the attention mechanism in an embodiment of the present invention.

[0017] Figure 4 This is a schematic diagram illustrating the merging and splitting of attention regions in an embodiment of the present invention.

[0018] Figure 5 This is a schematic diagram of the internal structure of PE in an embodiment of the present invention.

[0019] Figure 6 This is a schematic diagram of the PEArray workflow in an embodiment of the present invention.

[0020] Figure 7 This is a schematic diagram illustrating the process of PEArray reordering channels in an embodiment of the present invention.

[0021] Figure 8 This is a schematic diagram of the data flow of Activation data and Weight data of different precisions in the accelerator in an embodiment of the present invention.

[0022] Figure 9 This is a schematic diagram illustrating the performance improvement of not running and running the attention mechanism under videos with different target densities in this embodiment of the invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0024] In the attention mechanism proposed in this embodiment, the accelerator extracts multiple sub-images from an image, and these sub-images have different shapes and sizes. Although they use the same convolution kernel for convolution operations, the computation progress of these sub-images is different due to their size differences. Using a single high-performance, large-scale core to process these sub-images may have several drawbacks: first, the inconsistent computation progress of sub-images can cause idle waiting of computing resources; second, it will greatly increase the overhead of progress control. To solve these problems, it is desirable to process these sub-images relatively independently. The sub-images are referred to as "relatively independent" because in OneStage multi-object recognition like YOLO, the sub-images are eventually embedded into the original-sized image in the later stages, or in other words, the sub-images cannot run the YOLO model completely independently.

[0025] like Figure 1 The attention regions of the image are delivered to multiple nodes for parallel processing, and the accelerator performs sub-image parallelism. In the later stages of the DCNN model, the merged complete graph is broadcast to all nodes, and each node processes a portion of the kernel, with the accelerator switching to kernel-level parallelism.

[0026] Therefore, a multi-core accelerator architecture is proposed. Each core can process one sub-image; when the sub-image is large, it is decomposed into two parts and processed by two cores. Furthermore, these cores can communicate with each other. For One-Stage multi-object recognition models like YOLO, the parallelism of these cores can be modified: in the later stages of the model, the parallelism changes from sub-image parallelism to convolutional kernel-level parallelism.

[0027] The top-level architecture of the accelerator and the internal structure of the node are as follows: Figure 2 As shown, the accelerator consists of multiple Nodes and one Central Node. The Nodes are connected via NoC. The Central Node runs the top-level logic, including running the attention mechanism, allocating computational tasks to the Nodes, and subsequent data reshaping.

[0028] Each Node can independently run the basic operations of a CNN, and these Nodes are interconnected and communicate with each other through a network on-chip (NoC). Each Node consists of 8 PE arrays (PEArray), a weight register file (WeightRegFiles), a bottom register file (BottomRegFiles), a right register file (RightRegFiles), a zero-rate detector (ZeroDetector), a reordering buffer (RearrangeBuffer), 512KB and 256KB synapse / neural buffers (Synapses / Neural Buffers), a network interface (NetworkInterface), and a control unit (ControlUnit). The ControlUnit reads instructions from the instruction buffer (InstBuffer) to control the operation of the Node.

[0029] Each Node has 512KB and 256KB of Synapses / NeuralBuffer, respectively. This design allows their storage contents to be swapped: as will be discussed later, the space requirements for Weight and Activation data will change as the accelerator's parallelism changes during operation. Synapses / NeuralBuffer transfers Activation data to BottomRegFiles and RightRegFiles, and Weight data to WeightRegFiles. PE Arrays read computational data from these RegFiles. The output data of each PEArray passes through a ZeroDetector, which detects zero rates in the output data (ConvWindowSets). Data passing through the ZeroDetector is stored in a RearrangeBuffer. After passing through the RearrangeBuffer, the data becomes the input feature map for the next layer and is stored in the Synapses / NeuralBuffer. The NetworkInterface is used to connect the Node to the NoC.

[0030] Activation data refers to the feature map data in the YOLO network.

[0031] The eight registers in WeightRegFiles provide weight data to eight PEArrays, which are then broadcast to 64 PEs within each PEArray. RightRegFiles are connected to the rightmost PE of each PEArray, and BottomRegFiles are connected to the bottommost PE of each PEArray. This ensures that each PEArray receives the same activation data simultaneously. Data within a PEArray can flow from the rightmost adjacent PE to the leftmost adjacent PE, or from the bottommost adjacent PE to the topmost adjacent PE. These eight PEArrays, along with WeightRegFiles, RightRegFiles, and BottomRegFiles, form an OS dataflow system with "two-dimensional data sliding" functionality. This system was designed for two reasons: first, it maximizes the retention of data within the PEs, thus reusing weight and activation data in both time and space, reducing data transmission bandwidth and the additional power consumption caused by repetitive data transmission; second, it effectively supports real-time zero-rate detection using the proposed online convolutional window set.

[0032] Unlike discrete image detection, video object detection involves continuous movement. The relative position of an object changes only slightly between adjacent frames, and these frames are highly correlated. Therefore, on the one hand, the location of the object in the next frame can be largely predicted based on the previous frame; on the other hand, it's possible to identify which areas in the field of view are not valuable. Thus, methods entirely different from those used for image detection can be applied to video object detection.

[0033] like Figure 3 The diagram illustrates the computation process for frames 0-9 of a video stream. Every 5 frames, a complete frame image is detected and identified, and the accelerator updates the attention region based on the detection results. For the majority of the time, the accelerator only detects the attention region. This attention region includes: a portion slightly larger than the length and width of the identified target bounding box, and a portion of the image's edge regions. Using the image's edge regions as attention regions effectively handles targets that have just entered the field of view from the edge.

[0034] Since the input image size of the YOLO network is 416 pixels * 416 pixels, and the feature map size keeps decreasing, the length and width of the attention region sub-image need to be integer multiples of 32 to facilitate the embedding of attention regions in later stages of the model. For example... Figure 4 As shown in .a, when the target bounding box overlaps with the edge region of the image, the target bounding box extends all the way to the edge of the image as the attention region, at which point the image edge region will be segmented; as... Figure 4As shown in .b, when one target box overlaps with another target box, these target boxes are merged into a large rectangular target box, which contains all the original targets.

[0035] In frames where attention regions are detected, the accelerator assigns a single attention region to each node. For example... Figure 4 As shown in .c, if the attention region is large and there are idle nodes, then the attention region is split into two parts equal to the number of idle nodes. The two split images are then processed by two nodes. However, since the cutting plane splits the originally uniform image, it affects the path of the convolution kernel. Therefore, the edge data of the cutting plane needs to be transmitted between the two nodes before each layer computation. In addition, if the number of targets exceeds the number of nodes, the two smallest attention regions are assigned to a node in turn. The process is not clear, but specifically, for the first node, the two smallest attention regions are assigned to it, and then for the second node, the two smallest regions from the part after removing the two attention regions are assigned, thus assigning attention regions to nodes in pairs from smallest to largest.

[0036] In OneStage multi-object recognition like YOLO, sub-images are eventually embedded into the original image size. For example, in the CONV44 layer, the standard model's 416*416 image size is compressed to 26*26. Correspondingly, the 32*32 attention region sub-image is compressed to 2*2, and this 2*2 attention region sub-image needs to be embedded (overlaid) into the corresponding position within the 26*26 full image. Figure 3 As shown, at layer 44, the activation data of the attention region is embedded into the input feature map of layer 44 from frame 5. The blank areas in the input feature map of layer 44 in the figure are the locations to be covered by the attention region sub-image. Therefore, the accelerator needs to store the input feature map of layer 44 in the complete detection frames (frames divisible by 5) for later use.

[0037] When detecting frames of the complete image, the accelerator uses a similar method to accelerate the model. The accelerator segments the image into multiple sub-images and assigns them to different Nodes for processing. Each Node needs to transmit edge data of the cut surfaces via NoC to ensure proper extraction of edge features and maintain size.

[0038] Whether in frames detecting attention regions or frames detecting the complete image, sub-images are recombined in later stages. The combined complete feature map is broadcast to all nodes, and different nodes receive different convolutional kernels. Inter-node operations are performed as follows: Figure 1The diagram illustrates kernel-level parallelism. When the accelerator performs sub-image parallelism, the weight data requires more storage space; therefore, 512KB of Synapses / NeuralBuffer is used to store the weight data, and 256KB of Synapses / NeuralBuffer is used to store the activation data. When the accelerator switches from sub-image parallelism to kernel-level parallelism, the activation data requires more storage space, while the convolution kernels, distributed across 16 nodes, require less storage space per node. Therefore, 512KB of Synapses / NeuralBuffer is used to store the activation data, and 256KB of Synapses / NeuralBuffer is used to store the weight data.

[0039] In this detection mode, the computational load is drastically reduced. Furthermore, the computational load varies depending on the number of targets in the video. This is especially true in quiet scenes where targets are scarce or nonexistent most of the time. Attention-based detection is well-suited for such scenarios; the fewer targets in the frame, the less computation is required. Compared to traditional detection methods, this approach significantly reduces power consumption. In addition, multi-node parallel processing of different regions of the same image provides the accelerator with GPU-like multi-GPU parallel processing capabilities.

[0040] Before introducing online real-time precision quantization and zero-rate detection, we will first introduce the structure of PE and the data flow in PEArray.

[0041] The PE internally integrates MAC computation and storage functions. FIFO, FReg, and Psum RegStack support an OS data stream system with data memory capabilities and an Activation data buffer. This method maximizes data retention within the PE and reuses data in both time and space. The data memory data stream system significantly reduces the bandwidth requirements of the PEArray for input data.

[0042] The PE (Programmable Execution System) can perform 4x4 bit, 4x8 bit, 8x8 bit, and 8x16 bit calculations internally. For example... Figure 5As shown, the lines represent the data flow in 4*4bit, 4*8bit, and 8*8bit computation modes, respectively. In 4*4bit computation mode, each PE can simultaneously compute four pairs of Activations and weights with the same relative position on different channels within the same convolution window. In 4*8bit computation mode, using two bitshifters, each PE can simultaneously compute two pairs of Activations and weights with the same relative position on different channels within the same convolution window. The two multipliers on each side can respectively compute the 8-bit high four bits and low four bits of data multiplied by the 4-bit weight data. In 8*8bit computation mode, each PE computes only one pair of Activations and weights within the same convolution window; 8*8bit computation can be achieved by adding only one bitshifter to the 4*8bit model.

[0043] FReg receives feature data from the adjacent PE on the right. Based on control signals, the data in FReg is either passed to the adjacent PE or sent for computation, essentially allowing the data to slide within the PEArray. FIFO stores feature data from the adjacent PE below, so that when calculating the next row of the convolution window, it's not necessary to reread the data from the RGF or SRAM outside the PEArray. Based on control signals, the feature data in FIFO is also passed to the adjacent PE above or sent for computation in the next time step. The data selector MUX selects data from either FIFO or FReg based on control signals from the control unit.

[0044] PEArray employs an OS data flow architecture, where data from the same convolutional window set is simultaneously broadcast to eight PEArrays in a single computation round. The convolutional window set refers to a rectangular region composed of 16 aggregated 3x3 convolutional windows. For example... Figure 6 In the .a file, each PEArrray has 16 PEs. Each PE is responsible for computing a 3D convolutional window within the convolutional window set and outputting the activation data for the next layer corresponding to that convolutional window. Therefore, one round of computation yields the portion of the output feature map that is in the same relative position as the convolutional window set. Subsequent rounds of computation will sequentially select such a convolutional window set until all convolutional windows have been computed.

[0045] like Figure 6As shown in .b, if the convolution kernel size is 3*3 and the PEArray has 4*4 PEs, then the convolution window set on one channel appears as a 6*6 square. This data organization allows data to flow between PEs: at any given moment, the PEArray calculates the activation data at the same position in 16 convolution windows. For example, during the first calculation, the PEArray multiplies the data from the top left corner of the 16 convolution windows with the same weight data. The weight data is broadcast to these 16 PEs, allowing for full reuse of the weight data. Furthermore, each PE contains more than one set of MAC units, so more than one channel is being calculated at a time: each PE has 8 multipliers. In 4*4-bit mode, the 8 multipliers calculate the data for each channel, then accumulate the data through an addition tree and sum the results with a partial sum.

[0046] This embodiment proposes fine-grained precision quantization, which needs to be performed separately on the weight and activation aspects. Weight data is used to analyze the kernel granularity, requiring offline zero-rate detection for each kernel. First, the distribution of the absolute values ​​of all weight data in the DCNN accelerator is statistically analyzed, and the proportion G of data with absolute values ​​in the range [0, 0.0125] is recorded. When the zero rate of a convolutional kernel is higher than G, its computational precision is marked as 4 bits; when the zero rate of a convolutional kernel is lower than G, its computational precision is marked as 8 bits.

[0047] like Figure 7 As shown, the convolutional kernels are quantized and reordered offline to generate a mapping table. Since multiple convolutional kernels are computed with the same activation data simultaneously within the PEArray, the computational precision of the convolutional kernels entering the PEArray at the same time should be the same. Therefore, after determining their computational precision, the convolutional kernels are reordered according to precision order, and a mapping table is generated based on both the original order and the reordered order. The output of the PEArray is then reordered according to the mapping table. Only the reordered output feature maps can be used as input for the next layer.

[0048] The YOLO network is subordinate to the DCNN network, and the DCNN network is subordinate to the CNN network. This accelerator runs the YOLO network. Here, "next layer" refers to the layer below a certain layer in the YOLO network that the Node is currently running.

[0049] Activation data requires quantization at the convolution window set granularity. For example... Figure 8The diagram illustrates the data flow of Activation and Weight data at different precisions within the accelerator. The precision of the Weight data comes from off-chip pre-analysis, while the Activation data comes from on-chip online analysis. ZeroDetector performs zero-rate detection on the output convolutional window set. Each Node has 8 PEArrays and 8 ZeroDetectors. The 8 ZeroDetectors detect the zero rate of the output data of their respective PEArrays. Since each PEArray is an output partial convolutional window set, the results of the 8 ZeroDetectors need to be combined to determine the zero rate of the next layer's convolutional window set.

[0050] Two thresholds, TH1 and TH2, are set. If the proportion of near-zero data (i.e., the "zero rate" detected in the previous section) is between [0, TH1], the computational precision of this set of convolutional windows is marked as 16 bits; if the proportion is between [TH1, TH2], the computational precision of this set of convolutional windows is marked as 8 bits; and if the proportion is between [TH2, 1], it is marked as 4 bits. The next layer will be calculated according to the marked precision. The output convolutional window set is reordered by channel according to the mapping table (i.e., the mapping table generated when analyzing the kernel granularity of the weight data above) and then stored in on-chip memory as the input convolutional window set for the next layer.

[0051] When the default requirement is for the accelerator to perform all 4-bit calculations, both TH1 and TH2 are set to 0.

[0052] The accelerator operates at 500MHz, performing all calculations with 4x4-bit precision. Each PE performs two operations per clock cycle: multiplication and addition. The theoretical peak performance of a single node can reach 500MHz * 4PEs * 4PEs * 8 Arrays * 8 MACs * 2 = 1 TOPS, and the theoretical peak performance of an accelerator with 16 node cores can reach 1 TOPS * 16 Nodes = 16 TOPS. However, each PE has 4x4-bit, 4x8-bit, 8x8-bit, and 8x16-bit calculation modes, and the theoretical performance of the accelerator varies between 2 TOPS and 16 TOPS depending on the calculation precision within the node.

[0053] Experiments were conducted to investigate the impact of kernel granularity precision quantization and online convolution window set precision quantization:

[0054] Without running the attention mechanism, we will examine the performance improvements and impacts of kernel granularity precision quantization and online convolutional window set precision quantization on the accelerator. First, we will examine the impact of the choice of thresholds TH1 and TH2 on performance and accuracy. Controlling the choice of TH1 and TH2 can control whether the model's computation is biased towards high precision or low precision. The impact of different TH1 and TH2 values ​​on the accuracy of the model results is shown in the table below. Lower thresholds will cause the model to select more low-precision computations, while higher thresholds will cause the model to select more high-precision computations. Within an acceptable accuracy range, we will choose a lower threshold for the accelerator.

[0055]

[0056] With thresholds TH1 and TH2 at 40% and 70% respectively, this accelerator is compared to a model that only performs offline LayerLevel precision quantization (4-bit and 8-bit). The comparison of their runtime data is shown in the table below. It is evident that kernel-level precision quantization and online convolutional window set precision quantization did not cause a decrease in accuracy. Furthermore, compared to offline LayerLevel precision quantization and no quantization, this indicates that kernel-level precision quantization and online convolutional window set precision quantization can effectively reduce [accuracy issues].

[0057]

[0058] Experiments were conducted to study the impact on attention mechanisms:

[0059] The performance of the accelerator was compared with and without the attention mechanism. Without the attention mechanism, the accelerator detects the entire image in each frame. Four videos with different target densities were selected to test the advantages of the attention mechanism. The target density of the four videos was measured by the average number of targets per frame: d1 = 4.2, d2 = 9.9, d3 = 13.5, and d4 = 19.3. The comparison of their operational data is shown in the table below. Figure 9 As shown in the figure. Testing revealed that with the attention mechanism introduced into the accelerator, mAP remained essentially unchanged.

[0060] In video d1 with fewer targets, the accelerator's processing frame rate was improved by 5.1 times compared to without the attention mechanism. This improvement was also achieved in videos with more targets. This demonstrates that the attention mechanism plays a significant role in reducing the computational cost per frame. The improvements in the accelerator's computational cost and speed are related to the following factors:

[0061] 1) Target size. The larger the target, the larger the area that needs to be calculated;

[0062] 2) The proportion of frames without a target in the total number of frames in the video. The more blank frames there are, the less computation the accelerator needs to perform.

[0063]

[0064] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or variations made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. An attention region-based DCNN accelerator, characterized in that: the accelerator is based on a multi-core architecture, each core processing a sub-image; when the sub-image is large, it is divided into two parts for processing by two cores; data communication is carried out between the cores; when the accelerator performs target recognition, the parallel mode of the core is first sub-image parallel, and after the sub-image is recombined, the complete feature map is broadcast to all Nodes, different Nodes obtain different convolution kernels, and the Nodes change to convolution kernel-level parallel; for a video stream to be recognized, complete frame images are detected and recognized every fixed time, and the accelerator updates the attention region according to the detection result; at other times, the accelerator only detects the attention region; wherein the attention region includes a region slightly larger than the length and width of the target to be recognized and an edge region of the image; in the accelerator, fine-grained precision quantization is performed on the Weight data and Activation data of the sub-image, including convolution kernel granularity analysis based on zero rate proportion marking precision calculation and ordering output for Weight data, and convolution window set granularity quantization based on zero rate threshold marking precision calculation and ordering output for Activation data; in fine-grained precision quantization, Weight and Activation need to be processed respectively; Weight data is analyzed at the granularity of convolution kernel, which needs to be detected one by one offline; first, the distribution of the absolute values of all Weight data in the DCNN accelerator is counted, and the proportion G of the data with absolute values in [0, 0.0125] is recorded; when the zero rate of a convolution kernel is higher than G, its calculation precision is marked as 4 bits; when the zero rate of a convolution kernel is lower than G, its calculation precision is marked as 8 bits; after the calculation precision of the convolution kernel is determined, it is reordered according to the precision order, and a mapping table is generated according to the original order and the reordered order; the output of the PEArray will be reordered according to the mapping table; only the reordered output feature map can be used as the input of the next layer. 2.The attention region based DCNN accelerator of claim 1, wherein: the accelerator is composed of multiple core Nodes and a central core Central Node; the Nodes are connected and communicated with each other through the on-chip network NoC; the Central Node runs the top-level logic, including running the attention mechanism, assigning the calculation tasks of the Nodes, and reshaping the data later; Each Node independently runs a convolutional neural network CNN; each Node is composed of 8 PE arrays PEArray, weight register files Weight Reg Files, bottom register files Bottom Reg Files, right register files Right Reg Files, a zero rate detector Zero Detector, a rearrange buffer Rearrange Buffer, 512 KB and 256 KB synapses / neural buffers Synapses / Neural Buffer, a network interface Network Interface and a control unit Control Unit; the Control Unit reads instructions from an instruction buffer InstBuffer to control the running of the Node; the storage content of the 512 KB and 256 KB sized Synapses / Neural Buffer can be exchanged; the Synapses / Neural Buffer transmits Activation data to the Bottom Reg Files and the Right Reg Files, and transmits Weight data to the Weight Reg Files; each PE array is composed of 16 processing elements PE. 3.The attention region based DCNN accelerator of claim 1, wherein: When the accelerator is in sub-image parallel, the 512 KB Synapses / Neural Buffer is used to store Weight data, and the 256 KB Synapses / Neural Buffer is used to store Activation data; when the accelerator is switched from sub-image parallel to kernel-level parallel, the 512 KB Synapses / Neural Buffer is used to store Activation data, and the 256 KB Synapses / Neural Buffer is used to store Weight data. 4.The attention region based DCNN accelerator of claim 2, wherein: The PE Arrays read data participating in calculation from the Reg Files; the output data of each PE Array passes through a Zero Detector, which is used to detect the zero rate of the output data; the data passing through the Zero Detector is stored in the Rearrange Buffer, becoming the input feature map of the next layer and being stored in the Synapses / Neural Buffer; the Network Interface is used to connect the Node to the network on chip NoC.

5. The attention region based DCNN accelerator of claim 2, wherein: The data in the PEArray flows from the right adjacent PE to the left adjacent PE, or from the lower adjacent PE to the upper adjacent PE.

6. The attention region based DCNN accelerator of claim 1, wherein: The 8 registers in Weight RegFiles provide weight data for 8 PEArray respectively, and the weight data is broadcast to 64 PEs in each PEArray; Right Reg Files are connected to the rightmost PEs of each PEArray; Bottom Reg Files are connected to the bottommost PEs of each PEArray, that is, each PEArray obtains the same Activation data at the same time.

7. The attention region based DCNN accelerator of claim 1, wherein: When detecting the attention region, if the target frame overlaps with the edge region of the image, the target frame extends to the edge of the image as the attention region, and the edge region of the image is cut at this time; when a target frame overlaps with another target frame, the target frames are merged into a large rectangular target frame, and the large rectangular target frame contains all the original targets. 8.The attention region based DCNN accelerator of claim 1, wherein: When detecting the attention region, the accelerator allocates a single attention region for each Node; If the attention region is large and there are idle Nodes, then take the attention regions equal in number to the number of idle Nodes for splitting; the two parts of the image after splitting are delivered to two Nodes for processing, and the edge data of the cutting surface is transmitted between the two Nodes before each layer calculation; if the number of targets exceeds the number of Nodes, the smallest two attention regions are allocated to a Node in turn. 9.The attention region based DCNN accelerator of claim 1, wherein: In the fine-grained precision quantization processing, the Weight data is analyzed in the granularity of the convolution kernel, and the zero rate detection is performed offline on the convolution kernel one by one; first, the distribution of the absolute values of all Weight data in the DCNN accelerator is counted, and the proportion G of the data with absolute values in [0, 0.0125] is recorded; when the zero rate of a certain convolution kernel is higher than G, the calculation precision of the convolution kernel is marked as 4 bits; when the zero rate of a certain convolution kernel is lower than G, the calculation precision of the convolution kernel is marked as 8 bits; After the calculation precision of the convolution kernel is determined, the convolution kernel is reordered according to the precision order, and a mapping table is generated according to the original order and the reordered order; the output of the PEArray is reordered according to the mapping table, and the reordered output feature map is used as the input of the next layer. 10.The attention region based DCNN accelerator of claim 1, wherein: In the fine-grained precision quantization processing, the Activation data is quantized in the granularity of the convolution window set; each Node has 8 PEArray and 8 Zero Detector, and the 8 Zero Detctor detects the zero rate of the output data of the corresponding PEArray, and the results of the 8 ZeroDetector are integrated to judge the zero rate of the next convolution window set; Two threshold values TH1 and TH2 are set, if the zero rate is between [0, TH1], the calculation precision of the convolution window set is marked as 16 bits; if the zero rate is between [TH1, TH2], the calculation precision of the convolution window set is marked as 8 bits; if the zero rate is between [TH2, 1], the calculation precision of the convolution window set is marked as 4 bits; The next layer will be calculated with the precision of the tag; the output convolution window set is reordered by channel according to the mapping table and stored in the on-chip memory as the input convolution window set of the next layer.

Images