A winograd-based correlation algorithm accelerator storage method
By using a Winograd-based algorithm accelerator storage method, the problem of low computational efficiency of deep learning processors for related algorithms is solved, achieving high data throughput and low off-chip storage bandwidth requirements.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING AEROSPACE AUTOMATIC CONTROL RES INST
- Filing Date
- 2022-12-10
- Publication Date
- 2026-04-21
AI Technical Summary
Existing deep learning processors have low computational efficiency for related algorithms, resulting in high demand for off-chip storage bandwidth.
We adopt a Winograd-based algorithm-accelerated storage method, which reduces off-chip storage bandwidth requirements by using a regional storage and data reuse strategy for baseline and real-time tensor caches.
It achieves high-throughput data requirements while utilizing as few on-chip BRAM resources as possible, reducing the accelerator's bandwidth requirements for off-chip storage.
Smart Images
Figure CN115809024B_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of neural network storage technology, and specifically relates to a storage method for related algorithm accelerators based on Winograd. Background Technology
[0002] With the rapid development of deep learning technology, Convolutional Neural Networks (CNNs) have begun to be used in image matching and localization algorithms. This algorithm consists of a CNN and a correlation algorithm. The CNN is used for feature extraction from a baseline image and a real-time image, while the correlation algorithm performs correlation operations on the feature data extracted by the CNN to extract correlation maxima. Due to the massive computation and storage required for CNNs, the computation is generally performed by a Deep-Learning Processing Unit (DPU). However, because the computational characteristics of correlation algorithms differ significantly from those of deep learning algorithms, DPUs typically have extremely low computational efficiency for them, thus giving rise to FPGA-based correlation algorithm accelerators. Summary of the Invention
[0003] This disclosure is made based on the aforementioned needs of the prior art. The technical problem to be solved by this disclosure is to provide a Winograd-based algorithm accelerator storage method to reduce the accelerator's demand for off-chip storage bandwidth.
[0004] To address the aforementioned problems, the technical solutions provided in this disclosure include:
[0005] A storage method for a correlation algorithm accelerator based on Winograd is provided, including: S1 obtaining the sizes of the correlation result matrix block and the real-time graph matrix block, the sizes of the correlation result matrix and the real-time graph tensor, and the channel parallelism of the acceleration unit; S2 storing C from off-chip storage to the first region of the reference tensor. b ×(m+r-1)×P n The baseline tensor block, where C b Where m is the width of the baseline graph tensor, r is the width of the relevant result matrix block, and P is the width of the real-time graph matrix block. n To accelerate the channel parallelism of the unit; S3 is moved from off-chip storage to real-time tensor cache C. r A real-time graph tensor block of ×r×N, where C r S4 represents the width of the real-time graph tensor, and N represents the number of channels in the real-time graph tensor. S4 reads data from the first region of the baseline graph tensor buffer, writes the last two lines of data read into the first two lines of the second region of the baseline graph tensor buffer, and simultaneously reads the next four lines from external storage and writes them into the second region. This process is repeated in step C. b×r clock cycles; S5 reads a size of (r-1)×(m+r-1)×P from the reference tensor cache. n The tensor block is pre-written to the reference tensor register set; S6 writes r×r×P from the real-time graph tensor buffer to the real-time graph tensor register. n S7 moves the data from the front column of the reference register group to the back column, and reads the data from the reference tensor buffer to the front column of the register group; S8 processes and calculates the data between different register groups and then writes it to the tensor register group.
[0006] Preferably, in step S1, the reference tensor cache includes m×(m+r-1) independent storage units, each storage unit corresponding to one row of the reference tensor, and is divided into m groups.
[0007] Preferably, in step S2, when the data in the real-time tensor cache is exhausted, new data is loaded from off-chip storage to overwrite it.
[0008] Preferably, in step S5, the reference tensor buffer reads a value of (r-1)×(m+r-1)×P from groups m-1 and m-2. n The tensor block is pre-written into the first r-1 columns of the reference tensor register group.
[0009] Preferably, S5 is performed after S2.
[0010] Preferably, S6 and S5 are performed simultaneously.
[0011] Preferably, step S8 includes: transforming the data in the reference tensor register group and the real-time tensor register group and writing them into the V tensor register group and the U tensor register group respectively; multiplying the data in the V tensor register group and the U tensor register group point by point and then writing them into the E tensor register group.
[0012] Preferably, step S8 further includes: transforming the data in the E tensor register group and writing it into the S tensor register group; and simultaneously reading intermediate result data from the process tensor buffer into the bias tensor register group.
[0013] Preferably, step S8 further includes: accumulating the data in the S tensor register group along the channel direction, adding it point by point to the data in the bias register group, and writing it into the process tensor register group.
[0014] Preferably, step S8 ends after the current real-time tensor block has completed scanning a whole row of data from the reference graph tensor.
[0015] Compared with the prior art, this disclosure meets the high throughput data requirements of the computing unit while utilizing as few on-chip BRAM resources as possible to achieve the highest possible data reuse rate, thereby reducing the accelerator's demand for off-chip storage bandwidth. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings.
[0017] Figure 1 This disclosure presents a flowchart of the steps for a Winograd-based algorithm accelerator storage method;
[0018] Figure 2 This is an architecture diagram of the storage system for the Winograd-based algorithm accelerator.
[0019] Figure 3 This is a schematic diagram of the off-chip storage reference map data arrangement in an embodiment of this disclosure;
[0020] Figure 4 This is a schematic diagram illustrating the connection and data transmission of the reference graph cache and reference register group in this disclosure. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0022] In the description of the embodiments of this disclosure, it should be noted that, unless otherwise expressly specified and limited, the term "connected" should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.
[0023] Throughout the text, the terms “top,” “bottom,” “above,” “below,” and “on top” refer to the relative positions of components of the device, such as the relative positions of the top and bottom substrates within the device. It is understood that the device is multifunctional and independent of its spatial orientation.
[0024] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.
[0025] This embodiment provides a storage method for related algorithm accelerators based on Winograd, such as... Figures 1-4 As shown.
[0026] To overcome the limitations of DSP resources on FPGAs, this invention proposes to accelerate related algorithms using the Winograd algorithm.
[0027] The correlation algorithm, based on its fundamental principle, treats the real-time tensor as a correlation window. This correlation window sequentially scans the reference graph tensor to obtain a correlation result matrix. Within each correlation window, the real-time tensor and the reference graph tensor are multiplied and accumulated to obtain a correlation result value. This is represented as follows:
[0028]
[0029] 0 <c c <C c
[0030] 0 <r c <R c
[0031] 0 <n<N
[0032] Where C is the correlation result matrix, c c r is the column index of the element in the relevant result matrix. c Here, R represents the row index of an element in the relevant results matrix, N is the number of channels in the real-time graph tensor, and R is the number of channels in the tensor. r C represents the height of the real-time graph tensor. r B represents the width of the real-time graph tensor, and c represents the width of the reference graph tensor. b r is the column index of an element within the base graph tensor. b Here, n is the row index of an element within the baseline graph tensor, n is the index of the input channel, R represents the real-time graph tensor, and c is the index of the input channel. r r is the column index of elements within a real-time graph tensor. r This is the row index of the element within the real-time graph tensor.
[0033] Furthermore, the width of the relevant results matrix is expressed as follows:
[0034] Cc =C b -C r +1
[0035] Among them, C c C represents the width of the relevant result matrix. b C is the width of the reference graph tensor. r This is the width of the real-time image tensor.
[0036] The height of the correlation result matrix is represented as follows:
[0037] R c =R b -R r +1
[0038] Among them, R c R represents the height of the relevant results matrix. b R is the height of the reference plot tensor. r This represents the height of the real-time image tensor.
[0039] The basic principle of the Winograd algorithm can be expressed by the following expression:
[0040] y = A T [GgG] T ]e[B T dB]]A
[0041] Where y represents an m×m related result matrix block, g represents an r×r real-time graph matrix block, and d represents a (m+r-1)×(m+r-1) reference graph matrix block. That is, the real-time graph matrix block g and the reference graph matrix block d are transformed by matrices G and B respectively, multiplied point-by-point, and then subjected to an inverse transformation by matrix A to obtain the related result matrix block y. For convenience, the above expression is denoted as F(m×m, r×r).
[0042] Based on the fundamental principle of the Winograd algorithm to accelerate computation, the following is obtained:
[0043]
[0044] Among them, c C r is the column index of the block within the relevant results matrix. C c is the row index of the block within the relevant results matrix. B r is the column index for the block within the base graph tensor. B c is the row index of the block within the base graph tensor. R r is a column index for blocks within a real-time graph tensor. R This is the row index for blocks within a real-time graph tensor.
[0045] The block partitioning methods for real-time graph tensor R include RR With C R , represented as:
[0046]
[0047]
[0048] The partitioning method of the relevant result matrix C includes R C With C C , represented as:
[0049]
[0050]
[0051] The number of blocks R of the baseline tensor B B With C B It is the same as the related result matrix C, but the block size is different, and the overlap width between adjacent blocks is (r-1).
[0052] This method is implemented based on the storage system of the relevant algorithm accelerator, such as... Figure 2 As shown, the related algorithm accelerator storage system includes off-chip storage, on-chip tensor cache, and multiple registers. The on-chip tensor cache includes a base tensor cache, a real-time tensor cache, and a process tensor cache. The multiple registers include a base tensor register group, a real-time tensor register group, a process tensor register group, and bias tensor register groups, V tensor register groups, U tensor register groups, E tensor register groups, and S tensor register groups. The off-chip storage and on-chip tensor cache are connected via a data loading unit. Specifically, the off-chip storage is connected to the data loading unit via an AXI data bus. The data loading unit is connected to the base tensor cache and the real-time tensor cache via FIFO data lines. The base tensor cache is sequentially connected to the base tensor register group, the base graph transformation unit, and the V tensor register group. The real-time tensor cache is sequentially connected to the real-time tensor register group, the real-time graph transformation unit, and the U tensor register group. Both the V tensor register group and the U tensor register group are connected to the point-by-point multiplication unit. The point-by-point multiplication unit outputs data to the E tensor register. The E tensor register is connected to the result transformation unit and to the S tensor register group. The S tensor register group is connected to the channel accumulation unit. After processing by the channel accumulation unit, the data is input to the process tensor register group. The process tensor register group is connected to the process tensor cache and input to the data unloading unit via FIFO data lines. Finally, the data is output to the off-chip storage via the AXI data bus.
[0053] like Figure 1 As shown, the storage method includes:
[0054] S1 obtains the size of the relevant result matrix block and the real-time graph matrix block, the size of the relevant result matrix and the real-time graph tensor, and the channel parallelism of the acceleration unit.
[0055] S2 is stored from off-chip storage to the first region of the reference tensor. b ×(m+r-1)×P n The baseline tensor block, where C b Where m is the width of the baseline graph tensor, r is the width of the relevant result matrix block, and P is the width of the real-time graph matrix block. n To accelerate the channel parallelism of the unit
[0056] The benchmark tensor cache is used to cache the benchmark tensor blocks required for the current computation. Based on the data reuse strategy of this accelerator, the cache size is C each time. b ×(m+r-1)×P n The tensor block. Its structure is related to the Winograd algorithm used by the acceleration unit, and is represented as:
[0057] Buf B [m][m+r-1][D B ][P n ]
[0058] For F(m×m, r×r), a total of m×(m+r-1) independent storage units are required. Among them, Buf... B This serves as a cache for the baseline graph, where m represents the width of the relevant result matrix in Winograd, and (m+r-1) is the height of the baseline graph matrix block. The width of each storage unit is (P n ×BIT B ) bits, D B P is the depth of the storage bank. n It is the parallelism of the acceleration unit in the input channel direction, in bits. B This refers to the bit width of the baseline image feature data.
[0059] The depth of each memory bank must be greater than twice the maximum width of the baseline graph, that is:
[0060] D B ≥max(C b )×2
[0061] In other words, the maximum readable and writable size of this cache per clock cycle is m×(m+r-1)×P. n The baseline tensor block.
[0062] To fully utilize the BRAM resources in the FPGA, the reference map tensor cache is implemented using dual-port RAM, with ping-pong read / write regions set along the depth direction. Based on the depth requirements of the aforementioned memory, if the maximum width of the reference map is less than or equal to 256, then the depth of this cache is 512. The region from depth 0 to 255 is designated as the "ping region," and the region from 256 to 511 is designated as the "pong region." When the acceleration unit reads data from the "ping region" for calculation, the data loader can cache new data in the "pong region," and vice versa, thereby achieving parallel data computation and transmission. The "ping region" corresponds to the first region, and the "pong region" corresponds to the second region.
[0063] S3 transitions from off-chip storage to real-time tensor cache C r A real-time graph tensor block of ×r×N, where C r N represents the width of the real-time image tensor, and N represents the number of channels in the real-time image tensor.
[0064] The real-time graph tensor cache is used to cache the real-time graph tensor blocks required for the current computation. Based on the data reuse strategy of this accelerator, the cache size is C each time. r A tensor block of ×r×N. Its structure is related to the Winograd algorithm used in the acceleration unit, and is represented as:
[0065] Buf R [r][D R ][P n ]
[0066] For F(m×m, r×r), a total of r independent storage units are required, where Buf R For the real-time graph cache, r represents the height of the real-time graph matrix block, and D R This represents the depth of each memory bank. The width of each memory bank is (P... n ×BIT R Bit R The bit width of the real-time graph feature data means that the maximum read / write size of this cache per clock cycle is 1×r×P. n The baseline tensor block. The depth of each memory bank is greater than the maximum value of the product of the real-time graph width and the channel block, i.e.:
[0067]
[0068] S4 reads data from the first region of the benchmark tensor cache, writes the last two lines of data read into the first two lines of the second region of the benchmark tensor cache, and simultaneously reads the next four lines from external storage and writes them into the second region, repeating C. b ×r clock cycles.
[0069] Based on the fundamental principles of the Winograd algorithm and related algorithms, the overlap width between adjacent reference map matrix blocks is (r-1). Therefore, there are (r-1) rows of data that are identical between the "ping-pong" read / write regions of the reference map tensor cache. To reduce off-chip memory access bandwidth requirements, the last two rows, i.e., rows (m+r-2) and (m+r-3), can be written to the "ping-pong" region while reading data from the "ping-pong" region, and vice versa. Since all data in the "ping-pong" region needs to be retrieved from DDR when data is cached for the first time, the above method allows for caching only m rows of data from DDR to a single read / write region each time, except for the initial caching of data in the "ping-pong" region. This reduces the DDR transfer volume of the reference map data by (r-1) / (m+r-1).
[0070] S5 is read from the reference tensor buffer with a size of (r-1)×(m+r-1)×P. n The tensor block is pre-written into the reference tensor register group.
[0071] The reference tensor register set loads data from the reference graph tensor buffer and provides data to the reference graph transformation unit. The throughput of the reference graph transformation unit per clock cycle is (m+r-1)×(m+r-1)×P. n The tensor block is used, but since the overlap width between adjacent reference map matrix blocks is (r-1), each clock cycle only needs to read a tensor buffer of size m×(m+r-1)×P. n The remaining size of the tensor block is (r-1)×(m+r-1)×P n Tensor blocks are implemented by moving them within the register set.
[0072] S6 writes r×r×P from the real-time graph tensor buffer to the real-time graph tensor register. n Tensor blocks of size.
[0073] The real-time graph tensor cache is used to cache the real-time graph tensor blocks required for the current computation. Based on the data reuse strategy of this accelerator, the cache size is C each time. r A tensor block of ×r×N. Its structure is related to the Winograd algorithm used in the acceleration unit, and is represented as:
[0074] Buf R [r][D R ][P n ]
[0075] For F(m×m, r×r), a total of r independent storage units are required, where Buf R For the real-time graph cache, r represents the height of the real-time graph matrix block, and D R This represents the depth of each memory bank. The width of each memory bank is (P...n ×BIT R Bit R The bit width of the real-time graph feature data means that the maximum read / write size of this cache per clock cycle is 1×r×P. n The baseline tensor block. The depth of each memory bank is greater than the maximum value of the product of the real-time graph width and the channel block, i.e.:
[0076]
[0077] The real-time tensor register set loads data from the real-time graph tensor buffer and provides data to the real-time graph transformation unit. The size of the register set is r×r×P. n Based on the data reuse strategy designed for this accelerator, the real-time tensor register is used every (C... c It updates once every / m) clock cycles. Based on the maximum bandwidth of the real-time graph tensor buffer, this register group requires r clock cycles to complete one update.
[0078] S7 moves the data from the front column of the reference register group to the back column, and at the same time reads the data from the reference tensor buffer to the front column of the register group.
[0079] S8 processes and performs calculations across different register groups before writing the results to the tensor register group.
[0080] S8 includes: transforming the data in the reference tensor register group and the real-time tensor register group and writing them into the V tensor register group and the U tensor register group respectively; multiplying the data in the V tensor register group and the U tensor register group point by point and writing them into the E tensor register group.
[0081] The V tensor register set receives the output data from the reference graph transformation unit and provides data to the pointwise multiplication unit. The size of the register set is (m+r-1)×(m+r-1)×P. n .
[0082] The U tensor register set receives the output data from the real-time graph transformation unit and provides data to the pointwise multiplication unit. The size of the register set is (m+r-1)×(m+r-1)×P. n .
[0083] S8 further includes: transforming the data in the E tensor register group and writing it into the S tensor register group; and simultaneously reading intermediate result data from the process tensor buffer into the bias tensor register group.
[0084] The E tensor register set receives the output data from the pointwise multiplication unit and provides data to the result transformation unit. The size of the register set is (m+r-1)×(m+r-1)×P. n .
[0085] The S tensor register group receives the output data from the result transformation unit and provides data to the channel accumulation unit. The size of the register group is m×m×P. n .
[0086] S8 further includes: accumulating the data in the S tensor register group along the channel direction, adding it point by point to the data in the bias register group, and writing it into the process tensor register group.
[0087] The bias tensor register set loads data from the process tensor buffer and provides data to the channel accumulation unit. The size of the register set is m×m×1.
[0088] The process tensor register group receives data from the channel accumulation unit and writes it into the process tensor buffer. The size of the register is m×m×1.
[0089] The process tensor cache is used to cache intermediate results generated by the current computation. Based on the data reuse strategy of this accelerator, the required cache size is C. c ×R c This contains all intermediate result data of the relevant result matrix. Its structure is related to the Winograd algorithm used by the acceleration unit, and is represented as follows:
[0090] Buf C [m][m][D C ]
[0091] For F(m×m, r×r), a total of m×m independent storage units are required, where Buf C This is a cache for the process, where m represents the size of the relevant result matrix block. The width of each storage unit is (1×BIT). C Bit C This represents the bit width of the intermediate result data, meaning the maximum readable and writable size of the related result matrix block per clock cycle in this cache is m×m. The depth of each memory bank is greater than 1 / m of the area of the related result matrix. 2 .
[0092]
[0093] S8 continues until the current real-time tensor block has completed scanning an entire row of data from the reference graph tensor.
[0094] To make it easier to understand, an example is given here:
[0095] If the Winograd algorithm is F(4×4, 3×3), then the size of the relevant result matrix block output each time is 4×4.
[0096] The size of the S1 baseline tensor is 256×256×256, and the size of the real-time tensor is 128×128×256. Therefore, the size of the correlation result matrix is 129×129. The channel parallelism P used by the acceleration unit... n =32.
[0097] S2 caches a 256×6×32 reference graph tensor block from off-chip DDR to the "ping-pong region" cache of the reference tensor cache. For example... Figure 4 As shown, the reference tensor cache has 4×6 independent storage units, each corresponding to one row of the reference tensor, divided into 4 groups, each group corresponding to... Figure 3 Tensor data for columns marked with different colors.
[0098] S3 writes a real-time graph tensor block of size 128×3×256 from off-chip DDR to the real-time tensor cache. After a certain period of calculation, once some data in the cache has been reused, new data is loaded from DDR to overwrite it.
[0099] S4 Figure 4 As shown, during the calculation, while reading data from the "ping-pong region" of the benchmark tensor cache, the last two lines of data read are written to the first two lines of the "pong region". Simultaneously, the next four lines of data are read from DDR and written to the "pong region". This step requires C++. b ×3 clock cycles, performed concurrently with the subsequent calculation steps. Once the data in the "ping-pong area" has been reused, new data is written to the "ping-pong area" in the same way. In this way, the "ping-pong area" continuously reads and writes data in an alternating manner.
[0100] After S2 is completed, S5 reads a tensor block of size 2×6×32 from the third and fourth memory groups and pre-writes it into the first two columns of the reference tensor register group, corresponding to step0 marked by the green arrow in the figure.
[0101] While S6 is running S5, it uses 3 clock cycles to write a 3×3×32 tensor block from the real-time graph tensor buffer to the real-time tensor register.
[0102] S7, such as Figure 4 As shown, after S6 is completed, the first two columns of data in the reference tensor register group are moved to the last two columns, and at the same time, four columns of data are read from the reference tensor buffer and added to the first four columns of the register group.
[0103] S8: Transform the data in the reference tensor register group and the real-time tensor register group and write them into the V tensor register group and the U tensor register group, respectively. Multiply the data in the V tensor register group and the U tensor register group point by point and write them into the E tensor register group. Transform the data in the E tensor register group and write it into the S tensor register group. Read intermediate result data from the process tensor buffer into the bias tensor register group. If it is the first calculation and there is no intermediate result data in the buffer, directly set the register group to zero. Accumulate the data in the S tensor register group along the channel direction, add it point by point with the data in the bias register group, and write it into the process tensor register group. This process ends after the current real-time tensor block has scanned a whole row of data in the reference graph tensor.
[0104] Repeat steps S5-S8 above until all relevant calculations are completed.
[0105] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A Winograd-based correlation algorithm accelerator storage method, characterized by, include: S1 obtains the size of the relevant result matrix block and the real-time graph matrix block, respectively, obtains the size of the relevant result matrix and the real-time graph tensor, and obtains the channel parallelism of the acceleration unit; S2 stores C from off-chip memory to the reference tensor first region b × (m + r - 1) × P n where C b is the reference graph tensor width, m is the width of the correlation result matrix block, r is the width of the real-time graph matrix block, and P n is the channel parallelism of the acceleration unit; S3 is a real-time tensor cache C from off-chip memory r a real-time graph tensor block of size C x R x N, where C r is the width of the real-time graph tensor and N is the number of channels of the real-time graph tensor; S4 reads data from a first region in the reference map tensor cache, writes the last two rows of data read to the first two rows of a second region in the reference map tensor cache, while reading the next four rows from external storage and writing to the second region, repeating C b x r clock cycles; S5 reads a tensor block of size (r-1) x (m+r-1) x P from the reference tensor cache and pre-writes to the reference tensor register group; n S5 reads a tensor block of size (r-1) x (m+r-1) x P from the reference tensor cache and pre-writes to the reference tensor register group; S6 write r x r x P from real-time graph tensor cache to real-time graph tensor graph register n tensor blocks of size S7 moves the data from the front column of the reference register group to the back column, and at the same time reads the data from the reference tensor buffer to the front column of the register group; S8 processes and performs calculations across different register groups before writing the results to the tensor register group.
2. The Winograd-based correlation algorithm accelerator storage method of claim 1, wherein, In S1, the reference tensor cache includes m×(m+r-1) independent storage units, each storage unit corresponding to one row of the reference tensor, and is divided into m groups.
3. The Winograd-based correlation algorithm accelerator storage method of claim 1, wherein, In step S2, when the data in the real-time tensor cache is exhausted, new data is loaded from off-chip storage to overwrite it.
4. The Winograd-based correlation algorithm accelerator storage method of claim 2, wherein, In S5, the reference tensor cache reads a tensor block of size (r-1)x(m+r-1)xP from the m-1 group and the m-2 group, and pre- writes the first r-1 columns to the reference tensor register group. n In S5, the reference tensor cache reads a tensor block of size (r-1)x(m+r-1)xP from the m-1 group and the m-2 group, and pre- writes the first r-1 columns to the reference tensor register group.
5. The Winograd-based correlation algorithm accelerator storage method of claim 1, wherein, S5 is performed after S2.
6. The Winograd-based correlation algorithm accelerator storage method of claim 1, wherein, S6 and S5 are performed simultaneously.
7. The Winograd-based correlation algorithm accelerator storage method of claim 1, wherein, S8 includes: transforming the data in the reference tensor register group and the real-time tensor register group and writing them into the V tensor register group and the U tensor register group respectively; multiplying the data in the V tensor register group and the U tensor register group point by point and writing them into the E tensor register group.
8. The Winograd-based correlation algorithm accelerator storage method of claim 7, wherein, S8 further includes: transforming the data in the E tensor register group and writing it into the S tensor register group; and simultaneously reading intermediate result data from the process tensor buffer into the bias tensor register group.
9. The Winograd-based correlation algorithm accelerator storage method of claim 7, wherein, S8 further includes: accumulating the data in the S tensor register group along the channel direction, adding it point by point to the data in the bias register group, and writing it into the process tensor register group.
10. The Winograd-based correlation algorithm accelerator storage method of claim 1, wherein, S8 continues until the current real-time tensor block has completed scanning an entire row of data from the reference graph tensor.