An input data processing method based on non-continuous fc layer address of NNA

Abstract

The application provides an input data processing method based on fc layer address non-continuity of NNA, which carries redundant waste data through DMA to ensure that the carried data is 64-byte aligned, stores the carried data into ORAM space, and is used for NNA calculation. The input FeatureMap byte alignment problem caused by carrying data by using DMA is overcome, and the data is correctly carried by using DMA in the case of full connection layer data row alignment. The weight and input data are calculated in blocks by rows, so as to reduce the waiting time between data carrying and calculation.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a method for processing input data with non-contiguous addresses in the fully connected (FC) layer based on neural networks (NNA). Background Technology

[0002] The increasing prevalence of neural network technology has led chip manufacturers to create dedicated chips for neural network algorithms, especially inference chips, also known as Neural Network Accelerators (NNAs). In current technology, NNA 1.0 supports fast multi-channel convolution operations. A convolutional neural network mainly consists of an input layer, convolutional layers, pooling layers, and fully connected layers. The fully connected layer (fc) implements the logic: input data is multiplied by corresponding weights to obtain a number, which is then stored in the output space. The rules for DMA data transfer are that the source and destination addresses of the data transfer must be 64-bit aligned, and the amount of data transferred in DMA must be 64-byte aligned. Only when all three conditions are met can the correctness of the DMA data be guaranteed. The input layer operator uses line-by-line DMA data transfer, that is, each line of the input FeatureMap is treated as a DMA data block, and the input FeatureMap data required for convolution is transferred to the destination storage space. However, the input FeatureMap data of the input layer is aligned to 64 (iline_size). Continuous DMA data transfer, in order to ensure 64-byte data alignment, may result in invalid data at the end of the line participating in the calculation, leading to incorrect calculation results.

[0003] In addition, the commonly used terms in the prior art are as follows:

[0004] 1. NNA: A hardware accelerator on the CPU's SIMD pipeline. Its operation is controlled by special CPU / SIMD instructions and runs on a single thread. It is used to solve most convolution multiplication and addition operations.

[0005] 2. Fully connected layer (FC): This layer maps the features learned by the previous network to the sample label space and mainly plays a classification role.

[0006] 3. Feature_shape: The shape of the input data, which can be abstractly understood as multi-dimensional data, from left to right from high dimension to low dimension (NDHWC). N: The number of FeatureMaps processed at one time, set to 1; D: The number of input channels split into groups of 32; H: The height of the input data; W: The width of the input data; C: 32.

[0007] 4. Line alignment: Due to hardware limitations, the size of each processed data must be 64 bytes aligned. align64(iline_size) = (iline_size + 63) / 64 * 64.

[0008] 5. DMA: Short for Direct Memory Access, DMA transfer copies data from one address space to another, providing high-speed data transfer between peripherals and memory or between memory devices.

[0009] 6. ORAM: On-chip general-purpose RAM, with a size of 872kb per board.

[0010] 7. MAC: Convolutional multiplication and summation calculation based on NNA

[0011] 8. inline_size: This CSS property affects the width and height of an element to change the horizontal or vertical size of the element's box model. Summary of the Invention

[0012] To address the aforementioned issues, the purpose of this application is to overcome the input FeatureMap byte alignment problem caused by using DMA to move data, and to ensure correct data movement using DMA even with row-level data alignment in fully connected layers. Weights and input data are calculated in row-by-row blocks, reducing the waiting time between data movement and computation.

[0013] Specifically, this invention provides a method for processing non-contiguous input data at the FC layer based on NNA. In this method, address non-contiguousness requires address row alignment, and address jumps between rows need to be recalculated. A fixed number of rows are processed each time to ensure correct input address jumps. DMA is used to move redundant data to ensure 64-byte alignment of the moved data. The redundant data refers to data where the data at the end of each row is less than 64 bytes but 64 bytes need to be moved to ensure normal hardware operation. The moved data is stored in ORAM space for NNA calculation.

[0014] The method further includes:

[0015] S1, Initialization Settings:

[0016] Let the height H * width W * input channel IC of the FeatureMap be 3 * 3 * 256; feature_shape [1, 8, 3, 3, 32]; where, Feature_shape: the shape of the input data, which can be abstractly understood as multi-dimensional data, from left to right from high dimension to low dimension: NDHWC; where, N: the number of FeatureMaps processed at one time, set to 1;

[0017] D: The number of input channels that are divided into groups of 32;

[0018] H: High level of the input data;

[0019] W: Width of input data;

[0020] C: Input channel, divided into groups of 32;

[0021] S2: Calculate iline_size,

[0022] iline_size = align64(w * 32 * bit_width / 8); Since the addresses are not contiguous, we need to consider the address jumps between rows. The offset for each jump is defined as iline_size: the size of the input row in bytes; Due to hardware limitations, the size of the data processed each time must be guaranteed to be 64 bytes aligned.

[0023] S3: Grouped volume accumulation,

[0024] S3.1: Based on the ORAM space size, group the height values, as shown in the formula:

[0025] sub_h=oram_size / iline_size;

[0026] Group_num = (height + sub_h - 1) / sub_h; that is, under space constraints, the number of rows processed each time is looped; the height is obtained by grouping and looping.

[0027] This method ensures that all channels participate in the calculation;

[0028] DMA moves data sub_h*align64(iline_size); that is, the size of each line * the number of lines to be processed = the size of the data in bytes calculated each time;

[0029] 1 2 3 0 4 5 6 0 7 8 9 0

[0030] The table above is the FeatureMap, bit_width = 8, assuming the data bit width is 8 bits; as shown in the table above, the data rows are aligned, 0 is redundant data, and the data is not stored contiguously in order to ensure 64-byte alignment; S3.2: Based on NNA, the height is grouped, MAC is calculated for each row of pixels, and the MAC results are accumulated each time to obtain the final convolution result of all pixels;

[0031] Pixel_num=sub_h*align64(iline_size);

[0032] MAC: Accumulation: sum = temp0 + temp1 + ... + tempn

[0033] Result = sum;

[0034] S4: In the last sub_h loop, check if the sub_h loop is the last one. If it is, then sub_h = height % sub_h, and there is no need to move any extra elements. If not, the value of sub_h remains unchanged.

[0035] In step S3.1, when the ORAM space is insufficient to hold all the data, the input data is transported in groups, and the maximum space that can be loaded each time is transported. Conversely, if all the data can be placed, the data is transported to the ORAM space by DMA in one go.

[0036] Therefore, the advantage of this application is that it solves the problem of NNA-based convolution computation without row alignment by using a simple method, even when the input layer addresses are not contiguous. Attached Figure Description

[0037] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0038] Figure 1 This is a schematic diagram of the framework involving NNA, ORAM and DDR in this application.

[0039] Figure 2 This is a schematic diagram of the FeatureMap involved in this application.

[0040] Figure 3 This is a flowchart illustrating the method described in this application. Detailed Implementation

[0041] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0042] Due to certain rigid limitations, NNA only supports multi-channel convolution, limiting its versatility in practical applications. NNA 1.0 actually supports fast computation of ordinary unsigned input and the multiplication and accumulation of ordinary unsigned weights, while multi-channel convolution is the result of multiplying and accumulating ordinary unsigned input and ordinary signed weights. In actual use, configuring the relevant NNA registers is necessary to quickly obtain the correct convolution result.

[0043] This invention mainly uses DMA to move redundant and useless data to ensure that the moved data is aligned to 64 bytes, and stores the moved data in ORAM space for NNA calculation.

[0044] FRAM: Input data space; WRAM: Weight data space, such as... Figure 1 As shown.

[0045] The input data is stored from ORAM to FRAM, and the weight data is stored from ORAM to WRAM. After NNA calculation, the convolution result is obtained.

[0046] Example: The H, W, and IC of the FeatureMap are 3*3*256 respectively; feature_shape[1,8,3,3,32], such as Figure 2 As shown, the data is arranged in three dimensions, with address jumps in rows.

[0047] like Figure 3 As shown: This method further includes the following steps:

[0048] S1, Initialization Settings:

[0049] Let the height H * width W * input channel IC of the FeatureMap be 3 * 3 * 256; feature_shape [1, 8, 3, 3, 32]; where, Feature_shape: the shape of the input data, which can be abstractly understood as multi-dimensional data, from left to right from high dimension to low dimension: NDHWC; where, N: the number of FeatureMaps processed at one time, set to 1;

[0050] D: The number of input channels that are divided into groups of 32;

[0051] H: High level of the input data;

[0052] W: Width of input data;

[0053] C: Input channel, divided into groups of 32;

[0054] S2: Calculate iline_size,

[0055] iline_size = align64(w * 32 * bit_width / 8); Since the addresses are not contiguous, we need to consider the address jumps between rows. The offset for each jump is defined as iline_size: the size of the input row in bytes; Due to hardware limitations, the size of the data processed each time must be guaranteed to be 64 bytes aligned.

[0056] S3: Grouped volume accumulation,

[0057] S3.1: Based on the ORAM space size, group the height values, as shown in the formula:

[0058] sub_h=oram_size / iline_size;

[0059] Group_num = (height + sub_h - 1) / sub_h; that is, under space constraints, the number of rows processed each time is looped; the height is obtained by grouping and looping.

[0060] This method ensures that all channels participate in the calculation;

[0061] DMA moves data sub_h*align64(iline_size); that is, the size of each line * the number of lines to be processed = the size of the data in bytes calculated each time;

[0062] 1 2 3 0 4 5 6 0 7 8 9 0

[0063] The table above is the FeatureMap, bit_width = 8, assuming the data bit width is 8 bits; as shown in the table above, the data rows are aligned, 0 is redundant data, and the data is not stored contiguously to ensure 64-byte alignment; S3.2: Based on NNA (hardware acceleration to implement convolution calculation logic), the height is grouped, MAC calculation is performed on each row of pixels, and the MAC results are accumulated each time to obtain the final convolution result of all pixels;

[0064] Pixel_num=sub_h*align64(iline_size);

[0065] MAC:

[0066] Accumulation: sum = temp0 + temp1 + ... + tempn

[0067] Result = sum;

[0068] S4: The last sub_h loop,

[0069] Check if the sub_h loop is in its last iteration. If it is, then sub_h = height % sub_h, and there is no need to move any extra elements. If not, the value of sub_h remains unchanged.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for processing non-contiguous input data at the FC layer based on NNA, characterized in that, The method requires address row alignment for non-contiguous addresses, and the jump between row addresses needs to be recalculated. A fixed number of rows are processed each time to ensure correct input address jumps. DMA is used to move redundant data to ensure 64-byte alignment of the moved data. The redundant data refers to data where the data at the end of each row is less than 64 bytes but 64 bytes need to be moved to ensure normal hardware operation. The moved data is stored in ORAM space for NNA calculation. The method specifically includes: S1, Initialization Settings: Let the height H * width W * input channel IC of the FeatureMap be 3 * 3 * 256; feature_shape [1,8,3,3,32]; where, Feature_shape: the shape of the input data, which can be abstractly understood as multi-dimensional data, from left to right from high dimension to low dimension: NDHWC; where, N: the number of FeatureMaps processed at one time, set to 1; D: The number of input channels that are divided into groups of 32; H: High level of the input data; W: Width of input data; C: Input channel, divided into groups of 32; S2: Calculate iline_size, iline_size = align64(w * 32 * bit_width / 8); Since the addresses are not contiguous, we need to consider the address jumps between lines. The offset for each jump is defined as iline_size: the size of the input line in bytes; Due to hardware limitations, the size of each piece of data processed must be 64-byte aligned. S3: Grouped volume accumulation, S3.1: Based on the ORAM space size, group the height values, as shown in the formula: sub_h = oram_size / iline_size; Group_num = (height + sub_h - 1) / sub_h; that is, under space constraints, the number of rows processed each time is looped; the height is obtained by grouping and looping. This method ensures that all channels participate in the calculation; DMA moves data sub_h * align64(iline_size); that is, the size of each line * the number of lines to be processed = the size of the data in bytes calculated each time; S3.2: Based on NNA, group by height, perform MAC calculation on each row of pixels, and accumulate the MAC results to obtain the final convolution result of all pixels; Pixel_num = sub_h * align64(iline_size); MAC: Accumulation: sum = temp0 + temp1 + ... + tempn Result = sum; S4: The last sub_h loop, Check if the sub_h loop is in its last iteration. If it is, then sub_h = height % sub_h, and there is no need to move any extra elements. If not, the value of sub_h remains unchanged.

2. The method for processing non-contiguous input data at the FC layer based on NNA according to claim 1, characterized in that, In step S3.1, when the ORAM space is insufficient to hold all the data, the input data is moved in groups, and the maximum space that can be loaded each time is moved. Conversely, if the ORAM space can hold all the data, the data is moved to the ORAM space by DMA in one go.

Images