A Linear Vector Decomposition AI Processing Architecture and Its Application

Abstract

This invention discloses a linear vector factorization (LVM) AI processing architecture, including vector convolution units, vector fully connected units, and vector register accumulation units. There are no data caching units between these units, and no input data caching structure precedes the vector convolution units. Convolution calculations are performed using local convolution windows. The vector register accumulation unit consists of a state selection unit, a register addition unit, and an output selection unit. The state selection unit specifies the storage, accumulation, and output states. By specifying the location of the input data and configuring a valid data address, when the location of the input data matches the valid address, the vector register accumulation unit saves, adds, or outputs the data according to the current input state. The output unit then outputs the calculation result for its respective location; otherwise, the output is 0. This invention reduces the cache time between linear layers and their connections, ensures output integrity, and guarantees real-time data transmission and computation.

Description

Technical Field

[0001] This invention belongs to the field of neural network processing unit architecture technology, specifically relating to a linear vector decomposition AI processing architecture and its usage method. Background Technology

[0002] For computations in artificial intelligence neural networks, parallel operations are mostly performed by units such as NPUs, DSPs, GPUs, and FPGAs. Traditional architectures cache input data row-wise and then use complete convolutional window units for linear computation tasks such as convolution and fully connected layers, processing the output only after the current layer's output is complete before proceeding to the next layer. This architecture suffers from high storage consumption due to the large volume of input data requiring caching and the need for data storage between neural network layers. Furthermore, the fact that each layer completes its processing before the next results in long computation times from input completion to output, coupled with poor pipelined parallelism and consequently, poor real-time performance. In CPU-based parallel processors, this complete-output approach leads to poor real-time performance due to the lack of pipelined parallelism; while in FPGA-based processors, limited computing resources and storage result in poor caching and data parallelism between neural network layers, further hindering the achievement of adequate real-time performance requirements.

[0003] In addition, with the emergence of new neural network architectures, ASICs, represented by NPUs, have fixed structures and low execution efficiency for algorithm tasks that are not suitable for their own processor architecture, thus failing to meet the diverse application scenarios of algorithms. Summary of the Invention

[0004] The purpose of this invention is to provide a linear vector decomposition AI processing architecture and its usage method, which can reduce the cache occupation time of linear layers and linear layer connections, achieve output integrity, and ensure the real-time performance of data transmission and computation.

[0005] To achieve the above objectives, this invention provides a linear vector factorization (LVM) AI processing architecture, comprising a vector convolution unit, a vector fully connected unit, and a vector register accumulation unit. The vector convolution unit has no input data buffer structure before it, and the convolution calculation is performed using a local convolution window. There is no data buffer storage unit between the vector convolution unit, the vector fully connected unit, and the vector register accumulation unit. The vector register accumulation unit consists of a state selection unit, a register addition unit, and an output selection unit. The state selection unit specifies the storage, accumulation, and output states. By specifying the location of the input data, a valid data address is configured. When the location of the input data matches the valid address, the vector register accumulation unit saves, adds, or outputs the data according to the current input state. The output unit outputs the calculation result at its respective location; otherwise, the output is 0.

[0006] As a further aspect of the present invention: the input data of the vector convolution unit is a single-row normalized value of the length of a certain row of the RGB convolution window of the image data. The convolution stride of the convolution layer is the same as the length of the convolution window. There are a total of 4 batches of vector convolution multiply-accumulate arrays forming convolution vector groups in the vector convolution unit. The number of multipliers in each batch of vector convolution multiply-accumulate arrays is the same as the number of single rows of the input RGB convolution window. After the parameter multiplication operation is performed on each batch of vector convolution multiply-accumulate arrays, the addition tree operation is performed and the result is superimposed as the result of the vector multiply-accumulate operation of that batch.

[0007] As a further aspect of the present invention: the number of vector fully connected multiply-accumulate groups in the vector fully connected unit is three times the number of output channels of the convolutional layer, the amount of input data of each vector fully connected multiply-accumulate group matches the output data of the convolutional vector groups connected in front, and each vector fully connected multiply-accumulate group has 4 data inputs.

[0008] As a further aspect of the present invention: the vector register accumulation unit comprises 24 batches of state selection units, register addition units, and selection output units; driven by a clock and a valid flag position, the input data needs to specify the position of the input data and pre-specify the valid position to indicate the valid position. When the position of the input data is the same as the valid position, the data is saved, added, or output. The selection output unit outputs the calculation result of its respective position; otherwise, the output is 0. For different valid addresses, the calculation results of different output units are realized by addition to achieve single data output. Parallel data is converted into serial output in single data form through 0 numerical values ​​and OR logic.

[0009] The above-mentioned method for using a linear vector factorization AI processing architecture includes the following steps:

[0010] S31. Connect the data input and parameter input interfaces of the vector convolution unit to the input image matrix generator, which is used to simulate the generation of output image data; connect the data output of the convolutional layer ordering parameter output unit to the parameter input interface of the vector convolution unit; the matrix control unit controls the data multiplication and addition order of the input image matrix generator and the convolutional layer ordering parameter output unit to maintain consistency; driven by clock and reset signals, the output data consists of data and valid flag buses;

[0011] S32. Connect the parameter input of the vector fully connected unit to the parameter output of the Linear layer, connect the data input of the vector fully connected unit to the data output of the vector convolution unit, control the synchronization of data through the valid data flag bit of the vector convolution unit, and the output of the vector fully connected unit is the valid bit and the calculated data.

[0012] S33. Connect the data input terminal of the vector register accumulation unit to the vector fully connected unit of the preceding stage, and connect the control line to the accumulation processing control unit. The enable of the accumulation processing control unit is controlled in an open loop by the effective signal of the preceding network. The accumulation processing control unit realizes the output of the data storage location superimposed to the correct position. The output of the vector register accumulation unit is the valid data flag bit and the data output.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0014] This invention employs local row vector input computation, effectively eliminating the resource consumption of extensive data caching required by traditional computing architectures before neural network computation. Data is processed in vector form through convolution and fully connected operations. Bandwidth matching between network layers ensures real-time computation and significantly reduces storage consumption between network layers, minimizing cache time between linear layers. After linear layer computation, the dispersed vector forms are accumulated into a complete output structure via a vector register accumulation unit, ensuring computational integrity and correctness. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the linear vector decomposition AI processing architecture of the present invention;

[0016] Figure 2 for Figure 1 Enlarged schematic diagram of the vector convolution unit;

[0017] Figure 3 for Figure 1 Enlarged schematic diagram of the fully connected vector unit;

[0018] Figure 4 for Figure 1 A magnified schematic diagram of the middle vector register accumulation unit;

[0019] Figure 5 It is a vector convolution multiplication and addition array diagram;

[0020] Figure 6 It is a fully connected vector multiply-accumulate group;

[0021] Figure 7 This is a structural diagram of the basic unit in the vector register accumulation unit;

[0022] Figure 8 This is a diagram of the neural network model structure.

[0023] Figure 9 A timing diagram of the input data;

[0024] Figure 10 Input timing graph for convolution parameters;

[0025] Figure 11 This is the output timing diagram for the output layer of the vector register accumulation unit. Detailed Implementation

[0026] The present invention will be further illustrated by the following examples.

[0027] like Figures 1 to 4 As shown, a linear vector factorization (LVM) AI processing architecture includes vector convolution units, vector fully connected units, and vector registered accumulation units. The vector convolution units have no input data buffer structure before them, and convolution calculations are performed using local convolution windows. There are no data buffer storage units between the vector convolution units, vector fully connected units, and vector registered accumulation units. The vector registered accumulation unit consists of a state selection unit, a registered addition unit, and an output selection unit. The state selection unit specifies the storage, accumulation, and output states. By specifying the location of the input data and configuring the valid data address, when the location of the input data matches the valid address, the vector registered accumulation unit saves, adds, or outputs the data according to the current input state, and selects the output unit to output the calculation result at its respective location; otherwise, the output is 0. Different valid addresses can be addressed by using addition to output the calculation results of different output units for a single data point.

[0028] Furthermore, the input data of the vector convolution unit is a single-row normalized value of the length of a certain row of the RGB convolution window in the image data. The convolution stride of the convolutional layer is the same as the convolution window length. The vector convolution unit contains four batches of vector convolution multiply-accumulate arrays forming convolution vector groups. The vector convolution multiply-accumulate array is as follows: Figure 5 As shown, Figure 5 The module name is conv2d; i_clk is the input driving clock; one_line_mul implements the multiplication function; in the input, i_din_b, i_din_g, and i_din_r represent the three RGB components of the normalized image vector; i_weight_b, i_weight_g, and i_weight_r represent the RGB weight parameters of the convolutional layer in the neural network; mult_out_b, mult_out_g, and mult_out_r represent the multiplication output results corresponding to the RGB channels; zonghe_test is the addition unit, which adds mult_out_b, mult_out_g, and mult_out_r to obtain the output add_out; the number of multipliers in each batch of vector convolution multiply-accumulate array is the same as the single-row length of the input RGB convolution window; after the parameter multiplication operation is performed in each batch of vector convolution multiply-accumulate array, the addition tree operation is performed and the result is superimposed as the result of the vector multiply-accumulate operation in that batch.

[0029] Furthermore, the number of vector fully connected multiply-accumulate groups in the vector fully connected unit is three times that of the output channels of the convolutional layer, and the number of vector fully connected multiply-accumulate groups is as follows: Figure 6 As shown, Figure 6 The module is named linear_iclk_1_in_weight, consisting of a multiplication unit linear_mul_1clk_one_p_weight and an addition unit linear_add_1clk_one_p_weight. i_linear_weight and i_relu_date represent the weight parameters and input data, respectively, and o_linear_add represents the result after addition. i_clk and o_valid represent the driving clock and data validity signal flag, respectively. The input data volume of each fully connected vector multiply-add group matches the output data of the previously connected convolution vector groups, and each fully connected vector multiply-add group has 4 data inputs.

[0030] Furthermore, the vector register accumulation unit comprises 24 batches of state selection units, register addition units, and selection output units; such as Figure 7 The diagram shown is a basic unit structure diagram of a vector register accumulation unit. Figure 7 The name of the structural unit is represented as sum_x_one_data_unit_parameterized4; i_clk represents the driving clock signal, i_conv_nobias represents the input data signal, i_din_cnt represents the data position or position number corresponding to the input data signal, i_state represents the status flag signal indicating whether the input data is being written or accumulated, valid represents the valid signal flag bit of the input signal, o_conv_nobias_sum represents the final calculation result after accumulation, and o_valid represents the flag bit indicating that the final accumulated number is complete. In the detailed components, REG, ADD, MUX, ROM, EQ, and AND represent flip-flops, adders, multiplexers, read-only registers, data comparators, and AND gates, respectively. Driven by the clock and the valid flag position, the input data needs to specify the input data position and pre-specify the valid position. When the input data position is the same as the valid position, the data is saved, added, or output, and the output unit is selected to output the calculation result of each position; otherwise, the output is 0. When the valid addresses are different, the calculation results of different output units are achieved by addition, and the parallel data is converted into serial output in single data form through zero value and OR logic.

[0031] The above-mentioned method for using a linear vector factorization AI processing architecture includes the following steps:

[0032] S31. Connect the data input and parameter input interfaces of the vector convolution unit to the input image matrix generator, which is used to simulate the generation of output image data; in the actual working circuit, a camera sensor is used instead; the data output of the convolutional layer ordering parameter output unit is connected to the parameter input interface of the vector convolution unit; the matrix control unit controls the data multiplication and addition order of the input image matrix generator and the convolutional layer ordering parameter output unit to maintain consistency; driven by clock and reset signals, the output data is a data and valid flag bus;

[0033] S32. Connect the parameter input of the vector fully connected unit to the parameter output of the Linear layer, connect the data input of the vector fully connected unit to the data output of the vector convolution unit, control the synchronization of data through the valid data flag bit of the vector convolution unit, and the output of the vector fully connected unit is the valid bit and the calculated data.

[0034] S33. Connect the data input terminal of the vector register accumulation unit to the vector fully connected unit of the preceding stage, and connect the control line to the accumulation processing control unit. The enable of the accumulation processing control unit is controlled in an open loop by the effective signal of the preceding network. The accumulation processing control unit realizes the output of the data storage location superimposed to the correct position. The output of the vector register accumulation unit is the valid data flag bit and the data output.

[0035] Specifically, the image information of the RGB three channels of a single-row vector, the size of the convolution window, is input into four batches of multiplication arrays. These four batches of multiplication arrays have the same input data, but the parameters are the parameters of different channels in the neural network. The output single-row convolution vector is directly connected to the vector fully connected unit as data input. The vector fully connected unit contains 24 groups of vector fully connected multiply-accumulate groups. Each group has the same input data, but the parameters they compute are the parameters of the fully connected network, which are different from each other. The output of the fully connected layer is a 24-data-parallel output. The data is input to the vector register accumulation unit, and then to the 24 batches of state selection units, register addition units, and selection output units for corresponding superposition and output operations.

[0036] The vector convolution unit assumes the input RGB data are as follows:

[0037]

[0038] Assume that the input network parameters of a certain batch of vector convolution multiply-accumulate arrays are W. cn Where n represents the number of the batch of vector convolution multiplication-addition arrays, there are four batches in total, so n can be 1, 2, 3, or 4. Let W cn for:

[0039]

[0040] Let the output of the batch vector convolution multiplication-addition array after passing through the addition tree be yc. n ,but:

[0041]

[0042] The driving clock for the vector convolution unit is 1clk. Computation begins when data is valid. The output of a valid data point for the entire computation unit is yc, where yc is represented as:

[0043]

[0044] When the driving clock of the fully connected vector unit is active, assume the input weight is W. l Within one valid clock cycle, the vector parameters are set to l1, l2, ..., l 24 ,l n Represented as:

[0045] l n =[l n-1 ,l n-2 ,l n-3 ,l n-4 ];

[0046] Therefore, the input weights of the fully connected vector unit are W. l It can be represented as:

[0047]

[0048] Furthermore, the fully connected computation rule, i.e. the matrix multiplication rule, states that if the output data of a fully connected vector unit is yl, then yl is:

[0049] yl = W l ·yc;

[0050]

[0051] The initial and reset values ​​of the vector register accumulator are both 0. When the driving clock is valid, let the input state control be m. When m is 0, it means the input value is stored; when m is 1, it means the original state is superimposed on the input, but the result is incomplete; when m is 2, it means the input vector is complete. The output value of the register accumulator in the previous clock cycle is yo. At this time, the state of the register accumulator is s, and the output at this moment is yo'. Then:

[0052]

[0053] The neural network model structure used in this invention is as follows: Figure 8As shown, the algorithm model is designed with the entire image as RGB three-channel data, with a data tensor size of 3*9*9. The convolutional layer has the same stride and window size, and the convolution parameter tensor size is 8*3*3. After the data input undergoes the convolution operation, the data size is 8*3*3. After dimensionality reduction and concatenation, the model dimension is 9*8. The fully connected data dimension is 24*8, and after passing through the fully connected layer, it becomes 9*24. In the entire neural network, the linear layers do not contain bias terms.

[0054] Example 1:

[0055] The vector convolutional unit has a width of 4, and the convolutional layer has a width of 8. When the input data is valid, two clock cycles need to be kept constant. The same data is processed using two different parameters in two clock cycles. In the concatenated, dimensionality-reduced data tensor, two consecutive clock cycles represent a batch output of one row of data. That is, every two clock cycles, the input RGB data is changed, and each time three consecutive RGB pixels from a certain row of RGB data are transmitted. In each clock cycle, the corresponding positional parameters of the four adjacent width layers of the convolutional layer, matching the input data, are transmitted in parallel. Figure 9 , Figure 10 As shown.

[0056] Example 2:

[0057] The output of the vector convolution unit is used as the data input of the vector fully connected unit. The inputs of the multiply-accumulate units of the vector fully connected unit are the same as the outputs of the vector convolution units. The input parameters for each valid clock cycle are the weights of the fully connected parameters at their respective positions. The cycle repeats every two clock cycles. The output of the vector fully connected unit is then fed to the vector register accumulator. When the image input data is the first row corresponding to the convolution window, the vector register accumulator's control state is 0. When the image input data is the last row corresponding to the convolution window, the vector register accumulator's control state is 2; otherwise, the control state is 1. Data output is calculated when the control state is 2. Once the accumulation calculation is complete (control state 2), the data is considered valid. Figure 11 As shown. The data from rows 1 to 4 is output. The sum of two consecutive valid data points in each row is the result of convolution and fully connected processing, representing one valid data point. Every two clock cycles, one calculation result from the neural network for these four rows is output in parallel. The sum of the results from the vector register accumulation units is completely consistent with the calculation result executed by the software.

Claims

1. A linear vector decomposition AI processing system, comprising a vector convolution unit, a vector fully connected unit, and a vector register accumulation unit, characterized in that, There is no input data buffer structure before the vector convolution unit. The convolution calculation is performed through a local convolution window. There is no data buffer storage unit between the vector convolution unit, the vector fully connected unit, and the vector registered accumulator unit. The vector registered accumulator unit consists of a state selection unit, a registered addition unit, and an output selection unit. The state selection unit is used to specify the storage, accumulation, and output states. By specifying the position of the input data, a valid data address is configured. When the position of the input data matches the valid address, the vector registered accumulator unit saves, adds, or outputs the data according to the current input state. The output selection unit outputs the calculation result at its respective position; otherwise, the output is 0.

2. The linear vector decomposition AI processing system according to claim 1, characterized in that, The input data of the vector convolution unit is the normalized value of a single row of the RGB convolution window of the image data. The convolution stride of the convolutional layer is the same as the length of the convolution window. The vector convolution unit is composed of 4 batches of vector convolution multiply-accumulate arrays. The number of multipliers in each batch of vector convolution multiply-accumulate arrays is the same as the number of single rows of the input RGB convolution window. After the parameter multiplication operation is performed on each batch of vector convolution multiply-accumulate arrays, the result is superimposed after the addition tree operation and output as the result of the vector multiply-accumulate operation of that batch.

3. The linear vector decomposition AI processing system according to claim 2, characterized in that, The number of vector fully connected multiply-accumulate groups in the vector fully connected unit is three times the number of output channels of the convolutional layer. The amount of input data for each vector fully connected multiply-accumulate group matches the output data of the previously connected vector convolutional multiply-accumulate arrays. Each vector fully connected multiply-accumulate group has 4 data inputs.

Images