A convolution acceleration method and convolution hardware accelerator

Abstract

The present invention provides a convolution acceleration method and a convolution hardware accelerator, and relates to the field of hardware design. The convolution hardware accelerator includes a forward conversion module, an FNT module, a matrix dot multiplication module, an IFNT module, and an inverse conversion module. Based on the linear convolution property of the Fermat number transform, the convolution hardware accelerator is used to implement a forward Fermat number transform on two conversion matrices, and then perform dot multiplication on the result, and then perform an inverse Fermat number transform, thereby utilizing hardware resources to achieve convolution acceleration. Moreover, when the convolution hardware accelerator implements convolution acceleration, bit splitting, inversion, and bit splicing operations are used to replace the multiplication and modulo operations in the prior art during the forward Fermat number transform process and the inverse Fermat number transform process, thereby accelerating the convolution speed and effectively reducing the complexity of hardware implementation and the consumption of hardware resources.

Description

Technical Field

[0001] The present invention relates to the field of hardware design, and in particular to a convolution acceleration method and a convolution hardware accelerator. Background Art

[0002] Artificial intelligence (AI) technologies are widely used in the current field of image classification and recognition. Convolutional neural networks (CNNs) are one of the most widely used and efficient algorithms. As a key model in deep learning, CNNs are a deep learning method specifically designed for image classification and recognition, developed based on multi-layer neural networks. Compared to traditional image recognition algorithms, CNNs achieve higher accuracy in image classification tasks. Because convolutional layers account for the majority of CNN computation, and convolution is the primary operation, optimizing convolution operations can significantly impact CNN performance.

[0003] In order to improve the computing speed of convolutional neural networks, the existing technology proposes a hardware acceleration architecture based on Fermat Number Transform (FNT) to accelerate convolution calculations. This has become a solution to accelerate convolution operations in convolutional neural networks with lower hardware power consumption, and perform image recognition in a more efficient and high-speed manner.

[0004] Among them, when using the hardware acceleration architecture in the existing technology to achieve convolution acceleration, multiple matrix multiplication operation units are used in the Fermat number transform calculation process, the connection and use are relatively complicated, and multiple matrix multiplication operations and modulo operations are required in the Fermat number transform calculation. This makes the hardware implementation complexity of the hardware acceleration architecture higher and the hardware resource consumption is larger. Summary of the Invention

[0005] The purpose of the present invention is to provide a convolution acceleration method and a convolution hardware accelerator to improve the problems existing in the prior art.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] In a first aspect, the present invention provides a convolution acceleration method applied to a convolution hardware accelerator, wherein the convolution hardware accelerator includes: a forward conversion module, an FNT module, a matrix dot multiplication module, an IFNT module, and an inverse conversion module; the method includes:

[0008] The forward conversion module performs forward conversion processing on the first input matrix and the second input matrix in sequence according to a preset forward conversion formula to obtain a first conversion matrix and a second conversion matrix;

[0009] The FNT module sequentially performs positive Fermat number transformation on the first conversion matrix and the second conversion matrix according to a preset operation rule to obtain a transformed first conversion matrix and a transformed second conversion matrix; the preset operation rule includes a bit splitting operation, a negation operation, and a bit splicing operation;

[0010] The matrix dot multiplication module performs dot multiplication on the transformed first conversion matrix and the transformed second conversion matrix to obtain a dot product result matrix;

[0011] The IFNT module performs an inverse Fermat transformation on the dot product result matrix according to the preset operation rules to obtain an inverse dot product result matrix;

[0012] The inverse conversion module performs inverse conversion processing on the inverse dot product result matrix according to a preset inverse conversion formula to obtain a target matrix; the target matrix represents the convolution result of the first input matrix and the second input matrix.

[0013] Optionally, the forward conversion module sequentially performs forward conversion processing on the first input matrix and the second input matrix according to a preset forward conversion formula to obtain the first conversion matrix and the second conversion matrix, including:

[0014] The forward conversion module converts each element of the target input matrix according to the conversion formula to obtain a conversion matrix; wherein the target input matrix is the first input matrix or the second input matrix, and the conversion formula is:

[0015]

[0016] Where x is an element in the target input matrix, x new is the element corresponding to the element x in the target conversion matrix, and m represents the preset binary bit width;

[0017] The forward conversion module converts each element in the conversion matrix into binary data to obtain a target conversion matrix, where the target conversion matrix is the first conversion matrix or the second conversion matrix.

[0018] Optionally, the FNT module includes a positive row transformation module and a positive column transformation module;

[0019] The FNT module sequentially performs positive Fermat number transformation on the first conversion matrix and the second conversion matrix according to a preset operation rule to obtain the transformed first conversion matrix and the transformed second conversion matrix, including:

[0020] The forward row transformation module cuts the target conversion matrix by columns to obtain a plurality of column matrices of the target conversion matrix; the target conversion matrix is the first conversion matrix or the second conversion matrix;

[0021] The forward row transformation module processes each column matrix of the target transformation matrix based on the preset operation rule and the forward transformation matrix to obtain a forward operation matrix corresponding to each column matrix of the target transformation matrix;

[0022] The positive row transformation module performs horizontal splicing on all positive operation matrices of the target conversion matrix to obtain a target row transformation matrix;

[0023] The column-wise transformation module cuts the target row transformation matrix by columns to obtain a plurality of column matrices of the target row transformation matrix;

[0024] The positive column transformation module processes each column matrix of the target row transformation matrix based on the preset operation rule and the positive transformation matrix to obtain a positive operation matrix corresponding to each column matrix of the target row transformation matrix;

[0025] The positive column transformation module vertically splices all positive operation matrices of the target row transformation matrix to obtain a positive transformation result matrix; the positive transformation result matrix is the first transformation matrix after the transformation or the second transformation matrix after the transformation.

[0026] Optionally, the column matrix is an n×1 matrix; the forward transformation matrix is an n-order square matrix; and the forward operation matrix is a 1×n-order matrix.

[0027] Among them, the calculation process of the positive operation matrix corresponding to the target column matrix is:

[0028] Construct n data pairs based on the i-th row in the forward transformation matrix and the target column matrix; wherein the target column matrix is any column matrix of the target conversion matrix or any column matrix of the target row transformation matrix; a data pair includes a reference value and a binary sequence to be processed, wherein the reference value is any target element in the i-th row in the forward transformation matrix, and the binary sequence to be processed is an element in the target column matrix that matches the position of the target element;

[0029] For each of the data pairs, processing the data pair based on the preset operation rule to obtain a conversion sequence corresponding to each of the data pairs;

[0030] A result sequence is determined according to n conversion sequences and a sum sequence of the n conversion sequences, and the result sequence is used as an element of the i-th column of the positive operation matrix corresponding to the target column matrix, where i=1, 2, ... n.

[0031] Optionally, the IFNT module includes an inverse column transformation module and an inverse row transformation module;

[0032] The IFNT module performs an inverse Fermat number transformation on the dot product result matrix according to the preset operation rule to obtain an inverse dot product result matrix, including:

[0033] The inverse column transformation module cuts the dot product result matrix by rows to obtain a plurality of row matrices of the dot product result matrix;

[0034] The inverse column transformation module processes each row matrix of the dot product result matrix based on the preset operation rule and the inverse transformation matrix to obtain an inverse operation matrix corresponding to each row matrix of the dot product result matrix;

[0035] The inverse column transformation module vertically splices all inverse operation matrices of the dot product result matrix to obtain a target column transformation matrix;

[0036] The inverse transformation module cuts the target column transformation matrix by rows to obtain a plurality of row matrices of the target column transformation matrix;

[0037] The inverse transformation module processes each row matrix of the target column transformation matrix based on the preset operation rule and the inverse transformation matrix to obtain an inverse operation matrix corresponding to each row matrix of the target column transformation matrix;

[0038] The inverse transformation module horizontally splices all inverse operation matrices of the target column transformation matrix to obtain the inverse point product result matrix.

[0039] Optionally, the row matrix is a 1×n matrix; the inverse transformation matrix is an n-order square matrix; and the inverse operation matrix is an n×1 matrix;

[0040] Among them, the calculation process of the inverse operation matrix corresponding to the target row matrix is:

[0041] Construct n data pairs based on the j-th column in the inverse transformation matrix and the target row matrix; wherein the target row matrix is any row matrix of the dot product result matrix or any row matrix of the target column transformation matrix; a data pair includes a reference value and a binary sequence to be processed, wherein the reference value is any target element in the j-th column of the forward transformation matrix, and the binary sequence to be processed is an element in the target row matrix that matches the position of the target element;

[0042] For each of the data pairs, processing the data pair based on the preset operation rule to obtain a conversion sequence corresponding to each of the data pairs;

[0043] A result sequence is determined according to n conversion sequences and a sum sequence of the n conversion sequences, and the result sequence is used as an element of the j-th row of the inverse operation matrix corresponding to the target row matrix, where j=1, 2, ... n.

[0044] Optionally, the control value is 2 k or 2 -k , k∈[0,8], the bit width of the binary sequence to be processed is 9;

[0045] The step of processing the data pair based on the preset operation rules includes:

[0046] When k=0, the to-be-processed binary sequence of the data pair is used as the conversion sequence of the data pair;

[0047] When k≠0, the value position is determined based on the comparison value of the data pair; the value position is from the (8-k)th bit to the 7th bit from right to left in the binary sequence to be processed of the data pair;

[0048] Eliminating a specific sequence at a value position in the binary sequence to be processed to obtain a spliced sequence;

[0049] Inverting the specific sequence bit by bit to obtain a specific inverted sequence;

[0050] The specific inverted sequence is spliced to the spliced sequence to obtain a conversion sequence of the data pair.

[0051] Optionally, the step of determining a result sequence based on the n conversion sequences and the sum sequence of the n conversion sequences includes:

[0052] Adding the n conversion sequences to obtain the sum sequence;

[0053] Performing a preset operation on the sum sequence to obtain an intermediate sequence;

[0054] Determine a first control signal and a second control signal based on the intermediate sequence and the n conversion sequences, respectively; wherein the first control signal is the highest bit of the sum sequence, and the second control signal is the result of a bitwise AND operation on the highest bits of the n conversion sequences;

[0055] If the second control signal is at a high level, a preset sequence is used as the result sequence; wherein the bit width of the preset sequence is the same as the bit width of the conversion sequence, the most significant bit is 1, and the remaining bits are all 0;

[0056] If the second control signal is at a low level and the first control signal is at a low level, taking the intermediate sequence as the result sequence;

[0057] If the second control signal is at a low level and the first control signal is at a high level, a result of subtracting one from the intermediate sequence is used as the result sequence.

[0058] Optionally, the inverse conversion module performs inverse conversion processing on the inverse dot product result matrix according to a preset inverse conversion formula to obtain a target matrix, including:

[0059] The inverse conversion module converts each element in the inverse dot multiplication result matrix into decimal data;

[0060] The inverse conversion module converts each element of the inverse dot product result matrix according to the inverse conversion formula to obtain the target matrix; the inverse conversion formula is:

[0061]

[0062] Among them, y new is an element in the inverse dot multiplication result matrix, and y is the same as the element y in the target matrix. new The corresponding element, m represents the preset binary bit width.

[0063] In a second aspect, the present invention provides a convolution hardware accelerator, the convolution hardware accelerator comprising: a forward conversion module, an FNT module, a matrix dot multiplication module, an IFNT module and an inverse conversion module;

[0064] The forward conversion module is used to perform forward conversion processing on the first input matrix and the second input matrix according to a preset forward conversion formula to obtain a first conversion matrix and a second conversion matrix;

[0065] The FNT module is used to perform positive Fermat number transformation on the first conversion matrix and the second conversion matrix according to a preset operation rule to obtain a transformed first conversion matrix and a transformed second conversion matrix;

[0066] The matrix dot product module is used to perform dot product on the transformed first conversion matrix and the transformed second conversion matrix to obtain a dot product result matrix;

[0067] The IFNT module is used to perform an inverse Fermat transformation on the point product result matrix according to the preset operation rules to obtain an inverse point product result matrix;

[0068] The inverse transformation module is used to perform inverse transformation processing on the inverse dot product result matrix according to a preset inverse transformation formula to obtain a target matrix; the target matrix represents the convolution result of the first input matrix and the second input matrix.

[0069] Compared to the prior art, embodiments of the present invention provide a convolution acceleration method and convolution hardware accelerator. The convolution hardware accelerator includes a forward transformation module, an FNT module, a matrix dot product module, an IFNT module, and an inverse transformation module. The forward transformation module first performs forward transformation processing on two input matrices in sequence according to a preset forward transformation formula to obtain two transformation matrices. The FNT module then performs positive Fermat transforms on the two transformation matrices in sequence according to preset operation rules to obtain two transformed transformation matrices. The matrix dot product module then performs dot product on the two transformed transformation matrices to obtain a dot product result matrix. The IFNT module then performs an inverse Fermat transform on the dot product result matrix according to preset operation rules to obtain an inverse dot product result matrix. Finally, the inverse transformation module performs an inverse transform on the inverse dot product result matrix according to a preset inverse transformation formula to obtain a target matrix representing the convolution result between the two input matrices. Based on the linear convolution property of the Fermat transform, the convolution hardware accelerator performs positive Fermat transforms on the two transformation matrices, then performs dot product, and then performs an inverse Fermat transform on the result, thereby achieving convolution acceleration using hardware resources. In addition, when the convolution hardware accelerator implements convolution acceleration, it uses bit splitting, inversion, and bit splicing operations to replace multiplication and modulo operations in the process of positive Fermat number transform and inverse Fermat number transform, which speeds up the convolution speed and can effectively reduce the complexity of hardware implementation and the consumption of hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 A logical diagram of the linear convolution property of a Fermat number transform.

[0072] Figure 2 Schematic diagram of the structure of the hardware acceleration architecture in the prior art.

[0073] Figure 3 Schematic diagram of the process of performing positive Fermat number transformation on the input matrix X in the prior art.

[0074] Figure 4 Schematic diagram of the process of processing the column matrix X1 during the positive Fermat number transformation in the prior art.

[0075] Figure 5 The figure is a schematic diagram of the process of performing an inverse Fermat number transformation on a point product matrix in the prior art.

[0076] Figure 6A schematic diagram of the hardware modules of a convolution hardware accelerator provided in an embodiment of the present invention.

[0077] Figure 7 A flowchart of a convolution method provided by an embodiment of the present invention.

[0078] Figure 8 A schematic diagram of a logic circuit of a preset positive conversion formula provided in an embodiment of the present invention.

[0079] Figure 9 A schematic diagram of a process for performing a positive Fermat number transformation on a target transformation matrix provided by an embodiment of the present invention.

[0080] Figure 10 A schematic diagram of a process for performing an inverse Fermat transformation on a matrix resulting from a point product is provided in an embodiment of the present invention.

[0081] Figure 11 A schematic diagram of a logic circuit for obtaining a conversion sequence from a binary sequence to be processed is provided in an embodiment of the present invention.

[0082] Figure 12 A schematic diagram of a processing process for obtaining a conversion sequence from a binary sequence to be processed is provided in an embodiment of the present invention.

[0083] Figure 13 An exemplary schematic diagram of obtaining a conversion sequence from a binary sequence to be processed is provided in an embodiment of the present invention.

[0084] Figure 14 A schematic diagram of a logic circuit for obtaining a result sequence from n conversion sequences in an FNT module or an FNT module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0086] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0087] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0088] In addition, the terms "first", "second", etc., if used, are merely used to distinguish and describe, and should not be understood as indicating or implying relative importance.

[0089] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0090] Fermat number transform has the property of linear convolution: two input sequences are transformed into two positive sequences in a finite field after the positive Fermat number transform, and the inverse sequence of the two positive sequences after the point multiplication and the inverse Fermat number transform is equal to the convolution result of the two input sequences. Figure 1 , the formula of this property is expressed as follows:

[0091] y(n)=F -1 {F(x(n))×F(h(n))}

[0092] y(n)=x(n)*h(n)

[0093] Among them, x(n) represents an input sequence, h(n) represents another input sequence, F() represents the positive Fermat number transformation, F -1 () represents the inverse Fermat transformation, × represents the dot multiplication operation, * represents the convolution operation; y(n) represents the inverse sequence.

[0094] In conjunction with the background technology, this article first briefly introduces the process of implementing convolution acceleration using the hardware acceleration architecture of the existing technology.

[0095] The structural diagram of the existing hardware acceleration architecture is as follows Figure 2 As shown, the hardware acceleration architecture 100 includes a two-dimensional FNT module, a multiplier module, and a two-dimensional IFNT (Inverse Fermat Number Transform) module.

[0096] Among them, the two-dimensional FNT module is used to perform positive Fermat transform on the two N×N input matrices X and H in sequence to obtain two positive matrices; the multiplier module is used to perform point multiplication on the two positive matrices X and H to obtain a point product matrix; finally, the two-dimensional IFNT module performs an inverse Fermat transform on the point product matrix to obtain the inverse transformation matrix of the point product matrix, which is the convolution result between the input matrices X and H.

[0097] In the hardware acceleration architecture 100 , the implementation logic of the two-dimensional FNT module for performing positive Fermat number transformation on two input matrices X and H is the same. Here, the input matrix X is taken as an example to introduce the positive Fermat number transformation process in the prior art.

[0098] Assume that the input matrix X is of order 4×4, represented as X 4×4 , please refer to Figure 3 , Figure 3 Schematic diagram of the process of performing positive Fermat number transformation on the input matrix X in the prior art.

[0099] Assume that the preset Fermat number is 257, and the positive transformation matrix A is required in the process of positive Fermat number transformation 4×4 , A 4×4 The element in row i and column j is a ij :

[0100] a ij =2 k mod 257

[0101] Where k = (i-1)*(j-1)*α, 2 α ={2,4,8,…,256} and α is a positive integer, mod represents the modulo operation. Figure 3 , the process of performing positive Fermat number transformation on the input matrix X includes the following two steps:

[0102] The first step is to use the positive transformation matrix A 4×4 For the input matrix X 4×4 Processing is performed to obtain a row transformation matrix Y 4×4 The process includes the following (1) to (3):

[0103] (1) First, X 4×4 Cut by column to obtain 4 column matrices X1 to X4, each of which is 4×1 order;

[0104] (2) For any column matrix (denoted as XN), multiply the forward transformation matrix by the column matrix to obtain a 4×1 matrix XN-1. Modulate each element in XN-1 by 257 to obtain a new 4×1 matrix XN-2. Use the 4 elements of the new matrix XN-2 as a 1×4 row matrix XN-3. For example, taking the column matrix X1 as an example, the process of obtaining a 1×4 row matrix X1-3 from the column matrix X1 can be as follows: Figure 4 As shown, so, Figure 4 The operations in include multiplication and modulo operations.

[0105] (3) The four column matrices X1 to X4 are processed according to (2) to obtain four row matrices X1-3 to X4-3. Then, the four row matrices X1-3 to X4-3 are horizontally spliced to obtain the row transformation matrix Y 4×4 .

[0106] Step 2: Continue to refer to Figure 3 , using the positive transformation matrix A 4×4 Row transformation matrix Y 4×4 , get the input matrix X 4×4 A positive transformation matrix Z 4×4 This process is similar to the first step (1) to (3), including the following (4) to (6):

[0107] (4) First, Y 4×4 Cut by column to obtain 4 column matrices Y1 to Y4, each of which is 4×1 order;

[0108] (5) For any column matrix (denoted as YN), multiply the forward transformation matrix by the column matrix to obtain a 4×1 matrix YN-1. Modulo 257 is applied to each element in YN-1 to obtain a 4×1 new matrix YN-2. The four elements of the new matrix YN-2 are used as a 1×4 row matrix YN-3. Figure 4 Processing logic.

[0109] (6) The four column matrices Y1 to Y4 are processed according to (2) to obtain four row matrices Y1-3 to Y4-3. Then, the four row matrices Y1-3 to Y4-3 are horizontally spliced to obtain the forward transformation matrix Z. 4×4 .

[0110] Therefore, in the prior art, in the process of positive Fermat number transformation, multiplication and modulo operations are required twice (in the above steps (1) and (5)).

[0111] The process of performing inverse Fermat transformation on the dot product matrix is similar to the processing logic of steps (1) to (6) above, but the processing method is opposite. Figure 5 , the process of performing the inverse Fermat number transformation on the point product matrix is as follows:

[0112] The first step is to use the inverse transformation matrix to process the dot product matrix to obtain a column transformation matrix: first, cut the dot product matrix by row to obtain 4 row matrices, then for each row matrix, multiply the row matrix by the inverse transformation matrix and take each element modulo 257 to obtain a new matrix of order 1×4. The new matrices of the 4 row matrices are spliced vertically to obtain the column transformation matrix;

[0113] The second step is to use the forward transformation matrix to process the column transformation matrix to obtain the inverse transformation matrix of the dot product matrix. This process is similar to the first step and will not be repeated here.

[0114] Similarly, in the prior art, the inverse Fermat transform process also requires two multiplication and modulo operations. Furthermore, while the input matrix in the above example is 4×4, in actual applications, the input matrix can also be 16×16 or 32×32. The larger the input matrix, the more computing power and hardware resources are required for the multiplication and modulo operations.

[0115] In summary, the multiplication and modulo operations involved in the hardware acceleration architecture 100 of the prior art will result in a large amount of computational cost being consumed, and the logic circuit implementation is highly complex, consuming a large amount of hardware resources.

[0116] Based on the discovery of the above technical problems, the inventors have proposed the following technical solutions after creative work to solve or improve the above problems. It should be noted that the defects existing in the solutions in the above prior art are the results obtained by the inventors after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of this application below for the above problems should all be the contributions made by the inventors to this application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0117] In view of this, an embodiment of the present invention provides a convolution hardware accelerator. This convolution hardware accelerator utilizes bit splitting, inversion, and bit splicing operations to replace the multiplication and modulo operations used in the prior art during both the forward and inverse Fermat number transform processes. This accelerates convolution and effectively reduces hardware implementation complexity and resource consumption. This is described in detail below through an embodiment and in conjunction with the accompanying drawings.

[0118] Here we first briefly introduce the hardware structure of the convolution hardware accelerator.

[0119] Please refer to Figure 6 The convolution hardware accelerator 200 includes a forward transform module 210, an FNT module 220, a matrix dot product module 230, an IFNT module 240, and an inverse transform module 250. The FNT module 220 includes a forward row transform module 221 and a forward column transform module 222, and the IFNT module 240 includes an inverse column transform module 241 and an inverse row transform module 242.

[0120] The convolution hardware accelerator 200 takes two input matrices as input and outputs the convolution result of the two input matrices. The logic circuit structure of the convolution hardware accelerator 200 can be written in Verilog (ie, Verilog HDL, a hardware description language).

[0121] It should be noted that the meaning of the input matrix depends on the field to which the convolution calculation belongs. For example, in the field of image processing, both input matrices can be feature matrices obtained after preprocessing the image.

[0122] An embodiment of the present invention provides a convolution acceleration method, which is applied to the above-mentioned convolution hardware accelerator 200. The following is a detailed introduction to the process of implementing convolution acceleration using the convolution hardware accelerator 200.

[0123] Please refer to Figure 7 ,The convolution acceleration method includes the following steps S101 to S105.

[0124] S101. A forward conversion module performs forward conversion processing on a first input matrix and a second input matrix in sequence according to a preset forward conversion formula to obtain a first conversion matrix and a second conversion matrix.

[0125] In this embodiment, elements in the first conversion matrix and the second conversion matrix are all binary data.

[0126] S102 , the FNT module performs positive Fermat number transformation on the first transformation matrix and the second transformation matrix in sequence according to a preset operation rule to obtain a transformed first transformation matrix and a transformed second transformation matrix.

[0127] In this embodiment, the preset operation rules include a bit splitting operation, a negation operation, and a bit concatenation operation.

[0128] S103. The matrix dot product module performs dot product on the transformed first conversion matrix and the transformed second conversion matrix to obtain a dot product result matrix.

[0129] S104, the IFNT module performs an inverse Fermat transformation on the dot product result matrix according to a preset operation rule to obtain an inverse dot product result matrix.

[0130] S105. The inverse transformation module performs inverse transformation processing on the inverse dot multiplication result matrix according to a preset inverse transformation formula to obtain a target matrix.

[0131] In this embodiment, the target matrix represents the convolution result of the first input matrix and the second input matrix.

[0132] The convolution acceleration method adopted by the embodiment of the present invention adopts a preset forward conversion formula, a preset operation rule and a preset inverse conversion formula, so that when the convolution hardware accelerator implements convolution acceleration, bit splitting, inversion and bit splicing operations are used to replace multiplication and modulo operations in the positive Fermat number transformation and the inverse Fermat number transformation process, thereby accelerating the convolution speed and effectively reducing the complexity of hardware implementation and the consumption of hardware resources.

[0133] In an optional implementation, the forward conversion module has the same processing logic for the first input matrix and the second input matrix. Any one of the first input matrix and the second input matrix is regarded as the target input matrix. The above step S101 sub-steps may include S1011 to S1012:

[0134] S1011. The forward conversion module converts each element of the target input matrix according to the conversion formula to obtain a conversion matrix.

[0135] The conversion formula is:

[0136]

[0137] Among them, x is an element in the target input matrix, x new is the element in the target conversion matrix corresponding to the element x, and m represents the preset binary bit width.

[0138] The logic circuit of the positive conversion formula can be as follows Figure 8 As shown, the logic element corresponding to the conversion formula is a selector, where sel is the control signal. When x is 0, the control signal is high, and the selector outputs 2. m ; When 0 <x≤2 m , the control signal is low level, and the selector outputs x-1.

[0139] S1012. The forward conversion module converts each element in the conversion matrix into binary data to obtain a target conversion matrix.

[0140] In this embodiment, when the target input matrix is the first input matrix, the target conversion matrix is the first conversion matrix; when the target input matrix is the second input matrix, the target conversion matrix is the second conversion matrix.

[0141] The preset inverse conversion formula is opposite to the conversion formula. An original decimal number is converted into a new decimal number according to the conversion formula, and the new decimal number can be restored to the original decimal number according to the inverse conversion formula.

[0142] When the preset binary bit width m=4, the data conversion comparison table can be shown as Table (1):

[0143] Table (1)

[0144]

[0145]

[0146] Among them, the row where serial number 10 is located indicates that the original decimal number 9 and the original decimal number -8 have the same new binary representation and have no other meaning. Serial numbers 11 to 17 are similar.

[0147] In an optional implementation, the processing of the first conversion matrix and the second conversion matrix by the FNT module is consistent. The following describes the process of the FNT module processing the target conversion matrix. Figure 9 The sub-steps of the above step S102 may include S1021 to S1026.

[0148] S1021, the forward row transformation module cuts the target transformation matrix by column to obtain several column matrices of the target transformation matrix;

[0149] S1022: The forward row transformation module processes each column matrix of the target transformation matrix based on a preset operation rule and a forward transformation matrix to obtain a forward operation matrix corresponding to each column matrix of the target transformation matrix;

[0150] S1023, the forward row transformation module horizontally splices all the forward operation matrices of the target conversion matrix to obtain a target row transformation matrix;

[0151] S1024, the column-wise transformation module cuts the target row transformation matrix by columns to obtain a plurality of column matrices of the target row transformation matrix;

[0152] S1025. The positive column transformation module processes each column matrix of the target row transformation matrix based on a preset operation rule and a positive transformation matrix to obtain a positive operation matrix corresponding to each column matrix of the target row transformation matrix;

[0153] S1026. The positive column transformation module vertically splices all positive operation matrices of the target row transformation matrix to obtain a positive transformation result matrix.

[0154] In this embodiment, the forward transformation result matrix is the first transformation matrix after transformation or the second transformation matrix after transformation. Each column matrix of the target transformation matrix and each column matrix of the target row transformation matrix are n×1 matrices; the forward transformation matrix is an n-order square matrix; and the forward operation matrix corresponding to each column matrix of the target transformation matrix and each column matrix of the target row transformation matrix are 1×n matrices.

[0155] Optionally, the calculation logic for obtaining the positive operation matrix in steps S1022 and S1025 is the same. Any column matrix of the target conversion matrix is regarded as the target column matrix, or any column matrix of the target row transformation matrix is regarded as the target column matrix. Then, the calculation process of the positive operation matrix corresponding to the target column matrix may include the following steps S01 to S03:

[0156] S01. Construct n data pairs based on the i-th row in the forward transformation matrix and the target column matrix.

[0157] In this embodiment, a data pair may include a control value and a binary sequence to be processed, where the control value is any target element in the i-th row of the positive transformation matrix, and the binary sequence to be processed is an element in the target column matrix that matches the position of the target element.

[0158] The forward transformation matrix is represented by A n×n (n is 4, 16 or 32), A n×n The element in row i and column j is:

[0159] a ij =2 k mod F m

[0160] Among them, F m is the preset Fermat number, (i-1)*(j-1)*α, 2 α Less than the preset Fermat number and α is a positive integer. That is: 2 α ={2,4,8,…,F m -1}.

[0161] The target column matrix is denoted as S n×1 , S n×1 The binary element in row i is s i1 , then the i-th of n data pairs includes: a ij 、s i1 .

[0162] S02. For each data pair, based on a preset operation rule, the data pair is processed to obtain a conversion sequence corresponding to each data pair.

[0163] S03. Determine a result sequence according to the n conversion sequences and the sum sequence of the n conversion sequences, and use the result sequence as the element of the i-th column of the positive operation matrix corresponding to the target column matrix, where i=1, 2, ... n.

[0164] The following describes the detailed process of processing data pairs based on preset operation rules to obtain the corresponding conversion sequence of the data pairs. Next, the process of the IFNT module processing the dot product result matrix is introduced.

[0165] In an optional implementation, combined with Figure 10 The sub-steps of the above-mentioned step S103 may include S1031 to S1036.

[0166] S1031, the inverse column transformation module cuts the dot product result matrix by row to obtain several row matrices of the dot product result matrix;

[0167] S1032: The inverse column transformation module processes each row matrix of the dot product result matrix based on a preset operation rule and an inverse transformation matrix to obtain an inverse operation matrix corresponding to each row matrix of the dot product result matrix;

[0168] S1033, the inverse column transformation module vertically splices all inverse operation matrices of the dot product result matrix to obtain a target column transformation matrix;

[0169] S1034, the inverse transformation module cuts the target column transformation matrix by row to obtain several row matrices of the target column transformation matrix;

[0170] S1035. The inverse transformation module processes each row matrix of the target column transformation matrix based on a preset operation rule and an inverse transformation matrix to obtain an inverse operation matrix corresponding to each row matrix of the target column transformation matrix.

[0171] S1036. The inverse transformation module horizontally splices all inverse operation matrices of the target column transformation matrix to obtain an inverse point product result matrix.

[0172] In this embodiment, each row matrix of the dot product result matrix and each row matrix of the target column transformation matrix are 1×n matrices; the inverse transformation matrix is an n-order square matrix; the inverse operation matrix corresponding to each row matrix of the dot product result matrix and the inverse operation matrix corresponding to each row matrix of the target column transformation matrix are both n×1 matrices.

[0173] Optionally, the calculation logic for obtaining the inverse operation matrix in steps S1032 and S1035 is the same. Any row matrix of the dot product result matrix is regarded as the target row matrix, or any row matrix of the target column transformation matrix is regarded as the target row matrix. Then, the calculation process of the inverse operation matrix corresponding to the target row matrix may include the following steps S11 to S13:

[0174] S11. Construct n data pairs based on the j-th column in the inverse transformation matrix and the target row matrix.

[0175] In this embodiment, a data pair may include a control value and a binary sequence to be processed, where the control value is any target element in the j-th column of the positive transformation matrix, and the binary sequence to be processed is an element in the target row matrix that matches the position of the target element.

[0176] The inverse transformation matrix is represented as B n×n , B n×n The element in row i and column j is b ij ; The target row matrix is represented as R 1×n , R 1×n The binary element in the jth column is s 1j , then the jth of n data pairs includes: b ij 、s 1j .

[0177] S12. For each data pair, based on a preset operation rule, the data pair is processed to obtain a conversion sequence corresponding to each data pair.

[0178] S13. Determine a result sequence according to the n conversion sequences and the sum sequence of the n conversion sequences, and use the result sequence as the element of the j-th row of the inverse operation matrix corresponding to the target row matrix, where j=1, 2, ... n.

[0179] In particular, the calculation logic of the above steps S02 and S12 is consistent, and the calculation logic of the above steps S03 and S13 is consistent. The calculation logic of steps S02 and S12 is first introduced below, and then the calculation logic of steps S03 and S13 is introduced.

[0180] First, the calculation logic for obtaining the conversion sequence in steps S02 and S12 is as follows.

[0181] With the positive transformation matrix A n×n Similarly, in the inverse transformation matrix B n×n In, b ij =2 -k mod F m In the data pairs of steps S02 and S12, the control value of the former is 2 k The latter control value is 2 -k , the bit width of the binary sequence to be processed is 9.

[0182] Then, for any data pair in the above step S01 or S11, the process of processing the data pair based on the preset operation rules to obtain the conversion sequence corresponding to the data pair may include the following steps S001 to S005:

[0183] S001. When k=0, the binary sequence of the data pair to be processed is used as the conversion sequence of the data pair.

[0184] S002. When k≠0, determine the value location based on the comparison value of the data pair.

[0185] S003. Eliminate a specific sequence at the value position in the binary sequence to be processed to obtain a spliced sequence.

[0186] The binary sequence to be processed is from bit 0 to bit 8 from right to left, and the value position is from bit (8-k) to bit 7 from right to left in the binary sequence to be processed of the data pair. When (8-k)=7, the specific sequence has only one bit.

[0187] S004. Invert the specific sequence bit by bit to obtain a specific inverted sequence.

[0188] S005. After splicing the specific inverted sequence to the spliced sequence, a conversion sequence of the data pair is obtained.

[0189] Assuming k is 1, the logic circuit diagram involved in steps S003 to S005 can be as follows: Figure 11 As shown, Figure 11 In the data pair, a[8:0] represents the binary sequence to be processed, a[8] represents the 8th bit from right to left in the binary sequence to be processed, a[7] represents the 7th bit from right to left in the binary sequence to be processed, and a[6:0] represents bits 0 to 6 from right to left in the binary sequence to be processed. ~a[7] represents the bitwise inversion of a[7]. b[8:0] represents the conversion sequence. {a[8],a[6:0],~a[7]} represents the bitwise concatenation of a[8], a[6:0], and ~a[7] to obtain b[8:0].

[0190] In this embodiment, there are two processing situations for a data pair:

[0191] Case 1: In a data pair, when the control value satisfies k=0, the to-be-processed binary sequence of the data pair is directly used as the conversion sequence of the data pair.

[0192] Case 2: When the control value satisfies k≠0, the data pair is processed according to the above steps S002 to S005. The processing process is as follows: Figure 12 shown.

[0193] For example, suppose that in a data pair, the control value is 4 and the binary sequence to be processed is 000000010. Figure 13 , at this time k = 2, then the value position is from the 6th to the 7th bit from right to left, the specific sequence is 00, the specific sequence 00 is bitwise inverted, and the specific inverted sequence 11 is obtained, then the conversion sequence of the data pair is 000001011.

[0194] It should be noted that this example is only an example and is not limiting.

[0195] The above describes the calculation logic for obtaining the conversion sequence in steps S02 and S12. Next, the calculation logic of steps S03 and S13 will be introduced.

[0196] Optionally, the process of determining the result sequence according to the n conversion sequences and the sum sequence of the n conversion sequences in steps S03 and S13 may include the following steps S2001 to S2005:

[0197] S2001, add n conversion sequences to obtain a sum sequence;

[0198] S2002, performing a preset operation on the sum sequence to obtain an intermediate sequence;

[0199] S2003, determining a first control signal and a second control signal based on the intermediate sequence and the n conversion sequences respectively;

[0200] S2004: If the second control signal is at a high level, the preset sequence is used as the result sequence;

[0201] S2005: If the second control signal is at a low level and the first control signal is at a low level, the intermediate sequence is used as the result sequence;

[0202] S2006 : If the second control signal is at a low level and the first control signal is at a high level, subtract one from the intermediate sequence as the result sequence.

[0203] In this embodiment, when the bit width of the conversion sequence is 9, the first control signal is the 9th bit from the right to the left of the intermediate sequence, and the second control signal is the result of a bitwise AND operation of the most significant bits of the n conversion sequences. The bit width of the preset sequence is the same as the bit width of the conversion sequence, with the most significant bit being 1 and the remaining bits being all 0s (i.e., 100000000).

[0204] Please refer to Figure 14 , you can use two selector combinations to output the result sequence. Figure 14 In the example, Sel-1 in selector 1 is the first control signal, and Sel-2 in selector 2 is the second control signal. The output Out-1 of selector 1 is used as one of the inputs of selector 2, and the output Out-2 of selector 2 is the result sequence.

[0205] Figure 14 In the figure, the sum sequence is expressed as add[N:0], which means a binary sequence of N+bit width. When n is 4, the maximum bit width of the sum sequence is 11, and N=10. When n is 16, the maximum bit width of the sum sequence is 13, and N=12. When n is 32, the maximum bit width of the sum sequence is 14, and N=13.

[0206] The default operation is:

[0207] 1. Split the sum sequence add[N:0] into the front sequence add[7:0] and the back sequence add[N:8]; the front sequence add[7:0] is the first 8 bits of the sum sequence, and the back sequence add[N:8] is the part of the sum sequence without the first 8 bits;

[0208] 2. The front sequence add[7:0] is first subtracted from the back sequence add[N:8], and then the binary representation of (n-1) is subtracted.

[0209] from Figure 14 It can be seen that as long as Sel-2 is 1, the output of selector 2 is the preset sequence regardless of whether Sel-1 is 1 or 0. When Sel-2 is 0, the output of selector 2 is related to the intermediate sequence.

[0210] It can be understood that the logic circuits of the forward row conversion module and the forward column conversion module of the FNT module, and the reverse column conversion module and the reverse row conversion module of the FNT module can all adopt the following logic circuits: Figure 14 The structure shown implements the operation of obtaining a result sequence from n conversion sequences.

[0211] The above describes the process of obtaining the result sequence of n conversion sequences in steps S03 and S13.

[0212] In an optional implementation, the sub-steps of step S105 may include S1051 to S1052:

[0213] S1051. The inverse conversion module converts each element in the inverse dot multiplication result matrix into decimal data.

[0214] S1052. The inverse transformation module transforms each element of the inverse dot multiplication result matrix according to the inverse transformation formula to obtain a target matrix.

[0215] The inverse transformation formula is:

[0216]

[0217] Among them, y new is an element in the inverse dot multiplication matrix, and y is the same as the element y in the target matrix. new The corresponding element.

[0218] It should be noted that the sizes of the elements of the forward transformation matrix and the inverse transformation matrix are determined according to the requirements and are not limited here.

[0219] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0220] First, by transforming the input into a finite field and then performing matrix multiplication after sequence transformation, the present invention effectively reduces the number of multiplications compared to traditional convolution calculations and improves the operation speed. This invention simplifies the hardware acceleration architecture of the existing technology. When implementing convolution acceleration, the convolution hardware accelerator uses a new binary method to reconstruct the convolution operation, speeding up the convolution speed. It also uses bit splitting, negation, and bit splicing operations to replace multiplication and modulo operations, effectively reducing the complexity of hardware implementation and the consumption of hardware resources.

[0221] Second, compared with the traditional convolution operation based on the Fast Fourier Transform (Fast Fourier Transform) algorithm, the FNT algorithm used in the present invention has a very similar conversion formula to the FFT algorithm. The difference is that FNT uses a real number transformation kernel in a finite field to replace the complex number transformation kernel in FFT. Calculation in a finite field brings many advantages: First, compared with FFT, the intermediate storage requirements of FNT are reduced by nearly half, because FNT only needs to store unsigned integers instead of the complex numbers of FFT. Second, the present invention uses shift and addition instead of the complex number multiplication in FFT during the transformation process of convolution based on the FNT algorithm, which effectively simplifies the calculation. Finally, the complex number matrix dot multiplication operation in the frequency domain of FFT is replaced by the real number matrix dot multiplication operation in the finite field, which also greatly saves the required multiplication operation overhead.

[0222] Third, the convolution hardware accelerator of the present invention is different from the hardware acceleration architecture of the prior art in that the two have different operating methods. The present invention saves computing overhead in each calculation step. Taking a 9-bit 16×16 input matrix as an example, if the calculation method in the hardware acceleration architecture of the prior art is used, a modulo operation needs to be performed after the transformation is completed, then 9×16×16+16×16×2=2816 multiplications and 15×16×2=480 additions need to be performed. When the convolution hardware accelerator of the present invention is used, only the operations of base conversion, bit splicing and inversion are performed, that is, 256×2+15×16×2=992 additions are performed, which greatly speeds up the calculation speed.

[0223] Compared with the traditional convolution calculation using FFT, which corresponds to a 14×14 input matrix and uses a 3×3 convolution kernel calculation, a total of 12×12×3×3=1296 multiplications are required, while the present invention only requires 256 multiplications and 256×2+15×16×2×2=1472 additions to obtain the convolution result. Converting the multiplication into the corresponding number of additions, the traditional convolution requires 11664 additions, while the present invention only requires 3776, reducing the number of addition operations by 67.6%. Compared with the traditional convolution calculation, the present invention reduces the number of addition operations in the convolution calculation, that is, speeds up the convolution operation. Especially in the field of image processing, it can speed up the image processing speed and process image information more efficiently.

[0224] Based on the above convolution acceleration method, this solution also provides a convolution hardware accelerator 200, combined with Figure 6 ,in:

[0225] The forward conversion module 210 is used to perform forward conversion processing on the first input matrix and the second input matrix according to a preset forward conversion formula to obtain a first conversion matrix and a second conversion matrix;

[0226] The FNT module 220 is used to perform positive Fermat number transformation on the first conversion matrix and the second conversion matrix according to a preset operation rule to obtain a transformed first conversion matrix and a transformed second conversion matrix;

[0227] The matrix dot product module 230 is used to perform dot product on the transformed first conversion matrix and the transformed second conversion matrix to obtain a dot product result matrix;

[0228] The IFNT module 240 is used to perform an inverse Fermat number transformation on the point product result matrix according to a preset operation rule to obtain an inverse point product result matrix;

[0229] The inverse transformation module 250 is used to perform inverse transformation processing on the inverse dot product result matrix according to a preset inverse transformation formula to obtain a target matrix; the target matrix represents the convolution result of the first input matrix and the second input matrix.

[0230] In summary, embodiments of the present invention provide a convolution acceleration method and a convolution hardware accelerator. The convolution hardware accelerator includes a forward transformation module, an FNT module, a matrix dot multiplication module, an IFNT module, and an inverse transformation module. The forward transformation module first performs forward transformation processing on two input matrices in sequence according to a preset forward transformation formula to obtain two transformation matrices. The FNT module then performs positive Fermat transforms on the two transformation matrices in sequence according to preset operation rules to obtain two transformed transformation matrices. The matrix dot multiplication module then performs dot multiplication on the two transformed transformation matrices to obtain a dot product result matrix. The IFNT module then performs an inverse Fermat transform on the dot product result matrix according to preset operation rules to obtain an inverse dot product result matrix. Finally, the inverse transformation module performs an inverse transformation processing on the inverse dot product result matrix according to a preset inverse transformation formula to obtain a target matrix representing the convolution result between the two input matrices. Based on the linear convolution property of the Fermat transform, the convolution hardware accelerator performs positive Fermat transforms on the two transformation matrices, then performs dot multiplication on the result, and then performs an inverse Fermat transform on the result, thereby achieving convolution acceleration using hardware resources. In addition, when the convolution hardware accelerator implements convolution acceleration, it uses bit splitting, inversion, and bit splicing operations to replace multiplication and modulo operations in the process of positive Fermat number transform and inverse Fermat number transform, which speeds up the convolution speed and can effectively reduce the complexity of hardware implementation and the consumption of hardware resources.

[0231] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A convolution acceleration method, characterized in that: Applied to a convolution hardware accelerator, the convolution hardware accelerator includes: a forward conversion module, an FNT module, a matrix dot multiplication module, an IFNT module and an inverse conversion module; the method includes: The forward conversion module performs forward conversion processing on the first input matrix and the second input matrix in sequence according to a preset forward conversion formula to obtain a first conversion matrix and a second conversion matrix; The FNT module sequentially performs positive Fermat number transformation on the first conversion matrix and the second conversion matrix according to a preset operation rule to obtain a transformed first conversion matrix and a transformed second conversion matrix; the preset operation rule includes a bit splitting operation, a negation operation, and a bit splicing operation; The matrix dot multiplication module performs dot multiplication on the transformed first conversion matrix and the transformed second conversion matrix to obtain a dot product result matrix; The IFNT module performs an inverse Fermat transformation on the dot product result matrix according to the preset operation rules to obtain an inverse dot product result matrix; The inverse conversion module performs an inverse conversion process on the inverse dot product result matrix according to a preset inverse conversion formula to obtain a target matrix; the target matrix represents the convolution result of the first input matrix and the second input matrix; The FNT module includes a positive row transformation module and a positive column transformation module; the FNT module sequentially performs positive Fermat number transformation on the first transformation matrix and the second transformation matrix according to a preset operation rule to obtain the transformed first transformation matrix and the transformed second transformation matrix, including the following steps: The forward row transformation module cuts the target conversion matrix by columns to obtain a plurality of column matrices of the target conversion matrix; the target conversion matrix is the first conversion matrix or the second conversion matrix; The forward row transformation module processes each column matrix of the target transformation matrix based on the preset operation rule and the forward transformation matrix to obtain a forward operation matrix corresponding to each column matrix of the target transformation matrix; The positive row transformation module performs horizontal splicing on all positive operation matrices of the target conversion matrix to obtain a target row transformation matrix; The column-wise transformation module cuts the target row transformation matrix by columns to obtain a plurality of column matrices of the target row transformation matrix; The positive column transformation module processes each column matrix of the target row transformation matrix based on the preset operation rule and the positive transformation matrix to obtain a positive operation matrix corresponding to each column matrix of the target row transformation matrix; The positive column transformation module vertically splices all positive operation matrices of the target row transformation matrix to obtain a positive transformation result matrix; the positive transformation result matrix is the first transformation matrix after the transformation or the second transformation matrix after the transformation; Among them, the column matrix is Order matrix; the forward transformation matrix is Order square matrix, the positive operation matrix is Order matrix; the calculation process of the positive operation matrix corresponding to the target column matrix is: According to the first row and column matrices of the target, construct data pairs; wherein, the target column matrix is any column matrix of the target conversion matrix or any column matrix of the target row transformation matrix; a data pair includes a control value and a binary sequence to be processed, and the control value is the first column matrix in the forward transformation matrix. Any target element of a row, the binary sequence to be processed is an element in the target column matrix that matches the position of the target element; For each of the data pairs, processing the data pair based on the preset operation rule to obtain a conversion sequence corresponding to each of the data pairs; according to conversion sequence and the The sum sequence of the conversion sequences is determined, and the result sequence is used as the first positive operation matrix corresponding to the target column matrix. Elements of the column, where .

2. The method according to claim 1, characterized in that The forward conversion module sequentially performs forward conversion processing on the first input matrix and the second input matrix according to a preset forward conversion formula to obtain the first conversion matrix and the second conversion matrix, including: The forward conversion module converts each element of the target input matrix according to the conversion formula to obtain a conversion matrix; wherein the target input matrix is the first input matrix or the second input matrix, and the conversion formula is: in, is an element of the target input matrix, The target transformation matrix with elements The corresponding elements, Represents the preset binary bit width; The forward conversion module converts each element in the conversion matrix into binary data to obtain a target conversion matrix, where the target conversion matrix is the first conversion matrix or the second conversion matrix.

3. The method according to claim 1, characterized in that The IFNT module includes an inverse column transformation module and an inverse row transformation module; The IFNT module performs an inverse Fermat number transformation on the dot product result matrix according to the preset operation rule to obtain an inverse dot product result matrix, including: The inverse column transformation module cuts the dot product result matrix by rows to obtain a plurality of row matrices of the dot product result matrix; The inverse column transformation module processes each row matrix of the dot product result matrix based on the preset operation rule and the inverse transformation matrix to obtain an inverse operation matrix corresponding to each row matrix of the dot product result matrix; The inverse column transformation module vertically splices all inverse operation matrices of the dot product result matrix to obtain a target column transformation matrix; The inverse transformation module cuts the target column transformation matrix by rows to obtain a plurality of row matrices of the target column transformation matrix; The inverse transformation module processes each row matrix of the target column transformation matrix based on the preset operation rule and the inverse transformation matrix to obtain an inverse operation matrix corresponding to each row matrix of the target column transformation matrix; The inverse transformation module horizontally splices all inverse operation matrices of the target column transformation matrix to obtain the inverse point product result matrix.

4. The method according to claim 3, characterized in that The row matrix is The inverse transformation matrix is The inverse operation matrix is rank matrix; Among them, the calculation process of the inverse operation matrix corresponding to the target row matrix is: According to the inverse transformation matrix and the target row matrix, construct data pairs; wherein, the target row matrix is any row matrix of the dot product result matrix or any row matrix of the target column transformation matrix; a data pair includes a control value and a binary sequence to be processed, and the control value is the first row matrix in the forward transformation matrix. Any target element of a column, the binary sequence to be processed is an element in the target row matrix that matches the position of the target element; For each of the data pairs, processing the data pair based on the preset operation rule to obtain a conversion sequence corresponding to each of the data pairs; according to conversion sequence and the The sum sequence of the conversion sequences is determined, and the result sequence is used as the first inverse operation matrix corresponding to the target row matrix. Elements of a row, where .

5. The method according to claim 1 or 4, characterized in that The control value is or , , the bit width of the binary sequence to be processed is 9; The step of processing the data pair based on the preset operation rules includes: when , taking the to-be-processed binary sequence of the data pair as the conversion sequence of the data pair; when When the value position is determined based on the comparison value of the data pair; the value position is the ( 1st to 7th place; Eliminating a specific sequence at a value position in the binary sequence to be processed to obtain a spliced sequence; Inverting the specific sequence bit by bit to obtain a specific inverted sequence; The specific inverted sequence is spliced to the spliced sequence to obtain a conversion sequence of the data pair.

6. The method according to claim 1 or 4, characterized in that The basis conversion sequence and the The steps of determining the result sequence of the sum of the transformation sequences include: The Add the transformed sequences to obtain the sum sequence; Performing a preset operation on the sum sequence to obtain an intermediate sequence; Based on the intermediate sequence, the The conversion sequence determines the first control signal and the second control signal respectively; wherein the first control signal is the highest bit of the sum sequence, and the second control signal is the The result of the bitwise AND operation on the highest bit of the conversion sequence; If the second control signal is at a high level, a preset sequence is used as the result sequence; wherein the bit width of the preset sequence is the same as the bit width of the conversion sequence, the most significant bit is 1, and the remaining bits are all 0; If the second control signal is at a low level and the first control signal is at a low level, taking the intermediate sequence as the result sequence; If the second control signal is at a low level and the first control signal is at a high level, a result of subtracting one from the intermediate sequence is used as the result sequence.

7. The method according to claim 1, characterized in that The inverse conversion module performs inverse conversion processing on the inverse dot multiplication result matrix according to a preset inverse conversion formula to obtain a target matrix, including: The inverse conversion module converts each element in the inverse dot multiplication result matrix into decimal data; The inverse conversion module converts each element of the inverse dot product result matrix according to the inverse conversion formula to obtain the target matrix; the inverse conversion formula is: in, is an element in the inverse dot product result matrix, The target matrix with the element The corresponding elements, Represents the preset binary bit width.

8. A convolution hardware accelerator, characterized in that: The convolution hardware accelerator includes: a forward conversion module, an FNT module, a matrix dot multiplication module, an IFNT module and an inverse conversion module; The forward conversion module is used to perform forward conversion processing on the first input matrix and the second input matrix according to a preset forward conversion formula to obtain a first conversion matrix and a second conversion matrix; The FNT module is used to perform positive Fermat number transformation on the first conversion matrix and the second conversion matrix according to a preset operation rule to obtain a transformed first conversion matrix and a transformed second conversion matrix; The matrix dot product module is used to perform dot product on the transformed first conversion matrix and the transformed second conversion matrix to obtain a dot product result matrix; The IFNT module is used to perform an inverse Fermat transformation on the point product result matrix according to the preset operation rules to obtain an inverse point product result matrix; The inverse conversion module is used to perform inverse conversion processing on the inverse dot product result matrix according to a preset inverse conversion formula to obtain a target matrix; the target matrix represents the convolution result of the first input matrix and the second input matrix; The FNT module includes a positive row transformation module and a positive column transformation module; the FNT module is used to perform positive Fermat number transformation on the first transformation matrix and the second transformation matrix in sequence according to a preset operation rule to obtain the transformed first transformation matrix and the transformed second transformation matrix: The forward row transformation module is used to cut the target conversion matrix by column to obtain a plurality of column matrices of the target conversion matrix; the target conversion matrix is the first conversion matrix or the second conversion matrix; The forward row transformation module is further configured to process each column matrix of the target transformation matrix based on the preset operation rule and the forward transformation matrix, respectively, to obtain a forward operation matrix corresponding to each column matrix of the target transformation matrix; The positive row transformation module is further used to horizontally splice all positive operation matrices of the target conversion matrix to obtain a target row transformation matrix; The column-wise transformation module is used to cut the target row transformation matrix by columns to obtain a plurality of column matrices of the target row transformation matrix; The positive column transformation module is further configured to process each column matrix of the target row transformation matrix based on the preset operation rule and the positive transformation matrix to obtain a positive operation matrix corresponding to each column matrix of the target row transformation matrix; The positive column transformation module is further used to vertically splice all positive operation matrices of the target row transformation matrix to obtain a positive transformation result matrix; the positive transformation result matrix is the first transformation matrix after the transformation or the second transformation matrix after the transformation; Among them, the column matrix is Order matrix; the forward transformation matrix is Order square matrix, the positive operation matrix is Order matrix; the calculation process of the positive operation matrix corresponding to the target column matrix is: According to the first row and column matrices of the target, construct data pairs; wherein, the target column matrix is any column matrix of the target conversion matrix or any column matrix of the target row transformation matrix; a data pair includes a control value and a binary sequence to be processed, and the control value is the first column matrix in the forward transformation matrix. Any target element of a row, the binary sequence to be processed is an element in the target column matrix that matches the position of the target element; For each of the data pairs, processing the data pair based on the preset operation rule to obtain a conversion sequence corresponding to each of the data pairs; according to conversion sequence and the The sum sequence of the conversion sequences is determined, and the result sequence is used as the first positive operation matrix corresponding to the target column matrix. Elements of the column, where .

Images