Computing circuit, processor and computing device
By converting signed numbers into unsigned numbers in the calculation circuit and combining symmetric operations, the problem of complex structure and large power consumption in matrix multiplication operations is solved, and efficient matrix multiplication operations are realized.
Patent Information
- Application Number
- CN202410912414.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-09
AI Technical Summary
The existing circuits used for matrix multiplication operations have complex structures, large power consumption, and difficult to meet the timing. Especially when dealing with signed numbers, there is a problem of power consumption rising due to some flips.
A calculation circuit was designed to convert signed numbers into unsigned numbers through the symbol processing sub-circuit, and the internal product calculation was performed using the multiplication and accumulation sub-circuit. Combined with the symmetric operation sub-circuit, the multiplication operation was simplified, and the structural complexity of the operation circuit was reduced. Power consumption.
The structure of the calculation circuit is effectively simplified, power consumption is reduced, timing convergence is improved, and power consumption is further reduced by reducing the sign bit flip rate, improving the efficiency of matrix multiplication operation.
Smart Images

Figure CN118466899B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of electronic circuits, and more specifically, to an operation circuit, a processor, and a computing device. Background Art
[0002] In the process of data processing, matrix multiplication operations are often involved. However, the current operation circuits used to perform matrix multiplication have problems such as complex structure, high power consumption, and difficulty in meeting the timing requirements, and need to be improved. Summary of the invention
[0003] One of the objectives of the present disclosure is to provide an operation circuit, a processor, and a computing device.
[0004] According to a first aspect of the present disclosure, there is provided an operation circuit for performing an operation of multiplying a first matrix by a second matrix, wherein the number of columns of the first matrix and the number of rows of the second matrix are each N, and each first matrix element in the first matrix is a pre-known coefficient, and the operation circuit comprises:
[0005] a sign processing subcircuit configured to convert a second matrix element in the second matrix into a corresponding first operand, wherein the first operand is an unsigned number; and
[0006] A multiply-accumulate subcircuit, wherein an input end of the multiply-accumulate subcircuit is communicatively connected to an output end of the sign processing subcircuit, and the multiply-accumulate subcircuit is configured to at least calculate an inner product of a first vector and a second vector, wherein a k-th component of the first vector is a k-th first matrix element in a current operation row of the first matrix, and a k-th component of the second vector is a first operand obtained by converting a k-th second matrix element in a current operation column of the second matrix, and a dimension of the first vector and a dimension of the second vector are each N.
[0007] In some embodiments, the operation circuit includes N symbol processing sub-circuits, and in the same operation cycle, each symbol processing sub-circuit is configured to convert a corresponding second matrix element in the same column of the second matrix; and / or,
[0008] The operation circuit includes N' multiplication and accumulation sub-circuits. In the same operation cycle, each multiplication and accumulation sub-circuit is configured to calculate the inner product of the first vector and the second vector associated with a corresponding third matrix element in the same column of the third matrix, wherein N' is the number of rows of the first matrix, and the third matrix is a matrix obtained by multiplying the first matrix by the second matrix.
[0009] In some embodiments, the sign processing subcircuit is configured to invert a target sign bit of the second matrix element to generate a corresponding first operand, wherein the target sign bit of the second matrix element is the most significant bit of the second matrix element.
[0010] In some embodiments, the symbol processing subcircuit is configured to:
[0011] Determine whether the upper m bits of each second matrix element in the current operation column of the second matrix are all extended sign bits, where m is a preset positive integer and m is less than the total number of bits M of the second matrix elements;
[0012] In the case where the upper m bits of each second matrix element in the current operation column are extended sign bits, the upper m bits of the second matrix element are all set to zero, and the target sign bit of the second matrix element is inverted to generate a corresponding first operand, wherein the target sign bit of the second matrix element is the lower bit adjacent to the upper m bits of the second matrix element, or,
[0013] When the upper m bits of at least one second matrix element in the current operation column are not all extended sign bits, the target sign bit of the second matrix element is inverted to generate a corresponding first operand, wherein the target sign bit of the second matrix element is the highest bit of the second matrix element.
[0014] In some embodiments, the symbol processing subcircuit is configured to:
[0015] Determine the position of the real sign bit of each second matrix element in the current operation column of the second matrix, and use the highest real sign bit among the real sign bits of all the second matrix elements in the current operation column as the target sign bit; and
[0016] The bits higher than the target sign bit of the second matrix element are all set to zero, and the target sign bit of the second matrix element is inverted to generate the corresponding first operand.
[0017] In some embodiments, the sign processing subcircuit includes an inverter, an input terminal of the inverter being configured to receive a target sign bit of the second matrix element to invert the target sign bit.
[0018] In some embodiments, the sign processing subcircuit includes a sign bit determination unit, the sign bit determination unit including:
[0019] a first AND gate unit, wherein the first AND gate unit has a plurality of input terminals, each of which is configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in a current operation column, and an output terminal of the first AND gate unit is configured to output an indication signal indicating whether the received plurality of to-be-determined bits are all extended sign bits; or
[0020] An OR gate unit, wherein the OR gate unit has a plurality of input terminals, each of which is configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in the current operation column, and an output terminal of the OR gate unit is configured to output an indication signal indicating whether the received plurality of to-be-determined bits are all extended sign bits; or,
[0021] An XOR gate unit, wherein the XOR gate unit has multiple input terminals, each input terminal is configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in a current operation column, and the output terminal of the XOR gate unit is configured to output an indication signal indicating whether the received multiple to-be-determined bits are all extended sign bits.
[0022] In some embodiments, the sign processing subcircuit includes a second AND gate unit, wherein two input terminals of the second AND gate unit are configured to respectively receive a signal zero representing zero and a corresponding one to-be-set-to-zero bit of a corresponding second matrix element to set the to-be-set-to-zero bit to zero.
[0023] In some embodiments, the multiplication-accumulation subcircuit includes a shift-addition unit, and the shift-addition unit is configured to:
[0024] When the first matrix element multiplied by the first operand is a positive number, the first operand is shifted by a preset number of bits in a preset direction to generate one or more corresponding second operands, wherein the preset direction and the preset number of bits are determined based on the first matrix element multiplied by the first operand.
[0025] In some embodiments, the multiply-accumulate subcircuit further includes a bitwise inversion unit, wherein the bitwise inversion unit is configured to: invert the first operand bitwise to generate a corresponding third operand when the first matrix element to be multiplied by the first operand is a negative number;
[0026] The shift-add unit is further configured to: shift the third operand by a preset number of bits in a preset direction to generate one or more corresponding fourth operands, wherein the preset direction and the preset number of bits are determined according to a first matrix element multiplied by the first operand.
[0027] In some embodiments, the shift-and-add unit is further configured to:
[0028] A sum of all second operands or fourth operands generated from the N first operands is calculated.
[0029] In some embodiments, the first matrix element cik multiplied by the first operand Xkj satisfies cik=∑[(-1) s *2 p ]:
[0030] If s is even, the first operand is shifted left by p bits to produce the corresponding second operand;
[0031] If s is an odd number, the first operand is inverted bit by bit to produce the third operand, and the third operand is shifted left by p bits to produce the corresponding fourth operand;
[0032] Among them, the i-th row is the current operation row of the first matrix, and the j-th column is the current operation column of the second matrix.
[0033] In some embodiments, the multiplication-accumulation subcircuit is configured to calculate the inner product of the first vector and the second vector and the sum of a preset constant, wherein the preset constant is determined based on a conversion relationship between a second matrix element associated with the second vector and a corresponding first operand and a first matrix element associated with the first vector.
[0034] In some embodiments, when there are at least two first matrix elements having the same absolute value in at least one row of the first matrix, the operation circuit further comprises:
[0035] A symmetric operator circuit is configured to calculate a sum or difference of second matrix elements or corresponding first operands multiplied by at least two first matrix elements having the same absolute value in the same row of the first matrix.
[0036] In some embodiments, the symmetry operation subcircuit is arranged between the sign processing subcircuit and the multiplication and accumulation subcircuit.
[0037] In some embodiments, the symmetric operation subcircuit includes a single-stage butterfly operation unit or a plurality of cascade-connected butterfly operation units.
[0038] In some embodiments, the second matrix is a transposed matrix of the image matrix; or,
[0039] The second matrix is a transposed matrix of a fourth matrix, wherein the fourth matrix is a matrix obtained by multiplying the first matrix by the transposed matrix of the image matrix;
[0040] The matrix elements in the image matrix represent pixel parameters of pixels at corresponding positions in the image to be processed, or the matrix elements in the image matrix represent the difference between pixel parameters of pixels at corresponding positions in two adjacent frames of the image to be processed.
[0041] In some embodiments, the operation circuit is configured to perform a discrete cosine transform operation, and the first matrix is a discrete cosine transform coefficient matrix.
[0042] According to a second aspect of the present disclosure, a processor is provided, comprising the arithmetic circuit as described above.
[0043] According to a third aspect of the present disclosure, a computing device is provided, comprising the computing circuit as described above or the processor as described above.
[0044] Other features and advantages of the present disclosure will become more apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0046] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
[0047] Figure 1 A schematic diagram of the structure of an operation circuit for performing DCT8 operation is shown;
[0048] Figure 2 A schematic diagram of the structure of a computing circuit according to an exemplary embodiment of the present disclosure is shown;
[0049] Figure 3 A schematic diagram of the structure of a symbol processing subcircuit in a computing circuit according to a specific embodiment of the present disclosure is shown;
[0050] Figure 4 A schematic diagram showing the structure of a symbol processing subcircuit in an operation circuit according to another specific embodiment of the present disclosure is shown;
[0051] Figure 5 A schematic diagram showing the structure of a sign bit determination unit in an operation circuit according to a specific embodiment of the present disclosure is shown;
[0052] Figure 6 A schematic diagram showing the structure of a sign bit determination unit in an operation circuit according to another specific embodiment of the present disclosure is shown;
[0053] Figure 7 A schematic diagram of the structure of a sign bit determination unit in a computing circuit according to another specific embodiment of the present disclosure is shown;
[0054] Figure 8 A schematic diagram of the structure of a multiplication-accumulation subcircuit in a computing circuit according to a specific embodiment of the present disclosure is shown;
[0055] Fig. 9 A schematic structural diagram of an operation circuit according to another exemplary embodiment of the present disclosure is shown;
[0056] Fig.10 Another schematic diagram of the structure of a computing circuit for performing DCT8 operation is shown;
[0057] Fig.11 A schematic diagram of the structure of another operation circuit for performing DCT8 operation is shown;
[0058] Fig.12 A block diagram of a processor according to an exemplary embodiment of the present disclosure is shown;
[0059] Fig.13 A block diagram of a computing device according to an exemplary embodiment of the present disclosure is shown;
[0060] Fig.14 A block diagram of a computing device according to another exemplary embodiment of the present disclosure is shown.
[0061] Note that in the embodiments described below, the same reference numerals are sometimes used in common between different drawings to represent the same parts or parts with the same functions, and their repeated descriptions are omitted. In this specification, similar reference numerals and letters are used to represent similar items, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.
[0062] For ease of understanding, the positions, sizes, and ranges of various structures shown in the drawings and the like sometimes do not represent actual positions, sizes, and ranges, etc. Therefore, the disclosed invention is not limited to the positions, sizes, and ranges, etc. disclosed in the drawings and the like. In addition, the drawings are not necessarily drawn to scale, and some features may be exaggerated to show details of specific components. DETAILED DESCRIPTION
[0063] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure.
[0064] The following description of at least one exemplary embodiment is in fact merely illustrative and is in no way intended to limit the present disclosure and its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in the present disclosure. However, those skilled in the art will appreciate that they merely illustrate exemplary ways of the present disclosure that can be implemented, rather than exhaustive ways. In addition, the drawings need not be drawn to scale, and some features may be enlarged to illustrate the details of specific components.
[0065] In addition, technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0066] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0067] In various data processing processes, matrix multiplication is a common form of operation. For example, the multiplication operation of the first matrix C and the second matrix X can be expressed as:
[0068] Y=CX (1)
[0069] If cij is used to represent the first matrix element located at the i-th row and j-th column in the first matrix C, xij is used to represent the second matrix element located at the i-th row and j-th column in the second matrix X, and yij is used to represent the third matrix element located at the i-th row and j-th column in the third matrix Y obtained by multiplying the first matrix by the second matrix, wherein the values of i and j can be integers greater than or equal to zero respectively (it can be understood by those skilled in the art that the values of i and j can also be integers greater than or equal to 1 respectively, which does not affect the essence of the operation, so in this article, the values of i and j starting from zero will be taken as an example for detailed description), then, according to the rules of matrix multiplication operation, the third matrix element yij in the third matrix Y can be expressed as follows:
[0070] (2)
[0071] The number of columns of the first matrix C and the number of rows of the second matrix X are each N, and k is an integer.
[0072] In a specific example, the operation circuit for implementing matrix multiplication as described herein can be configured to perform discrete cosine transform (DCT) operations. Among them, DCT, as an important part of image or video encoding, can transform an image described in the form of pixels in the spatial domain to a transform domain. Considering that most images usually contain a large range of flat areas or areas with slowly changing content, the use of appropriate DCT operations can convert the relatively dispersed distribution of image energy in the spatial domain into a relatively concentrated distribution in the transform domain, thereby achieving the purpose of removing spatial redundancy and simplifying subsequent image or video data processing. For example, a two-dimensional DCT operation can be expressed in the form of matrix multiplication as shown below:
[0073] Z=CX 0 C T =C(CX 0 T ) T (3)
[0074] Here, X 0represents the image matrix corresponding to the image to be processed. In some embodiments, the matrix elements in the image matrix may represent the pixel parameters of the pixels at the corresponding positions in a certain frame of the image to be processed. The pixel parameters may include, for example, the RGB parameters of the pixels or other types of color parameters. In other embodiments, considering that the difference between two adjacent frames of images in a video is usually small, in order to optimize the compression rate of video encoding, two adjacent frames of images in a video may be subtracted for processing. In this case, the matrix elements in the image matrix may represent the difference between the pixel parameters of the pixels at the corresponding positions in the two adjacent frames of images to be processed. For example, the matrix elements may represent the difference in color parameters of the pixels at the same corresponding positions in the two adjacent frames of images. X in the above formula (3) 0 T Represents the image matrix X 0 In some operations (such as the first matrix multiplication of DCT operation), X 0 T The matrix multiplication operation is performed by using the coefficient matrix of the two-dimensional DCT operation as the second matrix X and the first matrix C. In addition, in some operation processes (such as the second matrix multiplication of the DCT operation), the first matrix C is multiplied by the transposed matrix X of the image matrix. 0 T The resulting fourth matrix CX 0 T , the transposed matrix of the fourth matrix (CX 0 T ) T As the second matrix X, the coefficient matrix of the two-dimensional DCT operation is used as the first matrix C to perform matrix multiplication operations, thereby completing the entire DCT operation. In other words, the two-dimensional DCT operation can be implemented by two one-dimensional DCT operations, and in each one-dimensional DCT operation, the multiplication of two matrices will be involved. In some specific examples below, the technical solution of the present disclosure will be described in detail using the DCT operation as an example. However, it can be understood by those skilled in the art that the operation circuit described herein and the processor and computing device including the operation circuit can be used for the multiplication operation of any matrix involved in any type of data processing process, without limitation here.
[0075] Taking DCT8 (where the dimensions of each matrix involved in the multiplication operation are 8*8) as an example, its coefficient matrix C can be expressed as follows:
[0076] ,
[0077] It can be seen that there are positive coefficients and negative coefficients in the coefficient matrix C of DCT8, so the operation circuit usually used to perform matrix multiplication should also be compatible with operations between signed numbers. In addition, it can be noted that the coefficient matrix C of DCT8 presents a high degree of symmetry, so these symmetric relationships can be used to simplify the operation circuit. In a specific example, the following can be used: Figure 1 The operation circuit shown is used to implement the matrix multiplication operation of each one-dimensional DCT operation in the DCT8 operation. The operation circuit may include a symmetric operation subcircuit 120' and a multiplication-accumulation subcircuit 110'. Among them, the symmetric operation subcircuit 120' can be used to execute the butterfly algorithm, which is designed based on the symmetric relationship in the coefficient matrix C of the DCT8 operation, as will be explained in detail later. The multiplication-accumulation subcircuit 110' can be formed by units or modules such as multipliers, adders or adder trees to implement the multiplication-accumulation operation in the above formula (2) to obtain the corresponding third matrix element in the third matrix Y. However, due to the existence of signed numbers in matrix multiplication, Figure 1 The operation circuit shown, especially the multiplication and accumulation subcircuit 110' must be able to perform multiplication and addition operations of signed numbers, that is, the sign bits of the operands involved in the operation must be processed during the operation, which results in Figure 1 The operation circuit shown has the disadvantages of complex structure, high power consumption, and difficulty in meeting the timing requirements.
[0078] In order to solve the above problem, the present disclosure provides an operation circuit for performing an operation of multiplying a first matrix C by a second matrix X, such as Figure 2 As shown, the operation circuit 100 may include a sign processing subcircuit 130 and a multiplication-accumulation subcircuit 110 .
[0079] As described above, according to the operation rules of matrix multiplication, the number of columns of the first matrix C should be equal to the number of rows of the second matrix X. For the convenience of description, the number of columns of the first matrix C and the number of rows of the second matrix X are each N. In addition, according to the operation rules of matrix multiplication, the number of rows of the third matrix Y obtained by multiplying the first matrix C by the second matrix X should be equal to the number of rows N' of the first matrix C, and the number of columns of the third matrix Y should be equal to the number of columns N'' of the second matrix X. In some detailed examples below, the first matrix, the second matrix and the third matrix are all square matrices. However, it can be understood by those skilled in the art that in some embodiments, the first matrix, the second matrix and the third matrix may not be square matrices, and there is no restriction here, as long as their number of rows and columns meet the limitations brought by the operation rules of the above-mentioned matrix multiplication. In addition, in the exemplary embodiments of the present disclosure, each first matrix element cij in the first matrix C can be a pre-known coefficient, so as to determine the specific structure of the operation circuit and the values of various parameters that may be involved therein according to these coefficients, as will be described in detail later. In addition, the second matrix X may be generated according to the data to be processed, and the data to be processed may include image data, video data, or other types of data, etc., which are not limited here.
[0080] The inventors of the present disclosure note that, in some embodiments, the above formula (2) involving the operation of the signed number xkj can be rewritten into the following form through the following mathematical derivation:
[0081] (4)
[0082] Where M is the bit width of the second matrix element in the second matrix X. By inverting the highest bit (or sign bit) of the signed number xkj, the signed number xkj can be converted to the unsigned number Xkj, that is, Xkj=xkj+2 M-1 .
[0083] Further, in the actual data processing process, most of the second matrix elements xkj involved may be a number with a small absolute value (for example, in the case where the matrix elements in the image matrix described above represent the difference in pixel parameters of pixels at corresponding positions in two adjacent frames of the image to be processed). In other words, multiple high bits in such a second matrix element may all be extended sign bits, or the highest bit of such a second matrix element is not its real sign bit, then these extended sign bits participate in the operation process, which will cause a large number of empty flips of related devices, units or modules in the operation circuit. The power consumption of the operation circuit can be divided into static power consumption and dynamic power consumption, wherein the static power consumption is generally related to the scale of the operation circuit, and the dynamic power consumption is not only related to the scale of the operation circuit, but also to the flip rate in the operation circuit. It can be seen that the large number of empty flips caused by the extended sign bit will lead to a significant increase in the power consumption of the operation circuit. In this regard, the power consumption of the operation circuit can be reduced by reducing the flip rate of the operation circuit. In other embodiments, the values of at least part of the extended sign bits of the qualified second matrix elements can be set to zero to reduce or even avoid the extended sign bits from participating in the matrix multiplication operation, thereby reducing the flip rate of the operation circuit. Corresponding to this situation, the above formula (2) involving the operation of the signed number xkj can be rewritten as follows:
[0084] (4')
[0085] Wherein, M is the bit width of the second matrix element in the second matrix X, and m is the number of bits of the extended sign bit set to zero of the second matrix element. Here, the lower bit of the second matrix element adjacent to the extended sign bit set to zero is used as the target sign bit of the second matrix element. By setting the upper m bits of the second matrix element to zero and inverting the target sign bit of the second matrix element, the signed number xkj can be converted to the unsigned number Xkj, that is, Xkj=xkj+2 M-m-1 It can be understood that, in order to simplify subsequent operations, the same sign bit processing is generally performed on a group of second matrix elements associated with the same third matrix element. Therefore, for each second matrix element, its target sign bit may be the real sign bit of the second matrix element, or it may be a certain extended sign bit of the second matrix element, as will be illustrated in detail later.
[0086] As described above, the sign processing subcircuit 130 may be configured to convert the second matrix element xkj in the second matrix X into a corresponding first operand Xkj in the form of an unsigned number. In the following text, for the sake of brevity, let m' be M or Mm, and the specific value of m' is as described above, so that Xkj=xkj+2 m’-1 .
[0087] Furthermore, the above formula (4) or (4') can be rewritten as follows:
[0088] (5)
[0089] in, Since cik, N and m' are all known constants, Ai1 is also a constant. Thus, Ai1 in the above formula (5) can be extracted as a constant term for subsequent processing, and the multiplication and accumulation subcircuit 110 can only calculate the first term in the above formula (5): It should be noted that, since Ai1 is related to cik, for different third matrix elements in the same column of the third matrix Y, the corresponding Ai1 values may be different and need to be processed separately.
[0090] Furthermore, since cik is a pre-known coefficient and Xkj is an unsigned number, when cik is a positive number, the value of cik*Xkj can be directly calculated according to the multiplication operation of unsigned numbers. When cik is a negative number, considering that cik is known in advance, cik*Xkj can be converted into an unsigned multiplication operation in advance, as will be explained in detail later.
[0091] like Figure 2 As shown, the input end of the multiplication-accumulation subcircuit 110 can be communicatively connected to the output end of the sign processing subcircuit 130, and the multiplication-accumulation subcircuit 110 can be configured to at least calculate the inner product of the first vector and the second vector, wherein the kth component of the first vector is the kth first matrix element cik in the current operation row (e.g., the i-th row) of the first matrix C, and the kth component of the second vector is the first operand Xkj obtained by converting the kth second matrix element xkj in the current operation column (e.g., the j-th column) of the second matrix X. It can be understood that the dimension of the first vector and the dimension of the second vector are N respectively.
[0092] In some embodiments, all third matrix elements in the same column of the third matrix Y or all vector inner products related to the third matrix elements as described above may be calculated in parallel or substantially in parallel in the same operation cycle of the operation circuit.
[0093] To this end, the operation circuit may include N symbol processing subcircuits 130, and in the same operation cycle, each symbol processing subcircuit 130 may be configured to respectively convert a corresponding second matrix element in the same column of the second matrix X. For example, in order to implement the DCT8 operation, the operation circuit may include a total of 8 symbol processing subcircuits 130, wherein the kth symbol processing subcircuit may be configured to receive the kth second matrix element xkj in the jth column (i.e., the current operation column) of the second matrix processed in the current operation cycle, where k is an integer and 0≤k<8.
[0094] Similarly, the operation circuit may include N' multiplication and accumulation subcircuits 110, as described above, where N' is the number of rows of the first matrix C. Thus, in the same operation cycle, each multiplication and accumulation subcircuit 110 may be configured to respectively calculate the inner product of the first vector and the second vector associated with a corresponding third matrix element in the same column of the third matrix Y. It is understandable that Figure 2 Each multiplication-accumulation subcircuit 110 shown in the figure can be communicatively connected with all the symbol processing subcircuits 130 to obtain all the second matrix elements in the same column of the second matrix X for calculating the inner product of the first vector and the second vector associated with the corresponding third matrix element, and for the sake of simplicity in the accompanying drawings, the communication connections between all the symbol processing subcircuits 130 and the multiplication-accumulation subcircuits 110 are not drawn one by one. In the case where the first matrix and the second matrix are square matrices, N'=N. For example, in order to implement the DCT8 operation, the operation circuit may include a total of 8 multiplication-accumulation subcircuits 110, wherein each multiplication-accumulation subcircuit 110 may be configured to calculate the inner product of the first vector and the second vector associated with the corresponding third matrix element ykj in the j-th column of the third matrix Y, where k is an integer and 0≤k<8.
[0095] Further, in an exemplary embodiment of the present disclosure, an operation circuit 100 including N symbol processing subcircuits 130 and N' multiplication and accumulation subcircuits 110 is used to complete the operation of multiplying the first matrix C by the second matrix X in a total of N'' operation cycles, and obtain all third matrix elements in the third matrix or vector inner products or other parameter values related to these third matrix elements, as described above, N'' is the number of columns of the second matrix. In the case where the first matrix and the second matrix are square matrices, N''=N.
[0096] In some other embodiments, the operation circuit may include more or fewer symbol processing subcircuits 130 or multiplication-accumulation subcircuits 110, and a buffer may be provided in the operation circuit to cache the intermediate results involved in the operation process, thereby completing the matrix multiplication operation, which is not limited here. In addition, as the scale of the operation circuit 100 increases (for example, the number of symbol processing subcircuits 130 or multiplication-accumulation subcircuits 110 increases), the number of operation cycles required for matrix operations may decrease, and the operation time may decrease accordingly, but at the same time, the structural complexity of the operation circuit 100 may increase. Therefore, a trade-off may be made according to actual needs to determine the appropriate number and arrangement of the symbol processing subcircuits 130 and the multiplication-accumulation subcircuits 110.
[0097] In some embodiments, as described above, the sign processing subcircuit 130 may be configured to invert the target sign bit of the second matrix element to generate the corresponding first operand. In a specific example, the target sign bit of the second matrix element may be the highest bit of the second matrix element, that is, the first operand Xkj=xkj+2 M-1 .
[0098] Accordingly, if Figure 3 As shown, the symbol processing subcircuit 130 may include an inverter 131, and the input end of the inverter 131 may be configured to receive the target sign bit (i.e., the highest bit) of the second matrix element to invert the target sign bit. In addition, the symbol processing subcircuit 130 may directly output other bits of the second matrix element, and the corresponding first operand may be obtained by splicing the inverted target sign bit with other bits in sequence. It is understandable that, in the case where the symbol processing subcircuit 130 processes only one second matrix element at a time, it may have only one inverter 131, and the symbol processing subcircuit 130 may also have M-1 paths to directly output other bits of the second matrix element except the target sign bit, where M is the total number of bits or the bit width of the second matrix element. In some other embodiments, if the symbol processing subcircuit 130 processes multiple second matrix elements at a time, it may include multiple inverters 131 and a corresponding number of other paths, etc., which are not limited here.
[0099] In other embodiments, as described above, the values of at least part of the extended sign bits of the second matrix element may be set to zero to suppress high-bit flipping, reduce flipping of the operation circuit, and thus reduce power consumption of the operation circuit.
[0100] Specifically, in some embodiments, the sign processing subcircuit 130 may be configured to determine whether the upper m bits of each second matrix element in the current operation column (e.g., the jth column) of the second matrix are all extended sign bits. Wherein, m may be a preset positive integer, and m is less than the total number of bits M of the second matrix element. For example, based on experience, m may be set to a maximum integer less than half of the total number of bits M of the second matrix element. Of course, m may also be set to other values based on experience in order to minimize the flipping of the operation circuit as much as possible, which is not limited here. Further, the sign processing subcircuit 130 may be configured to, when the upper m bits of each second matrix element in the current operation column are all extended sign bits, set the upper m bits of the second matrix element to zero, and invert the target sign bit of the second matrix element to generate a corresponding first operand, where the target sign bit of the second matrix element is the lower one adjacent to the upper m bits of the second matrix element; or, when the upper m bits of at least one second matrix element in the current operation column are not all extended sign bits, invert the target sign bit of the second matrix element to generate a corresponding first operand, where the target sign bit of the second matrix element is the highest bit of the second matrix element. That is to say, based on the experience accumulated in the data processing process, etc., a suitable value of m can be set in advance, and for each group of second matrix elements (that is, second matrix elements located in the same column in the second matrix) participating in the operation of each third matrix element, it can be determined whether their upper m bits are all extended sign bits: if so, the number of bits of this group of second matrix elements can be reduced, that is, the upper m bits of each second matrix element in this group of second matrix elements are set to 0, and the lower bit adjacent to the upper m bits is taken as the target sign bit for inversion; otherwise, the number of bits of this group of second matrix elements is not reduced, but the highest bit of each second matrix element in this group of second matrix elements is directly inverted for subsequent calculations.
[0101] In a specific example, assuming that M=8, and a total of eight second matrix elements in a column participating in the DCT8 operation include: 11110001, 11110101, 11111101, 11110000, 11111001, 11111111, 00000001, 00001001, if the value of m is set to 3, then a group of first operands obtained by adjusting the sign bit of this group of second matrix elements in the manner described above include: 00000001, 000001001, 0 ... 1, 00001101, 00000000, 00001001, 00001111, 00010001, 00011001; and if the value of m is set to 5, then a set of first operands obtained by adjusting the sign bits of the set of second matrix elements in the manner described above include: 01110001, 01110101, 01111101, 01110000, 01111001, 01111111, 10000001, 10001001.
[0102] To implement the sign bit adjustment as described above, Figure 4 As shown, the sign processing subcircuit 130 may further include a sign bit determination unit 133 to determine whether the upper m bits of each second matrix element in the current operation column of the second matrix are extended sign bits. The sign bit determination unit 133 may be configured in a variety of ways, which are not limited herein.
[0103] In some embodiments, Figure 5 As shown, the sign bit determination unit 133 may include a first AND gate unit, and the first AND gate unit may have multiple input terminals, each of which may be configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in the current operation column, and the output terminal of the first AND gate unit may be configured to output an indication signal indicating whether the received multiple to-be-determined bits are all extended sign bits. In a specific example, a corresponding first AND gate unit may be set for a second matrix element, and the first AND gate unit may include m+1 input terminals, respectively used to receive a corresponding one of the high m+1 bits of the second matrix element. If the output of the first AND gate unit is 1, it means that the values of the high m+1 bits of the second matrix element are all 1, so it can be determined that the high m bits of the second matrix element are all extended sign bits (corresponding to the case where the second matrix element is a negative number); otherwise, the high m bits of the second matrix element are not necessarily all extended sign bits.
[0104] In some embodiments, Figure 6As shown, the sign bit determination unit 133 may include an OR gate unit, and the OR gate unit may have multiple input terminals, each of which may be configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in the current operation column, and the output terminal of the OR gate unit may be configured to output an indication signal indicating whether the received multiple to-be-determined bits are all extended sign bits. In a specific example, a corresponding OR gate unit may be set for a second matrix element, and the OR gate unit may include m+1 input terminals, which are respectively used to receive a corresponding bit in the high m+1 bits of the second matrix element. If the output of the OR gate unit is 0, it means that the values of the high m+1 bits of the second matrix element are all 0, so it can be determined that the high m bits of the second matrix element are all extended sign bits (corresponding to the case where the second matrix element is a positive number); otherwise, the high m bits of the second matrix element are not necessarily all extended sign bits.
[0105] Since the value of the second matrix element may be a positive number or a negative number, in some embodiments, the first AND gate unit and the OR gate unit may jointly form a sign bit determination unit 133 to accurately determine whether the upper m bits of the second matrix element are all extended sign bits.
[0106] Or, if Figure 7 As shown, the sign bit determination unit 133 may include an XOR gate unit, which may be composed of one or more XOR gates, and the XOR gate unit may have multiple input terminals, each of which may be configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in the current operation column, and the output terminal of the XOR gate unit may be configured to output an indication signal indicating whether the received multiple to-be-determined bits are all extended sign bits. In a specific example, a corresponding XOR gate unit may be set for a second matrix element, and the XOR gate unit may include m+1 input terminals, which are respectively used to receive a corresponding one of the high m+1 bits of the second matrix element to determine whether the high m bits of the second matrix element are all extended sign bits.
[0107] In some other embodiments, the sign processing subcircuit 130 may be configured to determine the position of the real sign bit of each second matrix element in the current operation column (e.g., the jth column) of the second matrix, and use the highest real sign bit among the real sign bits of all the second matrix elements in the current operation column as the target sign bit; and set all the bits of the second matrix element higher than the target sign bit to zero, and invert the target sign bit of the second matrix element to generate the corresponding first operand. In other words, the real sign bit and the extended sign bit of each second matrix element in each group of second matrix elements may be determined one by one, so as to maximize the reduction of the number of bits or the adjustment of the sign bit for the group of second matrix elements, thereby reducing the flipping of the operation circuit and reducing power consumption.
[0108] In a specific example, assuming that M=8, and a group of eight second matrix elements participating in the DCT8 operation include: 11110001, 11110101, 11111101, 11110000, 11111001, 11111111, 00000001, 00001001, then the high 3 bits, high 3 bits, high 5 bits, high 3 bits, high 4 bits, high 7 bits, high 8 bits of each of the above second matrix elements are The 6th bit and the upper 3 bits are the extended sign bits respectively. Therefore, the 4th bit (the lowest bit is the 0th bit) of the second matrix element should be used as the target sign bit. The corresponding set of first operands obtained by adjusting the sign bit in the above manner include: 00000001, 00000101, 00001101, 00000000, 00001001, 00001111, 00010001, 00011001.
[0109] Similarly, to implement the sign bit adjustment as described above, Figure 4 As shown, the sign processing subcircuit 130 may include a sign bit determination unit 133 to determine the position of the real sign bit of each second matrix element in the current operation column of the second matrix. It is understandable that the sign bit determination unit 133 may also be formed by a first AND gate unit, an OR gate unit or an XOR gate unit. Different from the above embodiment, since the sign bit determination unit 133 here needs to determine the position of the real sign bit of each second matrix element in the current operation column of the second matrix, it may include M input terminals, where M is the bit width of the second matrix element, so as to determine the real sign bit of each second matrix element. It is understandable that in some other embodiments, other ways may be used to set the sign bit determination unit 133, which is not limited here.
[0110] As described above, in order to achieve the inversion of the determined target sign bit, such as Figure 4 As shown, the sign processing subcircuit 130 may include an inverter 131, and the input end of the inverter 131 may be configured to receive the target sign bit of the second matrix element to invert the target sign bit. Alternatively, other units or modules may be used to invert the target sign bit, which is not limited here.
[0111] Furthermore, if Figure 4As shown, the symbol processing subcircuit 130 may also include a second AND gate unit 132, and the two input ends of the second AND gate unit 132 may be configured to respectively receive a signal zero for indicating zero and a corresponding to-be-set-to-zero position of a corresponding second matrix element, so as to set the to-be-set-to-zero position to zero, thereby reducing the flip of the operation circuit and reducing power consumption. In a specific example, a corresponding symbol processing subcircuit 130 may be set for a second matrix element. If it is necessary to set all the high m bits of the second matrix element to zero, then the symbol processing subcircuit 130 may have m second AND gate units 132. In addition, in the above embodiment of determining the true sign bit of each second matrix element in each group of second matrix elements one by one, the symbol processing subcircuit 130 may have M-1 second AND gate units 132, and the activation or deactivation of each second AND gate unit 132 may be controlled according to the determined target sign bit to achieve the desired sign bit adjustment. Alternatively, other units or modules may be used to set at least part of the extended sign position of the second matrix element to zero, which is not limited here.
[0112] As described above, after generating the first operand in the form of an unsigned number according to the second matrix element, the multiplication-accumulation subcircuit 110 can be used to perform at least the calculation of the vector inner product. In some embodiments, the constant Ai1 described above can be incorporated into other operations other than the matrix multiplication operation, so that the multiplication-accumulation subcircuit 110 can only calculate the first term in the above formula (5), that is, . Alternatively, in some other embodiments, the multiplication-accumulation subcircuit 110 may be configured to calculate the sum of the inner product of the first vector and the second vector and a preset constant. In a specific example, the preset constant may be Ai1, which may be determined based on the conversion relationship between the second matrix element associated with the second vector and the corresponding first operand and the first matrix element associated with the first vector, that is, the multiplication-accumulation subcircuit 110 may calculate the result of the above formula (5). Alternatively, in some other specific examples, the preset constant may be other constants associated with Ai1, which depends on the specific data processing method and is not limited here. It can be understood that for each third matrix element in the same column of the third matrix Y, the value of the corresponding preset constant may be different, so the corresponding preset constants may be set respectively in the multiplication-accumulation subcircuit 110 used to calculate each third matrix element.
[0113] In some embodiments, the multiplication-accumulation subcircuit 110 may include a multiplier and an adder (or an adder tree, a subtractor, etc.) to calculate or Here, since the signed number xkj has been converted into the first operand Xkj in the form of an unsigned number, at least the multiplier in the multiplication-accumulation subcircuit 110 only needs to support the multiplication operation of unsigned numbers, thereby simplifying the structure of the operation circuit and helping to reduce the power consumption of the operation circuit. However, since the coefficient cik may be a negative number, a negative number may still be generated in the addition or subtraction operation after the multiplication operation, resulting in the need to process the sign. In order to further simplify the operation, the pre-known coefficient cik can be decomposed into the form shown below:
[0114] cik=∑[(-1) s *2 p ] (6)
[0115] That is to say, cik can be expressed as a power of 2 or the addition or subtraction of multiple powers of 2. For the sake of simplicity, the value of s can be 0 or 1.
[0116] In a specific example, the coefficient matrix C used for DCT8 operation is taken as an example, where:
[0117] c30=75=0B1001011=(-1) 0 *2 6 +(-1) 0 *2 3 +(-1) 0 *2 1 +(-1) 0 *2 0 , that is, the value of s in each item is 0, and the values of p are 0, 1, 3, and 6 respectively;
[0118] c31=-18=(-1) 1 *0B10010=(-1) 1 *2 4 +(-1) 1 *2 1 , that is, the value of s in each item is 1, and the values of p are 1 and 4 respectively;
[0119] c32=-89=(-1) 1 *0B1011001=(-1) 1 *2 6 +(-1) 1 *2 4 +(-1) 1 *2 3 +(-1) 1 *2 0 , that is, the value of s in each item is 1, and the values of p are 0, 3, 4, and 6 respectively;
[0120] c33 = -50 = (-1) 1 *0B110010 = (-1) 1 *2 5 +(-1) 1 *2 4 +(-1) 1 *2 1 That is, in each term, the value of s is 1, and the values of p are 1, 4, and 5 respectively.
[0121] Alternatively, some coefficients can have multiple representations. For example, taking the value of the coefficient as 31, it can be represented as (-1) 0 *2 4 +(-1) 0 *2 3 +(-1) 0 *2 2 +(-1) 0 *2 1 +(-1) 0 *2 0 in the form of, or can also be represented as (-1) 0 *2 5 +(-1) 1 *2 0 in the form of.
[0122] As will be described in detail later, each summation term in the above formula (6) can be associated with a shift of the operand. Therefore, if the number of summation terms can be reduced as much as possible, the number of shifts of the operand can be reduced accordingly, which helps to simplify the structure of the arithmetic circuit, reduce the arithmetic power consumption, and ensure timing convergence. Thus, in the specific example of the coefficient with a value of 31 above, representing it as (-1) 0 *2 5 +(-1) 1 *2 0 for processing is generally more preferable.
[0123] Furthermore, 2 p *Xkj is equivalent to shifting the first operand Xkj in binary form to the left by p bits. Thus, if p1 represents any value of p when s = 0, then 2 p1 *Xkj = Xkj << p1; and if p2 represents any value of p when s = 1, then based on the rule of representing negative numbers in two's complement, (-1)*Xkj = ~Xkj + 1, where ~Xkj is the bitwise inversion of Xkj. Then, (-1)*2 p2 *Xkj = 2 p2 *(~Xkj + 1) = (~Xkj << p2) + 2 p2 , where 2p2 It can be extracted as a constant term for subsequent processing. In this way, the above formula (5) can be rewritten as follows:
[0124] (7)
[0125] Among them, the constant term Ai2 is based on Ai1 and all the 2 p2 Determined.
[0126] According to the above formula (7), the multiplication-accumulation operation can be implemented by using shift calculation and accumulation calculation. Figure 8 As shown, the multiplication and accumulation subcircuit 110 may include a shift addition unit 111, and the shift addition unit 111 may be configured to shift the first operand by a preset number of bits in a preset direction to generate one or more corresponding second operands when the first matrix element multiplied by the first operand is a positive number, wherein the preset direction and the preset number of bits are determined according to the first matrix element multiplied by the first operand. Taking the coefficient c30=75=(-1) in the DCT8 operation 0 *2 6 +(-1) 0 *2 3 +(-1) 0 *2 1 +(-1) 0 *2 0 For example, the first operand Xkj multiplied by the coefficient can be shifted left by 0, 1, 3 and 6 bits respectively to obtain a total of four second operands, namely Xkj itself, Xkj<<1, Xkj<<3 and Xkj<<6, for further processing.
[0127] In some embodiments, Figure 8 As shown, the multiplication and accumulation subcircuit 110 may also include a bit-by-bit inversion unit 112. The bit-by-bit inversion unit 112 may be configured to invert the first operand bit by bit when the first matrix element multiplied by the first operand is a negative number to generate a corresponding third operand, that is, X3kj=~Xkj. In some embodiments, the bit-by-bit inversion unit 112 may be formed by a plurality of inverters. Alternatively, the bit-by-bit inversion unit 112 may be formed in other ways, which are not limited here. Further, the shift addition unit 111 may also be configured to shift the third operand by a preset number of bits in a preset direction to generate one or more corresponding fourth operands, wherein the preset direction and the preset number of bits are determined based on the first matrix element multiplied by the first operand. Taking the coefficient c31=-18=(-1) in the DCT8 operation 1 *2 4 +(-1) 1 *2 1For example, the first operand Xkj multiplied by the coefficient can be first bit-by-bit inverted by the bit-by-bit inversion unit 112 to generate a corresponding third operand X3kj=~Xkj, and then the third operand X3kj can be shifted left by 1 bit and 4 bits respectively by the shift and add unit 111 to obtain a total of two fourth operands, namely X3kj<<1 and X3kj<<4, for further processing.
[0128] Further, the shift addition unit 111 may also be configured to calculate the sum of all second operands or fourth operands generated according to the N first operands, that is, to calculate the sum of the first term and the second term in the above formula (7). In addition, in some embodiments, the shift addition unit 111 may also calculate the sum of all three terms in the above formula (7) to obtain the value of the third matrix element yij. It is understandable that for each third matrix element in the same column of the third matrix Y, the value of its corresponding constant term Ai2 may be different, so the corresponding constant term Ai2 may be set in the multiplication and accumulation subcircuit 110 used to calculate each third matrix element.
[0129] It can be understood that since all the coefficients in the first matrix C are known in advance, the number and arrangement of the shift addition units 111 and the bit-by-bit inversion units 112 in the multiplication and accumulation subcircuit 110 can be determined in advance based on the values of these coefficients, thereby simplifying the design of the operation circuit.
[0130] In some embodiments, the shift addition unit 111 can be implemented by a compression tree. The compression tree can efficiently compress multiple data into the form of the sum of two or three data. In addition, as needed, the shift operation of the related data can also be implemented inside the compression tree. In this way, by using the compression tree in combination with the bitwise inversion unit, the shift and addition operations as in the above formula (7) can be conveniently implemented, thereby compressing the shift sum of multiple numbers into the sum of two or three numbers. Common compression trees can include compression trees with two output ends such as 4:2 compression trees and 3:2 compression trees, and can also include compression trees with three output ends such as 5:3 compression trees, 6:3 compression trees, and 7:3 compression trees. The compression tree here can be a compression tree with any number of input ends and output ends that is currently available or developed later, or it can be a compression tree module with any number of input ends and output ends that is implemented by combining multiple existing compression trees. For simplicity, in the embodiments of the present disclosure, a compression tree with two output ends can be used, and then the two output data of the compression tree are added by an adder to obtain the operation result related to the third matrix element. In some other embodiments, the shift addition unit 111 may also be alternatively implemented as a full adder or a combination of a full adder and a half adder.
[0131] In most cases, especially in DCT operations, there are a large number of symmetric relationships in the first matrix or coefficient matrix, so these symmetric relationships can be used to further simplify the operation. Specifically, multiple second matrix elements multiplied by coefficients with the same absolute value can be added or subtracted in advance to reduce the number of multiplications, additions or subtractions, or shift calculations performed in the multiplication-accumulation subcircuit 110, thereby helping to simplify the circuit structure, reduce power consumption, and avoid problems such as difficulty in timing convergence caused by multiple cascade operations. Fig. 9 As shown, in the case where there are at least two first matrix elements with the same absolute value in at least one row of the first matrix, the operation circuit may further include a symmetric operation subcircuit 120, which may be configured to calculate the sum or difference of the second matrix element or the corresponding first operand multiplied by the at least two first matrix elements with the same absolute value in the same row of the first matrix. As required, the symmetric operation subcircuit 120 may be arranged before or after the sign processing subcircuit 130. Further, considering that the earlier the operands to be processed in the operation circuit are converted into unsigned numbers, the better the subsequent operations can be simplified, the symmetric operation subcircuit 120 is generally arranged between the sign processing subcircuit 130 and the multiplication and accumulation subcircuit 110.
[0132] In some embodiments, the symmetric operation subcircuit 120 may include a single-stage butterfly operation unit or a multi-stage butterfly operation unit connected in cascade, wherein each stage of the butterfly operation unit may include a plurality of adders or subtractors, and each adder or subtractor may be used to pre-calculate the sum or difference of a pair of second matrix elements or a corresponding pair of first operands. It is understood that the higher the symmetry in the coefficient matrix, the more stages of butterfly operation units may be set to simplify the calculation as much as possible.
[0133] Taking DCT8 operation as an example, observing the coefficient matrix of DCT8 operation shown above, we can see that the coefficients in the even rows (i.e., rows 0, 2, 4, and 6) have an even symmetric feature, that is, the value of the jth coefficient is equal to the value of the 7th-jth coefficient; while the coefficients in the odd rows (i.e., rows 1, 3, 5, and 7) have an odd symmetric feature, that is, the value of the jth coefficient is the negative of the value of the 7th-jth coefficient. Based on such characteristics, we can use Fig.10The four adders and four subtractors in the first-stage butterfly operation unit 121 shown in respectively calculate the sum and difference of the i-th first operand and the 7-i-th first operand corresponding to the i-th second matrix element and the 7-i-th second matrix element in the current operation column of the second matrix, i.e., X0j+X7j, X1j+X6j, X2j+X5j, X3j+X4j, X3j-X4j, X2j-X5j, X1j-X6j, X0j-X7j. Further, it can be noted that in the coefficient matrix of the DCT8 operation, for each even row, the coefficients in each half row also have the characteristics of even symmetry or odd symmetry. Accordingly, the following can be adopted: Fig.10 The second-stage butterfly operation unit 122 shown further calculates the sums and differences of these first operands, specifically including X0j+X7j+X3j+X4j, X1j+X6j+X2j+X5j, X0j+X7j-X3j-X4j and X1j+X6j-X2j-X5j.
[0134] Taking the calculation involving the coefficients of the second row in the coefficient matrix as an example, in the absence of the symmetric operator subcircuit 120, the multiplication and accumulation subcircuit 110 for calculating y2j may need to calculate the values of 83*X0j, 36*X1j, 36*(~X2j), 83*(~X3j), 83*(~X4j), 36*(~X5j), 36*X6j and 83*X7j respectively, and combine the addition operation to obtain the value associated with the third matrix element y2j; in the case where the symmetric operator subcircuit 120 is provided, it can be obtained by The symmetric operation subcircuit 120 first calculates the values of X0j+X7j-X3j-X4j and X1j+X6j-X2j-X5j, and then calculates the values of 83*(X0j+X7j-X3j-X4j) and 36*(X1j+X6j-X2j-X5j) in the multiplication and accumulation subcircuit 110, and combines the addition operation to obtain the value associated with the third matrix element y2j, thereby greatly reducing the number of multiplication operations or shift-addition operations required, thereby helping to reduce computing power consumption and ensure timing convergence.
[0135] In some embodiments, Fig.11 As shown, the symmetric operation subcircuit 120 may include an adder 123 and a compression tree 124, wherein the adder 123 may be used to calculate the addition or subtraction result of two first operands, and the compression tree 124 may be used to convert the addition or subtraction result of the four first operands into the form of the sum of two numbers, and further obtain the final addition or subtraction result through the adder 123. It can be understood that in some other embodiments, other methods may also be used to implement the above-mentioned symmetric operation subcircuit 120, which is not limited here.
[0136] The present disclosure also provides a processor, such as Fig.12 As shown, the processor 200 may include the arithmetic circuit 100 as described above. For example, such a processor 200 may be a neural network processor, a central processing unit, a coprocessor, a digital signal processor, a dedicated instruction processor, or other processors.
[0137] The present disclosure also proposes a computing device, such as Fig.13 and Fig.14 As shown, the computing device 300 may include the computing circuit 100 or the processor 200 as described above. Examples of the computing device 300 may include, but are not limited to, consumer electronic products, components of consumer electronic products, electronic testing equipment, image processing equipment, cellular communication infrastructure such as base stations, etc. Examples of the computing device 300 may include, but are not limited to, mobile phones such as smart phones, wearable computing devices such as smart watches or headphones, phones, televisions, computer monitors, computers, modems, handheld computers, laptop computers, tablet computers, personal digital assistants (PDAs), microwave ovens, refrigerators, in-vehicle electronic systems such as automotive electronic systems, stereo systems, DVD players, CD players, digital music players such as MP3 players, radios, portable cameras, cameras such as digital cameras, portable memory chips, washing machines, dryers, washing machines / dryers, peripheral devices, clocks, etc. In addition, the computing device 300 may include incomplete products.
[0138] In the technical solution disclosed in the present invention, by converting the signed number operation involved in the matrix multiplication operation into an unsigned number operation, the data path in the operation circuit is designed as a full unsigned number, thereby effectively simplifying the structure of the operation circuit, reducing the operation power consumption, and making the timing in the operation circuit easier to converge. Further, by processing at least part of the extended sign bit of the operand, the flip rate of the operation circuit can be effectively reduced, thereby further reducing the power consumption of the operation circuit. In addition, by converting the multiplication-accumulation operation into a shift and addition operation of the operand, the circuit structure can be further simplified. In some examples, based on the symmetry of the coefficient matrix in the matrix multiplication operation, some of the matrix elements participating in the operation can be added or subtracted in advance, thereby further reducing the number of multiplication operations involved and reducing the operation cost.
[0139] The words "left", "right", "front", "back", "top", "bottom", "up", "down", "high", "low", etc., in the specification and claims, if present, are used for descriptive purposes and are not necessarily used to describe an unchanging relative position. It should be understood that the words so used are interchangeable under appropriate circumstances, so that the embodiments of the present disclosure described herein, for example, can operate in other orientations than those shown or otherwise described herein. For example, when the device in the drawings is turned over, features originally described as "above" other features may now be described as "below" the other features. The device may also be oriented in other ways (rotated 90 degrees or in other orientations), and the relative spatial relationships will be interpreted accordingly.
[0140] In the specification and claims, when an element is referred to as being "on", "attached", "connected", "coupled", or "contacting" another element, the element may be directly on, directly attached, directly connected, directly coupled, or directly contacting another element, or one or more intermediate elements may be present. In contrast, when an element is referred to as being "directly" "on", "directly attached", "directly connected", "directly coupled", or "directly contacting" another element, there will be no intermediate elements. In the specification and claims, a feature being arranged "adjacent" to another feature may refer to a feature having a portion that overlaps with an adjacent feature or a portion that is located above or below an adjacent feature.
[0141] As used herein, the word "exemplary" means "serving as an example, instance, or illustration" rather than as a "model" to be exactly copied. Any implementation described as an example herein is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, the present disclosure is not to be limited by any expressed or implied theory given in the technical field, background, summary, or detailed description.
[0142] As used herein, the term "substantially" is intended to include any minor variations due to design or manufacturing imperfections, device or component tolerances, environmental influences, and / or other factors. The term "substantially" also allows for deviations from a perfect or ideal situation due to parasitic effects, noise, and other practical considerations that may exist in actual implementations.
[0143] In addition, the terms "first", "second" and the like may also be used herein for reference purposes only and are not intended to be limiting. For example, the terms "first", "second" and other such numerical terms referring to structures or elements do not imply a sequence or order unless the context clearly indicates otherwise.
[0144] It should also be understood that when the term “include / comprises” is used in this document, it indicates the presence of the specified features, integers, steps, operations, units and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, units and / or components and / or their combinations.
[0145] In addition, when used in this application, the words "herein", "above", "below", "hereunder", "above" and words of similar meaning shall refer to the entirety of this application rather than any particular portion of this application. In addition, unless otherwise expressly stated or understood in the context of use, conditional language used herein, such as "may", "might", "for example", "such as", etc., is generally intended to express that certain embodiments include, while other embodiments do not include certain features, elements and / or states. Therefore, such conditional language is generally not intended to imply that one or more embodiments require features, elements and / or states in any way, or whether these features, elements and / or states are included or performed in any particular embodiment.
[0146] In the present disclosure, the term "providing" is used in a broad sense to cover all ways of obtaining an object, so "providing an object" includes but is not limited to "purchasing", "preparing / manufacturing", "arranging / setting", "installing / assembling", and / or "ordering" an object, etc. In addition, in the present disclosure, the terms "circuit", "unit" and "module" can be used interchangeably.
[0147] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.
[0148] Those skilled in the art will appreciate that the boundaries between the above operations are merely illustrative. Multiple operations may be combined into a single operation, a single operation may be distributed in additional operations, and operations may be performed at least partially overlapping in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments. However, other modifications, changes, and substitutions are also possible. The aspects and elements of all embodiments disclosed above may be combined in any manner and / or in combination with aspects or elements of other embodiments to provide multiple additional embodiments. Therefore, this specification and the accompanying drawings should be regarded as illustrative, not restrictive. In fact, the novel devices, methods, and systems described herein may be embodied in various other forms. In addition, various omissions, substitutions, and changes may be made to the forms of the methods and systems described herein without departing from the spirit of the present disclosure. For example, although blocks are presented in a given arrangement, alternative embodiments may perform similar functions with different components and / or circuit topologies, and may delete, move, add, subdivide, combine, and / or modify some blocks. Each of these blocks may be implemented in various different ways.
[0149] The various embodiments of the present disclosure may be described in a progressive manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In the present disclosure, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present disclosure, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.
[0150] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. The various embodiments disclosed herein may be combined in any manner without departing from the spirit and scope of the present disclosure. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. A computing circuit for performing a multiplication operation of a first matrix by a second matrix, wherein: The number of columns of the first matrix and the number of rows of the second matrix are each N, and each first matrix element in the first matrix is a pre-known coefficient, and the operation circuit includes: a sign processing subcircuit configured to invert a target sign bit of a second matrix element in the second matrix to generate a corresponding first operand, wherein the first operand is an unsigned number; and a multiply-accumulate subcircuit, the input end of the multiply-accumulate subcircuit being communicatively connected to the output end of the sign processing subcircuit, and the multiply-accumulate subcircuit being configured to at least calculate the inner product of the first vector and the second vector , wherein the kth component of the first vector is the kth first matrix element in the current operation row of the first matrix , where the current operation row is the i-th row, and the k-th component of the second vector is the first operand obtained by converting the k-th second matrix element in the current operation column of the second matrix , wherein the current operation column is the j-th column, and the dimension of the first vector and the dimension of the second vector are each N, and in the third matrix obtained by multiplying the first matrix by the second matrix, the third matrix element located at the i-th row and the j-th column is the sum of the inner product Pij of the first vector and the second vector and a preset constant, wherein the preset constant is determined according to the conversion relationship between the second matrix element associated with the second vector and the corresponding first operand, and the first matrix element associated with the first vector.
2. The operation circuit according to claim 1, wherein: The operation circuit comprises N symbol processing sub-circuits, and in the same operation cycle, each symbol processing sub-circuit is configured to convert a corresponding second matrix element in the same column of the second matrix; and / or The operation circuit includes N' multiplication and accumulation subcircuits, each of which is configured to calculate the inner product of the first vector and the second vector associated with a corresponding third matrix element in the same column of the third matrix in the same operation cycle, wherein N' is the number of rows of the first matrix.
3. The operation circuit according to claim 1, wherein: The target sign bit of the second matrix element is the most significant bit of the second matrix element.
4. The operation circuit according to claim 1, wherein: The symbol processing subcircuit is configured as: Determine whether the upper m bits of each second matrix element in the current operation column of the second matrix are all extended sign bits, where m is a preset positive integer and m is less than the total number of bits M of the second matrix elements; In the case where the upper m bits of each second matrix element in the current operation column are extended sign bits, the upper m bits of the second matrix element are all set to zero, and the target sign bit of the second matrix element is inverted to generate a corresponding first operand, wherein the target sign bit of the second matrix element is the lower bit adjacent to the upper m bits of the second matrix element, or When the upper m bits of at least one second matrix element in the current operation column are not all extended sign bits, the target sign bit of the second matrix element is inverted to generate a corresponding first operand, wherein the target sign bit of the second matrix element is the highest bit of the second matrix element.
5. The operation circuit according to claim 1, wherein: The symbol processing subcircuit is configured as: Determine the position of the real sign bit of each second matrix element in the current operation column of the second matrix, and use the highest real sign bit among the real sign bits of all the second matrix elements in the current operation column as the target sign bit; as well as The bits higher than the target sign bit of the second matrix element are all set to zero, and the target sign bit of the second matrix element is inverted to generate the corresponding first operand.
6. The operation circuit according to any one of claims 3 to 5, wherein: The sign processing subcircuit includes an inverter, an input terminal of which is configured to receive a target sign bit of the second matrix element to invert the target sign bit.
7. The operation circuit according to claim 4 or 5, wherein: The sign processing subcircuit includes a sign bit determination unit, and the sign bit determination unit includes: a first AND gate unit, wherein the first AND gate unit has a plurality of input terminals, each of which is configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in a current operation column, and an output terminal of the first AND gate unit is configured to output an indication signal indicating whether the received plurality of to-be-determined bits are all extended sign bits; or An OR gate unit, wherein the OR gate unit has a plurality of input terminals, each of the input terminals is configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in the current operation column, and an output terminal of the OR gate unit is configured to output an indication signal indicating whether the received plurality of to-be-determined bits are all extended sign bits; or An XOR gate unit, wherein the XOR gate unit has multiple input terminals, each input terminal is configured to receive a corresponding to-be-determined bit of a corresponding second matrix element in a current operation column, and the output terminal of the XOR gate unit is configured to output an indication signal indicating whether the received multiple to-be-determined bits are all extended sign bits.
8. The operation circuit according to claim 4 or 5, wherein: The sign processing subcircuit comprises a second AND gate unit, wherein two input terminals of the second AND gate unit are configured to respectively receive a signal zero representing zero and a corresponding to-be-set-to-zero bit of a corresponding second matrix element, so as to set the to-be-set-to-zero bit to zero.
9. The operation circuit according to claim 1, wherein: The multiplication-accumulation subcircuit comprises a shift-addition unit, and the shift-addition unit is configured as follows: When the first matrix element multiplied by the first operand is a positive number, the first operand is shifted by a preset number of bits in a preset direction to generate one or more corresponding second operands, wherein the preset direction and the preset number of bits are determined based on the first matrix element multiplied by the first operand.
10. The operation circuit according to claim 9, wherein: The multiplication-accumulation subcircuit further includes a bitwise inversion unit, wherein the bitwise inversion unit is configured to: in the case where the first matrix element to be multiplied by the first operand is a negative number, invert the first operand bitwise to generate a corresponding third operand; The shift-add unit is further configured to: shift the third operand by a preset number of bits in a preset direction to generate one or more corresponding fourth operands, wherein the preset direction and the preset number of bits are determined according to a first matrix element multiplied by the first operand.
11. The operation circuit according to claim 9 or 10, wherein: The shift-and-add unit is further configured to: A sum of all second operands or fourth operands generated from the N first operands is calculated.
12. The operation circuit according to claim 9 or 10, wherein: The first matrix element cik multiplied by the first operand Xkj satisfies cik=∑[(-1) s *2 p ]: If s is even, the first operand is shifted left by p bits to produce the corresponding second operand; If s is an odd number, the first operand is inverted bit by bit to generate a third operand, and the third operand is shifted left by p bits to generate a corresponding fourth operand.
13. The operation circuit according to claim 1, wherein: The multiplication-accumulation subcircuit is configured to calculate a sum of an inner product of the first vector and the second vector and the preset constant.
14. The operation circuit according to claim 1, wherein: In the case where there are at least two first matrix elements having the same absolute value in at least one row of the first matrix, the operation circuit further comprises: A symmetric operator circuit is configured to calculate a sum or difference of second matrix elements or corresponding first operands multiplied by at least two first matrix elements having the same absolute value in the same row of the first matrix.
15. The operation circuit according to claim 14, wherein: The symmetric operation subcircuit is arranged between the sign processing subcircuit and the multiplication and accumulation subcircuit.
16. The operation circuit according to claim 14, wherein: The symmetric operator circuit includes a single-stage butterfly operation unit or a plurality of stages of butterfly operation units connected in cascade.
17. The operation circuit according to claim 1, wherein: The second matrix is a transposed matrix of the image matrix; or The second matrix is a transposed matrix of a fourth matrix, wherein the fourth matrix is a matrix obtained by multiplying the first matrix by the transposed matrix of the image matrix; The matrix elements in the image matrix represent pixel parameters of pixels at corresponding positions in the image to be processed, or the matrix elements in the image matrix represent the difference between pixel parameters of pixels at corresponding positions in two adjacent frames of the image to be processed.
18. The operation circuit according to claim 1, wherein: The operation circuit is configured to perform a discrete cosine transform operation, and the first matrix is a discrete cosine transform coefficient matrix.
19. A processor comprising the arithmetic circuit according to any one of claims 1 to 18. 20 . A computing device, comprising the computing circuit according to claim 1 or the processor according to claim 19 .
