Data processing device and electronic equipment

Abstract

The invention provides a data processing device and electronic equipment, and relates to the field of artificial intelligence. The device comprises a cache unit used for storing a first to-be-processed matrix and a second to-be-processed matrix; the pulse array comprises a plurality of pulse sub-arrays, and each pulse sub-array comprises a plurality of calculation units. The calculation unit is used for determining a target calculation result of the calculation unit according to the received first to-be-processed element and a second to-be-processed element for the calculation unit; and providing the first to-be-processed element to a rear hardware unit in a target external pulse sub-array, wherein the target external pulse sub-array is determined by circularly moving a first moving bit in a first direction from a source pulse sub-array where the computing unit is located. The to-be-processed vector data in the second to-be-processed matrix is obtained by circularly moving a plurality of elements in the initial vector data of the second initial matrix along a second direction by a second moving bit, and the second moving bit is determined according to the position information of the initial vector data in the second initial matrix and a preset value.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a data processing device and an electronic device. Background Art

[0002] With the rapid development of artificial intelligence (AI), deep learning models have been widely used in fields such as image recognition, natural language processing, and speech recognition. In these applications, matrix multiplication, as the core computational operation of deep learning models, has a direct impact on the performance of the entire system.

[0003] To accelerate matrix multiplication calculations, the industry generally uses dedicated hardware accelerators instead of general-purpose processors. Pulse arrays, a classic parallel computing architecture, have been widely used in matrix multiplication accelerator design. However, in the implementation of pulse arrays in related technologies, a large number of delay units are often required at the data input to ensure the correctness of data timing. The extensive use of these delay units leads to hardware resource consumption issues. Especially in resource-constrained devices, the additional hardware overhead severely limits the practical deployment and application of pulse arrays. Summary of the Invention

[0004] The present invention provides a data processing device and an electronic device, which can solve the problem of hardware resource consumption caused by the large-scale use of delay units.

[0005] According to one aspect of the present invention, a data device is provided, comprising: a cache unit for storing a first matrix to be processed and a second matrix to be processed, the first matrix to be processed including a plurality of first elements to be processed, the second matrix to be processed including a plurality of vector data to be processed, the vector data to be processed including a plurality of second elements to be processed; a pulse array including a plurality of pulse sub-arrays, the pulse sub-array including a plurality of calculation units, the calculation unit being configured to: determine a target calculation result of the calculation unit based on the received first elements to be processed and the second elements to be processed for the calculation unit; provide the first elements to be processed to a subsequent hardware unit for the calculation unit in a target external pulse sub-array, the target external pulse sub-array being determined by cyclically shifting a first number of shift bits along a first direction from a source pulse sub-array where the calculation unit is located in the plurality of pulse sub-arrays; wherein the second matrix to be processed includes a plurality of vector data to be processed, the vector data to be processed in the second matrix to be processed being obtained by cyclically shifting a plurality of elements in the initial vector data of the second initial matrix by a second number of shift bits along a second direction, the second number of shift bits being determined based on the position information of the initial vector data in the second initial matrix and a preset value.

[0006] According to another aspect of the present invention, an electronic device is provided, comprising: a memory; and the data processing device provided by the present invention.

[0007] The data processing device and electronic device provided by the embodiments of the present invention obtain a second matrix to be processed by cyclically shifting multiple elements in the initial vector data of the second initial matrix along the second direction by a second shifting bit number, thereby achieving preprocessing and rearrangement of the input data, so that the rearranged second matrix to be processed can match the data processing timing of the computing units in the pulse array. The second shifting bit number is determined based on the position information and preset value of the initial vector data in the second initial matrix. Therefore, the second matrix to be processed obtained after the data rearrangement ensures that each second element to be processed can reach the corresponding computing unit in the correct clock cycle, thereby eliminating the delay unit that must be set in the pulse array of the related art to ensure the correctness of the data timing.

[0008] At the same time, by redesigning the data flow between the computing units in the pulse array, the computing unit can provide the first element to be processed to the subsequent hardware unit for the computing unit in the target external pulse subarray, where the target external pulse subarray is determined by cyclically shifting the first number of shift bits along the first direction from the source pulse subarray. This reconstructed data propagation path is combined with the rearranged second matrix to be processed, which ensures the correctness of the matrix multiplication calculation while avoiding the setting of a delay unit.

[0009] Therefore, under the premise of ensuring the correctness of calculation, the embodiment of the present invention reduces the hardware resources required for the implementation of the pulse array, especially eliminates the use of a large number of delay units, reduces hardware costs and power consumption, and improves the deployment feasibility and application effect of the pulse array in resource-constrained devices. At the same time, since the waiting time of data in the delay unit is reduced, the startup delay of the entire matrix multiplication calculation can also be reduced, thereby improving the calculation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0011] Figure 1A Schematic diagram of data flow of a pulse array in the 0th clock cycle in the related art;

[0012] Figure 1B Schematic diagram of data flow of a pulse array in the first clock cycle in the related art;

[0013] Figure 1C Schematic diagram of data flow of a pulse array in the second clock cycle in the related art;

[0014] Figure 1D Schematic diagram of data flow of a pulse array in the third clock cycle in the related art;

[0015] Figure 2A schematic structural diagram of a data processing device provided by an embodiment of the present invention;

[0016] Figure 3 A schematic diagram of a second initial matrix data reordering operation provided by an embodiment of the present invention;

[0017] Figure 4A A schematic diagram of data flow of a pulse array in the first clock cycle provided by an embodiment of the present invention;

[0018] Figure 4B A schematic diagram of data flow of a pulse array in the second clock cycle provided by an embodiment of the present invention;

[0019] Figure 4C A schematic diagram of data flow of a pulse array in the third clock cycle provided by an embodiment of the present invention;

[0020] Figure 5 A schematic structural diagram of another data processing device provided by an embodiment of the present invention;

[0021] Figure 6 A schematic structural diagram of another data processing device provided by an embodiment of the present invention;

[0022] Figure 7 A schematic diagram of the structure of a computing unit provided in an embodiment of the present invention;

[0023] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] Before introducing the embodiments of the present invention, the implementation of a pulse array in related art is first introduced.

[0029] Please refer to Figure 1A to Figure 1D , Figure 1A to Figure 1D Schematic diagram of data flow of pulse array in related technology.

[0030] Please refer to Figure 1A , Figure 1A Schematic diagram of data flow of pulse array in the 0th clock cycle in the related art; Figure 1A As shown, the pulse array in the related art adopts a regular two-dimensional grid topology, in which each processing unit 110 can transmit data to the adjacent processing unit 110 through a fixed data channel. Taking the pulse array to implement the matrix multiplication operation between the A matrix 120 and the B matrix 130 as an example, based on the data channels between different processing units 110, the data flow of the A matrix 120 can be: input from the left boundary of the pulse array, and passed from left to right in sequence between the processing units 110 in the same row through the horizontal data channel. After receiving the element of the A matrix 120, each processing unit 110 will pass the element to the next processing unit 110 on the right side of the same row. At the same time, the data flow of the B matrix 130 can be: input from the upper boundary of the pulse array, and passed from top to bottom in sequence between the processing units 110 in the same column through the vertical data channel, so that all processing units 110 in the column can receive each element in a column of the B matrix 130. It can be understood that Figure 1A to Figure 1D The positions of the elements in the A matrix 120 and the B matrix 130 are determined in order to perform correct matrix multiplication calculations based on the multiple elements of the A matrix and the B matrix.

[0031] During the calculation process, each processing unit 110 receives elements of the A matrix 120 from the left and elements of the B matrix 130 from the top, performs multiplication and accumulation calculations, and then transmits the calculation results to the adjacent processing unit 110 below through the vertical downward data channel.

[0032] Based on the above-mentioned data flow of the A matrix 120 and the data flow of the B matrix 130, one or more delay units 140 can be set for the pulse array to ensure that different column data of the B matrix 130 need to be input into the pulse array at different time points, thereby ensuring that the corresponding elements of the A matrix 120 and the B matrix 130 can be calculated in the correct processing unit 110 and at the correct time point.

[0033] Please refer to Figure 1B , Figure 1B FIG. 1 is a schematic diagram of data flow of a pulse array in the first clock cycle in the related art; Figure 1B As shown, in the first clock cycle, the pulse array of the related art begins to perform the first step of the matrix multiplication calculation. Specifically, the element a11 in the first row and first column of matrix A 120 is input from the left edge of the pulse array and simultaneously arrives at the first row and first column processing unit 110. At this point, processing unit 110 receives the pre-loaded element b11 of matrix B 130 and the corresponding element c11 of matrix C, and performs a multiply-accumulate operation, obtaining the calculation result a11×b11+c11 of the first row and first column processing unit 110 in the first clock cycle.

[0034] Due to the delay unit 140, during the first clock cycle, the second column data b12, b22, and b32 and the third column data b13, b23, and b33 of the B matrix 130 have not yet been input into the corresponding processing units 110. This ensures that element a11 of the A matrix 120 can be calculated with the corresponding element of the B matrix 130 at the correct time. However, this also means that the parallelism of the pulse array is limited in the initial stage, and only the processing units 110 in the first column participate in the actual multiplication and accumulation calculations.

[0035] Please refer to Figure 1C , Figure 1C FIG. 1 is a schematic diagram of data flow of a pulse array in the second clock cycle in the related art; Figure 1CAs shown, in the second clock cycle, element a21 in the second row and first column of matrix A 120 is input from the left edge of the pulse array and arrives at processing unit 110 in row 1 and column 1. It then performs a multiplication-accumulation operation with the preloaded element b11 of matrix B 130 and the corresponding element c21 of matrix C, resulting in the calculation result a21 × b11 + c21 for processing unit 110 in row 1 and column 1 in the second clock cycle. Simultaneously, due to the control of delay unit 140, data b12 in column 2 of matrix B 130 begins to be input into processing unit 110 in row 1 and column 2 during this clock cycle. This processing unit 110 receives element a11 propagated from the left and the newly input element b12, along with the corresponding element c12 of matrix C, and performs a multiplication-accumulation operation on it, resulting in the calculation result a11 × b12 + c12 for processing unit 110 in row 1 and column 2 in the second clock cycle.

[0036] The first row and first column processing unit 110 may propagate the calculation result a11×b11+c11 of the first clock cycle downward to the first row and first column processing unit 110. The first row and first column processing unit 110 performs a multiplication-accumulation calculation based on the calculation result a11×b11+c11 propagated from above, the element a12 input from the left, and the pre-loaded element b21 of the B matrix 130, to obtain the calculation result a11×b11+c11+a12×b21 of the first row and first column processing unit 110 in the second clock cycle.

[0037] Please refer to Figure 1D , Figure 1D FIG. 1 is a schematic diagram of data flow of a pulse array in the third clock cycle in the related art; Figure 1D As shown, in the third clock cycle, the element a31 in the third row and first column of the A matrix 120 is input from the left boundary of the pulse array and reaches the first row and first column processing unit 110. It performs a multiplication-accumulation operation with the pre-loaded element b11 of the B matrix 130 and the corresponding element c31 of the C matrix, resulting in the calculation result a31×b11+c31 of the first row and first column processing unit 110 in the third clock cycle. It can be understood that the method for obtaining the calculation result a21×b12+c22 of the first row and second column processing unit 110 in the third clock cycle is the same or similar to the method for obtaining the calculation result a11×b12+c12 of the first row and second column processing unit 110 in the second clock cycle, and the present invention will not be repeated here.

[0038] Due to the two-clock-cycle delay control provided by delay unit 140, processing unit 110 in row 1, column 3 of matrix B 130 begins to input data b13. This processing unit 110 receives the element a11 propagated from the left and the newly input element b13, and performs a multiply-accumulate operation on these elements along with the corresponding element c13 of matrix C, obtaining the calculation result a11×b13+c13 for processing unit 110 in the third clock cycle.

[0039] It can be understood that the method of obtaining the calculation result a21×b11+c21+a22×b21 of the 2nd row and 1st column processing unit 110 in the 3rd clock cycle and the method of obtaining the calculation result a11×b12+c12+a12×b22 of the 2nd row and 2nd column processing unit 110 in the 3rd clock cycle are the same or similar to the method of obtaining the calculation result a11×b11+c11+a21×b21 of the 2nd row and 1st column processing unit 110 in the 2nd clock cycle, and the present invention will not be repeated here.

[0040] The second row and first column processing unit 110 may propagate the calculation result a11×b11+c11+a21×b21 of the second clock cycle downward to the third row and first column processing unit 110. The third row and first column processing unit 110 performs a multiplication-accumulation calculation based on the calculation result a11×b11+c11+a21×b21 propagated from above, the element a13 input from the left, and the pre-loaded element b31 of the B matrix 130, to obtain the calculation result a11×b11+c11+a21×b21+a13×b31 of the third row and first column processing unit 110 in the third clock cycle.

[0041] It is worth noting that in the third clock cycle, the pulse array has not yet reached a fully parallel state, and some processing units 110 have not received valid input data and are therefore still in a waiting state, which reflects the impact of the delay unit 140 of the pulse array on the overall computing efficiency during the startup phase in the related art.

[0042] Based on the analysis of the above-mentioned related technologies, it can be found that the implementation method of the pulse array in the related technology has technical problems such as serious hardware resource consumption and high startup delay, which will be explained below.

[0043] The extensive use of delay units 140 leads to serious consumption of hardware resources. In order to achieve timing control of data in different columns of the B matrix 130, the pulse array in the related art needs to deploy a large number of delay units 140 on the data input path. Each delay unit 140 requires a storage device, a clock control circuit, and corresponding control logic. In resource-constrained embedded devices or edge computing devices, the delay unit 140 occupies a large amount of valuable chip area and power consumption budget, forming a competitive relationship with the computing unit or processing unit 110 in hardware resource allocation, resulting in low chip area utilization efficiency. Under the same hardware resource constraints, the existence of the delay unit 140 limits the number of deployable processing units 110, thereby restricting the parallel computing capability and overall performance of the pulse array.

[0044] At the same time, the presence of delay units 140 increases the startup delay of the pulse array. Since different columns of data in the B matrix 130 need to pass through different numbers of delay units 140 to reach the corresponding processing unit 110, the entire pulse array must wait for the data on the longest delay path to be ready before it can begin effective calculation. This waiting time increases linearly with the increase in the size of the pulse array, resulting in idle computing resources during the startup phase and reducing overall computing efficiency. Especially in application scenarios that require frequent startup of matrix multiplication calculations, the cumulative effect of startup delays has a significant impact on system performance, limiting the actual deployment effect of the pulse array in a resource-constrained environment.

[0045] Based on the above problems, an embodiment of the present invention provides a data processing device.

[0046] Figure 2 A structural diagram of a data processing device provided by an embodiment of the present invention.

[0047] like Figure 2 As shown, the data processing device 200 includes a cache unit 210 and a pulse array 220 .

[0048] The cache unit 210 is configured to store a first matrix to be processed and a second matrix to be processed, wherein the first matrix to be processed includes a plurality of first elements to be processed and the second matrix to be processed includes a plurality of vector data to be processed and the vector data to be processed includes a plurality of second elements to be processed.

[0049] The pulse array 220 includes a plurality of pulse sub-arrays, each of which includes a plurality of calculation units. Figure 2As shown, pulse array 220 may include pulse sub-array 221, pulse sub-array 222, and pulse sub-array 223. The pulse sub-arrays may include multiple calculation units. For example, pulse sub-array 221 may include calculation unit 2211 and calculation unit 2212. Pulse sub-array 222 may include calculation unit 2221 and calculation unit 2222. Pulse sub-array 223 may include calculation unit 2231 and calculation unit 2232.

[0050] One or more computing units among the multiple computing units are used to: determine a target computing result of the computing unit based on a received first element to be processed and a second element to be processed for the computing unit; provide the first element to be processed to a subsequent hardware unit for the computing unit in a target external pulse subarray, where the target external pulse subarray is determined by cyclically shifting a first number of shift bits along a first direction from a source pulse subarray where the computing unit is located among the multiple pulse subarrays.

[0051] The vector data to be processed in the second matrix to be processed is obtained by the following operation: multiple elements in the initial vector data of the second initial matrix are circularly moved along the second direction by a second movement bit number, and the second movement bit number is determined based on the position information of the initial vector data in the second initial matrix and a preset value.

[0052] The cache unit 210 in the embodiment of the present invention may be a high-speed memory device located between the processor and the pulse array 220. In the embodiment of the present invention, it may be understood as an on-chip memory module with fast read and write capabilities, which may temporarily store input data and intermediate result data involved in matrix multiplication calculations.

[0053] Exemplarily, the cache unit 210 may be a static random-access memory (SRAM), a block random-access memory (BRAM), or other types of on-chip cache memory devices.

[0054] In an embodiment of the present invention, cache unit 210 can store a first matrix to be processed and a second matrix to be processed. The first matrix to be processed refers to a matrix used as a multiplicand in a matrix multiplication calculation. The matrix includes multiple first elements to be processed, each of which can be a value at a position in the matrix and is propagated through the data channel of pulse array 220 during the calculation process. For example, the first matrix to be processed can be the aforementioned matrix A.

[0055] The second matrix to be processed is the matrix used as the multiplier in the matrix multiplication calculation. After preprocessing and rearrangement, this matrix is ​​stored in cache unit 210 and contains multiple second elements to be processed, each of which represents the value of a position in the rearranged matrix. During the matrix multiplication calculation, the first element to be processed and the second element to be processed are the specific values ​​in different matrices participating in the multiplication and accumulation calculation. The matrix operation is completed through the coordinated operation of the computing units in pulse array 220.

[0056] To address the problem in the related art that the pulse array 220 requires a large number of delay units, resulting in severe hardware resource consumption and excessive startup delay, the embodiment of the present invention obtains a second matrix to be processed by reordering the second initial matrix. For example, the second initial matrix can be the aforementioned B matrix.

[0057] The second initial matrix in the embodiments of the present invention can be the original input matrix used as a multiplier in a matrix multiplication calculation. In the embodiments of the present invention, it can be understood as a weight matrix or parameter matrix of a deep learning model, which is used to perform multiplication operations with the first matrix to be processed. The second initial matrix organizes data in a row-column storage format, where each initial vector data can be a column of the matrix, including elements in different row positions in the column.

[0058] The second shift bit number refers to the step size by which the elements of the initial vector data need to be cyclically shifted along the second direction during the data reordering process. The second shift bit number can be determined based on the position information of the initial vector data in the second initial matrix and a preset value. The position information of the initial vector data in the second initial matrix can refer to the column number of the initial vector data in the matrix, which is used to indicate the position of the data in the original matrix structure. The column number is counted starting from 1, and the position information corresponding to the initial vector data in the i-th column can be i.

[0059] Exemplarily, the second shift bit number may be: the column number of the initial vector data minus a preset value. Different initial vector data will receive different shift bit numbers, thereby rearranging the entire matrix data distribution so that the rearranged data matches the data processing timing of the pulse array 220.

[0060] Correspondingly, the preset value in the embodiment of the present invention can be set to 1 as a benchmark reference value for calculating the second shift bit number.

[0061] Exemplarily, the i-th initial vector data may be the i-th column of the second initial matrix, and the second shifting bit number of the i-th initial vector data may be i-1. That is, in the second initial matrix, the shifting bit number of the first column of data may be 0 (no shifting), the shifting bit number of the second column of data may be 1, the shifting bit number of the third column of data may be 2, and so on. i may be an integer greater than or equal to 1.

[0062] Combine Figure 2 As shown, the pulse sub-array in the embodiment of the present invention may include multiple computing units in a column of the pulse array 220.

[0063] The pulse subarray structure has the following characteristics: the jth pulse subarray processes the data element in the jth column of the first matrix to be processed. j can be an integer greater than or equal to 1. Computational units within the same pulse subarray can receive a first element to be processed but separately process second elements to be processed from different columns of the second matrix to be processed. This organization enables the pulse subarray to receive and process multiple first elements to be processed from the first matrix to be processed, performing multiplication and accumulation calculations on these multiple first elements with corresponding elements of the second matrix to be processed.

[0064] The computing unit in the embodiment of the present invention refers to a basic processing module that performs specific multiplication and accumulation operations in the pulse subarray. In the embodiment of the present invention, the computing unit can be understood as a dedicated hardware unit that integrates a multiplier, an accumulator, and data transmission control logic. One or more computing units can be used to determine an element at a position in the result matrix of matrix multiplication, can receive a first element to be processed and a second element to be processed, perform multiplication and accumulation operations, and pass the elements received and the intermediate calculation results generated during the calculation process. In addition, the pulse subarray where the computing unit is located can serve as the source pulse subarray of the computing unit. For example, the source pulse subarray of computing unit 2211 can be pulse subarray 221. The source pulse subarray of computing unit 2222 can be pulse subarray 222. The source pulse subarray of computing unit 2231 can be pulse subarray 223.

[0065] Unlike related art techniques, where the first element to be processed is propagated horizontally only within the same row, the computational unit in this embodiment utilizes a reconstructed data propagation path. During the transmission of the first element to be processed, the computational unit can continue to propagate the received first element to be processed downward for use by subsequent computational units, but the propagation method is modified: the computational unit provides the first element to be processed to the subsequent hardware unit in the target external pulse subarray for that computational unit. This cross-pulse subarray data transmission method matches the rearranged second matrix to be processed, ensuring the timing correctness of the matrix multiplication calculation.

[0066] In embodiments of the present invention, the target external pulse subarray can be the target pulse subarray to which the current computing unit needs to send the first element to be processed during data transfer. The target external pulse subarray can be determined based on the location of the computing unit in the source pulse subarray and predefined movement rules. This establishes a data propagation topology different from that of pulse arrays in related technologies, allowing the target external pulse subarray to receive the first element to be processed transmitted by the computing unit in the source pulse subarray, thereby achieving data flow control across pulse subarrays.

[0067] The target external pulse subarray of the computing unit can be determined by cyclically shifting the source pulse subarray by the first shift bit number along the first direction. The first direction in the embodiment of the present invention refers to the direction of movement when performing position indexing between pulse subarrays in the pulse array 220. In the embodiment of the present invention, it can be understood as a direction reference for determining the target position of data propagation, usually pointing to the horizontal direction of the pulse array 220, specifically manifested as a left-right position shift in the array structure composed of multiple pulse subarrays, used to instruct the computing unit 2211 to transfer the first element to be processed to the correct target external pulse subarray. For example, when rearranging the second initial matrix, if the upward direction from the 2nd row to the 1st row is used as the second direction, the first direction can be the rightward direction from the 2nd column to the 1st column.

[0068] The first shift bit number in the embodiment of the present invention is related to the column position of the calculation unit 2211 in the source pulse subarray to ensure the correctness of the matrix multiplication operation. The first shift bit number can be preset, for example, 1.

[0069] The circular movement in the embodiment of the present invention refers to the circular movement method used in the process of determining the position of the pulse subarray, that is, when the source pulse subarray is the boundary of the pulse array 220, the target pulse subarray can be determined from the other end of the array based on the circular movement method. For example, taking the pulse array 220 including M pulse subarrays, the circular movement along the first direction is a circular movement to the right, and the first movement bit number is 1 as an example, if the source pulse subarray is the first column of the pulse array 220, the target external pulse subarray of the source pulse subarray can be the Mth column of the pulse array. M can be an integer greater than 1. Figure 2 As shown, for pulse subarray 221, the target outer pulse subarray determined by cyclically shifting by 1 bit along the first direction may be pulse subarray 223. For pulse subarray 222, the target outer pulse subarray determined by cyclically shifting by 1 bit along the first direction may be pulse subarray 221. For pulse subarray 223, the target outer pulse subarray determined by cyclically shifting by 1 bit along the first direction may be pulse subarray 222.

[0070] By performing a combined operation of circularly shifting the first shifting number in the first direction, that is, in the pulse array 220 comprising a plurality of pulse sub-arrays, starting from the position index of the source pulse sub-array, shifting the first shifting number of positions along the first direction, and using a circular shift method to handle boundary conditions, the specific position of the target external pulse sub-array is finally determined.

[0071] In the embodiments of the present invention, a subsequent hardware unit for a computing unit refers to a hardware module located after the current computing unit in the data transmission link and configured to receive the first element to be processed transmitted by the computing unit. In the embodiments of the present invention, this can be understood as a downstream receiving unit of a data stream. The subsequent hardware unit for a computing unit is located in the target external pulse subarray and can receive the data output of the current computing unit. For example, as described above, the target external pulse subarray of pulse subarray 221 can be pulse subarray 223. For computing unit 2211 in pulse subarray 221, the subsequent hardware unit can be computing unit 2232 in pulse subarray 223.

[0072] It should be noted that the principle for determining the number of first shift bits for cyclic shifting in the first direction is related to the principle for determining the number of second shift bits for cyclic shifting in the second direction. By coordinating data preprocessing and propagation path reconstruction, pulse array 220 computation without a delay unit is achieved. For example, the first direction and the second direction are perpendicular.

[0073] Specifically, because the vector data to be processed in the second matrix to be processed is obtained by cyclically shifting the initial vector data of the second initial matrix along the second direction by the second shifting number, the position of the second element to be processed is offset relative to the original position by a preset amount. In this embodiment of the present invention, the second matrix to be processed is obtained by preprocessing and rearranging the second initial matrix, thereby changing the distribution position of the second element to be processed in the matrix. Accordingly, the propagation path of the first element to be processed in pulse array 220 needs to be adjusted to ensure correct data timing matching.

[0074] It will be appreciated that the present invention has been described above using the example of rightward movement along the first direction and upward movement along the second direction. However, the present invention is not limited thereto. To ensure that the first element to be processed can be multiplied and accumulated with the corresponding second element to be processed at the correct time, the data propagation paths of the computing units in pulse array 220 can be adapted to this positional offset. Therefore, a correspondence can be established between the number of first and second shifted bits, ensuring that the rearranged second matrix to be processed and the reconstructed propagation paths of the first element to be processed are synchronized in time, thereby eliminating delay units while ensuring the correctness of the matrix multiplication calculation.

[0075] Analogous to the implementation of a 3×3 pulse array in related art, elements b11, b21, and b31 in the first column of matrix B are directly input at clock cycle 0, while elements b12, b22, and b32 in the second column need to be delayed by one clock cycle. Elements b13, b23, and b33 in the third column need to be delayed by two clock cycles. This time-sharing input method requires the configuration of three delay units. When the first row of data a11 in matrix A is input from the left, it needs to meet b11 in the first clock cycle, meet the delayed b12 in the second clock cycle, and meet the delayed b13 in the third clock cycle. Effective computation of the entire array needs to be delayed by two clock cycles.

[0076] Furthermore, multiple computation units in the pulse subarray can each receive multiple second elements to be processed in a column of the second matrix to be processed. The data flow direction of the second elements to be processed can be a third direction. For example, computation unit 2211 can provide the elements to be processed in the first column of the second matrix to be processed to computation unit 2212. The third direction is the direction from computation unit 2211 to computation unit 2212.

[0077] In addition, in the pulse subarray, the target calculation result generated by the previous calculation unit can be provided to the subsequent calculation unit. For example, the target calculation result of the calculation unit 2211 can be provided to the calculation unit 2212.

[0078] According to an embodiment of the present invention, a second matrix to be processed is obtained by cyclically shifting multiple elements in the initial vector data of the second initial matrix along a second direction by a second shifting bit number, thereby achieving preprocessing and rearrangement of the input data, so that the rearranged second matrix to be processed can match the data processing timing of the computing units in the pulse array 220. The second shifting bit number is determined based on the position information of the initial vector data in the second initial matrix and a preset value. Therefore, the second matrix to be processed obtained after data rearrangement ensures that each second element to be processed can reach the corresponding computing unit at the correct clock cycle, thereby eliminating the delay unit that must be provided in the pulse array of the related art to ensure the correctness of data timing.

[0079] At the same time, by redesigning the data flow between the computing units in the pulse array 220, the computing unit can provide the first element to be processed to the subsequent hardware unit for the computing unit 2211 in the target external pulse subarray, wherein the target external pulse subarray is determined by cyclically shifting the first shift bit number along the first direction from the source pulse subarray. This reconstructed data propagation path is combined with the rearranged second matrix to be processed, which ensures the correctness of the matrix multiplication calculation while avoiding the setting of the delay unit.

[0080] Therefore, under the premise of ensuring the correctness of calculation, the embodiment of the present invention reduces the hardware resources required for the implementation of the pulse array 220, especially eliminates the use of a large number of delay units, reduces hardware costs and power consumption, and improves the deployment feasibility and application effect of the pulse array 220 in resource-constrained devices. At the same time, since the waiting time of data in the delay unit is reduced, the startup delay of the entire matrix multiplication calculation can also be reduced, thereby improving the calculation efficiency.

[0081] It can be understood that the data processing device 200 of the present invention is described above. Figure 3 The second matrix to be processed of the present invention is further described.

[0082] Figure 3 A schematic diagram of a second initial matrix data reordering operation provided by an embodiment of the present invention.

[0083] like Figure 3 As shown, the original second initial matrix B310 includes 3 initial vector data, each of which can be a column data of the second initial matrix and includes 3 elements. The first initial vector data [b11, b21, b31] includes 3 elements, the second initial vector data [b12, b22, b32] includes 3 elements, and the third initial vector data [b13, b23, b33] includes 3 elements. According to the aforementioned calculation rule of the second shift bit number, the second shift bit number of the first initial vector data is 0, the shift bit number of the second initial vector data is 1, and the shift bit number of the third initial vector data is 2.

[0084] For the first initial vector data, since the number of shifted bits is 0, the original order can be kept unchanged, and the first vector data to be processed after rearrangement is still [b11, b21, b31];

[0085] For the second initial vector data, perform an upward circular shift of 1 bit. Element b12 moves to the third row after rearrangement, element b22 moves to the first row after rearrangement, and element b32 moves to the second row after rearrangement. The resulting second vector data to be processed is [b22, b32, b12].

[0086] For the third initial vector data, an upward circular shift of 2 bits is performed. The element b13 is moved to the second row after rearrangement, the element b23 is moved to the third row after rearrangement, and the element b33 is moved to the first row after rearrangement. The third vector data to be processed is [b33, b13, b23].

[0087] After the rearrangement operation is completed, the following Figure 3In the second matrix to be processed B320 shown, the vector data to be processed in each column is the result of cyclic shift of the corresponding initial vector data by a specific number of bits.

[0088] It can be understood that the second matrix to be processed of the present invention is described above. Figure 4A to Figure 4C The matrix multiplication operation of the present invention is further explained.

[0089] Figure 4A to Figure 4C A data flow diagram of a pulse array provided by an embodiment of the present invention.

[0090] like Figure 4A to Figure 4C As shown, the pulse array 220 may include multiple pulse sub-arrays. The pulse sub-array may include a head calculation unit, one or more intermediate calculation units, and an output unit. For example, the pulse sub-array 221 may include a calculation unit 2211, a calculation unit 2212, and a calculation unit 2213. The calculation unit 2211 may serve as a head calculation unit. The calculation unit 2212 may serve as an intermediate calculation unit. The calculation unit 2213 may serve as an output unit. Figure 4A The data flow of the pulse array in the first clock cycle of the current round is described.

[0091] In some embodiments, the head computing unit may perform the following operations: multiplying the first element to be processed for the head computing unit received from the cache unit with the second element to be processed for the head computing unit to obtain a multiplication result for the head computing unit. Based on the multiplication result of the head computing unit, a target calculation result for the head computing unit may be determined. Prior to the current round of the pulse array, the computing unit may preload the second element to be processed for the computing unit. That is, in the current round, the second element to be processed for the computing unit is already stored in the temporary storage module of the computing unit.

[0092] Please refer to Figure 4A , Figure 4A A schematic diagram of data flow of a pulse array in the first clock cycle provided by an embodiment of the present invention.

[0093] like Figure 4A As shown, the embodiment of the present invention first performs data rearrangement processing on the B matrix. In the first clock cycle, the first row of data of the first matrix to be processed includes elements a11, a12, and a13, and is simultaneously input into calculation units 2211, 2221, and 2231 (i.e., the first calculation unit of each pulse subarray). At the same time, Figure 3The first row of data for the second matrix to be processed B320 shown may include elements b11, b22, and b33, and may also be preloaded into computing units 2211, 2221, and 2231, respectively. Computing unit 2211 may perform a multiplication operation to obtain a multiplication result of a11×b11. Computing unit 2221 may perform a multiplication operation to obtain a multiplication result of a12×b22. Computing unit 2231 may perform a multiplication operation to obtain a multiplication result of a13×b33. After the calculation is completed, element a11 may be passed to computing unit 2232 in the second row and third column via the first channel, element a12 may be passed to computing unit 2212 in the second row and first column, and element a13 may be passed to computing unit 2222 in the second row and second column. It can be understood that the three computing units in the first row of pulse array 220 may serve as the head computing units of the three pulse sub-arrays, respectively. It can also be understood that for each computing unit in the first row, the subsequent hardware unit can be a computing unit.

[0094] Furthermore, the multiplication result of the head computation unit can serve as the target computation result of the head computation unit. The head computation unit can also be configured to: provide the second element to be processed to a subsequent computation unit of the source pulse subarray; and provide the target computation result of the head computation unit to the subsequent hardware unit. That is, the elements of the second matrix to be processed and the intermediate computation results can be transferred within the source pulse subarray. For example, element b11 can be transferred to computation unit 2221 in row 2 and column 1 via the second channel, element b22 can be transferred to computation unit 2222 in row 2 and column 2, and element b33 can be transferred to computation unit 2232 in row 2 and column 3. The target computation result of computation unit 2211 can be transferred to computation unit 2212 via the third channel. The target computation result of computation unit 2221 can be transferred to computation unit 2222 via the third channel. The target computation result of computation unit 2231 can be transferred to computation unit 2232 via the third channel. It is understood that three data transmission channels can be provided for the computation unit: the first element to be processed can be transferred via the first channel, the second element to be processed can be transferred via the second channel, and the target computation result can be transferred via the third channel.

[0095] It can be understood that the above description describes the calculation of the first clock cycle of the current round in conjunction with the head calculation unit, and the following description will describe the calculation of the second clock cycle of the current round.

[0096] In some embodiments, the computing unit may be further configured to: provide the second to-be-processed element for the subsequent hardware unit in the source pulse subarray to the subsequent hardware unit in the source pulse subarray.

[0097] Please refer to Figure 4B , Figure 4BA schematic diagram of data flow of a pulse array in the second clock cycle provided by an embodiment of the present invention.

[0098] like Figure 4B As shown, in the second clock cycle, the second row of data of the first matrix to be processed includes elements a21, a22, and a23, which can be input into calculation units 2211, 2221, and 2231 in the first row of pulse array 220, respectively. Calculation unit 2211 can perform a multiplication operation to obtain a multiplication result of a21 × b11. Calculation unit 2221 can perform a multiplication operation to obtain a multiplication result of a22 × b22. Calculation unit 2231 can perform a multiplication operation to obtain a multiplication result of a23 × b33. After the calculation is completed, element a21 can be passed to calculation unit 2232 in the second row and third column via the first channel, element a22 can be passed to calculation unit 2212 in the second row and first column, and element a23 can be passed to calculation unit 2222 in the second row and second column.

[0099] In some embodiments, to implement the basic computational logic of matrix multiplication, the computation unit may further perform a multiplication-accumulation operation and transfer the computational result within the pulse array 220. The computation unit may perform the following operations to determine a target computational result of the computation unit based on the received first element to be processed and the second element to be processed for the computation unit: multiplying the received first element to be processed and the second element to be processed for the computation unit to obtain a multiplication result of the computation unit; determining a target computational result of the computation unit based on the multiplication result of the computation unit and a previous computation result provided by a previous computation unit in the source pulse subarray; and providing the target computational result of the computation unit to a subsequent hardware unit in the source pulse subarray.

[0100] The rearranged second matrix to be processed B320 may include element b21, element b32, and element b13, which may be preloaded into computing unit 2212, computing unit 2222, and computing unit 2232, respectively. Since pulse array 220 employs a pipeline processing approach, computation results need to be sequentially transferred and accumulated among multiple computing units. Therefore, in each clock cycle, a computing unit may determine its target computation result based on its own multiplication result and the previous computation result provided by the previous computing unit in the source pulse subarray.

[0101] like Figure 4BAs shown, in the second clock cycle, the computing unit 2212 in the second row and first column receives the element a12 transmitted via the first channel and the target calculation result of the computing unit 2211 in the first clock cycle transmitted via the third channel. The computing unit 2212 performs a multiplication operation to obtain the multiplication result a12×b21 of the computing unit 2212 in the second clock cycle, and accumulates the multiplication result a12×b21 of the computing unit 2212 in the second clock cycle with the calculation result a11×b11 from the computing unit 2211 above in the first clock cycle to obtain the target calculation result a11×b11+a12×b21 of the computing unit 2212 in the second clock cycle. It can be understood that the method for obtaining the target calculation result a12×b22+a13×b32 of the second row and second column calculation unit 2222 in the second clock cycle and the method for obtaining the target calculation result a13×b33+a11×b13 of the second row and third column calculation unit 2232 in the second clock cycle are the same or similar to the method for obtaining the target calculation result a11×b11+a12×b21 of the calculation unit 2212 in the second clock cycle, and the present invention will not be repeated here. It can also be understood that the calculation unit 2211 can serve as the previous calculation unit of the calculation unit 2212 in the source pulse subarray. For the calculation unit 2212, the target calculation result provided by the calculation unit 2211 can be used as the previous calculation result. Due to the pre-adjustment of the data rearrangement operation, each calculation unit can receive the matching matrix elements at the correct time for multiplication and accumulation operations.

[0102] After each computing unit in the second row completes its calculation, it transmits the received first element to be processed and the target calculation result obtained in the second clock cycle to multiple computing units in the third row through the corresponding channels. For example, in the second clock cycle, element a11 can be transmitted to computing unit 2223 in the third row and second column via the first channel, element a12 can be transmitted to computing unit 2233 in the third row and third column, and element a13 can be transmitted to computing unit 2213 in the third row and first column. The target calculation result of computing unit 2212 can be transmitted to computing unit 2213 via the third channel. The target calculation result of computing unit 2222 can be transmitted to computing unit 2223 via the third channel. The target calculation result of computing unit 2232 can be transmitted to computing unit 2233 via the third channel.

[0103] By accumulating the multiplication result with the previous calculation result, the computing unit can obtain a target calculation result containing more complete accumulation information. For the target calculation result in the embodiment of the present invention, it can represent the matrix multiplication calculation that has been completed at the moment. In order to ensure that the matrix multiplication calculation can be continuously performed in the pulse array 220, the computing unit needs to provide the target calculation result of the computing unit to the subsequent hardware unit in the source pulse sub-array to ensure that the calculation result can continue to propagate according to the predetermined data flow path. For example, the target calculation result a11×b11+a12×b21 contains the product accumulation information of the first two elements of the first row of the first matrix to be processed and the first two elements of the first column of the second matrix to be processed, representing the partial calculation progress of the first row and first column of the result matrix.

[0104] According to an embodiment of the present invention, through the above-mentioned operational flow of multiplication calculation, accumulation operation and result transmission, the computing unit 2211 implements the processing logic of a single calculation position in matrix multiplication, and on the basis of ensuring the accuracy of the operation, it also achieves effective coordination with the overall data flow of the pulse array 220.

[0105] It can be understood that the above description is for the calculation of the second clock cycle of the current round, and the following description will be for the calculation of the third clock cycle of the current round.

[0106] Please refer to Figure 4C , Figure 4C A schematic diagram of data flow of a pulse array in the third clock cycle provided by an embodiment of the present invention.

[0107] like Figure 4C As shown, in the third clock cycle, the third row of data of the first matrix to be processed includes elements a31, a32, and a33, which can be input into calculation units 2211, 2221, and 2231 in the first row of pulse array 220, respectively. Calculation unit 2211 can perform a multiplication operation, obtaining a multiplication result of a31 × b11. Calculation unit 2221 can perform a multiplication operation, obtaining a multiplication result of a32 × b22. Calculation unit 2231 can perform a multiplication operation, obtaining a multiplication result of a33 × b33. After the calculation is completed, element a31 can be transmitted via the first channel to calculation unit 2232 in the second row and third column. Element a32 can be transmitted to calculation unit 2212 in the second row and first column. Element a33 can be transmitted to calculation unit 2222 in the second row and second column. It can be understood that for each intermediate calculation unit in the second row, the subsequent hardware unit can be an output unit.

[0108] Furthermore, in the third clock cycle, the second row, first column computation unit 2212 receives the element a22 delivered via the first channel and the target computation result of computation unit 2211 in the second clock cycle delivered via the third channel. Computation unit 2212 performs a multiplication operation, obtaining the multiplication result a22×b21 for computation unit 2212 in the third clock cycle. The multiplication result a22×b21 of computation unit 2212 in the second clock cycle is then added to the computation result a21×b11 from computation unit 2211 above in the second clock cycle, obtaining the target computation result a21×b11+a22×b21 for computation unit 2212 in the third clock cycle. It can be understood that the method of obtaining the target calculation result a22×b22+a23×b32 of the 2nd row and 2nd column calculation unit 2222 in the 3rd clock cycle and the method of obtaining the target calculation result a23×b33+a21×b13 of the 2nd row and 3rd column calculation unit 2232 in the 3rd clock cycle are the same or similar to the method of obtaining the target calculation result a31×b11+a22×b21 of the calculation unit 2212 in the 3rd clock cycle, and the present invention will not be repeated here.

[0109] Based on the above embodiment, as an optional embodiment, in order to complete the final calculation of the matrix multiplication and achieve effective output of the calculation result, the pulse subarray further includes an output unit; the output unit is configured to: multiply the received first element to be processed by the second element to be processed for the output unit to obtain a multiplication result of the output unit; add the multiplication result of the output unit to the previous calculation result provided by the previous calculation unit in the source pulse subarray to obtain an output result of the output unit; and store the output result in the cache unit 210.

[0110] The output unit in the embodiment of the present invention refers to a hardware module located at the end of each pulse sub-array 221 and outputting the final result.

[0111] For example, the rearranged second matrix to be processed B320 may include element b31, element b12, and element b23, which may be preloaded into calculation unit 2213, calculation unit 2223, and calculation unit 2233, respectively. Calculation unit 2213 in row 3, column 1 receives element a13 and the calculation result of calculation unit 2212 in the second clock cycle, provided by calculation unit 2222. Calculation unit 2213 may perform a multiplication operation to obtain a multiplication result a13×b31 of calculation unit 2213 in the third clock cycle, and add the multiplication result a13×b31 of calculation unit 2213 in the third clock cycle to the target calculation result of calculation unit 2212 in the second clock cycle, to obtain an output result a11×b11+a12×b21+a13×b31 of calculation unit 2213 in the third clock cycle, thereby completing the final calculation of the corresponding position in the matrix multiplication. It can be understood that the method of obtaining the output result a12×b22+a13×b32+a11×b12 of the 3rd row and 2nd column calculation unit 2223 in the 3rd clock cycle and the method of obtaining the target calculation result a13×b33+a11×b13+a12×b23 of the 3rd row and 3rd column calculation unit 2233 in the 3rd clock cycle are the same or similar to the method of obtaining the target calculation result a11×b11+a12×b21+a13×b13 of the calculation unit 2212 in the 2nd clock cycle, and the present invention will not be repeated here. At this point, at the end of the 3rd clock cycle, the three calculation units in the 3rd row have output the data of the first row in the result matrix of the matrix multiplication calculation. It can be understood that the calculation unit 2213, the calculation unit 2223, and the calculation unit 2233 can be used as three output units.

[0112] To ensure that the calculation results can be effectively utilized by the system and support subsequent data processing operations, the output unit needs to store the output results in the cache unit 210. By storing the final matrix multiplication result in the cache unit 210, the calculation results are buffered. It can also be understood that the method for obtaining the second row of data and the third row of data in the result matrix of the matrix multiplication calculation is the same or similar to the method for obtaining the first row of data in the result matrix of the matrix multiplication calculation, and the present invention will not be repeated here.

[0113] According to an embodiment of the present invention, by setting an output unit in the pulse sub-array and clarifying its calculation and storage functions, a complete closed loop of matrix multiplication calculation is achieved, which not only ensures the continuity of data flow and the accuracy of calculation results during the calculation process, but also realizes the effective preservation of calculation results by storing the results in the cache unit 210.

[0114] Based on the above embodiment, as another optional embodiment, in order to ensure that the second element to be processed in the pulse array 220 can be correctly propagated to the corresponding computing unit according to the rearranged data distribution, the computing unit can also assume the transmission function of the second element to be processed.

[0115] Specifically, the calculation unit is configured to provide the second to-be-processed element for the subsequent hardware unit in the source pulse subarray to the subsequent hardware unit in the source pulse subarray.

[0116] In the embodiment of the present invention, the second to-be-processed element for the subsequent hardware unit in the source pulse subarray refers to the second to-be-processed element received by the computing unit in the current computing cycle, but is not used for its own calculation but needs to be passed to the next computing unit 1 in the source pulse subarray.

[0117] Since the data elements in the rearranged second matrix to be processed need to be propagated among the different computing units of the pulse array 220 according to a specific timing and path, each computing unit, while performing its own multiplication and accumulation calculations, is also responsible for passing the received second elements to be processed downward to ensure that subsequent computing units can obtain the corresponding calculation data at the correct time.

[0118] During the specific transfer operation, the computing unit provides the received second element to be processed to the subsequent hardware unit in the source pulse subarray via a vertical downward data channel. In this embodiment of the present invention, the propagation of the second element to be processed must match the data distribution of the rearranged second matrix to be processed, ensuring that each second element to be processed reaches the target computing unit 2211 where multiplication and accumulation calculations are required according to the predetermined timing.

[0119] For example, the aforementioned 3×3 matrix multiplication scenario is used for further explanation. In the first clock cycle, the first row and first column computing unit 2211 receives element b11 for multiplication with element a11. Simultaneously, the computing unit 2211 can also pass element b11 to the second row and first column computing unit 2212, allowing the second row and first column computing unit 2211 to receive element b11 from above.

[0120] According to an embodiment of the present invention, it can be understood that elements b21, b32, and b13 are not intended for the second elements to be processed by computing units 2211, 2221, and 2231, and can be passed to the subsequent hardware units of computing units 2211, 2221, and 2231, respectively, without participating in the calculation. According to an embodiment of the present invention, by computing unit 2211 assuming the function of passing the second elements to be processed, the orderly propagation of the second elements to be processed in pulse array 220 is achieved, forming a complete data flow with the rearranged second matrix to be processed and the reconstructed propagation path of the first elements to be processed.

[0121] It can be understood that the elements a11, a12, a13, a21, a22, a23, a31, a32, and a33 in the above embodiment can be first elements to be processed, respectively. The elements b11, b12, b13, b21, b22, b23, b31, b32, and b33 in the above embodiment can be second elements to be processed, respectively.

[0122] From the above data flow comparison, it can be seen that in terms of hardware resource consumption, the embodiment of the present invention completely eliminates the use of delay units; in terms of startup delay, the present invention will almost completely eliminate the startup delay. For example, for a 3×3 pulse array, the use of 9 delay units can be eliminated, and the startup delay can be reduced from 2 clock cycles to 0 clock cycles. The data can be effectively calculated starting from the first clock cycle, and the results of each calculation unit 2211 in the first row can be obtained in the third clock cycle. It can be understood that the above description of the present invention uses a 3×3 pulse array and two 3×3 matrices as examples. However, the present invention is not limited to this, and the scale of the pulse array can be flexibly set according to the area of ​​the chip. The scale of the matrix that the pulse array can process may also be different from the scale of the pulse array.

[0123] Based on the above embodiment, as an optional embodiment, in order to support more complete matrix operation functions and improve the performance of the pulse array 220 in practical applications, the embodiment of the present invention expands the computing power to process matrix multiplication and addition operations including bias terms.

[0124] Specifically, the cache unit 210 is further configured to store a third matrix to be processed, where the third matrix to be processed includes a plurality of third elements to be processed;

[0125] The head calculation unit included in the pulse subarray is also used to perform the following operations to determine the target calculation result of the calculation unit based on the received first element to be processed and the second element to be processed for the calculation unit: multiplying the first element to be processed for the head calculation unit received from the cache unit and the second element to be processed for the head calculation unit to obtain the multiplication result of the head calculation unit; adding the third element to be processed for the head calculation unit received from the cache unit and the multiplication result of the head calculation unit to obtain the target calculation result of the head calculation unit.

[0126] The third matrix to be processed in the embodiment of the present invention refers to an input matrix used as a bias term or initial accumulated value in a matrix multiplication and addition operation. The matrix contains multiple third elements to be processed, and a linear transformation function is realized by adding the third elements to the matrix multiplication result.

[0127] In the embodiments of the present invention, the head calculation unit refers to the calculation unit located in the first row of each pulse subarray, which is at the starting position in the matrix multiplication and addition calculation chain. For example, the calculation unit 2211 of the pulse subarray 221 can serve as a head calculation unit. The calculation unit 2221 of the pulse subarray 222 can serve as a head calculation unit. The calculation unit 2231 of the pulse subarray 223 can serve as a head calculation unit.

[0128] For example, assume that the matrix multiplication and addition operation to be performed is Y=A×B+C, where A can be the first matrix to be processed, B can be the second matrix to be processed, and C can be the third matrix to be processed. Y can be the matrix of the matrix multiplication and addition operation result. In the first clock cycle, the calculation unit 2211 in the first row receives the first element to be processed a11, the second element to be processed b11, and the third element to be processed c11 from the cache unit 210. The head calculation unit 2211 first performs a multiplication operation to obtain a multiplication result a11×b11, and then adds the multiplication result a11×b11 to c11 to obtain the target calculation result a11×b11+c11. The target calculation result is passed to the 1st row and 2nd column calculation unit 2212, so that the calculation unit 2212 performs a multiplication operation based on the received first element to be processed a12 and the second element to be processed b22, obtains the multiplication result a12×b22 and adds it to the previous calculation result from the head calculation unit 2211 to obtain the target calculation result a11×b11+c11+a12×b22 of the calculation unit 2212, and so on until the calculation of the entire row is completed.

[0129] According to an embodiment of the present invention, by integrating the processing function of the third element to be processed in the head computing unit, the pulse array directly supports matrix multiplication and addition operations containing bias terms, thereby expanding the application scope of the pulse array 220.

[0130] Based on the above embodiment, as an optional embodiment, the first matrix to be processed represents the characteristic data of the neural network model, the second matrix to be processed represents the weight parameters of the neural network model, and the third matrix to be processed represents the bias term of the neural network layer in the neural network model.

[0131] In the neural network inference calculation process, the core operation is the linear transformation between neural network layers, which is usually expressed as a matrix multiplication calculation of feature data and weight parameters, and then adding a bias term to obtain the input features of the next layer.

[0132] In the above calculation process, the feature data carried by the first matrix to be processed changes dynamically with different input samples during the inference process, representing the input feature vector that needs to be processed currently or the output feature of the previous layer of neural network.

[0133] The weight parameters carried by the second matrix to be processed are fixed parameters learned during the training phase of the neural network model. They remain unchanged during the inference process and are often reused. They can be resident data.

[0134] The bias term carried by the third matrix to be processed is also a fixed parameter determined in the training phase, and is used to perform a linear offset adjustment on the matrix multiplication result.

[0135] Due to the permanent characteristics of weight parameters in the neural network inference process, the embodiment of the present invention pre-processes the weight parameters through a data reordering operation, and pre-loads the reordered second matrix to be processed into each computing unit of the pulse array 220, thereby realizing the pre-loading of weight parameters.

[0136] This preloading method maximizes the parallel computing capabilities of pulse array 220, avoiding the overhead of reloading weight parameters for each inference calculation. When new feature data is input, computation units 2211 in pulse array 220 can directly use the preloaded weight parameters for calculation, improving the computational efficiency of neural network inference.

[0137] According to an embodiment of the present invention, by combining the matrix multiplication computing capability of the pulse array 220 with the data characteristics of neural network inference, optimization for neural network applications is achieved, which not only ensures the accuracy of the calculation, but also gives full play to the advantages of weight parameter reuse, thereby improving the computing efficiency and resource utilization of the pulse array 220 in neural network inference tasks.

[0138] Figure 5 A schematic structural diagram of another data processing device provided by an embodiment of the present invention.

[0139] Based on the above embodiment, as an optional embodiment, the plurality of vector data to be processed is I. For the I vector data to be processed, the data processing device 200 further includes a processor 230, configured to perform the following operations: circularly shift multiple elements in the initial vector data of the second initial matrix by a second shifting bit number along the second direction; circularly shift multiple elements of the i-th initial vector data in the second initial matrix by i-1 bit numbers along the second direction to obtain a second matrix to be processed; and store the second matrix to be processed in the cache unit 210. I can be an integer greater than 1.

[0140] In this embodiment of the present invention, processor 230 refers to a general-purpose computing unit that works in conjunction with pulse array 220 and is responsible for performing data preprocessing and system control functions. Processor 230 reorders input matrix data by executing software algorithms, providing pulse array 220 with preprocessed data that meets its data processing timing requirements. Processor 230 can support various instruction sets, such as a Reduced Instruction Set Computer (RISC) processor.

[0141] After the processor 230 completes the data rearrangement operation, it stores the rearranged data in the cache unit 210. The system can ensure that the pulse array 220 can quickly access the required second element to be processed when performing matrix multiplication calculations, avoiding the impact of external memory access delay on calculation efficiency. The cache unit 210 serves as a data buffer between the processor 230 and the pulse array 220, supporting the batch data write operation of the processor 230 and meeting the data access timing requirements of the pulse array 220. The second initial matrix and the second matrix to be processed are respectively as follows: Figure 3 The second initial matrix B310 and the second matrix to be processed B320 are shown in FIG.

[0142] According to an embodiment of the present invention, the processor 230 stores the rearranged second matrix to be processed in advance in the cache unit 210, so that the pulse array 220 can directly and continuously read the timing-adjusted data from the cache, thereby avoiding data waiting and timing adjustment delays during the calculation process and improving the overall computational efficiency of the matrix multiplication calculation.

[0143] Figure 6 A schematic structural diagram of another data processing device provided by an embodiment of the present invention.

[0144] In practical matrix multiplication applications, particularly in deep learning inference and scientific computing scenarios, systems often need to perform a large number of matrix multiplications continuously, and the second matrix to be processed in each operation may come from different data batches or computational layers. If processor 230 needs to wait for pulse array 220 to complete the current calculation before preparing the data for the next round of calculation, pulse array 220 will be idle during the data preparation period, seriously affecting overall computing efficiency.

[0145] In view of the above problems, based on the above embodiments, as an optional embodiment, Figure 6 As shown, the above-mentioned data processing device 200 also includes a first storage module 240, which is used to store multiple state information and address information of the second matrix to be processed and the operating state information of the pulse array 220 sent by the processor 230; the pulse array 220 is further used to read the second matrix to be processed from the cache unit 210 according to the multiple state information and address information of the second matrix to be processed; the processor 230 is further used to generate the second matrix to be processed required for the next round of calculation of the pulse array 220 according to the operating state information of the pulse array 220.

[0146] The first storage module 240 in the embodiment of the present invention refers to a hardware module located between the processor 230 and the pulse array 220 for implementing status information exchange and data access control. This module can coordinate the working timing of the processor 230 and the pulse array 220 through multiple control registers.

[0147] By storing multiple pieces of status information about the second matrix to be processed, first storage module 240 enables pulse array 220 to accurately determine whether there is currently available rearranged data and its readiness status. Furthermore, first storage module 240 also stores the address information of the second matrix to be processed in cache unit 210, enabling pulse array 220 to directly locate and access the required data, avoiding the additional overhead of data lookup and address calculation.

[0148] In addition, the first storage module 240 is also responsible for storing the operation status information of the pulse array 220 , where the operation status information reflects at least one of whether the pulse array 220 is currently in a calculation state, calculation progress, and calculation completion status information.

[0149] By storing the operating status information in the first storage module 240 , the processor 230 can understand the working status of the pulse array 220 in real time and decide whether to start preparing the data required for the next round of calculations accordingly, thereby achieving timing coordination between the processor 230 and the pulse array 220 .

[0150] Specifically, after the processor 230 completes the rearrangement operation on the second initial matrix and stores the second matrix to be processed in the cache unit 210, the processor 230 updates the status information in the first storage module 240, indicating that the new rearranged data is ready, and writes the specific address information of the data in the cache unit 210 into the first storage module 240.

[0151] Pulse array 220 monitors the status information in first storage module 240 to determine whether new second matrix to be processed is available for reading. When the status information indicates that new data is available, pulse array 220 directly reads the second matrix to be processed from the specified location in cache unit 210 based on the address information stored in first storage module 240, avoiding the uncertain delays in data search and transmission.

[0152] At the start of a calculation, pulse array 220 updates the operating status information in first storage module 240, indicating that a matrix multiplication calculation is currently being performed. Using this operating status information, processor 230 can determine the operating status of pulse array 220 and, while pulse array 220 is executing the current calculation, simultaneously begin preparing the second matrix to be processed for the next round of calculations.

[0153] According to an embodiment of the present invention, by introducing state synchronization achieved by the first storage module 240, the waiting time between the processor 230 and the pulse array 220 is eliminated, so that when the pulse array 220 completes the current calculation, the data required for the next round of calculation is ready, and a new calculation task can be started immediately, thereby achieving parallelization of data preparation and calculation execution, avoiding idle waiting of hardware resources, and improving the efficiency in continuous matrix multiplication calculation scenarios.

[0154] In practical matrix multiplication systems, pulse array 220 must process matrix data of varying sizes, potentially originating from different computational tasks or data batches. If pulse array 220 relies solely on simple data-ready signals to read data, it may encounter issues such as incomplete data, incorrect addresses, or data size mismatches, leading to incorrect computational results or system anomalies.

[0155] In view of the above problems, based on the above embodiments, as an optional embodiment, Figure 6The pulse array 220 shown is further used to read the second matrix to be processed from the cache unit 210 based on multiple state information of the second matrix to be processed, address information of the second matrix to be processed in the cache unit 210, and the data volume of the second matrix to be processed; the first storage module 240 includes: a rearranged data state register 241, used to store at least one of the multiple state information of the second matrix to be processed sent by the processor 230; the multiple state information includes first state information and second state information, the first state information is used to indicate whether the processor 230 has generated the second matrix to be processed, and the second state information is used to indicate whether the processor 230 has stored the second matrix to be processed in the cache unit 210; an address synchronization register 242, used to store address information of the second matrix to be processed in the cache unit 210; and a length synchronization register 243, used to store data volume information of the second matrix to be processed.

[0156] The rearranged data status register 241 is used to store at least one of multiple pieces of status information regarding the second matrix to be processed sent by the processor 230, where the multiple pieces of status information include first status information and second status information. The first status information indicates whether the processor 230 has generated the second matrix to be processed, that is, whether the processor 230 has completed the rearrangement operation on the second initial matrix and generated the second matrix to be processed that meets the data processing timing requirements of the pulse array 220. The second status information indicates whether the processor 230 has stored the second matrix to be processed in the cache unit 210, ensuring that the rearranged data has been transferred from the working memory of the processor 230 to the cache unit 210 accessible to the pulse array 220. By distinguishing between the data generation status and the data storage status, the system can more precisely control data access timing, preventing the pulse array 220 from performing read operations when the data is not fully ready.

[0157] The address synchronization register 242 is used to store the address information of the second matrix to be processed in the cache unit 210. The address information indicates the starting position of the rearranged data in the cache unit 210. Since the cache unit 210 generally uses a continuous address space to store matrix data, the address information in the address synchronization register 242 provides an entry point for data access for the pulse array 220, allowing the pulse array 220 to directly locate the target data without performing address search or data scanning operations.

[0158] Length synchronization register 243 is used to store data size information for the second matrix to be processed, indicating the total number of data elements to be read from cache unit 210. By clarifying the data size information before data is read, pulse array 220 can verify the integrity of the read operation, ensuring that all required matrix elements are correctly read while avoiding reading invalid content beyond data boundaries.

[0159] In the specific data reading process, pulse array 220 first checks the status information in rearrangement data status register 241, verifies that both the first status information and the second status information are in the ready state, and confirms that processor 230 has completed data rearrangement and successfully stored it in cache unit 210. Subsequently, pulse array 220 obtains the starting address of the second matrix to be processed in cache unit 210 from address synchronization register 242 and obtains the data volume information to be read from length synchronization register 243. Based on this information, pulse array 220 executes a cache access operation starting from the specified starting address and reading the specified data volume, loading all elements of the second matrix to be processed into the internal storage units of pulse array 220. It will be understood that the data volume information can represent the amount of data that has been segmented so that the size of the second matrix to be processed is less than or equal to the size of the pulse array.

[0160] For example, assume that processor 230 completes the rearrangement of a 4×4 second initial matrix, generating a second matrix to be processed containing 16 elements and storing it in the contiguous space starting at address 0x1000 in cache unit 210. Processor 230 first sets the first status information of rearranged data status register 241 to 1, indicating that data generation is complete, and the second status information to 1, indicating that data storage is complete. It also sets address synchronization register 242 to 0x1000 and length synchronization register 243 to 16. After detecting the change in the status registers, pulse array 220 reads the contents of address synchronization register 242 and length synchronization register 243, confirming that 16 data elements need to be read starting at address 0x1000. Pulse array 220 then performs a cache access operation, sequentially reading the 16 data elements starting at address 0x1000, completing the loading process of the second matrix to be processed.

[0161] According to an embodiment of the present invention, by setting a status register, an address register, and a length register in the first storage module 240, comprehensive control and verification of the access process of the second matrix to be processed are achieved, ensuring that the pulse array 220 can accurately and completely obtain the data required for calculation, thereby improving the adaptability of the system when processing matrices of different sizes.

[0162] Based on the above embodiment, as an optional embodiment, in order to establish bidirectional state synchronization between the processor 230 and the pulse array 220, as shown in FIG. Figure 6 As shown, the first storage module 240 further includes an array status register 244 for storing operating status information of the pulse array 220 .

[0163] When the pulse array 220 starts to perform matrix multiplication calculation, the array state register 244 will be updated to the "calculation in progress" state. After the processor 230 detects this state change, it can start preparing for the data rearrangement required for the next round of calculation.

[0164] When the pulse array 220 completes the calculation and stores the result in the cache unit 210, it updates the array status register 244 to the "calculation completed" state and sends a task end signal to the processor 230. After receiving the status change, the processor 230 can trigger the next round of data transmission and calculation start process.

[0165] According to the embodiment of the present invention, the bidirectional status communication established through the array status register 244 realizes the real-time synchronization of the working status of the processor 230 and the pulse array 220, effectively improving the collaborative efficiency of continuous matrix multiplication calculations.

[0166] In actual continuous matrix multiplication scenarios, pulse array 220 typically needs to handle multiple rounds of computational tasks, each requiring different second matrix data to be processed. If computing unit 2211 were configured with only a single storage space to store the second matrix element to be processed, it would be unable to simultaneously receive and store the data required for the next round of computations during the execution of the current round of computations. Consequently, pulse array 220 must wait for the next round of data to be loaded and distributed after completing the current computation, resulting in idle waiting time for computing resources.

[0167] Figure 7 A schematic diagram of the structure of a computing unit provided in an embodiment of the present invention.

[0168] In view of the above problems, based on the above embodiments, as an optional embodiment, Figure 7 As shown, computing unit 2200 includes multiple temporary storage modules, each for storing the second to-be-processed elements for computing unit 2200 in different rounds. Computing unit 2200 may be any one of computing unit 2211, computing unit 2212, computing unit 2213, computing unit 2221, computing unit 2222, computing unit 2223, computing unit 2231, computing unit 2232, and computing unit 2233 described above.

[0169] The temporary storage module in the embodiment of the present invention refers to an independent storage unit inside the computing unit 2211 for temporarily storing the second element to be processed. Each temporary storage module has independent read-write control logic and data retention capabilities. By configuring multiple temporary storage modules in the computing unit, the computing unit can simultaneously store the second element to be processed from different computing rounds, thereby achieving time multiplexing and spatial isolation of data storage. The multiple temporary storage modules may include a temporary storage module 711 and a temporary storage module 712. It can be understood that the temporary storage module 711 can serve as a first temporary storage module, and the temporary storage module 712 can serve as a second temporary storage module.

[0170] Specifically, when pulse array 220 performs the first round of matrix multiplication, computing unit 2211 stores the second element to be processed in the current round in first temporary storage module 711 and reads data from temporary storage module 711 to participate in the multiplication-accumulation operation. For example, elements b11 through b33 in the current round can be stored in temporary storage modules 711 of different computing units.

[0171] While the first round of calculations is ongoing, processor 230 has already completed the rearrangement of the second matrix to be processed required for the second round of calculations, and pulse array 220 begins receiving the second elements to be processed in the second round. At this point, computing unit 2200 stores the received second elements to be processed in the second round of calculations in second temporary storage module 712, while the data in first temporary storage module 711 continues to be used for the current round of calculations.

[0172] After the first round of calculations is complete, computing unit 2200 immediately switches data sources, reading the second element to be processed required for the second round of calculations from second temporary storage module 712 to begin a new calculation task. Simultaneously, first temporary storage module 711 can receive data for the third round of calculations for preloading. By alternating between multiple temporary storage modules, computing unit 2200 achieves seamless switching between consecutive calculation rounds, eliminating data loading wait time.

[0173] According to an embodiment of the present invention, by configuring multiple temporary storage modules in the computing unit 2211 and realizing parallel storage and alternating use of data from different rounds, the data waiting problem in continuous matrix multiplication calculations is effectively solved, so that the pulse array 220 can prepare the data required for the next round of calculations in advance while completing the current calculation, realizing the parallelization of calculation execution and data preloading, and improving the hardware utilization and overall computing efficiency in continuous computing scenarios.

[0174] Based on the above embodiment, as an optional embodiment, Figure 7As shown, each computing unit 2200 also includes a second storage module 720, which is used to store at least one of a plurality of preload indication values ​​corresponding one-to-one to a plurality of temporary storage modules. The preload indication value can instruct the computing unit 2200 to store the second to-be-processed element for the computing unit 2200 in the next round to the temporary storage module corresponding to the preload indication value.

[0175] The second storage module 720 in the embodiment of the present invention refers to a storage unit inside the computing unit 2200 for storing usage status information and access control information of multiple temporary storage modules. The module implements the storage function of data storage and reading operations by maintaining control information corresponding to the number of temporary storage modules.

[0176] Specifically, the preload indication value indicates the working mode of the corresponding temporary storage module through binary state coding. The multiple preload indication values ​​include a first preload indication value and a second preload indication value. The first preload indication value indicates that the temporary storage module 711 is in an idle state or has completed the current round of data services. The second preload indication value indicates that the temporary storage module 712 is in an idle state or has completed the current round of data services. If the second storage module stores, for example, a second preload indication value for the temporary storage module 712, indicating that the temporary storage module 712 is currently in an idle state or has completed the current round of data services, it can receive the second to-be-processed element of the next round to perform a preload operation.

[0177] According to an embodiment of the present invention, by storing the preload indication value for the temporary storage module through the second storage module 720, the calculation unit 2211 can automatically identify and correctly route data of different rounds during continuous calculation, avoiding data storage errors and reading confusion, and ensuring the accuracy of data calculation in multiple rounds of calculation.

[0178] Based on the above embodiment, as an optional embodiment, Figure 7 As shown, the second storage module 720 further includes: a preload counter 721 for obtaining the number of second elements to be processed received by the computing unit 2200 in the current round; and a preload constant register 722 for storing the preload trigger threshold of the computing unit 2211 in the current round. The preload trigger threshold is the difference between the row number of the computing unit 2211 in the pulse array 220 and a preset value. The computing unit 2200 also includes a comparator 730 for: if the number of second elements to be processed received in the current round is equal to the preload trigger threshold, storing the received second elements to be processed that are consistent with the preload trigger threshold in one of the multiple temporary storage modules as the second elements to be processed for the computing unit 2200 in the next round.

[0179] The preload counter 721 is used to obtain the number of second elements to be processed received by the computing unit 2200 in the current round. The preload counter 721 is automatically reset to zero at the beginning of each computing round and then counts the number of second elements to be processed flowing through the computing unit 2200 in real time. For example, in the first clock cycle, the value of the preload counter of the computing unit 2211 is 0.

[0180] Preload constant register 722 is used to store the preload trigger threshold of the calculation unit 2200. This threshold is preset based on the row number of the calculation unit 2200 in the pulse array 220. Specifically, it is calculated as the difference between the row number of the calculation unit 2211 and a preset value, where the preset value can be set to 1. Through this setting, the preload trigger threshold of the calculation units in the first row is 0, the preload trigger threshold of the calculation units in the second row is 1, the preload trigger threshold of the calculation units in the third row is 2, and so on. This achieves triggering that matches the distribution timing of the rearranged second matrix to be processed.

[0181] The comparator 730 can compare the number of receptions recorded by the preload counter 721 with the preload trigger threshold stored in the preload constant register 722. When the two values ​​are equal, it indicates that the second element to be processed currently received is valid data for the computing unit 2200. At this time, the comparator 730 triggers a storage operation, storing the received second element to be processed that is consistent with the preload trigger threshold in one of the multiple temporary storage modules as the second element to be processed for the computing unit 2211 in the current round. For other second elements to be processed that do not meet the trigger condition, the computing unit 2211 only performs a transfer operation without storing them, ensuring that they can continue to flow to the downstream target computing unit 2211.

[0182] For example, in the first clock cycle, the value of the preload counter of computing unit 2211 is 0, the value of the preload constant register is 0, and the preload indication value indicates that the second element to be processed for computing unit 2211 in the next round can be stored in the second temporary storage module. The computing unit 2211 can receive a second element to be processed in the next round and store it in the second temporary storage module within computing unit 2211. In addition, the second element to be processed can also be provided to computing unit 2212, and the value of the preload counter of computing unit 2211 can also be set to 1.

[0183] For another example, in the second clock cycle, the value of the preload counter of the computing unit 2211 is 1, and the value of the preload constant register is 0. The preload indication value indicates that the second element to be processed for the computing unit 2211 in the next round can be stored in the second temporary storage module. The above-mentioned computing unit 2211 can receive the second element to be processed in the second next round. In the case where the value of the preload counter of the computing unit 2211 is different from the value of the preload constant register, the second element to be processed in the second next round may not be stored in the temporary storage module of the computing unit 2211, or the second element to be processed in the second next round may be provided to the computing unit 2212. In addition, in the second clock cycle, the value of the preload counter of the computing unit 2212 is 0, and the value of the preload constant register is 1. The above-mentioned computing unit 2212 can receive the second element to be processed in the first next round. If the value of the preload counter of computing unit 2212 is different from the value of the preload constant register, the second element to be processed in the first next round may not be stored in the temporary storage module of computing unit 2212, or the second element to be processed in the first next round may be provided to computing unit 2213. At the end of the second clock cycle, the value of the preload counter of computing unit 2211 may be set to 2, and the value of the preload counter of computing unit 2212 may be set to 1.

[0184] For another example, in the third clock cycle, the value of the preload counter of the computing unit 2211 is 2, and the value of the preload constant register is 0. The above-mentioned computing unit 2211 can receive the second element to be processed in the third next round. In the case that the value of the preload counter of the computing unit 2211 is different from the value of the preload constant register, the second element to be processed in the third next round may not be stored in the temporary storage module of the computing unit 2211, or the second element to be processed in the third next round may be provided to the computing unit 2212. In addition, in the second clock cycle, the value of the preload counter of the computing unit 2212 is 1, and the value of the preload constant register is 1. The preload indication value indicates that the second element to be processed for the computing unit 2212 in the next round can be stored in the second temporary storage module. The above-mentioned computing unit 2212 can receive the second element to be processed in the second next round. If the value of the preload counter of computing unit 2212 is consistent with the value of the preload constant register, the second element to be processed in the second next round can be stored in the second temporary storage module of computing unit 2212, or the second element to be processed in the second next round can be provided to computing unit 2213. At the end of the second clock cycle, the value of the preload counter of computing unit 2211 can be set to 2, and the value of the preload counter of computing unit 2212 can be set to 1. In the third clock cycle, the value of the preload counter of computing unit 2213 is 0, and the value of the preload constant register is 2. The above computing unit 2213 can receive the second element to be processed in the first next round. If the value of the preload counter of computing unit 2213 is different from the value of the preload constant register, the second element to be processed in the first next round can be not stored in the temporary storage module of computing unit 2213. When the third clock cycle is about to end, the preload counter of the calculation unit 2211 is 3, the preload counter of the calculation unit 2212 is 2, and the preload counter of the calculation unit 2213 is 1.

[0185] According to an embodiment of the present invention, through the precise matching method composed of the preload counter 721, the preload constant register 722 and the comparator 730, the computing unit 2211 is able to intelligently filter and accurately obtain the data flowing through, ensuring that each computing unit 2211 can obtain the correct second element to be processed at the correct time, thereby avoiding the problems of data mismatch and calculation errors.

[0186] Based on the above embodiment, as an optional embodiment, Figure 7As shown, the computing unit 2200 further includes a preload selection register 723, which is used to store a first preload indication value or a second preload indication value. The first preload indication value is used to instruct the second element to be processed for the computing unit in the next round to be stored in the first temporary storage module, and the second preload indication value is used to instruct the second element to be processed for the computing unit in the next round to be stored in the second temporary storage module. For example, the first preload indication value can be 0, and the second preload indication value can be 1.

[0187] Preload selection register 723 is a dedicated register within computing unit 2211 that controls the selection of temporary storage modules. It indicates the data storage target by maintaining the preload indicator bits corresponding to the temporary storage modules. The preload indicator bits are binary-coded, with each indicator bit corresponding to a temporary storage module, indicating whether the module is selected to receive the next round of data.

[0188] The calculation unit is further configured to perform at least one of the following operations: switching the value of the preload indicator bit from the first preload indicator value to the second preload indicator value when the pulse array completes the current round of calculations. Switching the value of the preload indicator bit from the second preload indicator value to the first preload indicator value when the pulse array completes the current round of calculations. The switching mode of the preload selection register 723 is associated with the operating state of the pulse array 220. When the pulse array 220 completes the current round of calculations, the value of the preload selection register may be flipped. The temporary storage module 711 used for calculations in the current round will switch to receiving new data in the next round, and the temporary storage module 712 used for preloading in the current round will switch to providing calculation data in the next round.

[0189] For example, during the first round of calculation, the preload selection register is set to the second preload indicator value. The second element to be processed by the calculation unit 2200 in the second round of calculation can be stored in the temporary storage module 712. After the pulse array 220 completes the first round of calculation, the preload selection register can be flipped to the first preload indicator value.

[0190] During the second round of calculation, the preload selection register is set to the first preload indication value. In the third round of calculation, the second element to be processed by the calculation unit 2200 can be stored in the temporary storage module 711. After the pulse array 220 completes the second round of calculation, the preload selection register can be flipped to the second preload indication value.

[0191] According to an embodiment of the present invention, by state-driven switching of the preload selection register 723, automatic storage of the computing unit 2200 between consecutive computing rounds is achieved, thereby ensuring the correctness of data flow in multiple rounds of computing.

[0192] Based on the above embodiment, as an optional embodiment, Figure 7 As shown, the second storage module 720 further includes: a calculation control register 724 for storing a calculation indication value, where the calculation indication value is used to instruct the calculation unit 2200 to select a second element to be processed from multiple temporary storage modules for multiplication calculation.

[0193] The calculation control register 724 is a register in the second storage module 720 that selects the calculation data source. It stores a calculation instruction value and instructs the calculation unit 2200 to read the second element to be processed from the corresponding temporary storage module. The calculation instruction value and the preload instruction value form a complementary control relationship. When the preload selection register 723 indicates one temporary storage module for preloading, the calculation control register 724 indicates another temporary storage module for calculation, avoiding data conflicts.

[0194] The calculation indicator value in the calculation control register 724 changes synchronously with the preload indicator value during the round switching state. When the preload indicator bit flips, the calculation indicator value also flips accordingly, switching the data read destination. This synchronous flipping operation ensures that the calculation unit 2211 always reads elements from the temporary storage module containing the correct data for the current round, while the other temporary storage module safely receives preloaded data.

[0195] For example, during the first round of calculation, the value of preload selection register 723 is 1, indicating that temporary storage module 712 is to be preloaded. The calculation control register 724 stores a first calculation indication value (e.g., 0), indicating that data should be read from temporary storage module 711 for calculation. When the round switches, the preload indication bit flips, indicating that temporary storage module 711 is to be preloaded. Furthermore, when the round switches, the value of the calculation control register flips to a second calculation indication value (e.g., 1), indicating that data should be read from temporary storage module 712 for the second round of calculation.

[0196] According to an embodiment of the present invention, the independent data source control established by the calculation control register 724 forms a complete bidirectional data flow control with the preload selection register 723, thereby achieving strict isolation and coordination between the calculation operation and the preload operation.

[0197] Based on the above embodiment, as an optional embodiment, Figure 7 As shown, the calculation unit 2200 also includes: a multiplication-accumulation calculator 750; a selector 740, which is used to determine the selected second element to be processed from multiple temporary storage modules according to the calculation indication value stored in the calculation control register 724; and provide the selected second element to be processed to the multiplication-accumulation calculator 750 for multiplication operation.

[0198] Selector 740 is a hardware module within computation unit 2211 that implements data path selection. This module receives the calculation indication value from computation control register 724 as a selection signal and, based on this signal, selects the corresponding data path from the outputs of multiple temporary storage modules. Selector 740 uses multiplexing logic to dynamically connect the temporary storage modules to the multiplication-accumulation calculator 750, ensuring that the correct second element to be processed is delivered to the multiplication-accumulation calculator in each computation cycle.

[0199] Among them, the multiplication and accumulation calculator 750 refers to a dedicated hardware unit that performs basic matrix multiplication operations inside the calculation unit 2211. The unit integrates the functions of the multiplier and accumulator, can receive the first to-be-processed element and the second to-be-processed element for multiplication operations, and add the result to the accumulated input to obtain the final calculation result.

[0200] Specifically, the selector 740 performs a data selection operation according to the calculation indicator value stored in the calculation control register 724. When the calculation indicator value is 0, the selector 740 connects the output of the first temporary storage module to the second to-be-processed element input terminal of the multiplication-accumulation calculator 750; when the calculation indicator value is 1, the selector 740 connects the output of the second temporary storage module to the second to-be-processed element input terminal of the multiplication-accumulation calculator 750.

[0201] In this manner, selector 740 ensures that multiply-accumulate calculator 750 always receives the correct data required for the current round. After selecting the data, selector 740 provides the selected second element to be processed to multiply-accumulate calculator 750 via a dedicated data path. Multiply-accumulate calculator 750 then performs a multiply-accumulate operation on the received first element to be processed and the accumulation input.

[0202] According to an embodiment of the present invention, a complete hardware path from data storage to operation execution is established, and accurate data transmission between the multi-temporary storage module and the computing unit 2200 is realized, ensuring that the computing unit 2200 can automatically select the correct data source and perform accurate operation operations in continuous matrix multiplication calculations.

[0203] Please refer to Figure 8 , Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0204] An embodiment of the present invention further discloses an electronic device 800, including a memory 250 and the above-mentioned data processing device.

[0205] like Figure 8 As shown, the memory 250 is used to store input data, intermediate result data, and output result data required for the data processing device 200 to perform matrix multiplication calculations.

[0206] Memory 250 in this embodiment of the present invention refers to a hardware module within electronic device 800 that provides large-capacity data storage. In this embodiment, it can be understood as an off-chip memory device that provides a data source and result storage space for computational operations performed by data processing device 200. Memory 250 works in conjunction with cache unit 210 within data processing device 200 to implement hierarchical data storage and efficient access.

[0207] Exemplarily, the memory 250 may be a dynamic random access memory (DRAM), a static random access memory (SRAM), a non-volatile memory (Non-volatile Memory), a solid state drive (SSD), a hard disk drive (HDD), or other types of data storage devices.

[0208] In this embodiment of the present invention, the processor 230 reads the original data of the second initial matrix from the memory 250, performs a data reordering operation to generate a second matrix to be processed, and writes the reordered data to the cache unit 210. Simultaneously, the pulse array 220 reads the data of the first matrix to be processed and the third matrix to be processed from the memory 250 to the cache unit 210 based on the status information in the first storage module 240. After the data processing device 200 completes the matrix multiplication calculation, it writes the calculation result from the cache unit 210 back to the memory 250 to store the calculation result.

[0209] The electronic device 800 in the embodiments of the present invention can be a computing system that integrates the aforementioned data processing device 200. In the embodiments of the present invention, it can be understood as a computing device with matrix multiplication computing capabilities, which implements data processing functions by carrying the data processing device 200 provided in the embodiments of the present invention. The electronic device 800 serves as a carrier platform for the data processing device 200, providing the necessary power supply, system control, input and output interfaces, and communication connection functions with external systems.

[0210] It should be noted that the electronic device 800 can simultaneously integrate one or more data processing devices 200 to meet matrix computing requirements of different scales and complexities. When the electronic device 800 is configured with multiple data processing devices 200, parallel computing can be achieved through task allocation and load balancing, and continuous computing tasks can be efficiently executed through pipeline processing.

[0211] Exemplarily, the electronic device 800 can be a server, a personal computer, a workstation, a mobile computing device, a tablet computer, a smart phone, an embedded computing device, an edge computing device, a dedicated computing accelerator, a graphics processing device, a digital signal processing device, or other electronic device 800 that needs to perform large-scale matrix operations.

[0212] The server in the embodiments of the present invention refers to a high-performance computing device deployed in a data center or cloud computing environment, which provides matrix computing services to multiple clients by integrating a data processing device 200. The personal computer in the embodiments of the present invention refers to a desktop or notebook computing device for individual users, which supports local artificial intelligence applications through a built-in data processing device 200. The mobile computing device in the embodiments of the present invention refers to a computing device with portability, such as a smartphone or tablet computer, which implements mobile computing functions by integrating a miniaturized data processing device 200.

[0213] The embedded computing device in the embodiments of the present invention refers to a dedicated computing module integrated into other systems or products, which provides matrix computing capabilities for the host system by carrying the data processing device 200. The edge computing device in the embodiments of the present invention refers to a computing device deployed at the edge of the network, which implements data processing and reasoning computing by configuring the data processing device 200.

[0214] In actual deployment applications, the electronic device 800 can select different system configurations and integration methods based on specific application requirements and usage scenarios. The electronic device 800 can integrate the data processing device 200 as a co-processing unit of the main processor, or it can use the data processing device 200 as an independent computing accelerator card for external expansion. The electronic device 800 can also establish a data transmission connection with the data processing device 200 via a system bus, a high-speed interconnect interface, or a dedicated data channel to achieve efficient data exchange between the main system and the data processing device 200.

[0215] In a feasible implementation, the electronic device 800 may also be configured with multiple data processing devices 200 to construct a distributed computing architecture, and further enhance the overall matrix computing processing capability through parallel deployment.

[0216] In another feasible implementation, the electronic device 800 may also dynamically configure and adjust the operating mode of the data processing device 200 according to the power consumption budget and performance requirements to achieve a balanced optimization between computing performance and energy efficiency.

[0217] It should be noted that the specific form and configuration of the electronic device 800 are merely exemplary, and the embodiments of the present invention do not limit the specific implementation of the electronic device. In actual applications, the electronic device may select an appropriate hardware platform and system architecture based on factors such as the specific application scenario, performance requirements, power consumption constraints, and cost budget.

[0218] It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or couplings fall within the scope of the present invention.

[0219] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data processing device, characterized in that: include: a cache unit, configured to store a first matrix to be processed and a second matrix to be processed, wherein the first matrix to be processed includes a plurality of first elements to be processed, and the second matrix to be processed includes a plurality of vector data to be processed, and the vector data to be processed includes a plurality of second elements to be processed; A pulse array, comprising a plurality of pulse sub-arrays, wherein the pulse sub-arrays include a plurality of computing units, and the computing units are configured to: determining a target calculation result of the calculation unit according to the received first element to be processed and the second element to be processed for the calculation unit; providing the first to-be-processed element to a subsequent hardware unit for the computing unit in a target external pulse subarray, wherein the target external pulse subarray is determined by cyclically shifting a source pulse subarray where the computing unit is located in the plurality of pulse subarrays by a first shift bit number in a first direction; The vector data to be processed in the second matrix to be processed is obtained by the following operations: A plurality of elements in the initial vector data of the second initial matrix are circularly shifted along a second direction by a second shifting bit number, where the second shifting bit number is determined according to position information of the initial vector data in the second initial matrix and a preset value.

2. The device according to claim 1, wherein The computing unit is configured to perform the following operations to determine a target computing result of the computing unit according to the received first element to be processed and the second element to be processed for the computing unit: multiplying the received first element to be processed and the second element to be processed for the computing unit to obtain a multiplication result of the computing unit; determining a target calculation result of the calculation unit according to a multiplication result of the calculation unit and a previous calculation result provided by a previous calculation unit in the source pulse subarray; The target calculation result of the calculation unit is provided to a subsequent hardware unit in the source pulse sub-array.

3. The device according to claim 1, wherein The computing unit is used for: A second to-be-processed element for a subsequent hardware unit in the source pulse subarray is provided to the subsequent hardware unit in the source pulse subarray.

4. The device according to claim 1, wherein The cache unit is further configured to store a third matrix to be processed, wherein the third matrix to be processed includes a plurality of third elements to be processed; The plurality of computing units included in the pulse subarray include a head computing unit, and the head computing unit is further configured to perform the following operations: multiplying a first to-be-processed element for the head calculation unit received from the cache unit and a second to-be-processed element for the head calculation unit to obtain a multiplication result of the head calculation unit; The third to-be-processed element for the head calculation unit received from the cache unit and the multiplication result of the head calculation unit are added to obtain a target calculation result of the head calculation unit.

5. The device according to claim 1, wherein The pulse sub-array further includes an output unit, and the subsequent hardware unit is a calculation unit or an output unit in the pulse sub-array. The output unit is used for: multiplying the received first element to be processed and the second element to be processed for the output unit to obtain a multiplication result of the output unit; Adding the multiplication result of the output unit to the previous calculation result provided by the previous calculation unit in the source pulse subarray to obtain the output result of the output unit; The output result is stored in the cache unit.

6. The device according to claim 1, characterized in that The plurality of vector data to be processed is I vector data to be processed, The processor is further included, configured to perform the following operations on the I vector data to be processed to circularly shift multiple elements in the initial vector data of the second initial matrix along the second direction by a second shifting bit number: Circularly shifting multiple elements of the i-th initial vector data in the second initial matrix by i-1 digits along the second direction to obtain the second matrix to be processed; as well as The second matrix to be processed is stored in the cache unit.

7. The device according to claim 6, characterized in that The system further comprises a first storage module for storing a plurality of state information and address information of the second matrix to be processed and the operation state information of the pulse array sent by the processor. The pulse array is further configured to read the second matrix to be processed from the cache unit according to a plurality of state information and address information of the second matrix to be processed; The processor is further configured to generate a second matrix to be processed required for the next round of calculation of the pulse array according to the operation status information of the pulse array.

8. The device according to claim 7, characterized in that The pulse array is further configured to read the second matrix to be processed from the cache unit according to a plurality of state information of the second matrix to be processed, address information of the second matrix to be processed in the cache unit, and a data amount of the second matrix to be processed; The first storage module includes: a reordering data status register, configured to store at least one of a plurality of status information of a second matrix to be processed sent by the processor; the plurality of status information comprising first status information and second status information, the first status information being configured to indicate whether the processor has generated the second matrix to be processed, and the second status information being configured to indicate whether the processor has stored the second matrix to be processed in the cache unit; An address synchronization register, configured to store address information of the second matrix to be processed in the cache unit; a length synchronization register, used to store data amount information of the second matrix to be processed; The array status register is used to store the operation status information of the pulse array.

9. The device according to claim 1, characterized in that Each of the computing units includes a plurality of temporary storage modules, each of which is used to store the second to-be-processed elements for the computing unit in different rounds.

10. The device according to claim 9, characterized in that Each of the computing units also includes a second storage module for storing at least one of a plurality of preloading indication values ​​corresponding one-to-one to the plurality of temporary storage modules, wherein the preloading indication value is used to instruct the computing unit to store the second to-be-processed element for the computing unit in the next round to the temporary storage module corresponding to the plurality of preloading indication values.

11. The device according to claim 10, characterized in that The second storage module further includes: A preload counter is used to obtain the number of second elements to be processed received by the computing unit in the current round; A preload constant register, used to store a preload trigger threshold of the calculation unit, wherein the preload trigger threshold is the difference between the row number of the calculation unit in the pulse array and a preset value; The calculation unit further includes a comparator, configured to: When the number of second elements to be processed received in the current round is equal to the preloading trigger threshold, the received second elements to be processed that are consistent with the preloading trigger threshold are stored in one of the multiple temporary storage modules as the second elements to be processed for the computing unit in the next round.

12. The device according to claim 11, characterized in that The multiple temporary storage modules include a first temporary storage module and a second temporary storage module, and the multiple preload indication values ​​include a first preload indication value and a second preload indication value. The computing unit further includes a preload selection register, the preload selection register being used to store the first preload indication value or the second preload indication value, the first preload indication value being used to instruct the second element to be processed for the computing unit in the next round to be stored in the first temporary storage module, and the second preload indication value being used to instruct the second element to be processed for the computing unit in the next round to be stored in the second temporary storage module; The computing unit is further configured to perform at least one of the following operations: When the pulse array completes the current round of calculation, switching the preload selection register from the first preload indication value to the second preload indication value; When the pulse array completes the current round of calculation, the preload selection register is switched from the second preload indication value to the first preload indication value.

13. The device according to claim 11, characterized in that The second storage module further includes: The calculation control register is used to store a calculation indication value, where the calculation indication value is used to instruct the calculation unit to select a second element to be processed from the multiple temporary storage modules for multiplication calculation.

14. The device according to claim 13, characterized in that The computing unit further includes: Multiplication and accumulation calculator; Selectors for: Determining a selected second element to be processed from the plurality of temporary storage modules according to the calculation indication value stored in the calculation control register; and The selected second element to be processed is provided to the multiplication-accumulation calculator for multiplication operation.

15. An electronic device, characterized in that: include: Memory; A data processing device as claimed in any one of claims 1 to 14.

Images