Systolic array, systolic array system and operation method, device and storage medium thereof

Abstract

The present disclosure provides a systolic array, a systolic array system and an operation method, device and storage medium thereof. According to a received work instruction, a work mode indicated by the work instruction is determined. When the work mode is a sorting mode, after different configuration values of control registers of a first basic operation unit and a second basic operation unit in a systolic array are assigned by a sorting control signal sent by an array controller, characteristic data of a characteristic cache and corresponding label data thereof are gradually input into the systolic array for a synchronous sorting operation, and after sorting is completed, sorted characteristic data and corresponding synchronous label data thereof are output via an output cache and returned to a system bus. The function of a complex sorting operator for direct calculation and tracking on the systolic array can be realized, the utilization rate of the systolic array is improved, and the problem of transmission bandwidth waste caused by data transfer is avoided.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a pulsating array, a pulsating array system, and its computing method, apparatus, and storage medium. Background Technology

[0002] With the rapid development of technology, Convolutional Neural Networks (CNNs) are neural network structures designed in the field of artificial intelligence for processing image data. Utilizing the translation invariance and locality of convolution operations in mathematics, they can effectively extract image features, and therefore have a wide range of applications in computer vision fields such as image classification and object detection.

[0003] The high performance of convolutional neural networks (CNNs) in image processing requires a massive amount of computation. In CNNs, besides convolution operations, sorting operations also consume a significant amount of computation. Sorting operations are commonly used in object detection algorithms, and as these algorithms evolve, many detection scenarios require knowing the position of each sorted data point within the original dataset. For example, when identical data appears, it's necessary to distinguish them based on their position in the original dataset. Another example is the Non-Maximum Suppression (NMS) operator in object detection algorithms, which requires sorting data according to a specific item, while reserving information for subsequent operations.

[0004] In existing technologies, there are two solutions for special sorting operators: one is to efficiently process sorting operators by adding additional heap sort hardware units, and the other is to support sorting operators through internal vector processing units. Adding additional hardware units increases unnecessary area, and the data to be sorted is generally generated by the systolic array inside the accelerator. Using additional hardware or vector units requires transferring the data from the systolic array to external processing, which cannot solve the bandwidth waste problem caused by data transfer. Furthermore, it cannot directly implement the tracking function of sorted data through the existing systolic array, resulting in poor utilization of the existing systolic array. Summary of the Invention

[0005] This disclosure provides a pulsating array, a pulsating array system, and a method, apparatus, and storage medium thereof, to at least solve the above-mentioned technical problems existing in the prior art.

[0006] According to a first aspect of this disclosure, a pulsating array is provided, comprising:

[0007] Multiple first-level basic computation units and multiple second-level basic computation units are connected in a row-corresponding matrix arrangement to perform synchronous sorting operations on labeled feature data in sorting mode; among them...

[0008] Each first basic arithmetic unit includes a first comparator, a first control register, and a first result buffer. The first comparator is used to compare the input feature data. The first control register is used to output the comparison sign result as a synchronization control signal to the second basic arithmetic unit corresponding to the current first basic arithmetic unit, and to output the comparison data result to the first result buffer and the first feature input register of the next basic arithmetic unit, respectively. After sorting, the temporary feature data stored in the first result buffer is output as sorted feature data.

[0009] Each second basic operation unit includes a second comparator, a second control register, a second result buffer, and a synchronization weight input register. The second comparator is used to compare the input tag data according to the synchronization control signal received by the synchronization weight input register. The second control register is used to output the comparison tag results to the second result buffer and the second feature input register of the next basic operation unit, respectively. After sorting, the temporary tag data stored in the second result buffer is used as the synchronization tag data of the sorting feature data for synchronous output.

[0010] In one embodiment, each first basic operation unit includes a first feature input register for storing feature data, and a first result buffer for temporarily storing temporary feature data; each second basic operation unit includes a second feature input register for storing tag data, and a second result buffer for temporarily storing temporary tag data.

[0011] Correspondingly, the first control register is also used to sequentially input the feature data of the first feature input register and the temporary feature data of the first result buffer into the first comparator; the second control register is also used to sequentially input the tag data of the second feature input register and the temporary tag data of the second result buffer into the second comparator.

[0012] In one possible implementation, the first comparator is specifically used for:

[0013] The size of the feature data input into the first feature input register and the temporary feature data input into the first result buffer are compared sequentially. According to the preset sorting rules, the feature data that meets the first sorting condition is used as the new temporary feature data, and the feature data that meets the second sorting condition is used as the feature data in the first feature input register of the next basic operation unit.

[0014] Accordingly, the first control register is further used to: output the new temporary feature data to the first result buffer, and output the feature data that meets the second sorting condition to the first feature input register of the next basic operation unit.

[0015] In one possible implementation, the second comparator is specifically used for:

[0016] Based on the synchronization control signal received by the synchronization weight input register, the tag data input to the second feature input register and the temporary tag data input to the second result buffer are compared. The tag data that meets the first sorting condition is used as the new temporary tag data, and the tag data that meets the second sorting condition is used as the tag data in the second feature input register of the next basic operation unit.

[0017] Accordingly, the second control register is also specifically used to: synchronously output the new temporary tag data to the second result buffer, and synchronously output the tag data that meets the second sorting condition to the second feature input register of the next basic operation unit.

[0018] In one possible implementation, the pulsating array further includes:

[0019] The plurality of first basic operation units and the plurality of second basic operation units are classified into the same convolution basic operation unit, which includes a convolution weight input register, a convolution feature input register, a convolution result buffer, a convolution control register, and a multiplier-accumulator; wherein...

[0020] The convolution weight input register is used to store convolution weight data;

[0021] The convolutional feature input register is used to store convolutional feature data;

[0022] The convolution result buffer is used to temporarily store convolutional temporary data;

[0023] The multiply-accumulator is used to take the convolutional temporary data temporarily stored in the convolutional result buffer as the cumulative addend, and successively calculate the multiplication operation of the convolutional feature data input to the convolutional feature input register and the convolutional weight data input to the convolutional weight input register, and use the calculation result as the new convolutional temporary data.

[0024] The convolution control register is used to input the convolution weight data from the convolution weight input register, the convolution feature data from the convolution feature input register, and the temporary convolution data temporarily stored in the convolution result buffer into the multiply-accumulator. After completing the current calculation cycle, the convolution feature data and the convolution weight data are respectively transferred to the convolution feature input register and the convolution weight input register of the next convolution basic operation unit. After the convolution operation is completed, the temporary convolution data stored in the convolution result buffer is output as the convolution data result.

[0025] According to a second aspect of this disclosure, a pulsating array system is provided, comprising: the aforementioned pulsating array, a system bus, an array controller, a feature buffer, and an output buffer, for performing synchronous sorting operations on labeled feature data in a sorting mode; wherein;

[0026] The system bus is connected to the array controller, the feature buffer, and the output buffer respectively, and is used to send sorting control instructions to the array controller, and after sorting is completed, receive the sorting feature data and its corresponding synchronization tag data uploaded by the output buffer;

[0027] The array controller is connected to the feature buffer, the systolic array, and the output buffer, respectively. After sending a sorting control signal according to the sorting control instruction, it inputs feature data and its corresponding label data into the feature buffer. After assigning different configuration values ​​to the control registers of the first and second basic operation units in the systolic array, it sequentially inputs the feature data and its corresponding label data in the feature buffer into the systolic array for synchronous sorting. After sorting is completed, it outputs the sorted feature data and its corresponding synchronous label data to the output buffer. The feature data consists of multiple candidate detection box scores generated by the neural network model, and the label data consists of position index information corresponding to the multiple candidate detection box scores.

[0028] In one embodiment, the pulsating array system further includes:

[0029] A weight buffer is connected to the system bus, the array controller, and the systolic array, respectively, and is used to perform convolution operations in convolution mode;

[0030] Correspondingly, the system bus is also used to send convolution control instructions to the array controller, and after the convolution operation is completed, to receive the convolution data results uploaded by the output buffer;

[0031] The array controller is further configured to, after sending a convolution control signal according to the convolution control instruction, allocate convolution configuration values ​​to the control registers of each basic operation unit in the systolic array, and respectively input the convolution feature data in the convolution feature buffer and the convolution weight data in the convolution weight buffer as two corresponding data columns into the systolic array in a preset order for convolution calculation, and after the convolution calculation is completed, transmit the convolution data result to the output buffer, wherein the convolution weight data is convolution window data arranged according to a first preset format, and the convolution feature data is image data arranged according to a second preset format.

[0032] According to a third aspect of this disclosure, a computational method for a pulsating array system is provided, applied to a pulsating array system, comprising:

[0033] Based on the received work instructions, determine the working mode indicated by the work instructions;

[0034] When the operating mode is sorting mode, after the sorting control signal sent by the array controller is assigned to different configuration values ​​of the control registers of the first basic operation unit and the second basic operation unit in the pulsating array, the feature data of the feature buffer and its corresponding tag data are gradually input into the pulsating array for synchronous sorting operation. After the sorting is completed, the sorted feature data and its corresponding synchronous tag data are output through the output buffer and sent back to the system bus. The first basic operation unit and the second basic operation unit in the pulsating array include comparators. The feature data of the feature buffer are multiple candidate detection box scores generated by the neural network model. The tag data are the position index information corresponding to the multiple candidate detection box scores.

[0035] In one possible implementation, determining the working mode indicated by the received working instruction includes:

[0036] Based on the received operating instructions, the system bus sends a sorting control instruction to the array controller, and the sorting control signal of the array controller is determined according to the sorting control instruction.

[0037] In one possible implementation, the computation method of the pulsating array system further includes:

[0038] When the working mode is convolution mode, after the convolution control signal sent by the array controller is allocated to the convolution configuration value of the control register of each basic operation unit in the systolic array, the convolution feature data of the feature buffer and the convolution weight data of the weight buffer are respectively input into the systolic array as two corresponding data columns in a preset order for convolution calculation. After the convolution calculation is completed, the convolution data result is output through the output buffer and sent back to the system bus. The convolution weight data of the weight buffer is convolution window data arranged according to a first preset format, and the convolution feature data of the feature buffer is image data arranged according to a second preset format.

[0039] According to a fourth aspect of this disclosure, a computing device for a pulsating array system is provided, the device comprising:

[0040] The mode determination module is used to determine the working mode indicated by the received working instruction based on the working instruction.

[0041] The sorting result output module is used to, when the working mode is sorting mode, after the sorting control signal sent by the array controller is assigned to different configuration values ​​of the control registers of the first basic operation unit and the second basic operation unit in the pulsating array, input the feature data of the feature buffer and its corresponding label data into the pulsating array step by step for synchronous sorting operation, and output the sorted feature data and its corresponding synchronous label data through the output buffer after sorting is completed, and send them back to the system bus. The first basic operation unit and the second basic operation unit in the pulsating array include comparators, the feature data of the feature buffer are multiple candidate detection box scores generated by the neural network model, and the label data are position index information corresponding to the multiple candidate detection box scores.

[0042] In one possible implementation, the pattern determination module is specifically used for:

[0043] Based on the received operating instructions, the system bus sends a sorting control instruction to the array controller, and the sorting control signal of the array controller is determined according to the sorting control instruction.

[0044] In one embodiment, the computing device of the pulsating array system further includes:

[0045] The convolution result output module is used to, when the working mode is convolution mode, after the convolution configuration value of the control register of each basic operation unit in the systolic array is allocated by the convolution control signal sent by the array controller, input the convolution feature data of the feature buffer and the convolution weight data of the weight buffer as two corresponding data columns into the systolic array one by one in a preset order for convolution calculation, and output the convolution data result through the output buffer after the convolution operation is completed and send it back to the system bus. The convolution weight data of the weight buffer is convolution window data arranged according to a first preset format, and the convolution feature data of the feature buffer is image data arranged according to a second preset format.

[0046] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0047] The systolic array, systolic array system, and its operation method, apparatus, and storage medium disclosed herein determine the operating mode indicated by the received operating instruction. When the operating mode is a sorting mode, after the sorting control signal sent by the array controller is assigned to different configuration values ​​of the control registers of the first and second basic operation units in the systolic array, the feature data of the feature buffer and its corresponding tag data are gradually input into the systolic array for synchronous sorting operation in groups. After the sorting is completed, the sorted feature data and its corresponding synchronization tag data are output through the output buffer and sent back to the system bus. This enables the direct calculation and tracking of complex sorting operators on the systolic array, improves the utilization rate of the systolic array, and avoids the problem of wasted transmission bandwidth due to data handling.

[0048] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0049] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0050] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0051] Figure 1A A schematic diagram of a pulsating array structure provided in Embodiment 1 of this disclosure is shown;

[0052] Figure 1B This illustration shows a schematic diagram of the structure of a basic arithmetic unit in a pulsating array under sorting mode, according to Embodiment 1 of this disclosure.

[0053] Figure 1C This illustration shows a schematic diagram of the structure of a basic operational unit in a systolic array in convolution mode, according to an embodiment of the present disclosure.

[0054] Figure 2A A schematic diagram of the structure of a pulsating array system provided in Embodiment 2 of this disclosure is shown;

[0055] Figure 2B A schematic diagram of a feature data sorting process for four labeled data points provided in Embodiment 2 of this disclosure is shown;

[0056] Figure 3 A flowchart of a computation method for a pulsating array system provided in Embodiment 3 of this disclosure is shown;

[0057] Figure 4 A schematic diagram of the structure of a computing device for a pulsating array system provided in Embodiment 4 of this disclosure is shown. Detailed Implementation

[0058] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0059] Example 1

[0060] Figure 1A This is a schematic diagram of a pulsating array structure provided in Embodiment 1 of the present disclosure, including: multiple first basic operation units and multiple second basic operation units connected in a row-corresponding matrix arrangement, used to complete the synchronous sorting operation of labeled feature data in a sorting mode. Figure 1A As shown, this embodiment consists of M x N configurable basic computing units.

[0061] The basic operation unit refers to the smallest component unit in the pulsation array. The first basic operation unit can be an operation unit used to sort the input feature data, and the second basic operation unit can be an operation unit used to sort the labels corresponding to the input feature data.

[0062] Specifically, the pulsation array provided in this embodiment can include two types of basic operation units, including a first basic operation unit and a second basic operation unit. The first basic operation unit and the second basic operation unit can be connected in a row-corresponding matrix arrangement, such as... Figure 1A As shown, for example, the first row can be a first basic operation unit, and the second row can be a second basic operation unit.

[0063] Figure 1B This is a schematic diagram of the structure of a basic arithmetic unit in a pulsating array under sorting mode, as provided in Embodiment 1 of this disclosure. Figure 1B As shown, in sorting mode, the weight input register and multiply-accumulate unit do not participate in operation; the first control register is connected to the first feature input register, the first result buffer, and the first comparator, but their connections are omitted in the figure; the second control register is connected to the second feature input register, the synchronization weight input register, the second result buffer, and the second comparator, but their connections are omitted in the figure.

[0064] Each first basic arithmetic unit includes a first comparator, a first control register, and a first result buffer. The first comparator is used to compare the input feature data. The first control register is used to control the output of the comparison sign result as a synchronization control signal to the second basic arithmetic unit corresponding to the current first basic arithmetic unit, and to output the comparison data result to the first result buffer and the first feature input register of the next basic arithmetic unit respectively. After sorting is completed, the temporary feature data stored in the first result buffer is output as sorted feature data.

[0065] Each second basic operation unit includes a second comparator, a second control register, a second result buffer, and a synchronization weight input register. The second comparator is used to compare the input tag data according to the synchronization control signal received by the synchronization weight input register. The second control register is used to control the output of the comparison tag results to the second result buffer and the second feature input register of the next basic operation unit, respectively. After sorting, the temporary tag data stored in the second result buffer is used as the synchronization tag data of the sorting feature data for synchronous output.

[0066] In this embodiment of the disclosure, each first basic operation unit includes a first feature input register for storing feature data and a first result buffer for temporarily storing temporary feature data; each second basic operation unit includes a second feature input register for storing tag data and a second result buffer for temporarily storing temporary tag data.

[0067] Correspondingly, the first control register is also used to sequentially input the feature data of the first feature input register and the temporary feature data of the first result buffer into the first comparator; the second control register is also used to sequentially input the tag data of the second feature input register and the temporary tag data of the second result buffer into the second comparator.

[0068] The first comparator can be an electronic component that compares the sizes of two features at its input and outputs the comparison result. The control register can be a memory that stores commands for execution in different operating modes, such as executing convolution-related instructions in convolution mode and sorting-related instructions in sorting mode; the first control register can be used to execute sorting-related instructions in sorting mode. The first result buffer can be a memory used to temporarily store feature data that meets the comparison result conditions. The first feature input register can be a register used to store feature data. The sorted feature data is feature data with a certain sorting rule.

[0069] The second comparator can be an electronic component that directly determines the comparison relationship between two input tag data based on the synchronization control signal and outputs the comparison result. In other words, the second comparator doesn't actually perform the comparison operation; it only needs to use the comparison symbol transmitted by the synchronization control signal as the comparison result. This facilitates the synchronous movement of the two tag data corresponding to the comparison result along with the feature data. The second control register can be a memory that executes tag sorting operations in tag mode. The second result buffer can be a memory used to temporarily store tag data that meets the comparison result conditions. The second feature input register can be a register used to store tag data. The synchronization weight input register can act as a "bridge" between the first and second basic operation units, storing the comparison symbol of the first comparator in the first basic operation unit. The synchronized tag data for sorting feature data is the tag data corresponding to the sorted feature data in the first basic operation unit. In the field of image processing, feature data can be multiple candidate detection box scores generated by a neural network model, and tag data can be the position index information corresponding to the multiple candidate detection box scores.

[0070] In this embodiment of the disclosure, the first comparator is specifically used to: successively compare the size of the feature data input to the first feature input register and the size of the temporary feature data input to the first result buffer; according to a preset sorting rule, take the feature data that meets the first sorting condition as the new temporary feature data, and take the feature data that meets the second sorting condition as the feature data in the first feature input register of the next basic operation unit;

[0071] Correspondingly, the first control register is also specifically used to: control the output of new temporary feature data to the first result buffer, and output feature data that meets the second sorting condition to the first feature input register of the next basic operation unit.

[0072] In this embodiment of the disclosure, the second comparator is specifically used to: compare the tag data input to the second feature input register and the temporary tag data input to the second result buffer according to the synchronization control signal received by the synchronization weight input register; take the tag data that meets the first sorting condition as the new temporary tag data; and take the tag data that meets the second sorting condition as the tag data in the second feature input register of the next basic operation unit.

[0073] Correspondingly, the second control register is also used to: control the synchronous output of new temporary tag data to the second result buffer, and synchronously output the tag data that meets the second sorting condition to the second feature input register of the next basic operation unit.

[0074] The first sorting condition, the second sorting condition, and the preset sorting rule can be set according to actual needs. For example, the first sorting condition can be set to data with larger sorting results, the second sorting condition can be set to data with smaller sorting results, and the preset sorting rule can require data to be sorted from largest to smallest, or input feature data with larger sorting results into the first result buffer and label data with larger sorting results into the second result buffer.

[0075] Specifically, this embodiment supports labeled sorting operators, also known as shadow mode. That is, the first basic operation unit compares the size of the input feature data, while the second basic unit does not actually compare the size of the label data. Instead, it uses the comparison sign of the second basic operation unit as a reference, aiming to move the label data of the second basic operation unit along with the feature data of the first basic operation unit. After sorting, the temporary feature data stored in the first result buffer is output as the sorted feature data; the temporary label data stored in the second result buffer is output synchronously as the synchronization label data of the sorted feature data.

[0076] For example, taking the first and second basic operation units as examples, the first and second result buffers can be set by default to directly store the first input data. The feature data is input sequentially as follows: feature data A and its corresponding label data 'a', as the first input data for the pulsating array; next, feature data B and its corresponding label data 'b'; and then feature data C and its corresponding label data 'c'. The preset sorting rule is to input the feature data with the larger sorting result into the first result buffer, and the label data with the larger sorting result into the second result buffer; and to input the feature data with the smaller sorting result into the next basic operation unit, and the label data with the smaller sorting result into the next basic operation unit. The first sorting condition is the larger data in the comparison results, and the second sorting condition is the smaller number in the comparison results.

[0077] For example, in the first basic operation unit, the first control register first temporarily stores the feature data A from the first feature input register directly into the first result buffer by default, without outputting a synchronization control signal. When the second feature data B is input into the first comparator, A in the first result buffer is also input into the first comparator for comparison. The first comparator first compares the input feature data B with the feature data A in the first result buffer. If the comparison result is "A>B" (with ">"), the feature data A that meets the first sorting condition is used as new temporary feature data, and the feature data B that meets the second sorting condition is used as feature data in the first feature input register of the next basic operation unit. The first control register outputs feature data A to the first result buffer, outputs feature data B to the first feature input register of the next basic operation unit, and outputs the comparison symbol ">" as a synchronization control signal to the second basic operation unit corresponding to the current first basic operation unit. Then, in this embodiment, the third feature data C and the feature data A temporarily stored in the result buffer are input into the first comparator for comparison. The comparison process is as described above, and the same type of comparison is performed in the next basic operation unit based on the sequentially input feature data, which will not be repeated here.

[0078] For example, in the second basic operation unit, the second control register first temporarily stores the tag data 'a' from the first second feature input register directly into the second result buffer. When the second tag data 'b' is input into the second comparator, it is compared with the tag data 'a' in the second result buffer. The second comparator directly compares the input tag data according to the synchronization control signal ">" received by the synchronization weight input register. The comparison result is "a>b". The tag data 'a' that meets the first sorting condition is used as the new temporary tag data, and the tag data 'b' that meets the second sorting condition is used as the feature data in the second feature input register of the next basic operation unit. The second control register controls the output of tag data 'a' to the second result buffer and the output of tag data 'b' to the second feature input register of the next basic operation unit, realizing the synchronous output of feature data A and tag data 'a', as well as feature data B and tag data 'b'. Then, in this embodiment, the third tag data 'c' and the 'a' temporarily stored in the result buffer are input into the second comparator for comparison. The comparison process is as described above, and the same type of comparison is performed in the next basic operation unit based on the sequentially input tag data, which will not be repeated here.

[0079] This embodiment classifies the basic computational units in a conventional systolic array by setting comparators within the array. The first basic computational unit is configured to sort feature data, and the second basic computational unit is configured to sort the corresponding tag data. The first and second control registers are configured to execute corresponding commands during the sorting operation, allowing a shadow mode. This mode transmits control signals through the weight input / output lines in the systolic array, and the data flow of the second basic computational unit is controlled by the computation results of adjacent units. This enables complex synchronous sorting of tagged data, avoiding the need to transfer data to other devices for sorting, saving computation time and bandwidth. Furthermore, it improves the utilization rate of the systolic array itself in computation, especially in multi-layer network computing and complex operators, reducing the bandwidth pressure on the external system bus and the need for additional external computing power.

[0080] In this embodiment of the disclosure, the systolic array further includes: a plurality of first basic operation units and a plurality of second basic operation units are classified into the same convolutional basic operation unit, wherein the convolutional basic operation unit includes a convolutional weight input register, a convolutional feature input register, a convolutional result buffer, a convolutional control register, and a multiplier-accumulator; wherein,

[0081] The convolution weight input register is used to store the convolution weight data;

[0082] Convolutional feature input register, used to store convolutional feature data;

[0083] A convolution result buffer is used to temporarily store convolutional data.

[0084] The multiply-accumulator is used to take the convolutional temporary data temporarily stored in the convolutional result buffer as the cumulative addend, and successively calculate the multiplication operation of the convolutional feature data input to the convolutional feature input register and the convolutional weight data input to the convolutional weight input register, and use the calculation result as the new convolutional temporary data.

[0085] The convolution control register is used to control the input of convolution weight data from the convolution weight input register, convolution feature data from the convolution feature input register, and temporary convolution data temporarily stored in the convolution result buffer into the multiply-accumulator. After the current calculation cycle is completed, the convolution feature data and convolution weight data are transferred to the convolution feature input register and convolution weight input register of the next convolution basic operation unit, respectively. After the convolution operation is completed, the temporary convolution data stored in the convolution result buffer is output as the convolution data result.

[0086] Figure 1C This is a schematic diagram illustrating the structure of a basic operational unit in a systolic array under convolution mode, as provided in an embodiment of this disclosure. Figure 1C As shown, in convolution mode, each basic operation unit also includes a weight input register and a multiplier-accumulator. The comparator does not participate in the operation. The convolution control register is connected to the convolution feature input register, the convolution weight input register, the convolution result buffer, and the multiplier-accumulator. The connection relationship is omitted in the figure. The basic operation unit provided in this embodiment is used to complete the convolution operation in convolution mode.

[0087] The convolution weight input register refers to the register used to store convolution weight data. In the field of image processing, convolution weight data can be convolution window data, while in other fields, weight data can be any data that requires convolution.

[0088] The convolutional feature data can be at least one piece of data stored in the convolutional feature input register in convolution mode. In image processing, the convolutional feature data can be image data, while in other fields, it can be any data that requires convolution. The convolutional temporary data can be the cumulative addends needed for each current computation cycle. The convolutional data result can be data with new features composed of the convolutional temporary data from the convolutional result buffers in multiple basic computation units.

[0089] Specifically, the basic operation unit in this embodiment can also realize the basic function of convolution operation. It stores two series of feature data with convolution requirements in the convolution weight input register and the convolution feature input register, and inputs them into the multiplier-accumulator to perform convolution operation to obtain data with new features.

[0090] Specifically, this embodiment uses the convolution operation of the current convolutional basic operation unit as an example. The convolution control register issues a control signal corresponding to the convolution mode, controlling the input of convolution weight data from the convolution weight input register, convolution feature data from the convolution feature input register, and temporary convolution data temporarily stored in the convolution result buffer into the multiplier-adder. In each calculation cycle, the temporary convolution data temporarily stored in the convolution result buffer is used as the cumulative addend to successively calculate the multiplication operation of the convolution feature data and the convolution weight data, and the calculation result is stored as new temporary convolution data. Furthermore, after completing the current calculation cycle, this embodiment transmits the convolution feature data and convolution weight data to the convolution feature input register and convolution weight input register of the next convolutional basic operation unit, and performs the same convolution operation on the newly input convolution feature data and convolution weight data. Similarly, the convolution operation of the next convolutional basic operation unit is as described above.

[0091] This embodiment can also implement basic convolution operation functions by setting relevant instructions for the convolution control register in convolution mode.

[0092] Example 2

[0093] Figure 2A This is a schematic diagram of a pulsating array system according to Embodiment 2 of the present disclosure, including: a pulsating array, a system bus, an array controller, a feature buffer, and an output buffer, used to perform sorting operations in sorting mode; wherein,

[0094] The system bus is connected to the array controller, feature buffer, and output buffer respectively. It is used to send sorting control commands to the array controller and receive the sorting feature data and its corresponding synchronization tag data uploaded by the output buffer after sorting is completed.

[0095] The array controller is connected to the feature buffer, the systolic array, and the output buffer. After sending a sorting control signal according to the sorting control instruction, it controls the input of feature data and its corresponding label data into the feature buffer. After assigning different configuration values ​​to the control registers of the first and second basic operation units in the systolic array, it sequentially inputs the feature data and its corresponding label data in the feature buffer into the systolic array for synchronous sorting. After the sorting is completed, it outputs the sorted feature data and its corresponding synchronous label data to the output buffer. The feature data consists of multiple candidate detection box scores generated by the neural network model, and the label data consists of the position index information corresponding to the multiple candidate detection box scores.

[0096] The system bus is a key component connecting the computer system. It receives work instructions and sortable data from the central controller and sends control instructions for different tasks to the array controller. When the pulse array processes data requiring sorting, the system bus sends sorting control instructions to the array. The array controller then controls and decodes these instructions to store, forward, and manage the entire array. The feature buffer stores sortable feature data, while the output buffer receives the sorting results uploaded by the pulse array and sends them back to the system bus.

[0097] The sorting control instruction is a sorting instruction obtained by parsing the working instruction from the system bus and sent to the array controller. When the task received by the pulsating array in this embodiment is a sorting operation, the array controller allocates a first configuration value to the first control register of the first basic arithmetic unit in the pulsating array and allocates a second configuration value to the second control register of the second basic arithmetic unit. The sorting control signal is a sorting execution signal for interaction between various devices in the array controller, used to implement the sorting operation. The first configuration value corresponds to the first control signal, used to enable the first basic arithmetic unit to complete the sorting operation of feature data; the second configuration value corresponds to the second control signal, used to enable the second basic arithmetic unit to complete the synchronous following sorting operation of tag data.

[0098] Specifically, in this embodiment, the sorting operation is completed in sorting mode. The system bus sends a sorting control command to the array controller, and the array controller sends a sorting control signal. After assigning different configuration values ​​to the control registers of the first and second basic arithmetic units in the pulsating array, the feature data stored in the feature buffer and its corresponding tag data are sequentially input into the pulsating array for synchronous sorting. After sorting is completed, the sorting feature data with sorting characteristics and its corresponding synchronous tag data output by the pulsating array are output to the output buffer and transmitted to the system bus via the output buffer.

[0099] Figure 2B This is a schematic diagram illustrating a feature data sorting process for four labeled data points provided in Embodiment 2 of this disclosure. This involves complex labeled data, specifically scenarios exceeding the data input bit width of a single basic computational unit. Figure 2BAs shown, this embodiment describes the sorting process of four randomly arranged data (4, 1, 8, 2) while retaining their position indices. This operation uses two rows of basic operation units: the first row uses sorting mode to sort the feature data (4, 1, 8, 2) and outputs synchronization control signals for the tags; the position indices, acting as a one-to-one pair between the tag data and feature data, enter the second row of basic operation units in shadow mode, where they are sorted (swapped) in relation to the feature data. After the sorting operation is completed, the original feature data are arranged in descending order in the first result buffer within the first row of basic operation units. The position of each feature data in the original data, i.e., the corresponding tag data, is arranged in the second result buffer within the second row of basic operation units. Finally, the pulsating array inputs the sorted feature data and the synchronized tag data.

[0100] Specifically, in this embodiment, the tagged feature data is progressively input into the systolic array from the feature buffer in groups. The feature data in the first row is input into the first basic operation unit of the systolic array one by one, and the tagged data in the second row is input into the second basic operation unit of the systolic array one by one. Each row of data enters the systolic array sequentially from the feature input device of the first column basic operation unit.

[0101] The first basic unit operation mode in sorting mode is as follows:

[0102] i) The weight input register and multiply-accumulator are disabled (bypass);

[0103] ii) When the first feature data enters the calculation unit, it is directly stored in the first result buffer without any synchronization control signal output; iii) Non-first feature data enters the first comparator from the first feature input, is compared with the data in the first result buffer, and the sorted results are input into the first result buffer and feature output respectively according to the preset sorting rules (e.g., output larger value or output smaller value), and the comparison symbol is passed to the next calculation unit from the weight output line as a synchronization control signal;

[0104] iv) After all data has been sorted, output the features in the first result buffer.

[0105] The operation mode of the second basic operation unit in shadow mode is as follows:

[0106] i) The multiply-accumulator is disabled (bypass);

[0107] ii) When the first label data enters the basic operation unit, it is directly stored in the second result buffer without any weight data input or output;

[0108] iii) When non-first tag data enters the second comparator from the second feature input register, according to the synchronization control signal entering the second comparator from the weight input register, the second control comparator inputs the tag data and the temporary tag data in the second result buffer into the second result buffer and the feature output respectively according to the comparison result of the first basic operation unit to complete the tag synchronous movement; if the lower basic operation unit also adopts the shadow mode, the control signal continues to be broadcast down from the weight output line to the next basic operation unit;

[0109] iv) After all feature data and label data are sorted, output the labels in the second result buffer.

[0110] like Figure 2A As shown in the embodiments of this disclosure, the systolic array system further includes: a weight buffer, which is connected to the system bus, the array controller and the systolic array respectively, and is used to perform convolution operations in convolution mode;

[0111] Correspondingly, the system bus is also used to send convolution control commands to the array controller and to receive the convolution data results uploaded by the output buffer after the convolution operation is completed;

[0112] The array controller is also used to allocate convolution configuration values ​​to the control registers of each basic operation unit in the systolic array after sending convolution control signals according to the convolution control instructions, and to input the convolution feature data in the convolution feature buffer and the convolution weight data in the convolution weight buffer as two corresponding data columns into the systolic array one by one in a preset order for convolution calculation. After the convolution operation is completed, the convolution data result is transmitted to the output buffer. The convolution weight data is convolution window data arranged in a first preset format, and the convolution feature data is image data arranged in a second preset format.

[0113] The system includes a weight buffer to store weight data requiring convolution, a system bus to receive work instructions and data for convolution operations from the central controller, and a convolution control command to the array controller. The preset order, first preset format, and second preset format are all arrangement formats conforming to convolution operations. The convolution configuration value corresponds to the convolution control signal, enabling the basic convolution operation unit to perform convolution operations on the convolution feature data and convolution weight data. For example, in image processing, the convolution weight data can be convolution window data arranged according to the first preset format, and the convolution feature data can be image data arranged according to the second preset format.

[0114] Specifically, in the convolution operation mode, in this embodiment, the array controller is also used to send convolution control signals according to the convolution control instructions sent by the system bus, allocate the convolution configuration values ​​to the convolution control registers in each convolution basic operation unit in the systolic array, and respectively input the convolution feature data in the convolution feature buffer and the convolution weight data in the convolution weight buffer as two corresponding data columns into the systolic array one by one in a preset order for convolution calculation, and after the convolution operation is completed, receive the convolution data results transmitted by the systolic array to the output buffer, and transmit them back to the system bus through the output buffer.

[0115] The pulsating array system provided in this embodiment can support the synchronous sorting of labeled data in the improved pulsating array, realize the function of direct calculation and tracking of complex sorting operators on the pulsating array, improve the utilization rate of the pulsating array, and avoid the problem of wasted transmission bandwidth due to data transfer.

[0116] Example 3

[0117] Figure 3 This is a flowchart illustrating a computational method for a pulsating array system according to Embodiment 3 of this disclosure. This method can be executed by the computational device of the pulsating array system provided in this embodiment, and the device can be implemented in software and / or hardware. Specifically, the method includes:

[0118] S310. Determine the working mode indicated by the received work instruction.

[0119] In this embodiment of the disclosure, determining the working mode indicated by the received working instruction includes: sending a sorting control instruction from the system bus to the array controller according to the received working instruction, and determining the sorting control signal of the array controller according to the sorting control instruction.

[0120] The working instructions can be instruction information containing task content. This embodiment has two working modes: sorting operation mode and convolution operation mode. When the working mode is sorting operation mode, the array controller sends sorting control signals to other devices in the systolic array system according to the received sorting control instructions to complete the corresponding sorting operation. When the working mode is convolution operation mode, the array controller sends convolution control signals to other devices in the systolic array system according to the received array control instructions to complete the corresponding convolution operation.

[0121] S320. When the working mode is sorting mode, after the sorting control signal sent by the array controller is assigned to the control registers of the first basic arithmetic unit and the second basic arithmetic unit in the pulsating array with different configuration values, the feature data of the feature buffer and its corresponding tag data are gradually input into the pulsating array for synchronous sorting operation. After the sorting is completed, the sorting feature data and its corresponding synchronous tag data are output through the output buffer and sent back to the system bus.

[0122] The first and second basic operation units in the pulsating array include comparators, the feature buffers contain feature data consisting of multiple candidate detection box scores generated by the neural network model, and the label data consists of position index information corresponding to the multiple candidate detection box scores.

[0123] In this embodiment of the disclosure, the systolic array system further includes: when the working mode is convolution mode, after the convolution control signal sent by the array controller is allocated to the convolution configuration value of the control register of each basic operation unit in the systolic array, the convolution feature data of the feature buffer and the convolution weight data of the weight buffer are respectively input into the systolic array as two corresponding data columns in a preset order for convolution calculation, and after the convolution operation is completed, the convolution data result is output through the output buffer and sent back to the system bus, wherein the convolution weight data of the weight buffer is convolution window data arranged according to a first preset format, and the convolution feature data of the feature buffer is image data arranged according to a second preset format.

[0124] Example 4

[0125] Figure 4 This is a schematic diagram of the structure of a computing device for a pulsating array system provided in an embodiment of this disclosure. The device specifically includes:

[0126] The mode determination module 410 is used to determine the working mode indicated by the received working instruction based on the working instruction.

[0127] The sorting result output module 420 is used to, when the working mode is sorting mode, after the sorting control signal sent by the array controller is assigned to different configuration values ​​of the control registers of the first basic operation unit and the second basic operation unit in the pulsating array, input the feature data of the feature buffer and its corresponding label data into the pulsating array for synchronous sorting operation in groups, and output the sorted feature data and its corresponding synchronous label data through the output buffer after sorting is completed, and send them back to the system bus. The first basic operation unit and the second basic operation unit in the pulsating array include comparators, the feature data of the feature buffer are the scores of multiple candidate detection boxes generated by the neural network model, and the label data are the position index information corresponding to the scores of multiple candidate detection boxes.

[0128] In one possible implementation, the pattern determination module is specifically used for:

[0129] Based on the received operating instructions, the system bus sends a sorting control instruction to the array controller, and the sorting control signal of the array controller is determined according to the sorting control instruction.

[0130] In one embodiment, the computing device of the pulsating array system further includes:

[0131] The convolution result output module is used to, when the working mode is convolution mode, distribute the convolution configuration values ​​of the control registers of each basic operation unit in the systolic array to the convolution control signal sent by the array controller, and then input the convolution feature data of the feature buffer and the convolution weight data of the weight buffer as two corresponding data columns into the systolic array one by one for convolution calculation according to a preset order. After the convolution operation is completed, the convolution data result is output through the output buffer and sent back to the system bus. The convolution weight data of the weight buffer is the convolution window data arranged according to the first preset format, and the convolution feature data of the feature buffer is the image data arranged according to the second preset format.

[0132] According to embodiments of this disclosure, this disclosure also provides a readable storage medium.

[0133] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0135] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A pulsating array, characterized in that, include: Multiple first-level basic computation units and multiple second-level basic computation units are connected in a row-corresponding matrix arrangement to perform synchronous sorting operations on labeled feature data in sorting mode; among them... Each first basic arithmetic unit includes a first comparator, a first control register, and a first result buffer. The first comparator is used to compare the input feature data. The first control register is used to control the output of the comparison sign result as a synchronization control signal to the second basic arithmetic unit corresponding to the current first basic arithmetic unit, and to output the comparison data result to the first result buffer and the first feature input register of the next basic arithmetic unit respectively. After sorting, the temporary feature data stored in the first result buffer is output as sorted feature data. Each second basic operation unit includes a second comparator, a second control register, a second result buffer, and a synchronization weight input register. The synchronization weight input register receives and stores the comparison sign of the synchronization control signal from the first basic operation unit. The second comparator does not perform a comparison operation but determines the comparison label result of the input label data based on the comparison sign of the synchronization control signal received by the synchronization weight input register, and synchronously moves the input label data corresponding to the comparison label result along with the corresponding feature data. The second control register controls the output of the comparison label result to the second result buffer and the second feature input register of the next basic operation unit, respectively. After sorting, the temporary label data stored in the second result buffer is synchronously output as the synchronization label data of the sorted feature data. The synchronization label data of the sorted feature data is the label data corresponding to the sorted feature data in the first basic operation unit.

2. The pulsating array according to claim 1, characterized in that, Each of the first basic arithmetic units includes a first feature input register for storing feature data, and a first result buffer for temporarily storing temporary feature data. Each of the second basic operation units includes a second feature input register for storing tag data, and a second result buffer for temporarily storing temporary tag data; Correspondingly, the first control register is also used to sequentially input the feature data of the first feature input register and the temporary feature data of the first result buffer into the first comparator; the second control register is also used to sequentially input the tag data of the second feature input register and the temporary tag data of the second result buffer into the second comparator.

3. The pulsating array according to claim 2, characterized in that, The first comparator is specifically used for: The size of the feature data input into the first feature input register and the temporary feature data input into the first result buffer are compared sequentially. According to the preset sorting rules, the feature data that meets the first sorting condition is used as the new temporary feature data, and the feature data that meets the second sorting condition is used as the feature data in the first feature input register of the next basic operation unit. Accordingly, the first control register is further used to: control the output of the new temporary feature data to the first result buffer, and output the feature data that meets the second sorting condition to the first feature input register of the next basic operation unit.

4. The pulsating array according to claim 3, characterized in that, The second comparator is specifically used for: Based on the synchronization control signal received by the synchronization weight input register, the tag data input to the second feature input register and the temporary tag data input to the second result buffer are compared. The tag data that meets the first sorting condition is used as the new temporary tag data, and the tag data that meets the second sorting condition is used as the tag data in the second feature input register of the next basic operation unit. Accordingly, the second control register is also specifically used to: control the synchronous output of the new temporary tag data to the second result buffer, and synchronously output the tag data that meets the second sorting condition to the second feature input register of the next basic operation unit.

5. The pulsating array according to claim 1, characterized in that, Also includes: The plurality of first basic operation units and the plurality of second basic operation units are classified into the same convolution basic operation unit, which includes a convolution weight input register, a convolution feature input register, a convolution result buffer, a convolution control register, and a multiplier-accumulator; wherein... The convolution weight input register is used to store convolution weight data; The convolutional feature input register is used to store convolutional feature data; The convolution result buffer is used to temporarily store convolutional temporary data; The multiply-accumulator is used to take the convolutional temporary data temporarily stored in the convolutional result buffer as the cumulative addend, and successively calculate the multiplication operation of the convolutional feature data input to the convolutional feature input register and the convolutional weight data input to the convolutional weight input register, and use the calculation result as the new convolutional temporary data. The convolution control register is used to control the input of convolution weight data from the convolution weight input register, convolution feature data from the convolution feature input register, and temporary convolution data temporarily stored in the convolution result buffer into the multiply-accumulator. After completing the current calculation cycle, the convolution feature data and the convolution weight data are respectively transferred to the convolution feature input register and convolution weight input register of the next convolution basic operation unit. After the convolution operation is completed, the temporary convolution data stored in the convolution result buffer is output as the convolution data result.

6. A pulsating array system, characterized in that, include: The pulsating array according to any one of claims 1-5, the system bus, the array controller, the feature buffer, and the output buffer are used to perform synchronous sorting operations on tagged feature data in sorting mode; wherein, The system bus is connected to the array controller, the feature buffer, and the output buffer respectively, and is used to send sorting control instructions to the array controller, and after sorting is completed, receive the sorting feature data and its corresponding synchronization tag data uploaded by the output buffer; The array controller is connected to the feature buffer, the systolic array, and the output buffer, respectively. After sending a sorting control signal according to the sorting control instruction, it controls the input of feature data and its corresponding label data into the feature buffer. After assigning different configuration values ​​to the control registers of the first and second basic arithmetic units in the systolic array, it progressively inputs the feature data and its corresponding label data from the feature buffer into the systolic array for synchronous sorting. After sorting is completed, it outputs the sorted feature data and its corresponding synchronous label data to the output buffer. The feature data consists of multiple candidate detection box scores generated by the neural network model, and the label data consists of position index information corresponding to the multiple candidate detection box scores.

7. The pulsating array system according to claim 6, characterized in that, Also includes: A weight buffer is connected to the system bus, the array controller, and the systolic array, respectively, and is used to perform convolution operations in convolution mode; Correspondingly, the system bus is also used to send convolution control instructions to the array controller, and after the convolution operation is completed, to receive the convolution data results uploaded by the output buffer; The array controller is further configured to, after sending a convolution control signal according to the convolution control instruction, allocate convolution configuration values ​​to the control registers of each convolution basic operation unit in the systolic array, and respectively input the convolution feature data in the convolution feature buffer and the convolution weight data in the convolution weight buffer as two corresponding data columns into the systolic array in a preset order for convolution calculation, and after the convolution calculation is completed, transmit the convolution data result to the output buffer, wherein the convolution weight data is convolution window data arranged according to a first preset format, and the convolution feature data is image data arranged according to a second preset format.

8. A computational method for a pulsating array system, characterized in that, Applied to the pulsating array system of claim 6 or 7, comprising: Based on the received work instructions, determine the working mode indicated by the work instructions; When the operating mode is sorting mode, after the sorting control signal sent by the array controller is assigned to different configuration values ​​of the control registers of the first basic operation unit and the second basic operation unit in the pulsating array, the feature data of the feature buffer and its corresponding tag data are gradually input into the pulsating array for synchronous sorting operation. After the sorting is completed, the sorted feature data and its corresponding synchronous tag data are output through the output buffer and sent back to the system bus. The first basic operation unit and the second basic operation unit in the pulsating array include comparators. The feature data of the feature buffer are multiple candidate detection box scores generated by the neural network model. The tag data are the position index information corresponding to the multiple candidate detection box scores.

9. The method according to claim 8, characterized in that, The step of determining the working mode indicated by the received working instruction includes: Based on the received working instructions, the system bus sends a sorting control instruction to the array controller, and the sorting control signal of the array controller is determined according to the sorting control instruction.

10. The method according to claim 9, characterized in that, Also includes: When the working mode is convolution mode, after the convolution control signal sent by the array controller is allocated to the convolution configuration value of the control register of each basic operation unit in the systolic array, the convolution feature data of the feature buffer and the convolution weight data of the weight buffer are respectively input into the systolic array as two corresponding data columns in a preset order for convolution calculation. After the convolution calculation is completed, the convolution data result is output through the output buffer and sent back to the system bus. The convolution weight data of the weight buffer is convolution window data arranged according to a first preset format, and the convolution feature data of the feature buffer is image data arranged according to a second preset format.

11. A computing device for a pulsating array system, characterized in that, Applied to the pulsating array system of claim 6 or 7, the device comprises: The mode determination module is used to determine the working mode indicated by the received working instruction based on the working instruction. The sorting result output module is used to, when the working mode is sorting mode, after the sorting control signal sent by the array controller is assigned to different configuration values ​​of the control registers of the first basic operation unit and the second basic operation unit in the pulsating array, input the feature data of the feature buffer and its corresponding label data into the pulsating array step by step for synchronous sorting operation, and output the sorted feature data and its corresponding synchronous label data through the output buffer after sorting is completed, and send them back to the system bus. The first basic operation unit and the second basic operation unit in the pulsating array include comparators, the feature data of the feature buffer are multiple candidate detection box scores generated by the neural network model, and the label data are position index information corresponding to the multiple candidate detection box scores.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 8-10.

Images