Data processing circuit, data processing method and related product

Abstract

The present disclosure discloses a data processing circuit, a data processing method and related products. The data processing circuit can be implemented as a computing device included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices, respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure provides a hardware implementation of data fusion related operations, which can simplify processing and improve the processing efficiency of the machine.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of data processing. More specifically, the present disclosure relates to a data processing circuit, a data processing method, a chip and a board. BACKGROUND

[0002] In recent years, with the rapid development of deep learning, the algorithm performance of a series of fields such as computer vision and natural language processing has made a leap. However, deep learning algorithm is a kind of computing-intensive and storage-intensive tool. With the increasing complexity of information processing tasks, the real-time performance and accuracy of the algorithm are constantly increasing. The neural network is often designed to be deeper and deeper, so that the computing amount and storage space requirement are larger and larger, which makes it difficult for existing artificial intelligence technology based on deep learning to be directly applied to mobile phones, satellites or embedded devices with limited hardware resources.

[0003] Therefore, the compression, acceleration and optimization of deep neural network model become particularly important. A large number of studies try to reduce the computing and storage requirements of neural network without affecting the model accuracy, which is of great significance for the engineering application of deep learning technology in embedded and mobile terminals. Sparsification is one of the methods for model lightening.

[0004] Network parameter sparsification is to reduce the redundant components in a large network by appropriate methods to reduce the demand of the network on computing amount and storage space. The existing hardware and / or instruction set cannot effectively support sparsification processing and / or processing related to sparsification. SUMMARY

[0005] In order to at least partially solve one or more technical problems mentioned in the background, the scheme of the present disclosure provides a data processing circuit, a data processing method, a chip and a board.

[0006] In a first aspect, the present disclosure discloses a data processing circuit, comprising a control circuit, a storage circuit and an operation circuit, wherein: the control circuit is configured to control the storage circuit and the operation circuit to perform merge sort accumulation processing on multiple paths of data to be fused; the storage circuit is configured to store information, the information at least including information before processing and / or information after processing; and the operation circuit is configured to, under the control of the control circuit, merge data elements in the multiple paths of data to be fused into one path of ordered fused data according to their corresponding indexes, and data elements with the same index in the fused data are merged into one fused data element.

[0007] In a second aspect, the present disclosure provides a chip comprising the data processing circuit of any one of the preceding first aspect.

[0008] In a third aspect, the present disclosure provides a board card comprising the chip of any one of the preceding second aspect.

[0009] In a fourth aspect, the present disclosure provides a method for processing data using a data processing circuit, the data processing circuit comprising a control circuit, a storage circuit and an operation circuit, the method comprising: reading, by the control circuit, a plurality of data to be fused from the storage circuit; merging, by the operation circuit, data elements in the plurality of data to be fused according to their corresponding indexes into one ordered fused data, wherein data elements having the same index in the fused data are merged into one fused data element; and outputting the fused data to the storage circuit.

[0010] By the data processing circuit, the method for processing data using a data processing circuit, the chip and the board card as provided above, the embodiments of the present disclosure provide a hardware circuit supporting data fusion operation, for performing merge sort accumulation processing on a plurality of data. In some embodiments, the data processing circuit can merge a plurality of ordered data into one ordered fused data according to index order, and data having the same index can be accumulated. By providing a hardware implementation of special data fusion related operation, the processing can be simplified and accelerated, thereby improving the processing efficiency of the machine. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is a structural diagram illustrating a board card of an embodiment of the present disclosure;

[0013] Figure 2 is a structural diagram illustrating a combined processing device of an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram illustrating the internal structure of a processor core of a single-core or multi-core computing device of an embodiment of the present disclosure;

[0015] Figure 4 is an exemplary schematic diagram illustrating data fusion processing according to an embodiment of the present disclosure;

[0016] Figure 5 is a structural schematic diagram of a data processing device of an embodiment of the present disclosure;

[0017] Figure 6 is an exemplary circuit diagram for data fusion processing of one embodiment of the present disclosure; and

[0018] Figure 7 is an exemplary flow chart illustrating a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of, rather than all of, the embodiments of the present disclosure. Based on the embodiments in the present disclosure, any other embodiments obtained by a person of ordinary skill in the art without creative effort fall within the protection scope of the present disclosure.

[0020] It should be understood that the terms “first”, “second”, “third”, and “fourth” and the like that can occur in the claims, specification, and drawings of the present disclosure are used to distinguish different objects, and are not used to describe a particular order. The terms “comprise” and “include” used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0021] It should also be understood that the terms used in the specification of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. As used in the specification and claims of the present disclosure, the singular forms “a”, “an” and “the” are intended to include the plural forms, unless the context clearly indicates otherwise. It should be further understood that the term “and / or” used in the specification and claims of the present disclosure means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0022] As used in the specification and claims of the present disclosure, the term “if’ can be interpreted as “when” or “upon” or “in response to a determination” or “in response to detecting” depending on the context.

[0023] The specific embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings.

[0024] Figure 1 A structural schematic diagram of a board card 10 according to an embodiment of the present disclosure is shown. As shown in FIG. 1, the board card 10 includes a board card body 11, a plurality of connectors 12, and a plurality of cables 13. Figure 1As shown, the board card 10 includes a chip 101, which is a system on chip (SoC) integrated with one or more combined processing devices, which is an artificial intelligence operation unit to support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in cloud intelligent fields. A significant feature of cloud intelligent applications is the large amount of input data, which has high requirements for the storage capacity and computing capacity of the platform. The board card 10 of this embodiment is suitable for cloud intelligent applications and has a large off-chip storage, on-chip storage and strong computing capacity.

[0025] The chip 101 is connected with an external device 103 through an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card or a wifi interface, etc. The data to be processed can be transmitted from the external device 103 to the chip 101 through the external interface device 102. The computing result of the chip 101 can be transmitted back to the external device 103 through the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface, etc.

[0026] The board card 10 further includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected and transmits data with the control device 106 and the chip 101 through a bus. The control device 106 in the board card 10 is configured to regulate the state of the chip 101. For this purpose, in one application scenario, the control device 106 can include a micro controller unit (MCU).

[0027] Figure 2 is a structural diagram of the combined processing device in the chip 101 of this embodiment. As Figure 2 As shown in the figure, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203 and a storage device 204.

[0028] The computing device 201 is configured to perform user-specified operations, mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations, which can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0029] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write into the storage device on the computing device 201. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write into the control buffer on the computing device 201. Alternatively or additionally, the interface device 202 can also read data from the storage device of the computing device 201 and transmit to the processing device 203.

[0030] The processing device 203 is a general-purpose processing device, which performs basic control including but not limited to data transfer, start and / or stop of the computing device 201, etc. Depending on the implementation, the processing device 203 can be one or more types of processors, including but not limited to a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors, including but not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., and the number thereof can be determined according to actual needs. As mentioned above, only in terms of the computing device 201 of the present disclosure, it can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0031] The storage device 204 is used to store data to be processed, which can be a DRAM, a DDR memory, usually with a size of 16G or more, for saving data of the computing device 201 and / or the processing device 203.

[0032] Figure 3 The internal structure of the processor core of the computing device 201 is shown when the computing device 201 is a single-core or multi-core device. The computing device 301 is used to process input data of computer vision, voice, natural language, data mining, etc. The computing device 301 includes three modules: a control module 31, a computation module 32, and a storage module 33.

[0033] The control module 31 is configured to coordinate and control the operations of the arithmetic module 32 and the storage module 33 to complete the task of deep learning, and includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 is configured to fetch instructions from the processing device 203, and the instruction decode unit 312 is configured to decode the fetched instructions and send the decoded results as control information to the arithmetic module 32 and the storage module 33.

[0034] The arithmetic module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is configured to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; and the matrix operation unit 322 is responsible for the core calculation of the deep learning algorithm, i.e., matrix multiplication and convolution.

[0035] The storage module 33 is configured to store or transfer related data, and includes a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access (DMA) 333. The NRAM 331 is configured to store input neurons, output neurons, and intermediate results after calculation; the WRAM 332 is configured to store the convolution kernel of the deep learning network, i.e., the weight; and the DMA 333 is connected to the DRAM 204 through the bus 34 and is responsible for data transfer between the computing device 301 and the DRAM 204.

[0036] Based on the foregoing hardware environment, embodiments of the present disclosure provide a data processing circuit supporting data fusion operations. As mentioned in the background, network parameter sparsification can effectively reduce the demand of the network for computing capacity and storage space. However, network parameter sparsification will also have a series of impacts on subsequent processing. For example, in sparse matrix multiplication operations, it may be necessary to sort and accumulate the vectors obtained in the middle of the operation to obtain the expected operation result. In view of this, embodiments of the present disclosure provide a hardware solution for data fusion processing to simplify and accelerate such processing.

[0037] Figure 4An exemplary principle of data fusion processing according to embodiments of the present disclosure is shown. Four pieces of data to be fused are exemplarily shown in the figure, each piece of data including six data elements. The data elements can be scalars, vectors or higher dimensional tensors, which are exemplarily shown as scalar values in the figure. Each data element has an associated index indicating the position information of the data element in the corresponding piece of data. For example, the original piece of data can include 1000 data elements, but only the data elements at some positions are valid. In this case, the valid elements can be extracted to form the above-mentioned data to be fused, and the indices corresponding to the valid elements are extracted to indicate their positions in the original data, which form the above-mentioned indices to be fused.

[0038] The corresponding four pieces of indices to be fused are schematically shown in the figure, each piece of index corresponding to a piece of data to be fused. The first piece of index is used to identify the position information of each data element in the first piece of data, the second piece of index is used to identify the position information of each data element in the second piece of data, and so on. Further, the index elements in each piece of index are stored in order and one-to-one correspond to the data elements in the corresponding piece of data. In the example shown in the figure, the index elements in each piece of index are arranged in a first order (for example, from small to large), and the data elements in each piece of data are also arranged in order according to the corresponding index. For example, the first index element in the first piece of index indicates that the index of the first data element in the first piece of data is 0, i.e., the first element; the second index element in the first piece of index indicates that the index of the second data element in the first piece of data is 2, i.e., the third element; and so on.

[0039] After data fusion, the four pieces of data are merged into one piece of ordered fused data according to their corresponding indices, and the data elements with the same index are merged into one fused data element. As shown in the figure, the fused index includes 16 index elements arranged in a second order (for example, from small to large), in which the repeated index elements in the index to be fused are removed, as shown by the dark squares in the figure. Correspondingly, the fused data also includes 16 data elements arranged in order according to the corresponding index, and the data elements with the same index are merged into one fused data element, as shown by the dark squares in the figure. In this example, the data elements with the same index are merged into one fused data element by accumulation. For example, for the fused index element "0", the corresponding fused data element is "25", which is the sum of the first data elements of each piece of data (2+1+10+12=25). For example, for the fused index element "9", the corresponding fused data element is "7", which is the sum of the fifth data element of the first piece of data and the third data element of the fourth piece of data (1+6=7).

[0040] Those skilled in the art can understand that the first order and the second order mentioned above can be the same or different, and both can be selected from any one of the following: an order from small to large, or an order from large to small. Those skilled in the art can also understand that although each lane of data is shown to have an equal number of data elements, the number of data elements in each lane of data can be the same or different, and the present disclosure is not limited in this regard.

[0041] In some embodiments of the present disclosure, the data elements are scalars, and thus, a lane of data including several data elements can be referred to as a data vector, and the vector length is equal to the number of contained data elements.

[0042] Figure 5 A structural block diagram of a data processing circuit 500 according to an embodiment of the present disclosure is shown. The data processing circuit 500 may, for example, be implemented in the computing device 201 of Figure 2 As shown, the data processing circuit 500 can include a control circuit 510, a storage circuit 520, and an operation circuit 530.

[0043] The function of the control circuit 510 can be similar to that of the control module 31 of Figure 3 which may, for example, include an instruction fetch unit to fetch instructions from the processing device 203, for example, and an instruction decode unit to decode the fetched instructions and send the decoding results as control information to the operation circuit 530 and the storage circuit 520. Figure 2

[0044] In one embodiment, the control circuit 510 can be configured to control the storage circuit 520 and the operation circuit 530 to perform merge sort accumulation processing on the multiple lanes of data to be fused.

[0045] The storage circuit 520 can be configured to store various information, including at least information before and / or after the merge sort accumulation processing. The storage circuit may, for example, be the WRAM 332 of Figure 3

[0046] The operation circuit 530 can be configured to merge, under the control of the control circuit, data elements in the multiple lanes of data to be fused according to their corresponding indices into one lane of ordered fused data, wherein data elements having the same index in the fused data are merged into one fused data element.

[0047] In one embodiment, the operation circuit 530 can further include an operation processing circuit (not shown) which can be configured to perform pre-processing on data before operation or post-processing on data after operation of the operation circuit according to operation instructions. In some application scenarios, the aforementioned pre-processing and post-processing may, for example, include data splitting and / or data splicing operations.​​

[0048] The operation circuit can be implemented in various ways. Figure 6 An exemplary circuit diagram for data fusion processing according to one embodiment of the present disclosure is shown.

[0049] As shown, in one embodiment, the storage circuit can be exemplarily divided into two parts: a first storage circuit 622 and a second storage circuit 624.

[0050] The first storage circuit 622 can be configured to store K-way data to be fused and K-way indexes corresponding to the K-way data, K > 1. The index elements in the K-way indexes indicate the index information of the corresponding data elements in the K-way data, i.e., the index elements and the data elements have a one-to-one correspondence. In addition, the index elements of each index in the K-way indexes are sequentially arranged in a first order, and the data elements of each data in the K-way data are sequentially arranged in the order of the corresponding index. Exemplarily shown in the figure are a 4-way index and corresponding 4-way data, where each data element is identified by the symbol D, and the specific values are not shown. In some embodiments, each index can be stored continuously, e.g., as a vector, so that the index / vector can be accessed according to the starting address of each index / vector. Correspondingly, each data can also be stored continuously, e.g., as a vector, so that the data / vector can be accessed according to the starting address of each data / vector. Figure 4

[0051] The second storage circuit 624 can be configured to store fused data output by the operation circuit and a fusion index corresponding to the fused data, where the fusion index elements in the fusion index are sequentially arranged in a second order, and the fusion data elements in the fused data are sequentially arranged in the order of the fusion index. As can be seen from the example in the figure, the 4-way data to be fused becomes one-way fused data, and the corresponding 4-way index also becomes one-way fusion index, where the fusion index elements are arranged in ascending order, and the index elements of the same size are removed. Correspondingly, the fusion data elements are arranged in the order of the fusion index, and the data elements with the same index are accumulated as the fusion data elements.

[0052] In some embodiments, the operation circuit can include a sorting circuit 632 and an accumulation circuit 636 to cooperatively implement the sorting and accumulation functions. Specifically, the sorting circuit 632 is configured to sort the K-way indexes according to the size of the index elements and sequentially output to the accumulation circuit 636. The accumulation circuit 636 then accumulates the data elements corresponding to the same index elements when receiving the same index elements from the sorting circuit, and removes the repeated index elements.

[0053] ​In some embodiments, the sorting circuit 632 can include a comparison circuit 631 and a buffer circuit 633. The comparison circuit 631 implements a comparison function that compares the size of the index elements in the plurality of to-be-merged indexes and submits the comparison result to the control circuit 610 for sorting. The control circuit 610 determines the insertion position of the index elements in the buffer circuit 633 according to the comparison result. The buffer circuit 633 is used to buffer the compared index elements and the information of the data elements corresponding thereto, and buffer them in order of the size of the index elements.

[0054] Specifically, the comparison circuit 631 can be configured to compare the index elements in the to-be-merged indexes with the index elements in the buffer circuit 633 that have not been output, and output the comparison result to the control circuit 610. The buffer circuit 633 can be configured to store the compared index elements and the information of the data elements corresponding thereto in order and output them in order according to the control of the control circuit 610.

[0055] In some embodiments, the buffer circuit 633 can be configured to buffer K index elements in order of size. Those skilled in the art can understand that the buffer circuit can also be configured to buffer more index elements, and the embodiments of the present disclosure are not limited in this respect. Depending on the sorting manner in the buffer circuit 633 and the desired output sorting manner, such as from small to large, or from large to small, the first or last index element in the current sequence can be output in the specified order each time. For example, in the example in the figure, the buffer circuit 633 buffers the index elements from left to right in order of from large to small, and outputs the rightmost index element each time, that is, the smallest index element in the current sequence, for example, “7”.

[0056] In these embodiments, the comparison circuit 631 can include K-1 comparators configured to compare the to-be-merged index elements with the K-1 index elements in the buffer circuit 633 that have not been output, that is, the K-1 index elements remaining after the first or last index element in the current sequence is output, generate a comparison result and output it to the control circuit 610.

[0057] For example, for 4-way data to be merged, 3-way comparators are shown in the figure, which compare the specified index element (at this time, 9) received from the first storage circuit 622 with the 3 index elements in the buffer circuit 633 that have not been output, that is, the three index elements 100, 10 and 9 on the left in the figure.

[0058] In some embodiments, the comparison result of the comparator can be represented using a bitmap. For example, if the index element to be merged (e.g., 9) is greater than or equal to the index element in the buffer circuit, the comparator can output "1", otherwise, output "0"; the opposite can also be true. In the example in the figure, the comparison result of the index element to be merged (9) with each index element in the buffer circuit (100, 10, and 9) is "001", which is output to the control circuit 610.

[0059] The control circuit 610 can be configured to determine the insertion position of the index element to be merged in the current sequence of the buffer circuit 633 according to the received comparison result. Specifically, the control circuit 610 can be further configured to determine the insertion position according to the changing position of the bit in the bitmap. In the example in the figure, the comparison result is "001", indicating that the current index element to be merged is less than the first and second index elements from the left in the buffer circuit, and greater than or equal to the third index element from the left, so the insertion position is between the second and third index elements, i.e., between "10" and "9".

[0060] In some embodiments, the buffer circuit 633 can be configured to insert the index element to be merged at the insertion position according to the indication of the control circuit 610. In the example in the figure, after the index element is inserted in the buffer circuit 633, the sequence becomes "100, 10, 9, 9".

[0061] In order to enable the data corresponding to the index to be obtained in the merge sort accumulation processing, in some embodiments, the buffer circuit 633 can be further configured to store the compared index elements and the data elements corresponding thereto in order according to the value of the index element. As shown in the figure, the buffer circuit 633 caches not only the index elements, but also the information of the data elements corresponding thereto. Therefore, after the insertion position of the index element is determined by comparison each time, the information of the data element corresponding to the index element can also be inserted into the buffer circuit. Those skilled in the art can understand that the information of the data element can be the data element itself, such as D32, D23, etc. shown in the example in the figure; the information of the data element can also be an address pointing to the data element, and the embodiments of the present disclosure are not limited in this respect.

[0062] Next, the buffer circuit 633 can output the rightmost index element "9". At this time, the control circuit 610 can be further configured to determine the memory access information of the next index element to be merged according to the output index element in the buffer circuit. Specifically, the control circuit determines which lane the output index element belongs to in the K-lane index, and takes the next index element to be merged from the lane to the comparator 631 for comparison.

[0063] Further, when outputting in order, the buffer circuit 633 can be configured to output the compared index elements in order of their values (e.g., from small to large) as fused indices, and output the data elements corresponding thereto as fused data in synchronization. The output data can be provided to the accumulation circuit 636 for further processing.

[0064] For clarity, the sequence of indices buffered in the buffer circuit 633 as the sorting progresses is also shown in the figure. As shown, initially, the first index elements of the K-way indices are stored in the buffer circuit 633 in order from large to small. In some implementations, the four index elements can be fetched, sorted, and stored in the buffer circuit at one time. In other implementations, the data in the buffer circuit can be initialized to negative numbers, and the first index elements of the K-way indices can be fetched one by one in order (e.g., in the order of the first way to the fourth way), compared with the data in the buffer circuit, and placed in the appropriate position. In this example, the first index elements of the four-way indices are all 0, and thus can be arranged in order of the way number according to the order of fetching, e.g., the "0" of the first way is placed at the rightmost position, the "0" of the second way is placed at the second position from the right, and so on.

[0065] Next, the "0" at the rightmost position in the buffer circuit, which belongs to the first way, is output. According to which way the output index element belongs to, the next index element to be fused, i.e., the second index element "2" of the first way, is fetched from the corresponding way. The "2" is sent to the comparison circuit to be compared with the remaining three "0"s in the buffer circuit, and the comparison result is "111", i.e., greater than all the three existing numbers "0" in the buffer circuit. Thus, the "2" is inserted at the tail end of the sequence, and the sequence in the buffer circuit becomes "2, 0, 0, 0".

[0066] Next, the "0" at the rightmost position in the buffer circuit, which belongs to the second way, is output. Thus, the second element "3" of the second way is fetched and compared with the remaining "2, 0, 0" in the buffer circuit, and the comparison result is "111". Thus, the "3" is inserted at the tail end of the sequence, and the sequence in the buffer circuit becomes "3, 2, 0, 0".

[0067] Next, the "0" at the rightmost position in the buffer circuit, which belongs to the third way, is output. Thus, the second element "100" of the third way is fetched and compared with the remaining "2, 0, 0" in the buffer circuit, and the comparison result is "111". Thus, the "100" is inserted at the tail end of the sequence, and the sequence in the buffer circuit becomes "100, 3, 2, 0".

[0068] Next, the "0" at the rightmost of the output buffer circuit belonging to the 4th path is taken out, and compared with the remaining "100, 3, 2" in the buffer circuit, and the comparison result is "001", so "2" is inserted after the first element at the rightmost of the sequence, and at this time the sequence in the buffer circuit becomes "100, 3, 2, 2".

[0069] By analogy, the index elements in the K paths of indexes can be compared one by one, and inserted into the appropriate position in the buffer circuit according to the size order, and then output by the buffer circuit. For example, the minimum index element output by the buffer circuit each time can be output to the accumulation circuit 636 in order. Those skilled in the art can understand that if the space of the buffer circuit is sufficient, the sorted elements after merging can also be output uniformly after sorting.

[0070] As can be seen from the output of the merging sorted index elements, when there are index elements of the same size, the sorting circuit 632 still retains these index elements of the same size and does not perform a deduplication operation, but provides them to the accumulation circuit 636 for processing.

[0071] In some embodiments, the accumulation circuit 636 can include a comparator 637, a buffer 635, and an adder 639.

[0072] The comparator 637 can be configured to compare the index elements output in order from the sorting circuit 632 with the previous fused index element and output the comparison result. The comparison result can be "1" indicating the same, "0" indicating not the same; or vice versa.

[0073] The buffer 635 can be configured to control the output of the index element according to the comparison result of the comparator 637. In some embodiments, the buffer 635 can output the current index element as a new fused index element only when the comparison result indicates that they are not the same. In other words, when the comparison result indicates that they are the same, the buffer 635 does not output the current index element, i.e., discards the index element that is duplicated with the previous fused index element. As shown, there is no duplicated fused index element in the fused index in the second storage circuit 624.

[0074] The adder 639 can be configured to control the accumulation of data elements according to the comparison result of the comparator 637. Specifically, when the comparison result indicates that they are not the same, the data element corresponding to the current index element is directly output as a new fused data element; when the comparison result indicates that they are the same, the data element corresponding to the current index element is accumulated to the fused data element corresponding to the previous fused index element.

[0075] By comparing the current index element with the previous fused index element each time the output of the sorting circuit 632 is received, the same index elements can be processed differently, thereby realizing the functions of index deduplication and data accumulation.

[0076] Figure 7 An exemplary flowchart of a data processing method 700 performed using the data processing circuit described above, according to an embodiment of this disclosure, is shown.

[0077] like Figure 7 As shown, in step 710, the control circuit reads multiple streams of data to be merged from the storage circuit. Next, in step 720, the arithmetic circuit groups the data elements from the multiple streams of data to be merged according to their corresponding indices, merging them into a single ordered merged data stream, where data elements with the same index are merged into one merged data element. Finally, in step 730, the arithmetic circuit outputs the merged data to the storage circuit. Although the method steps are shown sequentially in the figure, these steps are looped through data elements one by one, and some steps occur simultaneously within these loops. For example, when the arithmetic circuit outputs data, the control circuit simultaneously accesses the storage circuit to read the next data element to be merged.

[0078] In some embodiments, the storage circuit may include a first storage circuit and a second storage circuit. The first storage circuit is configured to store K-channel data to be fused and K-channel indices corresponding to the K-channel data, where K>1. The index elements in the K-channel indices indicate the index information of the corresponding data elements in the K-channel data. The index elements of each channel in the K-channel indices are arranged in a first order, and the data elements of each channel in the K-channel data are arranged in the order of their corresponding indices. The second storage circuit is configured to store the fused data output by the computation circuit and the fused index corresponding to the fused data. The fused index elements in the fused index are arranged in a second order, and the fused data elements in the fused data are arranged in the order of their fused indexes.

[0079] In some embodiments, the first order and the second order may be the same or different, and the first order and the second order are selected from either: an ascending order or a descending order. For example, multiple indexes arranged in ascending order can be merged into a single merged index arranged in descending order, or merged into a single merged index arranged in ascending order.

[0080] In some embodiments, the arithmetic circuit may include a sorting circuit and an accumulation circuit. In this case, the method may further include: the sorting circuit sorts the K-way indices according to the size of the index elements and outputs them in an orderly manner to the accumulation circuit; and when the same index element is received from the sorting circuit, the accumulation circuit accumulates the data elements corresponding to the same index element and removes duplicate index elements.

[0081] In some embodiments, the sorting circuit can include a comparison circuit and a buffer circuit. At this time, the method can further include: the comparison circuit comparing the index element to be sorted in the K-way index with the index elements not output in the buffer circuit, and outputting the comparison result to the control circuit; and the buffer circuit storing the compared index element and the information of the data element corresponding thereto in order according to the control of the control circuit, and outputting the compared index element and the information of the data element corresponding thereto in order.

[0082] In some embodiments, the comparison circuit can include a K-1 way comparator. At this time, the method can further include: the K-1 way comparator comparing the index element to be sorted in the K-way index with the K-1 index elements in the current sequence in the buffer circuit respectively, generating a comparison result and outputting it to the control circuit.

[0083] In some embodiments, the method further includes: the control circuit determining the insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison result.

[0084] In some embodiments, the comparison result is represented using a bitmap, and the method further includes: the control circuit determining the insertion position according to the change position of the bit in the bitmap.

[0085] In some embodiments, the method further includes: the buffer circuit inserting the index element to be sorted and the information of the data element corresponding thereto into the insertion position according to the indication of the control circuit.

[0086] In some embodiments, the method further includes: the buffer circuit outputting the first or last index element in the current sequence and the information of the data element corresponding thereto in the specified order.

[0087] In some embodiments, the method further includes: the control circuit determining the memory access information of the next index element to be sorted according to the index element output in the buffer circuit.

[0088] In some embodiments, the accumulation circuit includes a comparator, a buffer and an adder. At this time, the method can further include: the comparator comparing the index element output from the sorting circuit with the previous fusion index element and outputting the comparison result; the buffer outputting the index element as a new fusion index element only when the comparison result indicates that they are different; and the adder directly outputting the data element corresponding to the index element as a new fusion data element when the comparison result indicates that they are different, and accumulating the data element corresponding to the index element to the fusion data element corresponding to the previous fusion index element when the comparison result indicates that they are the same.

[0089] In some embodiments, the data elements in the multiple lanes of data to be fused are valid data elements in a sparse matrix, and the indices indicate location information of the valid data elements in the sparse matrix.

[0090] Those skilled in the art can understand that each step of the above method corresponds to each circuit described in the foregoing in connection with the example circuit diagram, and thus the features described in the foregoing can equally apply to the method steps, which are not repeated here.

[0091] As can be seen from the above description, the embodiments of the disclosure provide a hardware circuit for performing data fusion operations related to merge sort accumulation processing. By implementing merge sort accumulation with hardware, the processing speed can be accelerated, thereby better supporting operations related to post-sparse processing, such as sparse matrix multiplication and the like. In some embodiments, the hardware circuit can merge multiple lanes of ordered data into one lane of ordered fused data, and the same index data can be accumulated. By providing a hardware implementation of specialized data fusion-related operations, the processing can be simplified and accelerated, thereby improving the processing efficiency of the machine.

[0092] According to different application scenarios, the electronic device or apparatus of the present disclosure can include a server, a cloud server, a server cluster, a data processing apparatus, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a vehicle record device, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an autonomous driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship, and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric lamp, a gas stove, an oil smoke exhaust fan; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument, and / or an electrocardiograph. The electronic device or apparatus of the present disclosure can also be applied to the fields of Internet, Internet of Things, data center, energy, transportation, public management, manufacturing, education, power grid, telecommunications, finance, retail, construction site, medical treatment, etc. Further, the electronic device or apparatus of the present disclosure can also be used in cloud, edge, terminal, etc. application scenarios related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, the electronic device or apparatus with high computing power according to the present disclosure can be applied to a cloud device (such as a cloud server), and the electronic device or apparatus with small power consumption can be applied to a terminal device and / or an edge device (such as a smart phone or a camera). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or the edge device are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud device according to the hardware information of the terminal device and / or the edge device to simulate the hardware resources of the terminal device and / or the edge device, so as to complete the unified management, scheduling and collaborative work of end-cloud integration or cloud-edge integration.

[0093] It should be noted that for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art can understand that the schemes of the present disclosure are not limited by the order of the described actions. Therefore, those skilled in the art can understand that some steps can be executed in other orders or simultaneously according to the disclosure or teaching of the present disclosure. Further, those skilled in the art can understand that the described embodiments of the present disclosure can be regarded as optional embodiments, i.e. the actions or modules involved therein are not necessarily essential for the implementation of one or more schemes of the present disclosure. In addition, the description of some embodiments of the present disclosure also focuses on different schemes. Therefore, those skilled in the art can understand that the parts not described in detail in one embodiment of the present disclosure can also refer to the relevant description of other embodiments.

[0094] In terms of specific implementation, based on the disclosure and teaching of the present disclosure, those skilled in the art can understand that the several embodiments disclosed by the present disclosure can also be implemented in other manners not disclosed herein. For example, in terms of the units described in the foregoing electronic device or apparatus embodiments, the units can be split into other forms in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions of the units or components can be selectively disabled. In terms of the connection relationship between the units or components, the connection discussed above can be direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling refers to the communication connection using an interface, and the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0095] In the present disclosure, the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units. The foregoing components or units can be located in the same place or distributed on multiple network units. In addition, according to actual needs, part or all of the units can be selected to implement the purposes of the solutions described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure can be integrated into one unit or physically exist separately.

[0096] In some other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, i.e., specific hardware circuits, which can include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit can include but is not limited to physical devices, and the physical devices can include but are not limited to transistors or memristors, etc. In view of this, various devices (e.g., computing devices or other processing devices) described herein can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs, etc. Further, the aforementioned storage units or storage devices can be any appropriate storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM, etc.

[0097] The foregoing can be better understood in light of the following clauses:

[0098] Clause 1, a data processing circuit comprising a control circuit, a storage circuit and an operation circuit, wherein:

[0099] the control circuit is configured to control the storage circuit and the operation circuit to perform merge sort accumulation processing on a plurality of to-be-fused data;

[0100] the storage circuit is configured to store information, the information at least including pre-processing and / or post-processing information; and

[0101] the operation circuit is configured to, under the control of the control circuit, merge data elements in the plurality of to-be-fused data according to their corresponding indexes into one-way ordered fused data, and data elements with the same index in the fused data are merged into one fused data element.

[0102] Clause 2, the data processing circuit according to clause 1, wherein the storage circuit comprises a first storage circuit and a second storage circuit,

[0103] The first storage circuit is configured to store the K-way data to be fused and K-way indexes corresponding to the K-way data, K > 1, wherein an index element in the K-way index indicates index information of a corresponding data element in the K-way data, the index elements of each way index in the K-way index are sequentially arranged in a first order, and the data elements of each way data in the K-way data are sequentially arranged in the order of the corresponding index; and

[0104] The second storage circuit is configured to store the fused data output by the operation circuit and a fused index corresponding to the fused data, wherein a fused index element in the fused index is sequentially arranged in a second order, and a fused data element in the fused data is sequentially arranged in the order of the fused index.

[0105] Clause 3, the data processing circuit according to any one of clauses 2, wherein the first order is the same as or different from the second order, and the first order and the second order are selected from any one of the following: an order from small to large, or an order from large to small.

[0106] Clause 4, the data processing circuit according to any one of clauses 2-3, wherein the operation circuit comprises a sorting circuit and an accumulation circuit, wherein

[0107] The sorting circuit is configured to sort the K-way indexes according to the size of the index elements and sequentially output the K-way indexes to the accumulation circuit; and

[0108] The accumulation circuit is configured to, when receiving the same index element from the sorting circuit, accumulate the data elements corresponding to the same index element, and remove the repeated index elements.

[0109] Clause 5, the data processing circuit according to clause 4, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, wherein:

[0110] The comparison circuit is configured to compare the index element to be sorted in the K-way index with the index element not output in the buffer circuit, and output a comparison result to the control circuit; and

[0111] The buffer circuit is configured to, according to the control of the control circuit, sequentially store information of the compared index element and the data element corresponding thereto, and sequentially output the information of the compared index element and the data element corresponding thereto.

[0112] Clause 6, the data processing circuit according to clause 5, wherein the comparison circuit comprises:

[0113] K-1 way comparators configured to compare the index element to be sorted in the K way index with K-1 index elements in the current sequence in the buffer circuit respectively, generate comparison results and output to the control circuit.

[0114] Clause 7, the data processing circuit according to clause 6, wherein the control circuit is configured to determine the insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison results.

[0115] Clause 8, the data processing circuit according to clause 7, wherein the comparison results are represented using a bitmap, and the control circuit is further configured to determine the insertion position according to the changing position of a bit in the bitmap.

[0116] Clause 9, the data processing circuit according to any one of clauses 7-8, wherein the buffer circuit is configured to insert the index element to be sorted and the information of the data element corresponding thereto in the insertion position according to the indication of the control circuit.

[0117] Clause 10, the data processing circuit according to any one of clauses 5-9, wherein the buffer circuit is further configured to output the information of the first or last index element in the current sequence and the data element corresponding thereto in the specified order.

[0118] Clause 11, the data processing circuit according to clause 10, wherein the control circuit is further configured to determine the memory access information of the next index element to be sorted according to the index element output by the buffer circuit.

[0119] Clause 12, the data processing circuit according to any one of clauses 4-11, wherein the accumulation circuit comprises a comparator, a buffer and an adder, wherein:

[0120] the comparator is configured to compare the index element output from the sorting circuit with the last fused index element and output a comparison result;

[0121] the buffer is configured to output the index element as a new fused index element only when the comparison result indicates that they are different; and

[0122] the adder is configured to directly output the data element corresponding to the index element as a new fused data element when the comparison result indicates that they are different, and to accumulate the data element corresponding to the index element to the fused data element corresponding to the last fused index element when the comparison result indicates that they are the same.

[0123] Clause 13, the data processing circuit according to any one of clauses 1-12, wherein the data elements in the plurality of data to be fused are valid data elements in a sparse matrix, and the index indicates position information of the valid data elements in the sparse matrix.

[0124] Clause 14, a chip comprising the data processing circuit according to any one of clauses 1-13.

[0125] Clause 15, a board card comprising the chip according to clause 14.

[0126] Clause 16, a method for processing data using a data processing circuit, the data processing circuit comprising a control circuit, a storage circuit and an operation circuit, the method comprising:

[0127] the control circuit reads a plurality of data to be fused from the storage circuit;

[0128] the operation circuit merges data elements in the plurality of data to be fused according to their corresponding indexes into one ordered fused data, and data elements with the same index in the fused data are merged into one fused data element; and

[0129] outputting the fused data to the storage circuit.

[0130] Clause 17, the method according to clause 16, wherein the storage circuit comprises a first storage circuit and a second storage circuit,

[0131] the first storage circuit is configured to store the K pieces of data to be fused and K pieces of indexes corresponding to the K pieces of data, K>1, wherein an index element in the K pieces of indexes indicates index information of a corresponding data element in the K pieces of data, the index elements in each piece of index in the K pieces of indexes are arranged in a first order, and the data elements in each piece of data in the K pieces of data are arranged in an order corresponding to the indexes; and

[0132] the second storage circuit is configured to store the fused data output by the operation circuit and a fused index corresponding to the fused data, wherein fused index elements in the fused index are arranged in a second order, and fused data elements in the fused data are arranged in an order corresponding to the fused index.

[0133] Clause 18, the method according to clause 17, wherein the first order is the same as or different from the second order, and the first order and the second order are selected from any one of the following: an order from small to large, or an order from large to small.

[0134] Clause 19, the method of any of clauses 17-18, wherein the operation circuit comprises a sorting circuit and an accumulation circuit, and the method further comprises:

[0135] the sorting circuit sorts the K-way indexes according to the size of the index elements and outputs the sorted indexes to the accumulation circuit; and

[0136] the accumulation circuit accumulates the data elements corresponding to the same index elements when receiving the same index elements from the sorting circuit and removes the duplicated index elements.

[0137] Clause 20, the method of clause 19, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, and the method further comprises:

[0138] the comparison circuit compares the index element to be sorted in the K-way indexes with the index elements not outputted in the buffer circuit and outputs the comparison result to the control circuit; and

[0139] the buffer circuit stores the information of the compared index element and the data element corresponding thereto in order according to the control of the control circuit and outputs the information of the compared index element and the data element corresponding thereto in order.

[0140] Clause 21, the method of clause 20, wherein the comparison circuit comprises K-1 comparators, and the method comprises:

[0141] the K-1 comparators compare the index element to be sorted in the K-way indexes with the K-1 index elements in the current sequence in the buffer circuit respectively, generate the comparison result and output the comparison result to the control circuit.

[0142] Clause 22, the method of clause 21, further comprising:

[0143] the control circuit determines the insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison result.

[0144] Clause 23, the method of clause 22, wherein the comparison result is represented by a bitmap, and the method further comprises: the control circuit determines the insertion position according to the change position of the bit in the bitmap.

[0145] Clause 24, the method of any of clauses 22-23, further comprising:

[0146] the buffer circuit inserts the information of the index element to be sorted and the data element corresponding thereto in the insertion position according to the indication of the control circuit.

[0147] Clause 25, the method of any of clauses 20-24, further comprising:

[0148] The buffer circuit outputs information of the first or last index element in the current sequence and the data element corresponding thereto in a specified order.

[0149] Clause 26, the method of clause 25, further comprising:

[0150] The control circuit determines, according to the index element outputted by the buffer circuit, memory access information of the next index element to be sorted.

[0151] Clause 27, the method of any of clauses 19-26, wherein the accumulation circuit comprises a comparator, a buffer and an adder, and the method comprises:

[0152] The comparator compares the index element outputted from the sorting circuit with the last fused index element and outputs a comparison result;

[0153] The buffer outputs the index element as a new fused index element only when the comparison result indicates that they are different; and

[0154] When the comparison result indicates that they are different, the adder directly outputs the data element corresponding to the index element as a new fused data element, and when the comparison result indicates that they are the same, the adder accumulates the data element corresponding to the index element to the fused data element corresponding to the last fused index element.

[0155] Clause 28, the method of any of clauses 20-27, wherein the data element in the plurality of data to be fused is an effective data element in a sparse matrix, and the index indicates position information of the effective data element in the sparse matrix.

[0156] The above has been described in detail for the embodiments of the present disclosure, and the principles and implementation manners of the present disclosure are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present disclosure and its core idea; meanwhile, for those skilled in the art, according to the idea of the present disclosure, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the present disclosure.

Claims

1. A data processing circuit comprising a control circuit, a storage circuit and an operation circuit, wherein: the control circuit is configured to control the storage circuit and the operation circuit to perform merge sort accumulation processing on multiple channels of data to be fused; the storage circuit is configured to store information, the information comprising at least pre-processing and / or post-processing information; and the operation circuit is configured to, under the control of the control circuit, merge data elements in the multiple channels of data to be fused according to their corresponding indexes into one channel of ordered fused data, and remove duplicate index elements, wherein data elements with the same index in the fused data are merged into one fused data element, so as to reduce the demand of a neural network for calculation amount and storage space; wherein the storage circuit comprises a first storage circuit and a second storage circuit, the first storage circuit is configured to store the K channels of data to be fused and K channels of indexes corresponding to the K channels of data, K > 1, wherein an index element in the K channels of indexes indicates index information of a corresponding data element in the K channels of data, the index elements in each channel of indexes are arranged in a first order, and the data elements in each channel of data are arranged in an order corresponding to the indexes; and the second storage circuit is configured to store the fused data output by the operation circuit and a fused index corresponding to the fused data, wherein the fused index elements in the fused index are arranged in a second order, and the fused data elements in the fused data are arranged in an order corresponding to the fused indexes. 2.The data processing circuit of claim 1, wherein the first order is the same as or different from the second order, and the first order and the second order are selected from any one of the following: an order from small to large, or an order from large to small. 3.The data processing circuit of claim 1, wherein the operation circuit comprises a sorting circuit and an accumulation circuit, wherein: the sorting circuit is configured to sort the K channels of indexes according to the sizes of the index elements and output the sorted index elements to the accumulation circuit in an order; and the accumulation circuit is configured to, when receiving the same index element from the sorting circuit, accumulate the data elements corresponding to the same index element and remove the duplicate index elements. 4.The data processing circuit of claim 3, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, wherein: the comparison circuit is configured to compare an index element to be sorted in the K channels of indexes with an index element not yet output in the buffer circuit, and output a comparison result to the control circuit; and the buffer circuit is configured to, under the control of the control circuit, store information of the compared index element and the data element corresponding to the compared index element in an order, and output the information of the compared index element and the data element corresponding to the compared index element in an order. 5.The data processing circuit of claim 4, wherein the comparison circuit comprises: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ K-1 way comparators configured to compare the index element to be sorted in the K way index with K-1 index elements in the current sequence in the buffer circuit respectively, generate comparison results and output to the control circuit. 6.The data processing circuit of claim 5, wherein the control circuit is configured to determine an insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison results. 7.The data processing circuit of claim 6, wherein the comparison results are represented using a bitmap, and the control circuit is further configured to determine the insertion position according to a changing position of a bit in the bitmap. 8.The data processing circuit of claim 6, wherein the buffer circuit is configured to insert the index element to be sorted and information of the data element corresponding thereto in the insertion position according to an indication of the control circuit. 9.The data processing circuit of claim 4, wherein the buffer circuit is further configured to output the first or last index element in the current sequence and information of the data element corresponding thereto in a specified order. 10.The data processing circuit of claim 9, wherein the control circuit is further configured to determine memory access information of a next index element to be sorted according to the index element output in the buffer circuit. 11.The data processing circuit of claim 3, wherein the accumulation circuit comprises a comparator, a buffer and an adder, wherein: the comparator is configured to compare the index element output from the sorting circuit with a previous fused index element and output a comparison result; the buffer is configured to output the index element as a new fused index element only when the comparison result indicates that they are different; and the adder is configured to output the data element corresponding to the index element as a new fused data element directly when the comparison result indicates that they are different, and to accumulate the data element corresponding to the index element to the fused data element corresponding to the previous fused index element when the comparison result indicates that they are the same. 12.The data processing circuit of any of claims 1-11, wherein the data elements in the multiple ways of data to be fused are valid data elements in a sparse matrix, and the index indicates position information of the valid data elements in the sparse matrix. 13.A chip comprising the data processing circuit of any of claims 1-12. 14.A board card comprising the chip of claim 13. 15.A method of processing data using a data processing circuit, the data processing circuit comprising a control circuit, a storage circuit and an operation circuit, the method comprising: reading, by the control circuit, multiple ways of data to be fused from the storage circuit; ​ ​ ​ ​ The operation circuit merges data elements in the plurality of data to be fused according to their corresponding indexes into one ordered fused data, and removes repeated index elements, and data elements with the same index in the fused data are merged into one fused data element, so as to reduce the demand of the neural network for calculation amount and storage space; and output the fused data to the storage circuit; The storage circuit comprises a first storage circuit and a second storage circuit, The first storage circuit is configured to store the K pieces of data to be fused and K pieces of indexes corresponding to the K pieces of data, K > 1, wherein the index elements in the K pieces of indexes indicate index information of corresponding data elements in the K pieces of data, the index elements in each piece of index are arranged in a first order, and the data elements in each piece of data are arranged in order according to the corresponding indexes; and The second storage circuit is configured to store the fused data output by the operation circuit and a fused index corresponding to the fused data, wherein the fused index elements in the fused index are arranged in a second order, and the fused data elements in the fused data are arranged in order according to the fused index.

16. The method of claim 15, wherein the first order is the same as or different from the second order, and the first order and the second order are selected from any one of the following: an order from small to large, or an order from large to small.

17. The method of claim 15, wherein the operation circuit comprises a sorting circuit and an accumulation circuit, and the method further comprises: The sorting circuit sorts the K pieces of indexes according to the size of the index elements and outputs them to the accumulation circuit in order; and When receiving the same index element from the sorting circuit, the accumulation circuit accumulates the data elements corresponding to the same index element and removes the repeated index elements.

18. The method of claim 17, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, and the method further comprises: The comparison circuit compares the index elements to be sorted in the K pieces of indexes with the index elements not output in the buffer circuit, and outputs the comparison result to the control circuit; and The buffer circuit stores the compared index elements and the information of the data elements corresponding thereto in order and outputs the compared index elements and the information of the data elements corresponding thereto in order according to the control of the control circuit.

19. The method of claim 18, wherein the comparison circuit comprises K-1 comparators, and the method comprises: The K-1 comparators compare the index elements to be sorted in the K pieces of indexes with K-1 index elements in the current sequence in the buffer circuit respectively, generate a comparison result and output it to the control circuit.

20. The method of claim 19, further comprising: The control circuit determines the insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison result.

21. The method of claim 20, wherein the comparison result is represented using a bitmap, and the method further comprises: The control circuit determines the insertion position according to a change position of a bit in the bitmap.

22. The method of claim 20, further comprising: The buffer circuit inserts the index element to be sorted and information of the data element corresponding thereto in the insertion position according to an indication of the control circuit.

23. The method of claim 18, further comprising: The buffer circuit outputs the first or last index element in the current sequence and information of the data element corresponding thereto in the specified order.

24. The method of claim 23, further comprising: The control circuit determines access information of the next index element to be sorted according to the index element outputted by the buffer circuit.

25. The method of claim 17, wherein the accumulation circuit comprises a comparator, a buffer and an adder, and the method comprises: The comparator compares the index element outputted from the sorting circuit with the last fused index element and outputs a comparison result; The buffer outputs the index element as a new fused index element only when the comparison result indicates that they are different; and When the comparison result indicates that they are different, the adder directly outputs the data element corresponding to the index element as a new fused data element, and when the comparison result indicates that they are the same, the adder accumulates the data element corresponding to the index element to the fused data element corresponding to the last fused index element.

26. The method of any one of claims 15-25, wherein the data elements in the plurality of data to be fused are valid data elements in a sparse matrix, and the index indicates position information of the valid data elements in the sparse matrix.

Images