Data processing circuit, data processing method and related product

Abstract

The present disclosure discloses a data processing circuit, a data processing method and related products. The data processing circuit can be implemented as a computing device included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices, respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure provides a hardware implementation of data fusion related operations, which can simplify processing and improve the processing efficiency of the machine.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of data processing. More specifically, the present disclosure relates to data processing circuit, data processing method, chip and board card. BACKGROUND

[0002] In recent years, with the rapid development of deep learning, the algorithm performance of a series of fields such as computer vision and natural language processing has made a leap. However, deep learning algorithm is a kind of computing-intensive and storage-intensive tool. With the increasing complexity of information processing tasks, the real-time performance and accuracy of the algorithm are constantly increasing. The neural network is often designed to be deeper and deeper, so that the computational complexity and storage space requirement are increasing, which makes it difficult for existing artificial intelligence technology based on deep learning to be directly applied to mobile phones, satellites or embedded devices with limited hardware resources.

[0003] Therefore, the compression, acceleration and optimization of deep neural network model become particularly important. A large number of studies try to reduce the computational and storage requirements of neural network without affecting the model accuracy, which is of great significance for the engineering application of deep learning technology in embedded and mobile terminals. Sparsification is one of the methods for model lightening.

[0004] Network parameter sparsification is to reduce the redundant components in a large network by appropriate methods to reduce the demand for computational complexity and storage space. The existing hardware and / or instruction set cannot effectively support sparsification processing and / or processing related to sparsification. SUMMARY

[0005] In order to at least partially solve one or more technical problems mentioned in the background, the scheme of the present disclosure provides a data processing circuit, a data processing method, a chip and a board card.

[0006] In a first aspect, the present disclosure discloses a data processing circuit, comprising a control circuit, a storage circuit and an operation circuit, wherein: the control circuit is configured to control the storage circuit and the operation circuit to perform fusion processing on multiple pieces of data to be fused; the storage circuit is configured to store information, the information at least including pre-processing and / or post-processing information; and the operation circuit is configured to, under the control of the control circuit, merge data elements in the multiple pieces of data to be fused according to their corresponding indexes into one piece of ordered post-processing data, wherein data elements with the same index are merged into an operation structure element, and the data elements include any of scalar, vector or higher-dimensional data.

[0007] In a second aspect, the present disclosure provides a chip comprising the data processing circuit of any one of the preceding embodiments of the first aspect.

[0008] In a third aspect, the disclosure provides a board card comprising the chip of any one of the preceding second aspect.

[0009] In a fourth aspect, the disclosure provides a method for processing data using a data processing circuit, the data processing circuit comprising a control circuit, a storage circuit and an operation circuit, the method comprising: reading, by the control circuit, a plurality of data to be fused from the storage circuit; merging, by the operation circuit, data elements in the plurality of data to be fused according to their corresponding indexes into one ordered fused data, wherein data elements with the same index are merged into an operation structure element, and the data elements comprise any one of scalar, vector or higher dimensional data; and outputting the fused data to the storage circuit.

[0010] By the data processing circuit, the method for processing data using a data processing circuit, the chip and the board card provided as above, the embodiments of the disclosure provide a hardware circuit supporting data fusion operation, for performing merge sort fusion processing on a plurality of data. In some embodiments, the data processing circuit can merge a plurality of ordered data into one ordered fused data according to index order, and data elements with the same index can be merged and represented in the form of operation structure, thereby facilitating subsequent calculation processing. In some embodiments, the data elements to be fused can be vector or higher dimensional data. For example, the data elements in the data to be fused can be sparse effective data elements in radar-based object detection, so that the data fusion instructions and fusion operations provided by the embodiments of the disclosure can support related processing in radar algorithms. By providing a hardware implementation of special data fusion related operations, the processing can be simplified and accelerated, thereby improving the processing efficiency of the machine. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the disclosure will be readily understood through reading the detailed description of the exemplary embodiments of the disclosure when taken in conjunction with the accompanying drawings. In the drawings, several embodiments of the disclosure are illustrated by way of example and not limitation, in which like reference numerals refer to like elements, in which:

[0012] Figure 1 is a structural diagram illustrating a board card of an embodiment of the disclosure;

[0013] Figure 2 is a structural diagram illustrating a combined processing device of an embodiment of the disclosure;

[0014] Figure 3 is a schematic diagram illustrating the internal structure of a processor core of a single-core or multi-core computing device of an embodiment of the disclosure;

[0015] Figure 4is an exemplary schematic diagram illustrating data fusion processing according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram showing the structure of a data processing circuit according to an embodiment of the present disclosure;

[0017] Figure 6 is an exemplary circuit diagram illustrating a data fusion process according to an embodiment of the present disclosure;

[0018] Figure 7 is an example of an operation structure showing another embodiment of the present disclosure;

[0019] Figure 8 is an example of an operation structure illustrating yet another embodiment of the present disclosure; and

[0020] Figure 9 is an exemplary flow chart illustrating a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of this disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this disclosure, not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this disclosure.

[0022] It should be understood that the terms "first," "second," "third," and "fourth," etc., which may appear in the claims, specification, and drawings of this disclosure, are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of this disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0023] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0024] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0025] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC), or system-on-chip, integrated with one or more combined processing devices. The combined processing device is an artificial intelligence computing unit that supports various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud-based intelligent applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. Board 10 of this embodiment is suitable for cloud-based intelligent applications and has extensive off-chip storage, on-chip storage, and powerful computing capabilities.

[0027] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. Calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface formats, such as a PCIe interface.

[0028] Board 10 also includes a memory device 104 for storing data, which includes one or more storage units 105. Memory device 104 is connected to a control device 106 and chip 101 via a bus for data transmission. Control device 106 in board 10 is configured to control the state of chip 101. To this end, in one application scenario, control device 106 may include a microcontroller (MCU).

[0029] Figure 2 FIG. 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0030] The computing device 201 is configured to perform user-specified operations, and is implemented as a single-core or multi-core intelligent processor to perform deep learning or machine learning calculations, which can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0031] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it to the storage device on the computing device 201. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them to the control buffer on the computing device 201. Alternatively or optionally, the interface device 202 can also read data from the storage device of the computing device 201 and transmit it to the processing device 203.

[0032] The processing device 203 is a general-purpose processing device that performs basic controls including but not limited to data transfer, starting and / or stopping of the computing device 201, etc. Depending on the implementation, the processing device 203 can be one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors, including but not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., and the number thereof can be determined according to actual needs. As mentioned earlier, only in terms of the computing device 201 of the present disclosure, it can be considered as having a single-core or homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0033] The storage device 204 is used to store data to be processed, which can be a DRAM, a DDR memory, usually with a size of 16G or more, for saving data of the computing device 201 and / or the processing device 203.

[0034] Figure 3The internal structure of the processor core is shown when the computing device 201 is a single-core or multi-core device. The computing device 301 is used to process computer vision, speech, natural language, data mining, etc. input data, and the computing device 301 includes three modules: a control module 31, an operation module 32, and a storage module 33.

[0035] The control module 31 is used to coordinate and control the operation of the operation module 32 and the storage module 33 to complete the task of deep learning, which includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 is used to obtain instructions from the processing device 203, and the instruction decode unit 312 decodes the obtained instructions and sends the decoding results as control information to the operation module 32 and the storage module 33.

[0036] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 322 is responsible for the core calculation of the deep learning algorithm, i.e. matrix multiplication and convolution.

[0037] The storage module 33 is used to store or transport related data, including a neuron storage unit (NRAM) 331, a parameter storage unit (WRAM) 332, and a direct memory access module (DMA) 333. The NRAM 331 is used to store input neurons, output neurons, and intermediate results after calculation; the WRAM 332 is used to store the convolution kernel of the deep learning network, i.e. the weight; and the DMA 333 is connected to the DRAM 204 through the bus 34 and is responsible for data transportation between the computing device 301 and the DRAM 204.

[0038] Based on the foregoing hardware environment, embodiments of the present disclosure provide a data processing circuit that supports data fusion operations. As mentioned in the background, network parameter sparsification can effectively reduce the demand for computing power and storage space of the network. However, network parameter sparsification also brings a series of impacts on subsequent processing. For example, in sparse matrix multiplication operations, it may be necessary to sort and accumulate the vectors obtained during the operation to obtain the expected operation result. For another example, in radar algorithms, it is necessary to perform fusion processing on the sparsified data in radar-based object detection. In view of this, embodiments of the present disclosure provide a hardware solution for data fusion processing to simplify and accelerate such processing.

[0039] Figure 4An exemplary principle of data fusion processing according to embodiments of the present disclosure is shown. Four pieces of data to be fused are exemplarily shown in the figure, each piece of data including six data elements. The data elements can be scalars, vectors or higher dimensional tensors. The data elements are exemplarily shown as vectors, e.g. D11, D12, … D46. These vectors have a uniform vector length, e.g. D11 is (d1, d2, d3, …, dn) with length n. Each data element has an associated index indicating the position information of the data element in the corresponding piece of data. For example, the original piece of data can include 1000 data elements, but only the data elements at some positions are valid. In this case, the valid elements can be extracted to form the above-mentioned data to be fused, and the indices corresponding to these valid elements are extracted to indicate their positions in the original data. These indices form the above-mentioned indices to be fused.

[0040] The corresponding four pieces of indices to be fused are schematically shown in the figure, each piece of index corresponding to a piece of data to be fused. The first piece of index is used to identify the position information of each data element in the first piece of data, the second piece of index is used to identify the position information of each data element in the second piece of data, and so on. Further, the index elements in each piece of index are stored in order and one-to-one correspond to the data elements in the corresponding piece of data. In the example shown in the figure, the index elements in each piece of index are arranged in a first order (e.g. from small to large), and the data elements in each piece of data are also arranged in order according to the corresponding indices. For example, the first index element in the first piece of index indicates that the index of the first data element in the first piece of data is 0, i.e. the first element; the second index element in the first piece of index indicates that the index of the second data element in the first piece of data is 2, i.e. the third element; and so on.

[0041] After data fusion, the four pieces of data are merged into one piece of ordered fused data according to their corresponding indices, and the data elements with the same index are merged into one fused data element. As shown in the figure, the fused index includes 16 index elements arranged in a second order (e.g. from small to large), in which the repeated index elements in the indices to be fused are removed, as shown by the dark squares in the figure. Correspondingly, the fused data also includes 16 data elements arranged in order according to the corresponding indices, and the data elements with the same index are merged, as shown by the dark squares in the figure. Since the data elements can be vectors or higher dimensional tensors, in some embodiments of the present disclosure, at least for the merging of data elements with the same index, the form of operation structure element can be used to represent.

[0042] In this example, the merging of data elements with the same index is schematically represented by an addition operation. For example, for the post-fusion index element "0", its corresponding post-fusion data element is the sum of the first data elements of the lanes (D11+D21+D31+D41). For example, for the post-fusion index element "9", its corresponding post-fusion data element is the sum of the fifth data element of the first lane and the third data element of the fourth lane (D15+D43). When the data elements are vectors, the post-fusion data element is the sum of the corresponding vectors. The specific representation of the post-fusion data element will be described later.

[0043] Those skilled in the art can understand that the first order and the second order mentioned above can be the same or different, and both can be selected from any one of the following: an ascending order or a descending order. Those skilled in the art can also understand that although the lanes are shown to have the same number of data elements, the number of data elements in the lanes can be the same or different, and the present disclosure is not limited in this respect.

[0044] Figure 5 A structural block diagram of a data processing circuit 500 according to an embodiment of the present disclosure is shown. The data processing circuit 500 can be implemented in, for example, the computing device 201 of Figure 2 As shown, the data processing circuit 500 can include a control circuit 510, a storage circuit 520, and an operation circuit 530.

[0045] The control circuit 510 can have a function similar to the control module 31 of Figure 3 which can include, for example, an instruction fetch unit to fetch instructions from the processing device 203, for example, and an instruction decode unit to decode the fetched instructions and send the decoded results as control information to the operation circuit 530 and the storage circuit 520. Figure 2

[0046] In one embodiment, the control circuit 510 can be configured to control the storage circuit 520 and the operation circuit 530 to perform a fusion process on the multiple lanes of data to be fused.

[0047] The storage circuit 520 can be configured to store various information, including at least information before and / or after the fusion process. The storage circuit can be, for example, the WRAM 332 of Figure 3

[0048] The operation circuit 530 can be configured to merge, under the control of the control circuit, data elements in the multiple lanes of data to be fused according to their corresponding indices into one lane of ordered post-fusion data, wherein data elements with the same index are merged into an operation structure element. The data elements can be any one of a scalar, a vector, or a higher-dimensional data. ​​

[0049] In one embodiment, the operation circuit 530 can further include an operation processing circuit (not shown) which can be configured to pre-process the data before operation or post-process the data after operation according to operation instructions. In some application scenarios, the aforementioned pre-processing and post-processing can for example include data splitting and / or data concatenation operations.

[0050] The operation circuit can be implemented in various ways. Figure 6 An exemplary circuit diagram for data fusion processing according to one embodiment of the present disclosure is shown.

[0051] As shown, in one embodiment, the storage circuit can be exemplarily divided into two parts: a first storage circuit 622 and a second storage circuit 624.

[0052] The first storage circuit 622 can be configured to store K-way data to be fused and K-way indexes corresponding to the K-way data, K > 1. The index elements in the K-way indexes indicate the index information of the corresponding data elements in the K-way data, i.e., the index elements and the data elements have a one-to-one correspondence. In addition, the index elements of each way index in the K-way indexes are sequentially arranged in a first order, and the data elements of each way data in the K-way data are sequentially arranged in the order of the corresponding index.

[0053] Exemplarily shown in the figure are Figure 4 The 4-way indexes and the corresponding 4-way data are shown. Since the data elements can be vectors or higher dimensional data, for the sake of simplicity, the storage addresses of these data elements can be used instead, and each data element in the figure is identified by the symbol Pt, representing a pointer to the storage address of the corresponding data element (e.g., vector, three-dimensional tensor, etc.). It can be understood that the specific data elements are also stored in the storage circuit, and for the sake of clarity, they are not shown in the figure. In some embodiments, each way index can be stored continuously, e.g., as a vector, so that the way index / vector can be accessed according to the starting address of the way index / vector. Correspondingly, each way data can also be stored continuously, e.g., as a vector, so that the way data / vector can be accessed according to the starting address of the way data / vector, but in this case, the vector elements in the data vector are pointers to the final data elements.

[0054] The second storage circuit 624 can be configured to store the fusion-processed data output by the operation circuit. The data includes fusion indexes after fusion processing of K-way indexes, fusion data, and operation structures. The fusion index elements in the fusion indexes are ordered in a second order, and the fusion data elements in the fusion data correspond one-to-one to the fusion index elements and are ordered in the order of the fusion indexes. The operation structures include at least operation structure elements for indicating fusion data elements composed of data elements of the same index.

[0055] As can be seen from the examples in the figures, the 4-way data to be fused becomes one-way fusion data, and the corresponding 4-way indexes also become one-way fusion indexes, in which the fusion index elements are arranged in order from small to large, and index elements of the same size are removed. The corresponding fusion data elements are arranged in the order of the fusion indexes, and each fusion data element can be an address pointer pointing to the corresponding final data element. In some embodiments, the storage space of each final data element can be of a fixed size. Thus, when these data elements are stored continuously, the subsequent address can be determined by offsetting the first address by a fixed amount. For example, the first fusion data element can be an address base_addr, and assuming that each final data element occupies an address of size offset, the second fusion data element can be an address base_addr+offset, the third fusion data element can be an address base_addr+2*offset, and so on.

[0056] Among the final data elements pointed to by these addresses, some are original data elements because their indexes are unique in the indexes to be fused and do not need to be accumulated, and some need to be accumulated because there are multiple original data elements with the same index. In embodiments of the present disclosure, at least for fusion data elements composed of data elements with the same index, the final data elements pointed to by the fusion data elements can not be immediately obtained by operation, but can be represented by associated operation structure elements in the operation structures. Aspects of the operation structures will be described in detail later in combination with specific circuits.

[0057] In some embodiments, the operation circuit can include a sorting circuit 632 and an output circuit 636 to cooperatively implement the sorting and accumulation fusion function. Specifically, the sorting circuit 632 is configured to sort the K-way indexes according to the size of the index elements and output them in order to the output circuit 636. The output circuit 636 can generate operation structure elements representing the accumulation operation of the data elements corresponding to the same index elements and remove the duplicate index elements at least when receiving the same index elements from the sorting circuit.

[0058] In some embodiments, the sorting circuit 632 can include a comparison circuit 631 and a buffer circuit 633. The comparison circuit 631 implements a comparison function that compares the size of the index elements in the plurality of to-be-merged indexes and submits the comparison result to the control circuit 610 for sorting. The control circuit 610 determines the insertion position of the index elements in the buffer circuit 633 according to the comparison result. The buffer circuit 633 is configured to buffer the compared index elements and the information of the data elements corresponding thereto, and buffer them in order of the size of the index elements.

[0059] Specifically, the comparison circuit 631 can be configured to compare the index elements in the to-be-merged indexes with the index elements in the buffer circuit 633 that have not been output, and output the comparison result to the control circuit 610. The buffer circuit 633 can be configured to store the compared index elements and the information of the data elements corresponding thereto in order and output them in order according to the control of the control circuit 610.

[0060] In some embodiments, the buffer circuit 633 can be configured to buffer K index elements in order of the size. Those skilled in the art can understand that the buffer circuit can also be configured to buffer more index elements, and the embodiments of the present disclosure are not limited in this respect. Depending on the sorting manner in the buffer circuit 633 and the desired output sorting manner, such as from small to large or from large to small, the first or last index element in the current sequence can be output in the specified order each time. For example, in the example in the figure, the buffer circuit 633 buffers the index elements from left to right in order of from large to small, and outputs the rightmost index element each time, that is, the smallest index element in the current sequence, for example, “7”.

[0061] In these embodiments, the comparison circuit 631 can include K-1 comparators configured to compare the to-be-merged index elements with the K-1 index elements in the buffer circuit 633 that have not been output, that is, the K-1 index elements remaining after the first or last index element in the current sequence is output, generate the comparison result and output it to the control circuit 610.

[0062] For example, for 4-way data to be merged, a 3-way comparator is shown in the figure, which compares the specified index element (9 at this time) received from the first storage circuit 622 with the three index elements in the buffer circuit 633 that have not been output, that is, the three index elements 100, 10 and 9 on the left in the figure.

[0063] In some embodiments, the comparison result of the comparator can be represented using a bitmap. For example, if the index element to be fused (e.g., 9) is greater than or equal to the index element in the buffer circuit, the comparator can output "1", otherwise, output "0"; the opposite is also possible. In the example in the figure, the comparison result of the index element to be fused (9) with each index element in the buffer circuit (100, 10, and 9) is "001", which is output to the control circuit 610.

[0064] The control circuit 610 can be configured to determine the insertion position of the index element to be fused in the current sequence of the buffer circuit 633 according to the received comparison result. Specifically, the control circuit 610 can be further configured to determine the insertion position according to the changing position of the bit in the bitmap. In the example in the figure, the comparison result is "001", indicating that the current index element to be fused is less than the first and second index elements from the left in the buffer circuit, and greater than or equal to the third index element from the left, so the insertion position is between the second and third index elements, i.e., between "10" and "9".

[0065] In some embodiments, the buffer circuit 633 can be configured to insert the index element to be fused at the insertion position according to the indication of the control circuit 610. In the example in the figure, after the index element is inserted in the buffer circuit 633, the sequence becomes "100, 10, 9, 9".

[0066] In order to enable the data corresponding to the index to be obtained in the fusion process, in some embodiments, the buffer circuit 633 can be further configured to store the compared index elements and the data elements corresponding thereto in order according to the value of the index element. As shown in the figure, the buffer circuit 633 caches not only the index elements, but also the information of the data elements corresponding thereto. Therefore, after the comparison of the index elements determines the insertion position, the information of the data elements corresponding to the index elements can also be inserted into the buffer circuit. Since the data elements can be scalars, vectors, or higher-dimensional tensors, an address pointing to the data element can be used to represent the data element. For example, in the example in the figure, each element in the K-way data to be fused is an address pointing to the corresponding data element, regardless of whether the data element is a scalar, a vector, or a high-dimensional tensor. In the description herein, the data element can refer to an address, or to the final scalar, vector, or high-dimensional tensor data, and those skilled in the art can distinguish the meaning based on the context of the description.

[0067] Next, the buffer circuit 633 can output the rightmost index element "9". At this time, the control circuit 610 can be further configured to determine the memory access information of the next index element to be fused according to the index element output from the buffer circuit. Specifically, the control circuit determines which lane the output index element belongs to in the K-lane index, and takes the next index element to be fused from the lane and sends it to the comparison circuit 631 for comparison.

[0068] Further, when outputting in order, the buffer circuit 633 can be configured to output the compared index elements in order (e.g., from small to large) as fused indexes, and output the data elements corresponding thereto as fused data synchronously. The output data is provided to the output circuit 636 for further processing.

[0069] For clarity, the sequence of indexes buffered in the buffer circuit 633 along with the progress of the sorting is also shown in the figure. As shown, initially, the first index elements of each lane in the K-lane index are stored in the buffer circuit 633 in order from large to small. In some implementations, the four index elements can be taken out at once, sorted, and stored in the buffer circuit. In other implementations, the data in the buffer circuit can be initialized to negative numbers, and the first index elements of each lane are taken out one by one in order (e.g., in the order of lane 1 to lane 4), compared with the data in the buffer circuit, and placed in the appropriate position. In this example, the first index elements of the four lanes are all 0, so they can be arranged according to the order of taking them out, i.e., the first index element of lane 1 "0" is placed at the rightmost position, the first index element of lane 2 "0" is placed at the second position from the right, and so on.

[0070] Next, the rightmost "0" in the buffer circuit, which belongs to lane 1, is output. According to which lane the output index element belongs to, the next index element to be fused, i.e., the second index element of lane 1 "2", is taken from the corresponding lane. "2" is sent to the comparison circuit to compare with the remaining three "0"s in the buffer circuit, and the comparison result is "111", i.e., greater than the existing three numbers "0" in the buffer circuit. Therefore, "2" is inserted at the end of the sequence, and the sequence in the buffer circuit becomes "2, 0, 0, 0".

[0071] Next, the rightmost "0" in the buffer circuit, which belongs to lane 2, is output. Therefore, the second element of lane 2 "3" is taken out and compared with the remaining "2, 0, 0" in the buffer circuit, and the comparison result is "111". Therefore, "3" is inserted at the end of the sequence, and the sequence in the buffer circuit becomes "3, 2, 0, 0".

[0072] Next, the "0" belonging to the 3rd lane at the rightmost of the buffer circuit is output, thus the 2nd element "100" of the 3rd lane is taken out and compared with the remaining "2, 0, 0" in the buffer circuit, and the comparison result is "111", thus "100" is inserted at the end of the sequence, and at this time the sequence in the buffer circuit becomes "100, 3, 2, 0".

[0073] Next, the "0" belonging to the 4th lane at the rightmost of the buffer circuit is output, the 2nd element "2" of the 4th lane is taken out and compared with the remaining "100, 3, 2" in the buffer circuit, and the comparison result is "001", thus "2" is inserted after the 1st element at the rightmost of the sequence, and at this time the sequence in the buffer circuit becomes "100, 3, 2, 2".

[0074] By analogy, the index elements in the K-lane index can be compared one by one, and inserted into the appropriate position in the buffer circuit according to the size order, and then output by the buffer circuit. For example, the minimum index element output by the buffer circuit each time can be output to the output circuit 636 in order. Those skilled in the art can understand that if the space of the buffer circuit is sufficient, the sorted elements after merging can also be output uniformly after sorting.

[0075] As can be seen from the outputted merging-sorted index elements, when there are index elements of the same size, the sorting circuit 632 still retains these index elements of the same size and does not perform a deduplication operation, but provides them to the output circuit 636 for processing.

[0076] In some embodiments, the output circuit 636 can include a comparator 637, a buffer 635, and a structure generator 639.

[0077] The comparator 637 can be configured to compare the index element output in order from the sorting circuit 632 with the previous fused index element and output the comparison result. The comparison result can be "1" indicating the same, and "0" indicating different; or vice versa.

[0078] The buffer 635 can be configured to control the output of the index element according to the comparison result of the comparator 637. In some embodiments, the buffer 635 can output the current index element as a new fused index element only when the comparison result indicates that it is different. In other words, when the comparison result indicates that it is the same, the buffer 635 does not output the current index element, i.e., discards the index element that is duplicated with the previous fused index element. As shown, there is no duplicated fused index element in the fused index in the second storage circuit 624.

[0079] The structure generator 639 can be configured to generate the corresponding operation structure element according to the comparison result of the comparator 637, to control accumulation of the data elements. Specifically, at least when the comparison result indicates the same, the operation structure element is generated based on the data element corresponding to the index element, and the operation structure element indicates an accumulation operation of adding the current data element to the fused data element corresponding to the last fused index element.

[0080] Figure 6 A way of generating the operation structure element is shown in the embodiment. In this embodiment, the operation structure element is generated for other fused data elements in addition to the fused data element generated for the data elements with the same index. Specifically, in this embodiment, when the comparison result of the comparator 637 indicates the different, the operation structure element is generated based on the data element corresponding to the current index element, and the operation structure element indicates an accumulation operation of adding the current data element to 0. In this way, the operation structure element can be generated for all fused data elements, so as to unify the expression manner, simplify the operation, and provide flexible processing for subsequent operations.

[0081] Figure 6 The operation structure generated in the current embodiment is exemplarily shown in the table. As shown in the table, when the output circuit 636 receives the first "0" output by the sorting circuit 632, since it is the first element, the comparator 637 outputs, for example, "0" to indicate the different from the last fused index element (for example, initialized as a negative number). The buffer 635 outputs the index "0" as the first fused index element. The structure generator 639 assigns an address for the new fused data element based on the result of the comparator 637, for example, the address of the first fused data element is base_addr. At this time, even if the index is considered to be different, the associated operation structure element is generated for the fused data element, which is only an accumulation operation of adding the current data element to 0, for example, {add_zero, Pt11, base_addr}, where add_zero indicates the address of the value of 0, Pt11 indicates the address of the current data element, and base_addr indicates the storage address of the result, which corresponds to the address of the first fused data element assigned above. The generated operation structure element can be stored in the operation structure of the second storage circuit 624.

[0082] Each operation structure element can include three elements, respectively indicating two addends and an addition result, for example, {src0_addr, src1_addr, dst_addr}, where src0_addr indicates the address of the first addend, src1_addr indicates the address of the second addend, and dst_addr indicates the address of the sum of the addition.

[0083] Next, the sorting circuit 632 outputs a second "0", and since the previous fusion index element is "0", the comparator 637 outputs, for example, "1" to indicate the same as the previous fusion index element ("0"). At this time, the buffer 635 does not output. The structure generator 639 considers that no new fusion data element is generated based on the result of the comparator 637, and thus does not allocate an address, i.e., there is no new element in the fusion data. At this time, the structure generator 639 also generates an operation structure element indicating accumulation of data elements, which accumulates the data element into the fusion data element corresponding to the previous fusion index element, for example, {base_addr, Pt21, base_addr}, where base_addr represents the address of the fusion data element corresponding to the previous fusion index element, i.e., the address of the first fusion data element allocated in the previous step, Pt21 represents the address of the current data element, and base_addr represents the storage address of the result, i.e., the data element is accumulated into the first fusion data element allocated above. The generated operation structure element is also stored in the operation structure of the second storage circuit 624.

[0084] When the sorting circuit 632 continues to output a third and a fourth "0", the fusion index does not increase, and the fusion data element in the corresponding fusion data also does not increase, and the structure generator outputs the associated structure operation elements: {base_addr, Pt31, base_addr} and {base_addr, Pt41, base_addr}.

[0085] Next, the sorting circuit 632 outputs a first "2", and at this time, the buffer 635 outputs the index "2" as the second fusion index element with reference to the previous step. The structure generator 639 allocates an address for the new fusion data element based on the result of the comparator 637, for example, by adding an offset to the address of the first fusion data element. The offset is the storage address size of the corresponding final data element (for example, a scalar, a vector, or a high-dimensional tensor). At this time, the associated operation structure element is generated for the new fusion data element, for example, {add_zero, Pt12, base_addr+offset}. The generated operation structure element is stored in the operation structure of the second storage circuit 624.

[0086] When the sorting circuit 632 outputs a second "2", the fusion index does not increase, and the fusion data element in the corresponding fusion data also does not increase, and the structure generator outputs the associated structure operation elements: {base_addr+offset, Pt42, base_addr+offset}.

[0087] According to the above description, other fusion results can be similarly deduced by those skilled in the art, which are not described here one by one.

[0088] Figure 7 An example of generating operation structure elements according to another embodiment is shown. In this embodiment, for the data elements with no repetition in the index of the data to be fused, no operation structure element can be generated, thus avoiding redundant calculation.

[0089] Specifically, when the comparator 637 indicates that the index element of the current output is the same as the last fused index element, an operation structure element representing the accumulation of the current corresponding data element to the fused data element corresponding to the last fused index element can be generated. When the comparator 637 indicates that the index element of the current output is different from the last fused index element, the current index element can be directly output as a new fused index element, and the new fused data element can be directly assigned an address where the corresponding original data element is stored.

[0090] The generation process of the operation structure elements in the above embodiment will be described below in combination with the data example shown in Figure 6 Figure 7 As shown in the figure, when the output circuit 636 receives the first "0" output by the sorting circuit 632, since it is the first element, the comparator 637 will output, for example, "0" to indicate that it is different from the last fused index element (for example, initialized as a negative number). The buffer 635 outputs the index "0" as the first fused index element. The structure generator 639 assigns an address to the new fused data element based on the result of the comparator 637, for example, the address of the first fused data element is base_addr. At this time, since the index is different, the original data element corresponding to the index element can be directly moved to the assigned address base_addr without accumulation. In this way, the address of the fused data can be continuous, which can facilitate subsequent access.

[0091] ​Next, the sorting circuit 632 outputs a second "0", and since the previous fusion index element is "0", the comparator 637 outputs, for example, "1" to indicate that it is the same as the previous fusion index element ("0"). At this time, the buffer 635 does not output. The structure generator 639 considers that no new fusion data element is generated based on the result of the comparator 637, and thus does not allocate an address, i.e., there is no new element in the fusion data. At this time, the structure generator 639 generates an operation structure element indicating that the data element is accumulated, which accumulates the data element into the fusion data element corresponding to the previous fusion index element, for example, {base_addr, Pt21, base_addr}, where base_addr represents the address of the fusion data element corresponding to the previous fusion index element, i.e., the address of the first fusion data element allocated in the previous step, and please note that at this time, the base_addr has been put into the data element pointed to by Pt11 in the previous step. Pt21 represents the address of the current data element, and base_addr represents the storage address of the result, i.e., the data element is accumulated into the first fusion data element allocated above. The generated operation structure element is stored in the operation structure element of the second storage circuit.

[0092] When the sorting circuit 632 continues to output the third and fourth "0", the fusion index does not increase, and the fusion data element in the corresponding fusion data also does not increase, and the structure generator outputs the associated structure operation element: {base_addr, Pt31, base_addr}, {base_addr, Pt41, base_addr}.

[0093] Next, the sorting circuit 632 outputs a first "2", and at this time, the buffer 635 outputs the index "2" as the second fusion index element with reference to the previous step. The structure generator 639 allocates an address for the new fusion data element based on the result of the comparator 637, for example, adding an offset to the address of the first fusion data element. The offset is the storage address size of the corresponding final data element (for example, a scalar, a vector, or a high-dimensional tensor). At this time, since the indexes are not the same, there is no need for accumulation, and the original data element corresponding to the index element (i.e., the data element pointed to by Pt12) can be directly moved to the allocated address base_addr+offset.

[0094] When the sorting circuit 632 outputs a second "2", the fusion index does not increase, and the fusion data element in the corresponding fusion data also does not increase, and the structure generator outputs the associated structure operation element: {base_addr+offset, Pt42, base_addr+offset}.

[0095] According to the above description, those skilled in the art can similarly deduce other fusion results. For example, for the fusion indexes "3" and "5", there is no situation where the indexes are the same. Therefore, only data transfer needs to be performed without generating operation structure elements.

[0096] Figure 7 The embodiment ensures that the addresses of the fused data elements in the fused data are continuous by performing data transfer during the fusion process, and for data elements that do not need to be accumulated, there is no need to operate the structure elements to represent them, thereby saving the subsequent possible amount of calculation.

[0097] Figure 8 An example of an operation structure element generated according to another embodiment is shown. Figure 7 Similarly to the embodiment of FIG, in this embodiment, for data elements whose indexes do not have duplicates in the data to be fused, operation structure elements may not be generated, thereby avoiding redundant calculations.

[0098] Specifically, when the comparator 637 indicates that the currently output index element is the same as the previous fused index element, an operation structure element may be generated to indicate that the currently corresponding data element is added to the fused data element corresponding to the previous fused index element. When the comparator 637 indicates that the currently output index element is different from the previous fused index element, the current index element may be directly output as the new fused index element, and the address of the corresponding data element may be retained. In other words, the fused data element is the same as the data element before fusion, and both point to the same address.

[0099] The following combination Figure 6 Data example description Figure 8 The process of generating the elements of the operation structure in the process. As shown in the figure, when the output circuit 636 receives the first "0" output by the sorting circuit 632, since this is the first element, the comparator 637 will output, for example, "0" to indicate that it is different from the previous fusion index element (for example, initialized to a negative number). The buffer 635 outputs the index "0" as the first fusion index element. Based on the result of the comparator 637, the structure generator 639 uses the data element corresponding to the current index element as the new fusion data element, for example, Pt11 at this time. At this time, since the indexes are different, accumulation is not required.

[0100] Then, the sorting circuit 632 outputs the second "0", and since the last fusion index element is "0", the comparator 637 outputs, for example, "1" to indicate the same as the last fusion index element ("0"). At this time, the buffer 635 does not output. The structure generator 639 considers that no new fusion data element is generated based on the result of the comparator 637, and thus does not perform any operation on the fusion data element, but only generates an operation structure element indicating the accumulation of data elements, which accumulates the data element into the fusion data element corresponding to the last fusion index element, for example, {Ptll, Pt21, Ptll}, where Ptll represents the address of the fusion data element corresponding to the last fusion index element. Pt21 represents the address of the current data element, and the accumulation result is still placed in the initial address of the fusion data element, i.e., Ptll. The generated operation structure element is stored in the operation structure element of the second storage circuit.

[0101] When the sorting circuit 632 continues to output the third and fourth "0", the fusion index does not increase, and the fusion data element in the corresponding fusion data also does not increase, and the structure generator outputs the associated structure operation element: {Ptll, Pt31, Ptll}, {Ptll, Pt41, Ptll}.

[0102] Then, the sorting circuit 632 outputs the first "2", and the buffer 635 outputs the index "2" as the second fusion index element at this time according to the previous step. The structure generator 639 considers the data element corresponding to the current index element as a new fusion data element based on the result of the comparator 637, for example, Pt12 at this time. At this time, since the indexes are not the same, no accumulation is needed.

[0103] When the sorting circuit 632 outputs the second "2", the fusion index does not increase, and the fusion data element in the corresponding fusion data also does not increase, and the structure generator outputs the associated structure operation element: {Pt12, Pt42, Pt12}.

[0104] According to the above description, those skilled in the art can similarly deduce other fusion results, for example, for the fusion indexes "3" and "5", there is no case of the same index, and thus only the corresponding data element needs to be taken as a new fusion data element, for example, Pt22 and Pt13, without the need to generate an operation structure element.

[0105] Figure 8 The embodiment of the application compared with the prior art Figure 7 The data transfer in the fusion process is reduced, and the price is that the addresses of the fusion data elements in the fusion data are not continuous. In addition, for the data elements that do not need to be accumulated, no operation structure element is needed to represent, and the subsequent possible operation amount is saved.

[0106] Figure 9 An exemplary flowchart of a data processing method 900 performed by the data processing circuit described above is shown according to embodiments of the present disclosure.

[0107] As shown in FIG. 9, in step 910, the control circuit reads the plurality of data to be fused from the storage circuit. Then, in step 920, the operation circuit merges the data elements in the plurality of data to be fused according to their corresponding indexes into one ordered fused data, in which the data elements with the same index are merged into an operation structure element. Finally, in step 930, the operation circuit outputs the fused data to the storage circuit. Although the method steps are shown in sequence in the figure, these steps are looped for each data element, and some steps occur simultaneously in these loops. For example, when the operation circuit outputs data, the control circuit can simultaneously access the storage circuit to read the next data element to be fused. Figure 9

[0108] Those skilled in the art can understand that each step of the above method corresponds to each circuit described above in connection with the exemplary circuit diagram, and thus the features described above can be equally applicable to the method steps, which are not repeated here.

[0109] As can be seen from the above description, the embodiments of the present disclosure provide a hardware circuit for performing data fusion operations related to merge sort fusion processing. By implementing merge sort fusion in hardware, processing speed can be accelerated, thereby better supporting operations related to processing after sparsification, such as supporting operations in radar algorithms. In some embodiments, the hardware circuit can merge a plurality of ordered data into one ordered fused data, and data with the same index can be merged and represented in the form of an operation structure, thereby facilitating subsequent calculation processing. By providing a hardware implementation of special data fusion related operations, processing can be simplified and accelerated, thereby improving the processing efficiency of the machine.

[0110] ​According to different application scenarios, the electronic device or apparatus of the present disclosure can include a server, a cloud server, a server cluster, a data processing apparatus, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a vehicle record device, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an autonomous driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship, and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric lamp, a gas stove, an oil smoke exhaust fan; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument, and / or an electrocardiograph. The electronic device or apparatus of the present disclosure can also be applied to the fields of Internet, Internet of Things, data center, energy, transportation, public management, manufacturing, education, power grid, telecommunications, finance, retail, construction site, medical treatment, etc. Further, the electronic device or apparatus of the present disclosure can also be used in cloud, edge, terminal, etc. application scenarios related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, the electronic device or apparatus with high computing power according to the present disclosure can be applied to a cloud device (such as a cloud server), and the electronic device or apparatus with small power consumption can be applied to a terminal device and / or an edge device (such as a smart phone or a camera). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or the edge device are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud device according to the hardware information of the terminal device and / or the edge device to simulate the hardware resources of the terminal device and / or the edge device, so as to complete the unified management, scheduling and collaborative work of end-cloud integration or cloud-edge integration.

[0111] It should be noted that for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art can understand that the schemes of the present disclosure are not limited by the order of the described actions. Therefore, those skilled in the art can understand that some steps can be executed in other orders or simultaneously according to the disclosure or teaching of the present disclosure. Further, those skilled in the art can understand that the described embodiments of the present disclosure can be regarded as optional embodiments, i.e. the actions or modules involved therein are not necessarily essential for the implementation of one or more schemes of the present disclosure. In addition, the description of some embodiments of the present disclosure also focuses on different schemes. Therefore, those skilled in the art can understand that the parts not described in detail in one embodiment of the present disclosure can also refer to the relevant description of other embodiments.

[0112] In terms of specific implementation, based on the disclosure and teaching of the present disclosure, those skilled in the art can understand that the several embodiments disclosed by the present disclosure can also be implemented in other manners not disclosed herein. For example, in terms of the units described in the foregoing electronic device or apparatus embodiments, the units can be split into other forms in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions of the units or components can be selectively disabled. In terms of the connection relationship between the units or components, the connection discussed above can be direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling refers to the communication connection using an interface, and the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0113] In the present disclosure, the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units. The foregoing components or units can be located in the same place or distributed on multiple network units. In addition, according to actual needs, part or all of the units can be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure can be integrated into one unit or physically exist separately.

[0114] In some other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, i.e., specific hardware circuits, which can include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit can include but is not limited to physical devices, and the physical devices can include but are not limited to transistors or memristors, etc. In view of this, various devices described herein (e.g., computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs, etc. Further, the aforementioned storage units or storage devices can be any appropriate storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM, etc.

[0115] The foregoing can be better understood in light of the following clauses:

[0116] Clause 1, a data processing circuit comprising a control circuit, a storage circuit and an operation circuit, wherein:

[0117] the control circuit is configured to control the storage circuit and the operation circuit to perform fusion processing on multiple channels of data to be fused;

[0118] the storage circuit is configured to store information, the information at least including pre-processing and / or post-processing information; and

[0119] the operation circuit is configured to, under the control of the control circuit, merge data elements in the multiple channels of data to be fused according to their corresponding indexes into one channel of ordered post-processing data, wherein data elements with the same index are merged into operation structure elements, and the data elements include any of scalars, vectors or higher-dimensional data.

[0120] Clause 2, the data processing circuit according to Clause 1, wherein the storage circuit comprises a first storage circuit and a second storage circuit,

[0121] The first storage circuit is configured to store the K-way data to be fused and K-way indexes corresponding to the K-way data, K>1, wherein an index element in the K-way index indicates index information of a corresponding data element in the K-way data, the index elements of each way index in the K-way index are sequentially arranged in a first order, and the data elements of each way data in the K-way data are sequentially arranged in the order of the corresponding index; and

[0122] The second storage circuit is configured to store the fused data output by the operation circuit after the fusion processing, including a fused index after the fusion of the K-way index, fused data, and an operation structure, wherein the fused index elements in the fused index are sequentially arranged in a second order, the fused data elements in the fused data correspond one by one to the fused index elements, and at least the fused data elements composed of data elements of the same index are represented by the associated operation structure elements in the operation structure.

[0123] Clause 3, the data processing circuit according to any one of clauses 2-3, wherein the first order and the second order are the same or different, and the first order and the second order are selected from any one of the following: an order from small to large, or an order from large to small.

[0124] Clause 4, the data processing circuit according to any one of clauses 2-3, wherein the operation circuit comprises a sorting circuit and an output circuit, wherein

[0125] The sorting circuit is configured to sort the K-way indexes according to the size of the index elements and sequentially output the K-way indexes to the output circuit; and

[0126] The output circuit is configured to generate an operation structure element representing an accumulated operation of data elements corresponding to the same index element when receiving the same index element from the sorting circuit, and to remove the repeated index elements.

[0127] Clause 5, the data processing circuit according to clause 4, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, wherein:

[0128] The comparison circuit is configured to compare the index elements to be sorted in the K-way index with the index elements not output in the buffer circuit, and output the comparison result to the control circuit; and

[0129] The buffer circuit is configured to store the compared index elements and the information of the corresponding data elements in order according to the control of the control circuit, and to output the compared index elements and the information of the corresponding data elements in order.

[0130] Clause 6, the data processing circuit according to clause 5, wherein the comparison circuit comprises:

[0131] K-1 way comparators configured to compare the index element to be sorted in the K way index with K-1 index elements in the current sequence in the buffer circuit respectively, generate comparison results and output to the control circuit.

[0132] Clause 7, the data processing circuit according to clause 6, wherein the control circuit is configured to determine the insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison results.

[0133] Clause 8, the data processing circuit according to clause 7, wherein the comparison results are represented using a bitmap, and the control circuit is further configured to determine the insertion position according to the changing position of a bit in the bitmap.

[0134] Clause 9, the data processing circuit according to any one of clauses 7-8, wherein the buffer circuit is configured to insert the index element to be sorted and the information of the data element corresponding thereto in the insertion position according to the indication of the control circuit.

[0135] Clause 10, the data processing circuit according to any one of clauses 5-9, wherein the buffer circuit is further configured to output the information of the first or last index element in the current sequence and the data element corresponding thereto in the specified order.

[0136] Clause 11, the data processing circuit according to clause 10, wherein the control circuit is further configured to determine the memory access information of the next index element to be sorted according to the index element output in the buffer circuit.

[0137] Clause 12, the data processing circuit according to any one of clauses 4-11, wherein the output circuit comprises a comparator, a buffer and a structure generator, wherein:

[0138] the comparator is configured to compare the index element output from the sorting circuit with the last fused index element and output a comparison result;

[0139] the buffer is configured to output the index element as a new fused index element only when the comparison result indicates that they are different; and

[0140] the structure generator is configured to generate an operation structure element based on the data element corresponding to the index element when the comparison result indicates that they are the same, the operation structure element representing an accumulation operation of accumulating the data element to the fused data element corresponding to the last fused index element.

[0141] Clause 13, the data processing circuit according to clause 12, wherein the structure generator is further configured to, when the comparison result indicates difference, generate an operation structure element based on the data element corresponding to the index element, wherein the operation structure element represents an accumulation operation of adding the data element and 0.

[0142] Clause 14, the data processing circuit according to any one of clauses 12-13, wherein each operation structure element comprises three elements respectively indicating two addends and an addition result.

[0143] Clause 15, the data processing circuit according to any one of clauses 1-14, wherein each data element in the data to be fused points to a data vector with a predetermined length, and each fused data element in the fused data points to a fused data vector with the predetermined length, at least part of the fused data vector being determined based on the associated operation structure element.

[0144] Clause 16, the data processing circuit according to any one of clauses 1-15, wherein the data element in the data to be fused is a sparsified valid data element in radar-based object detection, and the index indicates position information of the valid data element in data before sparsification.

[0145] Clause 17, a chip comprising the data processing circuit according to any one of clauses 1-16.

[0146] Clause 18, a board card comprising the chip according to clause 17.

[0147] Clause 19, a method for processing data using a data processing circuit, the data processing circuit comprising a control circuit, a storage circuit and an operation circuit, the method comprising:

[0148] the control circuit reads multiple pieces of data to be fused from the storage circuit;

[0149] the operation circuit merges data elements in the multiple pieces of data to be fused according to their corresponding indexes into one piece of ordered fused processed data, wherein data elements with the same index are merged into an operation structure element, and the data elements include any one of scalars, vectors or higher-dimensional data; and

[0150] outputting the fused processed data to the storage circuit.

[0151] Clause 20, the method according to clause 19, wherein the storage circuit comprises a first storage circuit and a second storage circuit,

[0152] The first storage circuit is configured to store the K-way data to be fused and K-way indexes corresponding to the K-way data, K>1, wherein an index element in the K-way index indicates index information of a corresponding data element in the K-way data, the index elements of each way index in the K-way index are sequentially arranged in a first order, and the data elements of each way data in the K-way data are sequentially arranged in the order of the corresponding index; and

[0153] The second storage circuit is configured to store the fused data output by the operation circuit after the fusion processing, including a fused index after the fusion of the K-way index, fused data, and an operation structure, wherein the fused index elements in the fused index are sequentially arranged in a second order, the fused data elements in the fused data correspond one by one to the fused index elements, and at least the fused data elements composed of data elements of the same index are represented by the associated operation structure elements in the operation structure.

[0154] Clause 21. The method of clause 20, wherein the first order is the same as or different from the second order, and the first order and the second order are selected from any one of the following: an order from small to large, or an order from large to small.

[0155] Clause 22. The method of any one of clauses 20-21, wherein the operation circuit includes a sorting circuit and an output circuit, and the method further comprises:

[0156] The sorting circuit sorts the K-way indexes according to the size of the index elements and sequentially outputs them to the output circuit; and

[0157] At least when receiving the same index element from the sorting circuit, the output circuit generates an operation structure element representing the cumulative operation of the data elements corresponding to the same index element, and removes the duplicate index elements.

[0158] Clause 23. The method of clause 22, wherein the sorting circuit includes a comparison circuit and a buffer circuit, and the method further comprises:

[0159] The comparison circuit compares the index elements to be sorted in the K-way index with the index elements not output in the buffer circuit, and outputs the comparison result to the control circuit; and

[0160] The buffer circuit sequentially stores the compared index elements and the information of the data elements corresponding thereto according to the control of the control circuit, and sequentially outputs the compared index elements and the information of the data elements corresponding thereto.

[0161] Clause 24, the method of clause 23, wherein the comparison circuit comprises K-1 way comparators, and the method comprises:

[0162] The K-1 way comparators compare the index element to be sorted in the K way index with K-1 index elements in the current sequence in the buffer circuit respectively, generate comparison results and output to the control circuit.

[0163] Clause 25, the method of clause 24, further comprising:

[0164] The control circuit determines the insertion position of the index element to be sorted in the current sequence in the buffer circuit according to the comparison results.

[0165] Clause 26, the method of clause 25, wherein the comparison results are represented using a bitmap, and the method further comprises: the control circuit determines the insertion position according to the changing position of the bit in the bitmap.

[0166] Clause 27, the method of any one of clauses 25-26, further comprising:

[0167] The buffer circuit inserts the index element to be sorted and the information of the data element corresponding thereto in the insertion position according to the indication of the control circuit.

[0168] Clause 28, the method of any one of clauses 23-27, further comprising:

[0169] The buffer circuit outputs the first or last index element in the current sequence and the information of the data element corresponding thereto in the specified order.

[0170] Clause 29, the method of clause 28, further comprising:

[0171] The control circuit determines the memory access information of the next index element to be sorted according to the index element output in the buffer circuit.

[0172] Clause 30, the method of any one of clauses 22-29, wherein the output circuit comprises a comparator, a buffer and a structure generator, and the method comprises:

[0173] The comparator compares the index element output from the sorting circuit with the last fusion index element and outputs the comparison results;

[0174] The buffer outputs the index element as a new fusion index element only when the comparison results indicate that they are different; and

[0175] When the comparison result indicates the same, the structure generator generates an operation structure element based on the data element corresponding to the index element, the operation structure element representing an accumulation operation of accumulating the data element to a last fused data element corresponding to a fused index element.

[0176] Clause 31, the method according to clause 30, further comprising:

[0177] When the comparison result indicates the same, the structure generator generates an operation structure element based on the data element corresponding to the index element, the operation structure element representing an accumulation operation of accumulating the data element to a last fused data element corresponding to a fused index element.

[0178] Clause 32, the method according to any one of clauses 30-31, wherein each operation structure element comprises three elements respectively indicating two addends and an addition result.

[0179] Clause 33, the method according to any one of clauses 19-32, wherein each data element in the data to be fused points to a data vector with a predetermined length, and each fused data element in the fused data points to a fused data vector with the predetermined length, at least part of the fused data vector being determined based on an associated operation structure element.

[0180] Clause 34, the method according to any one of clauses 19-22, wherein the data element in the data to be fused is a sparsified valid data element in radar-based object detection, and the index indicates position information of the valid data element in data before sparsification.

[0181] The above detailed description of the embodiments of the present disclosure is provided, and the principles and implementation manners of the present disclosure are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present disclosure and its core idea; meanwhile, for those skilled in the art, according to the idea of the present disclosure, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the present disclosure.

Claims

1. A data processing circuit, comprising a control circuit, a storage circuit, and an operation circuit, wherein: The control circuit is configured to control the storage circuit and the operation circuit to perform fusion processing on multiple channels of data to be fused; The storage circuit is configured to store information, the information including at least pre-processed and / or post-processed information; as well as The operation circuit is configured to, under the control of the control circuit, merge the data elements in the multiple channels of data to be fused into one channel of ordered fused data according to their corresponding indexes, wherein data elements with the same index are merged into operation structure elements, and the data elements include any of scalars, vectors, or higher-dimensional data. The storage circuit includes a first storage circuit and a second storage circuit, The first storage circuit is configured to store the K-way data to be fused and the K-way index corresponding to the K-way data, K>1, wherein the index element in the K-way index indicates the index information of the corresponding data element in the K-way data, the index element of each index in the K-way index is arranged in order according to the first order, and the data elements of each data in the K-way data are arranged in order according to the order of the corresponding index; and The second storage circuit is configured to store the fused data output by the operation circuit, including a fused index after sorting and fusion of the K-way index, fused data, and an operation structure, wherein the fused index elements in the fused index are arranged in order according to a second order, the fused data elements in the fused data correspond one-to-one to the fused index elements, and the fused data elements composed of at least data elements with the same index are represented by associated operation structure elements in the operation structure. 2 . The data processing circuit according to claim 1 , wherein the first order is the same as or different from the second order, and the first order and the second order are selected from either the following: an order from small to large, or an order from large to small.

3. The data processing circuit according to claim 1 , wherein the operation circuit comprises a sorting circuit and an output circuit, wherein The sorting circuit is configured to sort the K-way indexes according to the size of the index elements and output them to the output circuit in order; and The output circuit is configured to generate, at least when receiving the same index element from the sorting circuit, an operation structure element representing an accumulation operation of data elements corresponding to the same index element, and remove duplicate index elements.

4. The data processing circuit according to claim 3, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, wherein: The comparison circuit is configured to compare the index elements to be sorted in the K-way index with the index elements not outputted in the buffer circuit, and output a comparison result to the control circuit; and The buffer circuit is configured to sequentially store information of the compared index elements and the corresponding data elements under the control of the control circuit, and sequentially output information of the compared index elements and the corresponding data elements.

5. The data processing circuit according to claim 4 , wherein the comparison circuit comprises: The K-1 comparators are configured to compare the index elements to be sorted in the K-way index with the K-1 index elements of the current sequence in the buffer circuit, generate a comparison result, and output it to the control circuit. 6 . The data processing circuit according to claim 5 , wherein the control circuit is configured to determine an insertion position of the index element to be sorted in a current sequence in the buffer circuit according to the comparison result. 7 . The data processing circuit according to claim 6 , wherein the comparison result is represented by a bitmap, and the control circuit is further configured to determine the insertion position according to a changed position of a bit in the bitmap. 8 . The data processing circuit according to claim 6 , wherein the buffer circuit is configured to insert the information of the index elements to be sorted and the data elements corresponding thereto into the insertion position according to an instruction of the control circuit.

9. The data processing circuit according to claim 4, wherein the buffer circuit is further configured to output information of the first or last index element and the corresponding data element in a current sequence in a specified order. 10 . The data processing circuit according to claim 9 , wherein the control circuit is further configured to: determine memory access information of a next index element to be sorted according to the index element output from the buffer circuit.

11. The data processing circuit according to claim 3, wherein the output circuit comprises a comparator, a buffer, and a structure generator, wherein: The comparator is configured to compare the index element output from the sorting circuit with the last fused index element and output a comparison result; The buffer is configured to output the index element as a new fused index element only when the comparison result indicates a difference; and The structure generator is configured to generate an operation structure element based on the data element corresponding to the index element when the comparison result indicates the same, and the operation structure element represents an accumulation operation of adding the data element to the fused data element corresponding to the last fused index element.

12. The data processing circuit according to claim 11, wherein the structure generator is further configured to, when the comparison result indicates inequality, generate an operation structure element based on the data element corresponding to the index element, wherein the operation structure element represents an accumulation operation of adding the data element to 0. 13 . The data processing circuit according to claim 11 , wherein each operation structure element comprises three elements, respectively indicating two addends and an addition result.

14. The data processing circuit according to claim 1, wherein each data element in the data to be fused points to a data vector having a predetermined length, and each fused data element in the fused data points to a fused data vector having the predetermined length, and at least part of the fused data vector is determined based on associated operation structure elements.

15. The data processing circuit according to claim 1, wherein the data elements in the data to be fused are valid data elements after sparsification in radar-based object detection, and the index indicates position information of the valid data elements in the data before sparsification.

16. A chip comprising the data processing circuit according to any one of claims 1-15.

17. A board comprising the chip according to claim 16.

18. A method for processing data using a data processing circuit, the data processing circuit comprising a control circuit, a storage circuit, and an operation circuit, the method comprising: The control circuit reads multiple channels of data to be fused from the storage circuit; The operation circuit merges the data elements in the multiple channels of data to be fused into one channel of ordered fused data according to their corresponding indexes, wherein data elements with the same index are merged into an operation structure element, wherein the data element includes any of scalars, vectors, or higher-dimensional data; and Outputting the fused data to the storage circuit, The storage circuit includes a first storage circuit and a second storage circuit, The first storage circuit is configured to store the K-way data to be fused and the K-way index corresponding to the K-way data, K>1, wherein the index element in the K-way index indicates the index information of the corresponding data element in the K-way data, the index element of each index in the K-way index is arranged in order according to the first order, and the data elements of each data in the K-way data are arranged in order according to the order of the corresponding index; and The second storage circuit is configured to store the fused data output by the operation circuit, including a fused index after sorting and fusion of the K-way index, fused data, and an operation structure, wherein the fused index elements in the fused index are arranged in order according to a second order, the fused data elements in the fused data correspond one-to-one to the fused index elements, and the fused data elements composed of at least data elements with the same index are represented by associated operation structure elements in the operation structure.

19. The method according to claim 18, wherein the first sequence is the same as or different from the second sequence, and the first sequence and the second sequence are selected from any one of the following: a sequence from small to large, or a sequence from large to small.

20. The method according to claim 18, wherein the operation circuit comprises a sorting circuit and an output circuit, and the method further comprises: The sorting circuit sorts the K-way indexes according to the size of the index elements and outputs them to the output circuit in order; and At least when receiving the same index element from the sorting circuit, the output circuit generates an operation structure element representing an accumulation operation of the data elements corresponding to the same index element, and removes duplicate index elements.

21. The method of claim 20, wherein the sorting circuit comprises a comparison circuit and a buffer circuit, and the method further comprises: The comparison circuit compares the index elements to be sorted in the K-way index with the index elements not outputted in the buffer circuit, and outputs the comparison result to the control circuit; as well as The buffer circuit sequentially stores the information of the compared index elements and the corresponding data elements under the control of the control circuit, and sequentially outputs the information of the compared index elements and the corresponding data elements.

22. The method of claim 21 , wherein the comparison circuit comprises a K-1 comparator, and the method comprises: The K-1-way comparator compares the index elements to be sorted in the K-way index with the K-1 index elements of the current sequence in the buffer circuit respectively, generates a comparison result and outputs it to the control circuit.

23. The method of claim 22, further comprising: The control circuit determines an insertion position of the index element to be sorted in a current sequence in the buffer circuit according to the comparison result.

24. The method according to claim 23, wherein the comparison result is represented using a bitmap, and the method further comprises: The control circuit determines the insertion position according to the changed position of the bit in the bitmap.

25. The method of claim 23, further comprising: The buffer circuit inserts the index elements to be sorted and the information of the data elements corresponding thereto into the insertion position according to the instruction of the control circuit.

26. The method of claim 21, further comprising: The buffer circuit outputs information of the first or last index element and the corresponding data element in the current sequence in a specified order.

27. The method of claim 26, further comprising: The control circuit determines the memory access information of the next index element to be sorted according to the index element output from the buffer circuit.

28. The method of claim 20, wherein the output circuit comprises a comparator, a buffer, and a structure generator, and the method comprises: The comparator compares the index element output from the sorting circuit with the previous fused index element and outputs a comparison result; The buffer outputs the index element as a new fused index element only when the comparison result indicates that the index element is not identical; and When the comparison result indicates the same, the structure generator generates an operation structure element based on the data element corresponding to the index element, and the operation structure element represents an accumulation operation of adding the data element to the fused data element corresponding to the last fused index element.

29. The method of claim 28, further comprising: When the comparison result indicates that the data elements are not identical, the structure generator generates an operation structure element based on the data element corresponding to the index element, wherein the operation structure element represents an accumulation operation of adding the data element to 0.

30. The method according to claim 28, wherein each operation structure element comprises three elements, respectively indicating two addends and an addition result.

31. A method according to claim 18, wherein each data element in the data to be fused points to a data vector with a predetermined length, and each fused data element in the fused data points to a fused data vector with the predetermined length, and at least part of the fused data vector is determined based on the associated operation structure elements.

32. The method according to any one of claims 18 to 31, wherein the data elements in the data to be fused are valid data elements after sparsification in radar-based object detection, and the index indicates position information of the valid data elements in the data before sparsification.

Images